Video signal processing method and apparatus using quadratic transformation
The use of quadratic transformation and low-band non-separable transforms in video signal processing addresses inefficiencies in existing methods, enhancing coding efficiency and compression performance.
Patent Information
- Application Number
- JP2023149119
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Priority Date
- 2019-07-07
- Filing Date
- 2023-09-14
- Publication Date
- 2024-11-06
- Estimated Expiration
- 2040-06-25
AI Technical Summary
Existing video signal processing methods lack efficiency in coding, particularly in handling spatial and temporal correlations of video signals.
The method employs quadratic transformation, including a low-band non-separable transform (LFNST) applied to transform blocks based on specific conditions such as coefficient positions, block sizes, and syntax elements, to enhance coding efficiency.
This approach increases coding efficiency by effectively transforming and encoding video signals, particularly for intra-predicted blocks, reducing quantization errors and improving compression performance.
Smart Images

Figure 0007578373000008 
Figure 0007578373000009 
Figure 0007578373000010
Abstract
Description
[Technical field]
[0001] The present invention relates to a method and apparatus for processing a video signal, and more particularly to a method and apparatus for processing a video signal for encoding or decoding a video signal. [Background technology]
[0002] Compression coding refers to a series of signal processing techniques for transmitting digitized information via a communication line or storing it in a form suitable for a storage medium. The objects of compression coding include voice, video, text, etc., and the technology for compressing and coding video in particular is called video compression. Compression coding of video signals is performed by removing redundant information by considering spatial correlation, temporal correlation, probabilistic correlation, etc. However, with the recent development of various media and data transmission media, there is a demand for more efficient video signal processing methods and devices. Summary of the Invention [Problem to be solved by the invention]
[0003] An object of the present invention is to increase the efficiency of coding of video signals.
[0004] The present invention has the objective of increasing the coding efficiency via quadratic transforms. [Means for solving the problem]
[0005] This specification provides a method for processing a video signal that utilizes a quadratic transformation.
[0006] In detail, the video signal decoding apparatus includes a processor, and the processor parses syntax elements related to a secondary transform of a coding unit from a bitstream of a video signal if one or more preset conditions are satisfied, and determines whether the secondary transform is applied to a transform block included in the coding unit based on the parsed syntax elements. If the secondary transform is applied to the transform block, the processor performs a secondary inverse transform based on one or more coefficients of a first sub-block, which is one of one or more sub-blocks constituting the transform block, to obtain one or more inverse transform coefficients for the first sub-block, and performs a primary inverse transform based on the one or more inverse transform coefficients to obtain residual samples for the transform block. The secondary transform is a low frequency non-separable transform. In the present invention, the transform block is a block to which a first transform that can be performed separately for a vertical transform and a horizontal transform is applied, and a first condition of the one or more preset conditions is that an index value indicating a position of a first coefficient among the one or more coefficients of the first sub-block is greater than a preset threshold value.
[0007] Also, in this specification, the syntax element is characterized in that it includes information indicating whether the secondary transform is applied to the coding unit, and information indicating a transform kernel to be used for the secondary transform.
[0008] In the present specification, the first coefficient is a last significant coefficient according to a preset scan order, and the significant coefficient is a non-zero coefficient.
[0009] In the present specification, the first sub-block is a first sub-block in a preset scan order.
[0010] In addition, in the present specification, a second condition of the one or more preset conditions is that the width and height of the transformation block are 4 pixels or more.
[0011] In the present specification, the preset threshold value is 0.
[0012] In addition, in this specification, the preset scan order is an up-right diagonal scan order.
[0013] In addition, in this specification, a third condition among the one or more pre-set conditions is a case where a transform skip flag value included in the bitstream is not a specific value, and if the transform skip flag value has the specific value, the transform skip flag indicates that the first transform and the second transform are not applied to the transform block.
[0014] Also, in the present specification, a fourth condition among the one or more preset conditions is characterized in that at least one of the one or more coefficients of the first sub-block is not 0, and the at least one coefficient is present at a position other than a first position according to a preset scan order.
[0015] In addition, in this specification, the coding unit is composed of a plurality of coding blocks, and if at least one of the transformation blocks corresponding to each of the plurality of coding blocks satisfies one or more of the preset conditions, syntax elements related to the secondary transformation are parsed.
[0016] Also, in the present specification, a video signal decoding apparatus includes a processor, the processor performing a first transform on residual samples of a block included in a coding unit to obtain a plurality of first transform coefficients for the block, performing a second transform based on one or more coefficients of the plurality of first transforms to obtain one or more second transform coefficients for a first sub-block, which is one of sub-blocks constituting the block, and encoding information on the one or more second transform coefficients and a syntax element related to the second transform of the coding unit to obtain a bitstream, the second transform being a low band non-separable transform (LFNST), the first transform being separated into a vertical transform and a horizontal transform, and the syntax element related to the second transform of the coding unit is encoded if one or more predetermined conditions are satisfied, and a first condition of the one or more predetermined conditions is a case where an index value indicating a position of a first coefficient of the one or more second transform coefficients is greater than a predetermined critical value.
[0017] Also, in this specification, the syntax element is characterized in that it includes information indicating whether the secondary transform is applied to the coding unit, and information indicating a transform kernel to be used for the secondary transform.
[0018] In the present specification, the first coefficient is a last significant coefficient according to a preset scan order, and the significant coefficient is a coefficient other than 0.
[0019] In the present specification, the first sub-block is a first sub-block in a preset scan order.
[0020] In the present specification, a second condition among the one or more preset conditions is that the width and height of the primary transformation block are 4 pixels or more.
[0021] In the present specification, the preset threshold value is 0.
[0022] In the present specification, the preset scan order is a top right diagonal scan order.
[0023] Also, in this specification, a third condition among the one or more pre-set conditions is a case where a transform skip flag value included in the bitstream is not a specific value, and if the transform skip flag value has the specific value, the transform skip flag indicates that the first transform and the second transform are not applied to the block.
[0024] In addition, in the present specification, a fourth condition among the one or more preset conditions is characterized in that at least one of the one or more secondary transform coefficients is not 0, and the one or more coefficients are present at a position other than a first position according to a preset scan order.
[0025] Also, in the present specification, in a non-transitory computer-readable medium for storing a bitstream, the bitstream is encoded through an encoding method including: performing a primary transform on residual samples of a block included in a coding unit to obtain a plurality of primary transform coefficients for the block; performing a secondary transform based on at least one coefficient of the plurality of primary transforms to obtain one or more secondary transform coefficients for a first sub-block that is one of sub-blocks constituting the block; and encoding information on the one or more secondary transform coefficients and syntax elements related to the secondary transform of the coding unit, the secondary transform being a low band non-separable transform (LFNST), the primary transform being separated into a vertical transform and a horizontal transform, and the syntax elements related to the secondary transform being encoded if one or more predetermined conditions are satisfied, and a first condition of the one or more predetermined conditions is a case where an index value indicating a position of a first coefficient of the one or more secondary transform coefficients is greater than a predetermined threshold value. Effect of the Invention
[0026] One embodiment of the present invention provides a method and apparatus for processing a video signal using a quadratic transformation. [Brief description of the drawings]
[0027] [Figure 1] 1 is a schematic block diagram of a video signal encoding apparatus according to an embodiment of the present invention; [Diagram 2] 1 is a schematic block diagram of a video signal decoding device according to an embodiment of the present invention; [Diagram 3] FIG. 2 shows an example of how coding tree units are divided into coding units within a picture. [Figure 4] FIG. 1 illustrates an embodiment of a method for signaling splits of quad trees and multi-type trees. [Diagram 5] FIG. 2 illustrates in more detail an intra-prediction method according to an embodiment of the present invention; [Figure 6] FIG. 2 illustrates in more detail an intra-prediction method according to an embodiment of the present invention; [Figure 7] FIG. 2 is a diagram showing in more detail how the encoder converts the residual signal. [Figure 8] 2 is a diagram illustrating in detail how the encoder and decoder inverse transform the transform coefficients to obtain a residual signal. [Figure 9] FIG. 1 illustrates basis functions for multiple transform kernels that can be used in a linear transform. [Figure 10] 4 is a block diagram showing a process of restoring a residual signal in a decoder performing a secondary transform according to an embodiment of the present invention; [Figure 11] 4 is a diagram illustrating a process of restoring a residual signal at a block level in a decoder that performs a secondary transform according to an embodiment of the present invention. [Figure 12] FIG. 2 illustrates a method of applying a quadratic transform using a reduced number of samples according to an embodiment of the present invention. [Figure 13] FIG. 13 illustrates a method for determining the upper right diagonal scan order according to one embodiment of the present invention. [Figure 14] FIG. 13 illustrates a top right diagonal scan order according to block size according to one embodiment of the present invention. [Figure 15] FIG. 1 illustrates a method for indicating secondary transforms at the coding unit level. [Figure 16] A diagram showing a residual_coding syntax structure according to one embodiment of the present invention. [Figure 17] 4 illustrates a method for indicating a secondary transformation at a coding unit level according to an embodiment of the present invention. [Figure 18] 4 illustrates a method for indicating a secondary transformation at a coding unit level according to an embodiment of the present invention. [Figure 19] A diagram showing a residual_coding syntax structure according to one embodiment of the present invention. [Figure 20] A diagram showing a residual_coding syntax structure according to another embodiment of the present invention. [Figure 21] 13 illustrates a method for indicating a secondary transformation at a coding unit level according to another embodiment of the present invention. [Figure 22] A diagram showing a residual_coding syntax structure according to another embodiment of the present invention. [Diagram 23] FIG. 2 illustrates a method for indicating a secondary transform at the transform unit level according to an embodiment of the present invention. [Figure 24] FIG. 10 illustrates a method for indicating a secondary transformation at a transform unit level according to another embodiment of the present invention. [Diagram 25] A diagram showing coding unit syntax according to one embodiment of the present invention. [Figure 26] FIG. 10 illustrates a method for indicating a secondary transformation at a transform unit level according to another embodiment of the present invention. [Figure 27] A diagram showing a syntax structure regarding the position of the last significant coefficient in the scan order according to an embodiment of the present invention. [Figure 28] A diagram showing a residual_coding syntax structure according to another embodiment of the present invention. [Figure 29] 2 is a flow chart illustrating a method for processing a video signal according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0028] The terms used in this specification are selected from currently widely used general terms as much as possible while taking into consideration the functions in the present invention, but this may vary depending on the intentions of the engineers in this field, customs, or the emergence of new technologies. In addition, in certain cases, the applicant may arbitrarily select terms, and in this case, the meanings of the terms will be described in the corresponding embodiment of the invention. Therefore, it is made clear that the terms used in this specification should be interpreted based on the substantial meanings of the terms and the contents of this specification as a whole, rather than simply the names of the terms.
[0029] In this specification, some terms are interpreted as follows. Coding may be interpreted as encoding or decoding depending on the circumstances. In this specification, an apparatus for encoding a video signal to generate a video signal bitstream is referred to as an encoding apparatus or encoder, and an apparatus for decoding a video signal bitstream to restore a video signal is referred to as a decoding apparatus or decoder. In this specification, a video signal processing apparatus is used as a term including both an encoder and a decoder. Information is a term including values, parameters, coefficients, elements, etc., and may be interpreted differently depending on the circumstances, so the present invention is not limited thereto. A "unit" is used to mean a basic unit of image processing or a specific position of a picture, and refers to an image area including at least one of a luma component and a chroma component. In addition, a "block" refers to an image area including a specific component of a luma component and a chroma component (i.e., Cb and Cr). However, depending on the embodiment, the terms "unit", "block", "partition", and "area" may be used interchangeably. In this specification, a unit is used as a concept including a coding unit, a prediction unit, and a transform unit, and a picture refers to a field or a frame, and the terms are used interchangeably depending on the embodiment.
[0030] 1 is a schematic block diagram of a video signal encoding apparatus 100 according to an embodiment of the present invention. Referring to FIG. 1, the encoding apparatus 100 of the present specification includes a transform unit 110, a quantization unit 115, an inverse quantization unit 120, an inverse transform unit 125, a filtering unit 130, a prediction unit 150, and an entropy coding unit 160.
[0031] The transform unit 110 transforms a residual signal, which is a difference between an input video signal and a prediction signal generated by the prediction unit 150, to obtain a transform coefficient value. For example, a discrete cosine transform (DCT), a discrete sine transform (DST), or a wavelet transform is used. The discrete cosine transform and the discrete sine transform divide the input picture signal into blocks and perform the transform. In the transform, the coding efficiency may vary depending on the distribution and characteristics of values in the transform domain. The quantizer 115 quantizes the values of the transform coefficients output from the transform unit 110.
[0032] In order to improve coding efficiency, instead of coding the picture signal as it is, a method is used in which a picture is predicted using a pre-coded area through the prediction unit 150, and a residual value between the original picture and the predicted picture is added to the predicted picture to obtain a reconstructed picture. In order to prevent mismatch between the encoder and the decoder, when the encoder performs prediction, information that can also be used by the decoder should be used. For this purpose, the encoder performs a process of further reconstructing the current block that has been coded. The inverse quantization unit 120 inversely quantizes the transform coefficient values, and the inverse transform unit 125 reconstructs the residual values using the inversely quantized transform coefficient values. Meanwhile, the filtering unit 130 performs a filtering operation to improve the quality of the reconstructed picture and the coding efficiency. For example, the filtering unit 130 may include a deblocking filter, a sample adaptive offset (SAO), an adaptive loop filter, and the like. The filtered picture is stored in a decoded picture buffer (DPB) 156 to be output or used as a reference picture.
[0033] In order to improve coding efficiency, instead of coding the picture signal as it is, a method is used in which a picture is predicted using a previously coded area through the prediction unit 150, and a residual value between the original picture and the predicted picture is added to the predicted picture to obtain a restored picture. The intra prediction unit 152 performs intra prediction within the current picture, and the inter prediction unit 154 predicts the current picture using a reference buffer stored in the decoded picture buffer 156. The intra prediction unit 152 performs intra prediction from a restored area within the current picture, and transfers intra coding information to the entropy coding unit 160. The inter prediction unit 154 includes a second motion estimation unit 154a and a motion compensation unit 154b. The motion estimation unit 154a acquires a motion vector value of the current area by referring to a specific restored area. The motion estimation unit 154a transfers position information of the reference area (reference frame, motion vector, etc.) to the entropy coding unit 160 so that it is included in the bitstream. The motion compensation unit 154b performs inter-motion compensation using the motion vector values transmitted from the motion estimation unit 154a.
[0034] The prediction unit 150 includes an intra prediction unit 152 and an inter prediction unit 154. The intra prediction unit 152 performs intra prediction within a current picture, and the inter prediction unit 154 performs inter prediction to predict the current picture using a reference buffer stored in a decoded picture buffer 156. The intra prediction unit 152 performs intra prediction from reconstructed samples within the current picture, and transmits intra coding information to the entropy coding unit 160. The intra coding information includes at least one of an intra prediction mode, a Most Probable Mode (MPM) flag, and an MPM index. The intra coding information includes information regarding the reference sample. The inter prediction unit 154 includes a motion estimation unit 154a and a motion compensation unit 154b. The motion estimation unit 154a obtains a motion vector value of the current region by referring to a specific region of the reconstructed reference signal picture. The motion estimation unit 154a transmits a motion information set (reference picture index, motion vector information) for the reference region to the entropy coding unit 160. The motion compensation unit 154b performs motion compensation using the motion vector value transmitted from the motion compensation unit 154a. The inter prediction unit 154 transmits inter coding information including the motion information for the reference region to the entropy coding unit 160.
[0035] According to a further embodiment, the prediction unit 150 includes an intra block copy (BC) prediction unit (not shown). The intra BC prediction unit performs intra BC prediction from reconstructed samples in the current picture and transmits intra BC coding information to the entropy coding unit 160. The intra BC prediction unit obtains block vector values indicating a reference region to be used for predicting the current region by referring to a specific region in the current picture. The intra BC prediction unit performs intra BC prediction using the obtained block vector values. The intra BC prediction unit transmits the intra BC coding information to the entropy coding unit 160. The intra BC prediction unit includes the block vector information.
[0036] When the above-mentioned picture prediction is performed, the transform unit 110 converts residual values between the original picture and the predicted picture to obtain transform coefficient values. At this time, the transform is performed in units of specific blocks within the picture, and the size of the specific blocks varies within a preset range. The quantization unit 115 quantizes the transform coefficient values generated by the transform unit 110 and transfers the quantized values to the entropy coding unit 160.
[0037] The entropy coding unit 160 generates a video signal bit stream by entropy coding information indicating quantized transform coefficients, intra-coding information, and inter-coding information. The entropy coding unit 160 uses a variable length coding (VLC) method, an arithmetic coding method, and the like. The variable length coding (VLC) method converts input symbols into consecutive code words, but the length of the code words is variable. For example, frequently occurring symbols are represented by short code words, and infrequently occurring symbols are represented by long code words. As the variable length coding method, a context-based adaptive variable length coding (CAVLC) method is used. In arithmetic coding, consecutive data symbols are converted into one prime number, and arithmetic coding obtains optimal prime number bits required to represent each symbol. As the arithmetic coding, a context-based adaptive binary arithmetic coding (CABAC) method is used. For example, the entropy coding unit 160 binarizes information indicating quantized transform coefficients, and then performs arithmetic coding on the binarized information to generate a bitstream.
[0038] The generated bitstream is encapsulated in a basic unit of Network Abstraction Layer (NAL) units. An NAL unit includes an integer number of coded coding tree units. In order for a video decoder to decode the bitstream, the bitstream must first be separated into NAL units and then each separated NAL unit must be decoded. Meanwhile, information required for decoding the video signal bitstream is transmitted via Raw Byte Sequence Payload (RBSP) of higher level sets such as a Picture Parameter Set (PPS), a Sequence Parameter Set (SPS), a Video Parameter Set (VPS), etc.
[0039] 1 illustrates an encoding apparatus 100 according to one embodiment of the present invention, with separate blocks illustrating logically distinct elements of encoding apparatus 100. Thus, the elements of encoding apparatus 100 described above may be implemented on a single chip or multiple chips depending on the device design. According to one embodiment, the operation of each element of encoding apparatus 100 described above is performed by a processor (not shown).
[0040] 2 is a schematic block diagram of a video signal decoding device 200 according to an embodiment of the present invention. Referring to FIG. 2, the decoding device 200 of the present invention includes an entropy decoding unit 210, an inverse quantization unit 220, an inverse transform unit 225, a filtering unit 230, and a prediction unit 250.
[0041] The entropy decoding unit 210 performs entropy coding on the video signal bitstream to extract transform coefficient information, intra-coding information, inter-coding information, etc. for each region. For example, the entropy decoding unit 210 obtains a binary code for transform coefficient information of a specific region from the video signal bitstream. The entropy decoding unit 210 also obtains quantized transform coefficients by de-binarizing the binary code. The inverse quantization unit 220 inverse quantizes the quantized transform coefficients, and the inverse transform unit 225 restores residual values using the inverse quantized transform coefficients. The video signal processing device 200 adds the residual values obtained from the inverse transform unit 225 to the predicted values obtained from the prediction unit 250 to restore original pixel values.
[0042] Meanwhile, the filtering unit 230 performs filtering on the picture to improve image quality, including a deblocking filter for reducing block distortion and / or an adaptive loop filter for removing distortion of the entire picture, etc. The filtered picture is output or stored in a decoded picture buffer (DPB) 256 to be used as a reference picture for the next picture.
[0043] The prediction unit 250 includes an intra prediction unit 252 and an inter prediction unit 254. The prediction unit 250 generates a predicted picture using the coding type, the transform coefficient for each region, the intra / inter coding information, etc., decoded through the above-mentioned entropy decoding unit 210. To reconstruct the current block to be decoded, the current picture including the current block or a region in which another picture is decoded is used. A picture (or tile / slice) that uses only the current picture for reconstruction, i.e., performs intra prediction or intra BC prediction, is called an intra picture or I picture (or tile / slice), and a picture (or tile / slice) that performs all of intra prediction, inter prediction, and intra BC prediction is called an inter picture (or tile / slice). Among interpictures (or tiles / slices), a picture (or tile / slice) that uses at most one motion vector and reference picture index to predict sample values of each block is called a predictive picture or P picture (or tile / slice), and a picture (or tile / slice) that uses at most two motion vectors and reference picture indexes is called a bi-predictive picture or B picture (or tile / slice). That is, a P picture (or tile / slice) uses at most one motion information set to predict each block, and a B picture (or tile / slice) uses at most two motion information sets to predict each block. Here, a motion information set includes one or more motion vectors and one reference picture index.
[0044] The intra prediction unit 252 generates a prediction block using intra coding information and reconstructed samples in the current picture. As described above, the intra coding information includes at least one of an intra prediction mode, a MOST Probable Mode (MPM) flag, and an MPM index. The intra prediction unit 252 predicts sample values of the current block using reconstructed samples located to the left and / or above the current block as reference samples. In the present disclosure, the reconstructed samples, the reference samples, and the samples of the current block refer to pixels. Also, a sample value refers to a pixel value.
[0045] In one embodiment, the reference samples are samples included in neighboring blocks of the current block. For example, the reference samples are samples adjacent to the left boundary and / or the top boundary of the current block. Also, the reference samples are samples located on a line within a preset distance from the left boundary of the current block and / or samples located on a line within a preset distance from the top boundary of the current block among samples of neighboring blocks of the current block. In this case, the neighboring blocks of the current block include at least one of a left (L) block, an upper (A) block, a below left (BL) block, an above right (AR) block, or an above left (AL) block adjacent to the current block.
[0046] The inter prediction unit 254 generates a prediction block using the reference picture and inter coding information stored in the decoded picture buffer 256. The inter coding information includes a motion information set (reference picture index, motion vector, etc.) of a current block with respect to a reference block. Inter prediction includes L0 prediction, L1 prediction, and bi-prediction. L0 prediction means a prediction using one reference picture included in an L0 picture list, and L1 prediction means a prediction using one reference picture included in an L1 picture list. For this purpose, one set of motion information (e.g., motion vector and reference picture index) is required. In the bi-prediction method, a maximum of two reference regions are used, and the two reference regions may exist in the same reference picture or may exist in different pictures. That is, in the bi-prediction method, a maximum of two sets of motion information (e.g., motion vector and reference picture index) are used, and the two motion vectors may correspond to the same reference picture index or may correspond to different reference picture indexes. In this case, the reference picture may be displayed (or output) either before or after the current picture in time. According to one embodiment, in a bi-predictive scheme, the two reference regions used are regions selected from the L0 picture list and the L1 picture list, respectively.
[0047] The inter prediction unit 254 obtains a current reference block using a motion vector and a reference picture index. The reference block exists in a reference picture corresponding to the reference picture index. In addition, a sample value of a block identified by the motion vector or an interpolated value thereof is used as a predictor of the current block. For motion prediction with pixel accuracy in sub-pel units, for example, an 8-tab interpolation filter is used for a luminance signal, and a 4-tab interpolation filter is used for a chrominance signal. However, the interpolation filter for sub-pel unit motion prediction is not limited thereto. In this way, the inter prediction unit 254 performs motion compensation to predict the texture of the current unit from a previously restored picture. In this case, the inter prediction unit uses a motion information set.
[0048] According to a further embodiment, the prediction unit 250 includes an intra BC prediction unit (not shown). The intra BC prediction unit reconstructs the current region by referring to a specific region including reconstructed samples in the current picture. The intra BC prediction unit obtains intra BC coding information for the current region from the entropy decoding unit 210. The intra BC prediction unit obtains block vector values of the current region indicating the specific region in the current picture. The intra BC prediction unit performs intra BC prediction using the obtained block vector values. The intra BC prediction unit includes block vector information.
[0049] A reconstructed video picture is generated by adding the predicted value output from the intra prediction unit 252 or the inter prediction unit 254 and the residual value output from the inverse transformation unit 225. That is, the video signal decoding apparatus 200 reconstructs a current block using the predicted block generated from the prediction unit 250 and the residual value obtained from the inverse transformation unit 225.
[0050] 2 shows a block diagram of a decoding apparatus 200 according to an embodiment of the present invention, with separate blocks illustrating the logically distinct elements of the decoding apparatus 200. Thus, the elements of the decoding apparatus 200 described above may be implemented on a single chip or multiple chips depending on the device design. According to an embodiment, the operation of each element of the decoding apparatus 200 described above is performed by a processor (not shown).
[0051] FIG. 3 illustrates an embodiment in which a coding tree unit (CTU) is divided into coding units (CUs) in a picture. In the process of coding a video signal, a picture is divided into a sequence of coding tree units (CTUs). A coding tree unit consists of two blocks, an NXN block of luma samples and a corresponding chroma sample. A coding tree unit is divided into a plurality of coding units. A coding tree unit may be a leaf node without being divided. In this case, the coding tree unit itself may be a coding unit. A coding unit refers to a basic unit for processing a picture in the above-mentioned video signal processing process, i.e., in the process of intra / inter prediction, transformation, quantization, and / or entropy coding. In one picture, the size and pattern of the coding units are not constant. The coding units have a square or rectangular pattern. A rectangular coding unit (or rectangular block) includes a vertical coding unit (or vertical block) and a horizontal coding unit (or horizontal block). In this specification, a vertical block is a block whose height is greater than its width, and a horizontal block is a block whose width is greater than its height. In addition, in this specification, a non-square block refers to a rectangular block, but the present invention is not limited to this.
[0052] Referring to Figure 3, the coding tree unit is first divided into a quad tree (QT) structure. That is, in the quad tree structure, one node having a size of 2N x 2N is divided into four nodes having a size of N x N. In this specification, the quad tree is also referred to as a quaternary tree. The quad tree division is performed recursively, and it is not necessary that all nodes are divided to the same depth.
[0053] Meanwhile, the leaf node of the above-mentioned quad tree is further divided into a Multi-Type Tree (MTT) structure. According to an embodiment of the present invention, in the Multi-Type Tree structure, one node is divided into a horizontally or vertically divided binary or ternary tree structure. That is, the Multi-Type Tree structure has four division structures, namely, vertical binary division, horizontal binary division, vertical ternary division, and horizontal ternary division. According to an embodiment of the present invention, in each of the tree structures, the width and height of the node are both powers of 2. For example, in a Binary Tree (BT) structure, a node of size 2N×2N is divided into two N×2N nodes by vertical binary division and divided into two 2N×N nodes by horizontal binary division. In addition, in a Ternary Tree (TT) structure, a node of size 2N x 2N is divided into (N / 2) x 2N, N x 2N, and (N / 2) x 2N nodes by vertical ternary division, and into 2N x (N / 2), 2N x N, and 2N x (N / 2) nodes by horizontal ternary division. Such multi-type tree division is performed recursively.
[0054] A leaf node of a multi-type tree can be a coding unit. If a coding unit is not larger than the maximum transform length, the coding unit can be used as a unit of prediction and / or transformation without further division. In one embodiment, if the width or height of a current coding unit is larger than the maximum transform length, the current coding unit is divided into multiple transform units without explicit signaling regarding division. Meanwhile, in the above-mentioned quad tree and multi-type tree, at least one of the following parameters is predefined or transmitted via the RBSP of a higher level set such as PPS, SPS, VPS, etc. 1) CTU size: the size of the root node of the quad tree, 2) MinQtSize: the size of the smallest QT leaf node allowed, 3) MaxBtSize: the size of the largest BT root node allowed, 4) MaxTtSize: the size of the largest TT root node allowed, 5) MaxMttDepth: the maximum allowed depth of MTT split from a QT leaf node, 6) MinBtSize: the size of the smallest BT leaf node allowed, 7) MinTT size: the size of the smallest TT leaf node allowed.
[0055] 4 is a diagram showing an embodiment of a method for signaling the split of a quad tree and a multi-type tree. In order to signal the split of the quad tree and the multi-type tree, a preset flag is used. Referring to FIG. 4, at least one of a flag "split_cu_flag" indicating whether a node is split, a flag "split_qt_flag" indicating whether a quad tree node is split, a flag "mtt_split_cu_vertical_flag" indicating the split direction of a multi-type tree node, and a flag "mtt_split_binarycu_flag" indicating the split pattern of a multi-type tree node is used.
[0056] According to an embodiment of the present invention, a flag 'split_cu_flag' indicating whether the current node is split or not is signaled first. If the value of 'split_cu_flag' is 0, it indicates that the current node is not split and the current node becomes a coding unit. If the current node is a coding tree unit, the coding tree unit contains one coding unit that is not split. If the current node is a quad tree node 'QT node', the current node is the leaf node of the quad tree node 'QT leaf node' and becomes a coding unit. If the current node is a multi-type tree node 'MTT node', the current node is the leaf node of the multi-type tree 'MTT leaf node' and becomes a coding unit.
[0057] If the value of 'split_cu_flag' is 1, the current node is split into a quad tree or a multi-type tree node according to the value of 'split_qt_flag'. The coding tree unit is the root node of the quad tree and is preferentially split into a quad tree structure. In the quad tree structure, 'split_qt_flag' is signaled for each node 'QT node'. If the value of 'split_qt_flag' is 1, the corresponding node is split into four square nodes, and if the value of 'qt_split_flag' is 0, the corresponding node becomes a leaf node 'QT leaf node' of the quad tree and is split into a multi-type node. According to an embodiment of the present invention, the quad tree split may be restricted according to the type of the current node. If the current node is a coding tree unit (root node of the quad tree) or a quad tree node, the quad tree split is allowed, and if the current node is a multi-type tree unit, the quad tree split is not allowed. Each quad tree leaf node, "QT leaf node", is further split into a multi-type tree structure. As mentioned above, if "split_qt_flag" is 0, the current node is split into a multi-type node. "mtt_split_cu_vertical_flag" and "mtt_split_cu_binary_flag" are signaled to indicate the split direction and split pattern. If "mtt_split_cu_vertical_flag" has a value of 1, a vertical split of the node "MTT node" is indicated, and if "mtt_split_cu_vertical_flag" has a value of 0, a horizontal split of the node "MTT node" is indicated. Also, if "mtt_split_cu_binary_flag" has a value of 1, the node "MTT node" is split into two rectangular nodes, and if "mtt_split_cu_binary_flag" has a value of 0, the node "MTT node" is split into three rectangular nodes.
[0058] Picture prediction (motion compensation) for coding is a code that cannot be divided further. The coding unit tree is the leaf node of the coding unit. The basic unit for performing such prediction is called a prediction unit. This is called a prediction block or a block of a block of a signal.
[0059] Hereinafter, the term "unit" used in this specification is used as an alternative term to the prediction unit, which is a basic unit for performing prediction, but the present invention is not limited thereto, and in a broader sense, the term "unit" is understood as a concept including the coding unit.
[0060] 5 and 6 are diagrams illustrating in more detail an intra prediction method according to an embodiment of the present invention. As described above, the intra prediction unit predicts sample values of a current block using reconstructed samples located to the left and / or above the current block as reference samples.
[0061] First, Fig. 5 shows an example of a reference sample used to predict a current block in an intra prediction mode. According to an example, the reference sample is a sample adjacent to the left boundary and / or the top boundary of the current block. As shown in Fig. 5, if the size of the current block is W x H and a sample of a single reference line adjacent to the current block is used for intra prediction, the reference sample is set using up to 2W + 2H + 1 neighboring samples located to the left and / or top of the current block.
[0062] Also, if at least some samples used as reference samples have not yet been restored, the intra prediction unit performs a reference sample padding process to obtain reference samples. Also, the intra prediction unit performs a reference sample filtering process to reduce an error in intra prediction. That is, filtering is performed on the neighboring samples and / or the reference samples obtained by the reference sample padding process to obtain filtered reference samples. The intra prediction unit predicts samples of the current block using the reference samples obtained in this manner. The intra prediction unit predicts samples of the current block using unfiltered reference samples or filtered reference samples. In the present disclosure, the neighboring samples include samples on at least one reference line. For example, the neighboring samples may include neighboring samples on a line adjacent to a boundary of the current block.
[0063] Next, Fig. 6 is a diagram showing an embodiment of a prediction mode used for intra prediction. For intra prediction, intra prediction mode information indicating an intra prediction direction is signaled. The intra prediction mode indicates one of a plurality of intra prediction modes constituting an intra prediction mode set. If the current block is an intra prediction block, the decoder receives the intra prediction mode information of the current block from the bitstream. The intra prediction unit of the decoder performs intra prediction on the current block based on the extracted intra prediction mode information.
[0064] According to an embodiment of the present invention, the intra prediction mode set includes all intra prediction modes (e.g., a total of 67 intra prediction modes) used in intra prediction. More specifically, the intra prediction mode set includes a planar mode, a DC mode, and a plurality of (e.g., 65) angle modes (i.e., directional modes). Each intra prediction mode is indicated by a preset index (i.e., intra prediction mode index). For example, as shown in FIG. 6, an intra prediction mode index 0 indicates a planar mode, and an intra prediction mode index 1 indicates a DC mode. In addition, intra prediction mode indexes 2 to 66 indicate different angle modes. The angle modes indicate different angles within a preset angle range. For example, the angle modes indicate angles within an angle range between 45 degrees and -135 degrees clockwise (i.e., a first angle range). The angle modes are defined based on 12 orientations. In this case, intra-prediction mode index 2 indicates horizontal diagonal (HDIA) mode, intra-prediction mode index 18 indicates horizontal (HOR) mode, intra-prediction mode index 34 indicates diagonal (DIA) mode, intra-prediction mode index 50 indicates vertical (VER) mode, and intra-prediction mode index 66 indicates vertical diagonal (VDIA) mode.
[0065] Meanwhile, the preset angle ranges are set differently according to the pattern of the current block. For example, if the current block is a rectangular block, a wide angle mode that indicates an angle that exceeds 45 degrees or is less than -135 degrees clockwise is further used. If the current block is a horizontal block, the angle mode indicates an angle within an angle range between (45+offset1) degrees and (-135+offset1) degrees clockwise (i.e., a second angle range). At this time, angle modes 67 to 76 that deviate from the first angle range are further used. Also, if the current block is a vertical block, the angle mode indicates an angle within an angle range between (45-offset2) degrees and (-135-offset2) degrees clockwise (i.e., a third angle range). At this time, angle modes -10 to -1 that deviate from the first angle range are further used. According to an embodiment of the present invention, the values of offset1 and offset2 are determined differently according to the ratio between the width and height of the rectangular block. Also, offset1 and offset2 are positive numbers.
[0066] According to a further embodiment of the present invention, the plurality of angle modes constituting the intra prediction mode set includes a base angle mode and an extension angle mode, where the extension angle mode is determined based on the base angle mode.
[0067] According to an embodiment, the basic angle mode is a mode corresponding to an angle used in intra prediction of the conventional High Efficiency Video Coding (HEVC) standard, and the extended angle mode is a mode corresponding to an angle newly added in intra prediction of the next-generation video codec standard. More specifically, the basic angle mode is an angle mode corresponding to any one of intra prediction modes {2, 4, 6, ..., 66}, and the extended angle mode is an angle mode corresponding to any one of intra prediction modes {3, 5, 6, ..., 65}. That is, the extended angle mode is an angle mode between the basic angle modes within the first angle range. Thus, the angle indicated by the extended angle mode is determined based on the angle indicated by the basic angle mode.
[0068] According to another embodiment, the basic angle mode is a mode corresponding to an angle within a first angle range that is preset, and the extension angle mode is a wide angle mode that deviates from the first angle range. That is, the basic angle mode is an angle mode corresponding to any one of intra prediction modes {2, 3, 4, ..., 66}, and the extension angle mode is an angle mode corresponding to any one of intra prediction modes {-10, -9, ..., -1} and {67, 68, ..., 76}. The angle indicated by the extension angle mode is determined to be an angle opposite to the angle indicated by the corresponding basic angle mode. Thus, the angle indicated by the extension angle mode is determined based on the angle indicated by the basic angle mode. Meanwhile, the number of extension angle modes is not limited thereto, and further extension angles are defined according to the size and / or pattern of the current block. For example, the extension angle mode may be defined as an angle mode corresponding to any one of intra prediction modes {-14, -13, ..., -1} and {67, 68, ..., 80}. Meanwhile, the total number of intra prediction modes included in the intra prediction mode set varies depending on the configuration of the base angle mode and the extended angle mode.
[0069] In the above embodiment, the interval between the extension angle modes is set based on the interval between the corresponding basic angle modes. For example, the interval between the extension angle modes {3, 5, 7, ..., 65} is determined based on the interval between the corresponding basic angle modes {2, 4, 6, ..., 66}. Also, the interval between the extension angle modes {-10, -9, ..., -1} is determined based on the interval between the corresponding opposite basic angle modes {56, 57, ..., 65}, and the interval between the extension angle modes {67, 68, ..., 76} is determined based on the interval between the corresponding opposite basic angle modes {3, 4, ..., 12}. The angle interval between the extension angle modes is set to be the same as the angle interval between the corresponding basic angle modes. Also, the number of the extension angle modes in the intra prediction mode set is set to be equal to or less than the number of the basic angle modes.
[0070] According to an embodiment of the present invention, the extension angle mode is signaled based on the base angle mode. For example, the wide angle mode (i.e., the extension angle mode) replaces at least one angle mode (i.e., the base angle mode) within the first angle range. The replaced base angle mode is an angle mode corresponding to the opposite side of the wide angle mode. That is, the replaced base angle mode is an angle mode corresponding to an angle in the opposite direction of the angle indicated by the wide angle mode, or an angle that is different from the opposite angle by a preset offset index. According to an embodiment of the present invention, the preset offset index is 1. The intra prediction mode index corresponding to the replaced base angle mode is further mapped to the wide angle mode to signal the corresponding wide angle mode. For example, the wide angle modes {-10, -9, ... -1} are signaled by intra prediction mode indexes {57, 58, ... , 66}, respectively, and the wide angle modes {67, 68, ... , 76} are signaled by intra prediction mode indexes {2, 3, ... , 11}, respectively. In this way, by making the intra prediction mode index for the base angle mode signal the extended angle mode, even if the configurations of the angle modes used for intra prediction of each block are different, the same set of intra prediction mode indexes can be used to signal the intra prediction mode, thereby minimizing signaling overhead due to changes in the intra prediction mode configuration.
[0071] Meanwhile, whether or not to use the extended angle mode is determined based on at least one of the shape and size of the current block. According to one embodiment, if the size of the current block is larger than a preset size, the extended angle mode is used for intra prediction of the current block, otherwise, only the basic angle mode is used for intra prediction of the current block. According to another embodiment, if the current block is a non-square block, the extended angle mode is used for intra prediction of the current block, and if the current block is a square block, only the basic angle mode is used for intra prediction of the current block.
[0072] Meanwhile, in order to improve coding efficiency, a method is used in which the residual signal is not directly coded, but the transform coefficient value obtained by converting the residual signal is quantized, and the quantized transform coefficient is coded. As described above, the transform unit converts the residual signal to obtain transform coefficient values. At this time, the residual signal of a specific block may be distributed throughout the entire area of the current block. Therefore, the residual signal is frequency domain transformed to concentrate energy in a low frequency area, thereby improving coding efficiency. Hereinafter, a method of transforming or inverse transforming the residual signal will be described in detail.
[0073] FIG. 7 is a detailed diagram illustrating how an encoder transforms a residual signal. As described above, the spatial domain residual signal is transformed into the frequency domain. The encoder transforms the obtained residual signal to obtain transform coefficients. First, the encoder obtains at least one residual block including a residual signal for a current block. The residual block is either the current block or a block divided from the current block. In this disclosure, the residual block is referred to as a residual array or a residual matrix including residual samples of the current block. In this disclosure, the residual block refers to a block having the same size as the size of a transform unit or transform block.
[0074] Next, the encoder transforms the residual block using a transform kernel. The transform kernel used for transforming the residual block is a transform kernel having separable characteristics of vertical transform and horizontal transform. In this case, the transform for the residual block is performed separately into vertical transform and horizontal transform. For example, the encoder applies a transform kernel to the vertical direction of the residual block to perform a vertical transform. Also, the encoder applies a transform kernel to the horizontal direction of the residual block to perform a horizontal transform. In this disclosure, the transform kernel is used as a term to refer to a set of parameters used for transforming the residual signal, such as a transform matrix, a transform array, a transform function, or a transform. According to an embodiment, the transform kernel is one of a plurality of available kernels. Also, transform kernels based on different transform types may be used for the vertical transform and the horizontal transform, respectively.
[0075] The encoder transmits the transform block, which is transformed from the residual block, to a quantization unit for quantization. In this case, the transform block includes a plurality of transform coefficients. More specifically, the transform block is made up of a plurality of transform coefficients arranged in a two-dimensional array. The size of the transform block is the same as that of the residual block, either the current block or one of the blocks divided from the current block. The transform coefficients transmitted to the quantization unit are represented by quantized values.
[0076] The encoder also performs a further transform before the transform coefficients are quantized. As shown in FIG. 7, the above transform method is called a primary transform, and the further transform is called a secondary transform. The secondary transform is selective for each residual block. According to an embodiment, the encoder performs a secondary transform on a region where it is difficult to concentrate energy in a low frequency region by only the primary transform, thereby improving coding efficiency. For example, a secondary transform may be added to a block whose residual value is largely expressed in a direction other than the horizontal or vertical direction of the residual block. The residual value of an intra predicted block is more likely to change in a direction other than the horizontal or vertical direction than the residual value of an inter predicted block. Therefore, the encoder further performs a secondary transform on the residual signal of the intra predicted block. The encoder may also omit the secondary transform on the residual signal of the inter predicted block.
[0077] As another example, whether to perform the secondary transform is determined according to the size of the current block or the residual block. Also, transform kernels having different sizes are used according to the size of the current block or the residual block. For example, an 8×8 secondary transform is applied to a block whose short side of the width or height is equal to or greater than a first preset length. Also, a 4×4 secondary transform is applied to a block whose short side of the width or height is equal to or greater than a second preset length and smaller than the first preset length. In this case, the first preset length may be greater than the second preset length, but the present disclosure is not limited thereto. Also, unlike the primary transform, the secondary transform does not have to be separated into a vertical transform and a horizontal transform. Such a secondary transform is called a low-band non-separable transform (LFNST).
[0078] Also, in the case of a video signal of a specific region, high frequency band energy does not decrease even if frequency conversion is performed due to a sudden change in brightness. As a result, compression performance due to quantization may be degraded. Also, when conversion is performed on a region where residual values rarely exist, encoding and decoding times may be increased unnecessarily. As a result, conversion of the residual signal of the specific region may be omitted. Whether or not to convert the residual signal of the specific region is determined by a syntax element related to the conversion of the specific region. For example, the syntax element includes transform skip information. The transform skip information is a transform skip flag. If the transform skip information for a residual block indicates a transform skip, the conversion of the corresponding residual block is not performed. In this case, the encoder immediately quantizes the residual signal of the corresponding region that has not been converted. The operation of the encoder described with reference to FIG. 7 is performed through the conversion unit of FIG. 1.
[0079] The above-mentioned syntax elements related to transform are information parsed from a video signal bitstream. A decoder entropy decodes the video signal bitstream to obtain the syntax elements related to transform, and an encoder entropy codes the syntax elements related to transform to generate a video signal bitstream.
[0080] FIG. 8 is a diagram illustrating in detail how the encoder and the decoder inversely transform the transform coefficients to obtain a residual signal. For convenience of explanation, it will be described below that the inverse transform operation is performed through the inverse transform unit of each of the encoder and the decoder. The inverse transform unit inversely transforms the inverse quantized transform coefficients to obtain a residual signal. First, the inverse transform unit detects whether an inverse transform for a particular region is to be performed from a syntax element related to the transform of the particular region. According to an embodiment, if a syntax element related to the transform of a particular transform block indicates a transform skip, the transform for the particular transform block is omitted. In this case, both the primary inverse transform and the secondary inverse transform for the transform block are omitted. Also, the inverse quantized transform coefficients are used as a residual signal. For example, the decoder restores a current block using the inverse quantized transform coefficients as a residual signal. The above-mentioned primary inverse transform refers to an inverse transform for a primary transform and is called an inverse primary transform. The secondary inverse transform refers to an inverse transform for a secondary transform and is called an inverse secondary transform or inverse LFNST. In the present invention, the primary (inverse) transform is referred to as the first (inverse) transform, and the secondary (inverse) transform is referred to as the second (inverse) transform.
[0081] In another embodiment, a syntax element related to the transform for a particular transform block may not indicate a transform skip. In this case, the inverse transform unit determines whether to perform a secondary inverse transform for the secondary transform. For example, if the transform block is a transform block of an intra-predicted block, a secondary inverse transform is performed for the transform block. Also, a secondary transform kernel to be used for the corresponding transform block is determined based on an intra-prediction mode for the transform block. As another example, whether to perform a secondary inverse transform may be determined according to the size of the transform block. The secondary inverse transform is performed after the inverse quantization process and before the primary inverse transform is performed.
[0082] The inverse transform unit performs a primary inverse transform on the inverse quantized transform coefficients or the transform coefficients that have been subjected to the secondary inverse transform. In the case of the primary inverse transform, the primary inverse transform is performed by separating it into a vertical transform and a horizontal transform, as in the primary transform. For example, the inverse transform unit performs a vertical inverse transform and a horizontal inverse transform on the transform block to obtain a residual block. The inverse transform unit inversely transforms the transform block based on a transform kernel used for transforming the transform block. For example, the encoder explicitly or implicitly signals information indicating a transform kernel that is currently applied to the transform block among a plurality of available transform kernels. The decoder uses the signaled information indicating the transform kernel to select a transform kernel to be used for inverse transforming the transform block among a plurality of available transform kernels. The inverse transform unit restores the current block using a residual signal obtained through an inverse transform on the inverse transform coefficients.
[0083] Meanwhile, the distribution of residual signals of a picture may differ from region to region. For example, the distribution of values of residual signals in a specific region may differ depending on a prediction method. When a same transform kernel is used to transform a plurality of different transform regions, the coding efficiency may differ from one transform region to another according to the distribution and characteristics of values in the transform region. Therefore, the coding efficiency can be further improved by adaptively selecting a transform kernel to be used for transforming a specific transform block from among a plurality of available transform kernels. That is, the encoder and the decoder are configured to further use a transform kernel other than the base transform kernel in transforming a video signal. The method of adaptively selecting a transform kernel is referred to as adaptive multiple core transform (ATM) or multiple transform selection (MTS). In this disclosure, for convenience of explanation, transform and inverse transform are collectively referred to as transform. Also, transform kernel and inverse transform kernel are collectively referred to as transform kernel.
[0084] Residual signals, which are the difference between an original signal and a predicted signal generated through inter-frame prediction or intra-frame prediction, have energy distributed throughout the entire pixel domain, so if the pixel values of the residual signal themselves are coded, the compression efficiency will decrease. Therefore, a process is required to concentrate the energy of the pixel domain residual signal in the low frequency region of the frequency domain through transform coding.
[0085] In the high efficiency video coding (HEVC) standard, efficient DCT-II (discrete cosine transform type-II) is mostly used when the signal is uniformly distributed in the pixel domain (when adjacent pixel values are similar), and DCT-VII (discrete sine transform type-VII) is used only for predicted 4x4 blocks in a screen to convert the pixel domain residual signal to the frequency domain. DCT-II transform is suitable for residual signals generated through inter-screen prediction (when energy is uniformly distributed in the pixel domain). However, in the case of residual signals generated through intra-screen prediction, the energy of the residual signal tends to increase the further away it is from the reference sample due to the characteristics of intra-screen prediction, which predicts using restored reference samples around the current coding unit. Therefore, high coding efficiency cannot be achieved when only DCT-II transform is used to convert the residual signal to the frequency domain.
[0086] AMT is a transformation technique that adaptively selects a transformation kernel from among a number of pre-defined kernels according to a prediction method. Since the pixel domain pattern (horizontal signal characteristics, vertical signal characteristics) of the residual signal differs depending on which prediction method is used, higher coding efficiency is expected than when only DCT-II is simply used to transform the residual signal. In the present invention, AMT is not limited to its name and may be called MTS (multiple transform selection).
[0087] FIG. 9 is a diagram showing basis functions for a number of transformation kernels that can be used in a linear transformation.
[0088] In detail, FIG. 9 is a diagram showing basis functions of transform kernels used in AMT, and shows the mathematical formulas of DCT-II, DCT-V (discrete cosine transform type-V), DCT-VIII (discrete cosine transform type-VIII), DST-I (discrete sine transform type-I), and DST-VII kernels applied to AMT.
[0089] DCT and DST are expressed by cosine and sine functions, respectively. When the basis function of the transformation kernel for the number of samples N is expressed as Ti(j), the index i indicates an index in the frequency domain, and the index j indicates an index within the basis function. In other words, the smaller the i, the lower the frequency basis function, and the larger the i, the higher the frequency basis function. If the basis function Ti(j) is expressed as a two-dimensional matrix, it indicates the j-th element of the i-th row, but since all the transformation kernels shown in FIG. 9 have separable characteristics, it is possible to perform transformations on the residual signal X in the horizontal and vertical directions. In other words, if the residual signal block is X and the transformation kernel matrix is T, the transformation on the residual signal X is expressed as TXT'. In this case, T' means the transpose matrix of the transformation kernel matrix T.
[0090] The transformation matrix values defined by the basis functions shown in FIG. 9 are in the prime number form rather than the integer form. The prime number form values may be difficult to implement in hardware in a video encoding device and a video decoding device. Therefore, an integer-approximated transformation kernel from an original transformation kernel including prime number form values is used in encoding and decoding a video signal. The approximated transformation kernel including integer form values is generated through scaling and rounding of the original transformation kernel. The integer values included in the approximated transformation kernel are values within a range that can be represented by a preset number of bits. The preset number of bits is 8-bit or 10-bit. The orthogonal properties of DCT and DST may not be maintained due to the approximation. However, since the loss of coding efficiency due to this is not large, approximating the transformation kernel to the integer form is advantageous in terms of hardware implementation.
[0091] In the case of the linear transform domain and the inverse linear transform described in FIG. 7 and FIG. 8, since the separable transform kernel is expressed as a two-dimensional matrix and transformed in the vertical direction and the horizontal direction, respectively, it is considered that two-dimensional matrix multiplication operations are performed twice. This may be problematic in terms of implementation since it involves a large amount of calculations. Therefore, in terms of implementation, it may be an important issue whether the amount of calculations can be reduced by using a combination structure of a butterfly structure or a half butterfly structure and a half matrix multiplier as in DCT-II, or whether the corresponding transform kernel can be decomposed into a transform kernel with low implementation complexity (whether the corresponding channel can be represented by a product of matrices with low complexity). In addition, since the elements of the transform kernel (matrix elements of the transform kernel) should be stored in memory for calculation, the memory capacity for storing the kernel matrix should also be considered in implementation. From this perspective, since the implementation complexity of DST-VII and DCT-VIII is relatively high, a transform that shows similar characteristics to DST-VII and DCT-VIII but has low implementation complexity can replace DST-VII and DCT-VIII.
[0092] DST-IV (discrete sine transform type-IV) and DCT-IV (discrete cosine transform type-IV) are considered to be candidates that can replace DST-VII and DCT-VIII, respectively. The DCT-II kernel for 2N samples contains the DCT-IV kernel for N samples, and the DST-IV kernel for N samples can be realized by performing a simple operation of sign inversion from the DCT-IV kernel for N samples and rearranging the corresponding basis functions in reverse order, so that the DST-IV and DCT-IV for N samples can be easily derived from the DCT-II for 2N samples.
[0093] Since the residual signal, which is the difference between the original signal and the predicted signal, exhibits a characteristic that the signal energy distribution changes depending on the prediction method, the coding efficiency can be improved by adaptively selecting a transform kernel according to the prediction method, such as AMT or MTS. Also, as described in FIG. 7 to FIG. 8, the coding efficiency can be improved by performing a secondary transform and an inverse secondary transform (an inverse transform corresponding to the secondary transform), which are additional transforms, in addition to the primary transform and the inverse primary transform (an inverse transform corresponding to the primary transform). Such a secondary transform improves energy compaction, especially for an intra-screen predicted residual signal block in which strong energy is likely to exist in a direction other than the horizontal and vertical directions of the residual signal. As described above, such a secondary transform is called a low-band non-separable transform (LFNST). And, the primary transform is called a core transform.
[0094] FIG. 10 is a block diagram showing a process of restoring a residual signal in a decoder that performs a secondary transform according to an embodiment of the present invention. First, an entropy coder parses syntax elements related to a residual signal from a bitstream, and obtains quantized coefficients through de-binarization. A decoder performs inverse quantization on the restored quantized coefficients to obtain transform coefficients, and performs inverse transform on the transform coefficients to restore a residual signal block. The inverse transform is applied to blocks to which transform skip (TS) is not applied. The inverse transform is performed in the decoder in the order of secondary inverse transform and primary inverse transform. In this case, the secondary inverse transform may be omitted. The secondary inverse transform may be omitted without being performed on inter-predicted blocks. Alternatively, the secondary inverse transform may be omitted depending on a condition of the block size. The restored residual signal includes a quantization error, and the secondary transform changes the energy distribution of the residual signal, thereby reducing the quantization error more than when only the primary transform is performed.
[0095] FIG. 11 is a diagram illustrating a process of restoring a residual signal at a block level in a decoder that performs a secondary transform according to an embodiment of the present invention. Restoration of a residual signal is performed in units of a transform unit (TU) or a sub-block within a TU. FIG. 11 illustrates a process of restoring a residual signal block to which a secondary transform is applied, in which a secondary inverse transform is first performed on a dequantized transform coefficient block. The decoder may perform a secondary inverse transform on all W×H (W: width, number of horizontal samples; H: height, number of vertical samples) samples within a TU, or may perform a secondary inverse transform only on a left-top sub-block of size W'×H', which is a low-frequency region having the greatest influence, in consideration of complexity. In this case, W' is equal to or smaller than W. H' is equal to or smaller than H. The size of the upper left sub-block, W'×H', is set differently depending on the size of the TU. For example, if min(W,H)=4, both W' and H' are set to 4. If min(W,H)>=8, both W' and H' are set to 8. min(x,y) indicates an operation that returns x if x is equal to or smaller than y, and returns y if x is equal to or smaller than y. After performing the second inverse transform, the decoder obtains the left-to-top W'×H' size sub-block transform coefficients in the TU, and performs a first inverse transform on the entire W×H size transform coefficient block to restore the residual signal block.
[0096] Whether or not secondary transformation is activated or applicable is indicated by being included in at least one of High Level Syntax (HLS) RBSPs such as a sequence parameter set (SPS), a picture parameter set (PPS), a picture header, a slice header, a tile group header, etc., in the form of a 1-bit flag. Furthermore, if secondary transformation is applicable, the size of the upper left sub-block to be considered in secondary transformation may be indicated in the form of a 1-bit flag in at least one of the HLS RBSPs. For example, whether a sub-block of 8×8 size can be used in secondary transformation considering sub-blocks of 4×4 and 8×8 size is indicated by a 1-bit flag in at least one of the HLS RBSPs.
[0097] If the activation or applicability of the secondary transform is indicated at a higher level (e.g., HLS), whether the secondary transform is applied is indicated by a 1-bit flag at a coding unit (CU) level. Also, if the secondary transform is applied to the current block, an index indicating a transform kernel used for the secondary transform is indicated at a coding unit level. The decoder performs a secondary inverse transform on the block to which the secondary transform is applied, using a transform kernel indicated by the corresponding index in a transform kernel set preset according to a prediction mode. The index indicating the transform kernel is binarized using a truncated unary or fixed length binarization method. The 1-bit flag indicating whether the secondary transform is applied at the CU level and the index indicating the transform kernel used for the secondary transform may be indicated using one syntax element, which is referred to as lfnst_idx[x0][y0] or lfnst_idx in the present invention, but is not limited thereto. As an example, the first bit of lfnst_idx[x0][y0] indicates whether or not a secondary transform is applied at the CU level. The remaining bits indicate an index indicating a transform kernel used in the secondary transform. That is, lfnst_idx[x0][y0] indicates whether or not a secondary transform (LFNST) is applied, and an index indicating a transform kernel used when the secondary transform is applied. Such lfnst_idx[x0][y0] is coded through an entropy coder such as CABAC (context-based adaptive binary arithmetic coding) or CAVLC (context-based adaptive variable length coding) that adaptively codes according to a context. If a current CU is divided into a number of TUs smaller than the CU size, the secondary transform is not applied, and lfnst_idx[x0][y0], which is a syntax element related to the secondary transform, is set to 0 without signaling. For example, if lfnst_idx[x0][y0] is 0, it indicates that the secondary transform is not applied.On the other hand, if lfnst_idx[x0][y0] is greater than 0, it indicates that a secondary transform is applied, and the transform kernel used for the secondary transform is selected based on lfnst_idx[x0][y0].
[0098] As described above, a coding tree unit, a leaf node of a quad tree, and a leaf node of a multi-type tree can be a coding unit. If a coding unit is not larger than the maximum transform length, the coding unit is not further divided and is used as a unit of prediction and / or transformation. In one embodiment, if the width or height of a current coding unit is larger than the maximum transform length, the current coding unit is divided into a plurality of transform units without explicit signaling regarding the division. If the size of a coding unit is larger than the maximum transform size, the current coding unit is divided into a plurality of transform blocks without signaling. In this case, the maximum coding block (or the maximum size of a coding block) to which a secondary transform is applied is limited because the performance is degraded and the complexity is increased when a secondary transform is applied. The size of the maximum coding block is the same as the maximum transform size. Alternatively, the size of the maximum coding block is defined as a preset coding block size. In one embodiment, the preset value may be 64, 32, or 16, but the present invention is not limited thereto. In this case, the value compared with the preset value (or the maximum transform size) is defined as the length of the long side or the number of samples.
[0099] Meanwhile, the transform kernel based on the DCT-II, DST-VII, and DCT-VIII basis functions used in the primary transform has a separable characteristic. Therefore, two transforms in the vertical and horizontal directions are performed on the samples in the residual block of size N×N, and the size of the transform kernel is N×N. In contrast, in the case of the secondary transform, the transform kernel has a non-separable characteristic. Therefore, if the number of samples considered in the secondary transform is n×n, one transform is performed. In this case, the size of the transform kernel is (n^2)×(n^2). For example, when performing the secondary transform on the left-top 4×4 coefficient block, a transform kernel of size 16×16 is applied. And, when performing the secondary transform on the left-top 8×8 coefficient block, a transform kernel of size 64×64 is applied. The 64×64 transform kernel involves a large number of multiplication operations, which may place a heavy burden on the encoder and decoder. Therefore, if the number of samples considered in the secondary transform is reduced, the amount of calculations and the memory required to store the transform kernel can be reduced.
[0100] FIG. 12 is a diagram illustrating a method of applying a secondary transformation using a reduced number of samples according to an embodiment of the present invention. According to an embodiment of the present invention, the secondary transformation is expressed as a product of a secondary transformation kernel matrix and a linearly transformed coefficient vector, and the linearly transformed coefficients are interpreted as mapping to another space. In this case, if the number of coefficients to be secondarily transformed is reduced, that is, the number of basis vectors constituting the secondary transformation kernel is reduced, the amount of calculation required for the secondary transformation and the memory capacity required to store the transformation kernel can be reduced. For example, when performing a secondary transformation on a left-top 8×8 coefficient block, if the number of coefficients to be secondarily transformed is reduced to 16, a secondary transformation kernel of 16 (rows)×64 (columns) size (or 16 (rows)×48 (columns) size) is applied. The transformation unit of the encoder obtains a secondarily transformed coefficient vector through an inner product of each row vector constituting the transformation kernel matrix and the linearly transformed coefficient vector. The inverse transform units of the encoder and decoder obtain linearly transformed coefficient vectors by performing an inner product of each column vector constituting a transform kernel matrix and the quadratically transformed coefficient vector.
[0101] Referring to FIG. 12, the encoder first performs a primary transform on the residual signal block to obtain a primary transformed coefficient block. If the size of the primary transformed coefficient block is M×N, then for an intra predicted block with a min(M,N) value of 4, a 4×4 secondary transform is performed on the left-top 4×4 samples of the primary transformed coefficient block. For an intra predicted block with a min(M,N) value of 8 or more, an 8×8 secondary transform is performed on the left-top 8×8 samples of the primary transformed coefficient block. Since the 8×8 secondary transform involves a large amount of calculation and memory, only a portion of the 8×8 samples may be used. In one embodiment, in order to improve coding efficiency, for a rectangular block with a min(M,N) value of 4 and M or N greater than 8 (e.g., a rectangular block of 4×16 or 16×4 size), a 4×4 secondary transform may be performed on each of the two left-top 4×4 sub-blocks in the primary transformed coefficient block.
[0102] Since the secondary transformation is calculated by multiplying the secondary transformation kernel matrix and the input vector, the encoder first configures the coefficients in the left-top sub-block of the primary transformed coefficient block into the form of vectors. The method of configuring the vectors depends on the intra prediction mode. For example, if the intra prediction mode is the 34th angle mode or less among the intra prediction modes shown in FIG. 6, the encoder horizontally scans the left-top sub-block of the primary transformed coefficient block and configures the coefficients into vectors. If the element of the i-th row and j-th column of the left-top n×n block of the primary transformed coefficient block is represented as x(i,j), the vectorized coefficients are represented as [X(0,0), X(0,1), ..., X(0,n-1), X(1,0), X(1,1), ..., X(1,n-1), ..., X(n-1,0), X(n-1,1), ..., X(n-1,n-1)]. On the other hand, if the intra prediction mode is greater than the 34th angle mode, the left-top subblock of the primary transformed coefficient block is scanned vertically to construct the coefficients into vectors. The vectorized coefficients are represented as [X(0,0), X(1,0), ..., X(n-1,0), X(0,1), X(1,1), ..., X(n-1,1), ..., X(0,n-1), X(1,n-1), ..., X(n-1,n-1)]. In order to reduce the amount of calculation, if only a portion of the 8x8 samples is used in the 8x8 secondary transform, the coefficients x_ij where i>3 and j>3 may not be included in the above vector construction method. In this case, 16 primary transformed coefficients may be input for secondary transform in the 4x4 secondary transform. 48 primary transformed coefficients may be input for secondary transform in the 8x8 secondary transform.
[0103] The encoder obtains quadratically transformed coefficients through the multiplication of the left-top subblock samples of the vectorized primary transform coefficient block and the quadratically transformed kernel matrix. The quadratically transformed kernel applied to the quadratically transformed coefficients is determined according to the size of the transform unit or transform block, the intra mode, and a syntax element indicating the transform kernel. As described above, if the number of coefficients to be quadratically transformed is reduced, the amount of calculation and the memory required to store the transform kernel can be reduced. Thus, the number of coefficients to be quadratically transformed is determined according to the size of the current transform block. For example, in the case of a 4×4 block, the encoder obtains a coefficient vector of length 8 through the multiplication of a vector of length 16 and an 8 (row)×16 (column) transform kernel matrix. The 8 (row)×16 (column) transform kernel matrix is obtained based on the first to eighth basis vectors constituting the 16 (row)×16 (column) transform kernel matrix. For 4×N or M×4 blocks (N and M are 8 or more), the encoder obtains a coefficient vector of length 16 through the multiplication of a vector of length 16 and a 16 (row) × 16 (column) transformation kernel matrix. For 8×8 blocks, the encoder obtains a coefficient vector of length 8 through the multiplication of a vector of length 48 and an 8 (row) × 48 (column) transformation kernel matrix. The 8 (row) × 48 (column) transformation kernel matrix is obtained based on the first to 8th basis vectors that constitute the 16 (row) × 48 (column) transformation kernel matrix. For M×N blocks (M and N are 8 or more) except 8×8, the encoder obtains a coefficient vector of length 16 through the multiplication of a vector of length 48 and a 16 (row) × 48 (column) transformation kernel matrix.
[0104] According to an embodiment of the present invention, the doubly transformed coefficients are in the form of vectors and are therefore represented as two-dimensional data. According to a preset scan order, the doubly transformed coefficients are arranged into left-top coefficient sub-blocks. In one embodiment, the preset scan order is a top right diagonal scan order. The present invention is not limited thereto, and the top right diagonal scan order is determined based on a method described below with reference to FIGS. 13 and 14.
[0105] According to an embodiment of the present invention, the transform coefficients of the entire transform unit including the secondarily transformed coefficients are quantized and then transmitted in a bitstream. The bitstream includes syntax elements related to the secondarily transform. In particular, the bitstream includes information indicating whether the secondarily transform is applied to the current block and information indicating the transform kernel to be applied to the current block.
[0106] The decoder first parses the quantized transform coefficients from the bitstream and obtains the transform coefficients through de-quantization. De-quantization is called scaling. The decoder determines whether the secondary inverse transform is applied to the current block based on the syntax element related to the secondary transform. If the secondary inverse transform is applied to the current transform unit or transform block, 8 or 16 transform coefficients may be input to the secondary inverse transform depending on the size of the transform unit or transform block. The number of coefficients that are input to the secondary inverse transform is equal to the number of coefficients output by the secondary transform of the encoder. For example, if the size of the transform unit or transform block is 4×4 or 8×8, 8 transform coefficients are input to the secondary inverse transform, otherwise 16 transform coefficients are input to the secondary inverse transform. If the size of the transform unit is M×N, for an intra-predicted block with a min(M,N) value of 4, a 4×4 secondary inverse transform is applied to 16 or 8 coefficients of the left-top 4×4 sub-block of the transform coefficient block. For intra-predicted blocks with min(M,N) ≥ 8, an 8×8 quadratic transform is performed on the 16 or 8 coefficients of the left-top 4×4 sub-blocks of the transform coefficient block. In one embodiment, to improve coding efficiency, if min(M,N) is 4 and M or N is greater than 8 (e.g., rectangular blocks of size 4×16, 16×4), then a 4×4 quadratic inverse transform may be performed on each of the two left-top 4×4 sub-blocks in the coefficient block.
[0107] According to an embodiment of the present invention, since the second inverse transform is calculated by multiplying the second inverse transform kernel matrix and the input vector, the decoder configures the previously input dequantized transform coefficient block into the form of a vector according to a preset scan order. According to an embodiment, the preset scan order is the upper right diagonal scan order, but the present invention is not limited thereto, and the upper right diagonal scan order is determined based on a method described in FIG. 13 and FIG. 14 below.
[0108] According to an embodiment of the present invention, a decoder obtains a first-order transformed coefficient through a multiplication of vectorized transform coefficients and a second-order inverse transform kernel matrix. In this case, the second-order inverse transform kernel is determined according to a syntax element indicating a size of a transform unit or transform block, an intra mode, and a transform kernel. The second-order inverse transform kernel matrix is a transposed matrix of a second-order transform kernel matrix. In consideration of the complexity of implementation, the elements of the kernel matrix are integers represented with 10-bit or 8-bit accuracy. The length of a vector that is the output of the second-order inverse transform is determined according to the size of a current transform block. For example, in the case of a 4×4 block, a coefficient vector of length 16 is obtained through a multiplication of a vector of length 8 and an 8 (row)×16 (column) transform kernel matrix. The 8 (row)×16 (column) transform kernel matrix is obtained based on the first to eighth basis vectors constituting a 16 (row)×16 (column) transform kernel matrix. For 4×N or M×N blocks (N and M are 8 or more), a coefficient vector of length 16 is obtained through the multiplication of a vector of length 16 with a 16 (row) × 16 (column) transformation kernel matrix. For 8×8 blocks, a coefficient vector of length 48 is obtained through the multiplication of a vector of length 8 with an 8 (row) × 48 (column) transformation kernel matrix. The 8 (row) × 48 (column) transformation kernel matrix is obtained based on the first to 8th basis vectors that constitute the 16 (row) × 48 (column) transformation kernel matrix. For M×N blocks (M and N are 8 or more) other than 8×8, a coefficient vector of length 48 is obtained through the multiplication of a vector of length 16 with a 16 (row) × 48 (column) transformation kernel matrix.
[0109] In one embodiment, since the primary conversion coefficients obtained through the secondary inverse conversion are in the form of a vector, the decoder can further represent this as two-dimensional data, which is intra-mode dependent. At this time, the mapping relationship based on the intra-mode applied by the encoder is also applied. As described above, if the intra prediction mode is 34th angle mode or less, the decoder scans the coefficient vector obtained by the secondary inverse conversion horizontally to obtain a two-dimensional conversion coefficient array. If the intra prediction mode is greater than the 34th angle mode, the decoder scans the coefficient vector obtained by the secondary inverse conversion vertically to obtain a two-dimensional conversion coefficient array. The decoder performs a primary inverse conversion on all conversion units or conversion coefficient blocks of the conversion block size including the conversion coefficients obtained by performing the secondary inverse conversion to obtain a residual signal.
[0110] Although not shown in FIG. 12, a scaling process using a bit shift operation may be included when applying the conversion or inverse conversion in order to correct the scale increased by the conversion kernel after the change or inverse conversion.
[0111] FIG. 13 is a diagram showing a method for determining the upper right diagonal scan order according to an embodiment of the present invention. According to an embodiment of the present invention, during encoding or decoding, a process of initializing the scan order is performed. Initialization of an array including scan order information is performed according to the block size. Specifically, for the combination of log2BlockWidth and log2BlockHeight, the array initialization process of the upper right diagonal scan order shown in FIG. 13 with 1<<log2BlockWidth and 1<<log2BlockHeight as inputs is called (or performed). The output of the array initialization process of the upper right diagonal scan order is assigned to DiagScanOrder[log2BlockWidth][log2BlockHeight]. Here, log2BlockWidth and log2BlockHeight are variables indicating the values obtained by taking the base-2 logarithm of the width and height of the block, respectively, and are values in the range of [0, 4].
[0112] Through the array initialization process of the upper right diagonal scan order shown in FIG. 13, the encoder / decoder outputs the array diagScan[sPos][sComp] for blkWidth, the width of the input block, and blkHeight, the height of the block. The sPos, which is the index of the array, indicates the scan position (scan index) and is a value in the range of [0, blkWidth*blkHeight-1]. If the sComp, which is the index of the array, is 0, sPos indicates the horizontal component (x), and if sComp is 1, sPos indicates the vertical component (y). The algorithm shown in FIG. 13 is interpreted such that the x-coordinate value and y-coordinate value on the two-dimensional coordinates at the scan position sPos in the upper right diagonal scan order are assigned to diagScan[sPos][0] and diagScan[sPos][1], respectively. That is, the value stored in the DiagScanOrder[log2BlockWidth][log2BlockHeight][sPos][sComp] array (or array) means the coordinate value corresponding to sComp at the sPos scan position (scan index) in the upper right diagonal scan order of the block whose block width and height are 1<<log2BlockWidth and 1<<log2BlockHeight, respectively.
[0113] FIG. 14 is a diagram illustrating the upper right diagonal scan order according to an embodiment of the present invention, depending on the block size. Referring to FIG. 14(a), if log2BlockWidth and log2BlockHeight are both 2, it means a block of 4×4 size. Referring to FIG. 14(b), if log2BlockWidth and log2BlockHeight are both 3, it means a block of 8×8 size. In FIG. 14, the numbers shown in the gray shaded area indicate the scan position (scan index) sPos. The x and y coordinate values at the sPos position are assigned to DiagScanOrder[log2BlockWidth][log2BlockHeight][sPos][0] and DiagScanOrder[log2BlockWidth][log2BlockHeight][sPos][1], respectively.
[0114] The encoder / decoder codes the transform coefficient information based on the above-mentioned scan order. Although the present invention will be described mainly in terms of an embodiment based on the case where the top right scan method is used, the present invention is not limited thereto and can be applied to other known scan methods.
[0115] The decoding process for the quadratic transform will now be described in detail. For ease of explanation, the process for the quadratic transform will be described primarily with respect to the decoder, but the embodiments described below apply to the encoder in substantially the same manner.
[0116] FIG. 15 illustrates a method of indicating a secondary transform at a coding unit level. The secondary transform is indicated at a coding unit level, and syntax elements related to the 22nd transform are included in the coding_unit syntax structure. The coding_unit syntax structure includes syntax elements related to a coding unit. In this case, the inputs of the coding_unit syntax structure are (x0, y0) which is the coordinates of the left-top luma sample of the current block based on the left-top luma sample of the picture, cbWidth which is the width of the block, cbHeight which is the height of the block, and treeType which is a variable indicating the type of coding tree. Since there is a correlation between luma and chroma, efficient video compression can be achieved by encoding the luma and chroma with the same coding structure. In addition, in order to increase the coding efficiency, the luma and chroma may be encoded with different coding structures. If the variable treeType is SINGLE_TREE, it means that the luma and chroma are encoded with the same coding tree structure, and the coding unit includes a luma coding block and a chroma coding block according to the color format. If treeType is DUAL_TREE_LUMA, it means that luma and chroma are coded with different coding trees, and the currently processed tree is a tree for luma. In this case, the coding unit includes only luma coding blocks. If treeType is DUAL_TREE_CHROMA, it means that luma and chroma are coded with different coding trees, and the currently processed tree is a tree for chroma. In this case, the coding unit includes chroma coding blocks according to the color format.
[0117] In the coding_unit syntax structure, a prediction method for a current coding unit is indicated, and the variable CuPredMode[x0][y0] indicates a prediction method for a current block. If CuPredMode[x0][y0] is MODE_INTRA, it indicates that an intra prediction method is applied to the current block, and if it is MODE_INTER, it indicates that an inter prediction method is applied to the current block. Also, if CuPredMode[x0][y0] is MODE_IBC, it indicates that an Intra Block Copy (IBC) prediction, which generates a reference block from an area where reconstruction of the current picture is completed and performs prediction, is applied to the current block. Syntax elements related to the prediction method are processed according to the value of the variable CuPredMode[x0][y0]. For example, if the variable CuPredMode[x0][y0] indicates intra prediction, the decoder parses syntax elements including information on an intra prediction mode, a reference line index, and an ISP (Intra Sub-Partitions) prediction, or sets variables related to the intra prediction mode according to a preset method.
[0118] After processing the syntax elements related to the prediction method, the syntax elements related to the residual signal are processed. The transform_tree() syntax structure is a syntax structure for a transform tree, in which the transform tree is divided into nodes with a size smaller than that of the root node, with the root node having the same size as the coding unit, and the leaf nodes of the transform tree become transform units. The transform_tree syntax structure includes information about the division of the transform tree.
[0119] One of the intra prediction methods is PCM (Pulse Code Modulation) prediction. If PCM prediction is used for prediction of the current coding unit, no transformation and quantization are performed, and therefore no transform_tree syntax structure exists. That is, since no transform_tree syntax structure exists, the decoder does not operate on the transform_tree syntax structure. If intra prediction is indicated for the current coding unit, PCM prediction is indicated by pcm_flag[x0][y0]. That is, if pcm_flag[x0][y0] is 1, the decoder does not operate on the transform_tree syntax structure. Meanwhile, whether or not the transform_tree syntax structure exists for the current coding unit is indicated by a 1-bit flag, which is referred to as cu_cbf in the present invention, but is not limited thereto. The decoder parses cu_cbf, or if cu_cbf is not parsed, sets cu_cbf according to a preset method. If cu_cbf is 1, the decoder operates on the transform_tree syntax structure. If inter prediction or IBC prediction is used for predicting the current coding unit, merge prediction can also be used for predicting the current coding unit. Whether merge prediction is used is indicated by merge_flag[x0][y0]. If it is indicated that merge prediction is used for the current block (merge_flag[x0][y0]=1), cu_cbf is not parsed and the value of cu_cbf is determined according to a preset method. The preset method is based on cu_skip_flag[x0][y0] which indicates the skip mode. For example, if cu_skip_flag[x0][y0] is 1, cu_cbf is inferred to 0, and otherwise cu_cbf is inferred to 1. If cu_cbf is 1, the transform_tree syntax structure is processed and a counter value for measuring the number of non-zero significant coefficients is initialized to 0.
[0120] The numSigCoeff variable indicates the number of non-zero quantization coefficients present in the transform unit of the current coding unit, and the processing of syntax elements related to secondary transform may differ depending on the value of numSigCoeff.
[0121] The numZeroOutSigCoeff variable indicates the number of non-zero quantized coefficients present at a particular position in a transform unit included in the current coding unit, and the processing of syntax elements related to secondary transform may differ depending on the value of numZeroOutSigCoeff.
[0122] In transform_tree, the transform tree is divided and the leaf nodes of the transform tree are transform units. transform_tree includes a trasnform_unit syntax structure which is a syntax structure related to the transform unit which is the leaf node. trasnform_unit processes syntax elements related to the transform unit and includes a residual_coding syntax structure if the corresponding transform unit includes one or more non-zero coefficients. The residual_coding syntax structure includes a syntax structure related to the quantized transform coefficients and processing related thereto. The transform blocks constituting the transform unit may differ depending on the type of tree currently being processed. If treeType is SINGLE_TREE, the current transform unit includes a luma transform block and a chroma transform block depending on the color format. If treeType is DUAL_TREE_LUMA, the current transform unit includes a luma transform block. If treeType is DUAL_TREE_CHROMA, the current transform unit includes a chroma transform block. The transform_unit syntax structure includes coded block flag (CBF) information indicating whether the transform block of the current transform unit includes one or more non-zero coefficients according to treeType. The CBF information is information indicated for each color component. For example, if the CBF value for the luma transform block of the current transform unit indicates that the luma transform block does not include one or more non-zero coefficients, the residual_coding syntax structure for the luma transform block is not processed since all the coefficients of the luma transform block are zero. As another example, if the CBF value for the chroma Cb transform block of the current transform unit indicates that the chroma Cb transform block includes one or more non-zero coefficients, the residual_coding syntax structure for the Cb transform block of the current transform unit exists.
[0123] Whether or not a secondary transform is applied to the current block is indicated at the CU level. If a secondary transform is applied, an index indicating a transform kernel used for the secondary transform may be further indicated. As described in FIG. 11, whether or not a secondary transform is applied to the current block is indicated using the lfnst_idx[x0][y0] syntax element. The first bit of lfnst_idx[x0][y0] indicates whether or not a secondary transform is applied to the current coding unit. If the first bit of lfnst_idx[x0][y0] is 0, that is, if lfnst_idx[x0][y0] is 0, it indicates that a secondary transform is not applied to the current block. On the other hand, if the first bit of lfnst_idx[x0][y0] is 1, that is, if lfnst_idx[x0][y0] is greater than 0 (lfnst_idx[x0][y0]>0), it indicates that a secondary transform is applied to the current block. In this case, an additional bit is used to indicate the transform kernel used for the secondary transform, and an index indicating the secondary transform kernel is signaled via the additional bit.
[0124] The lfnst_idx[x0][y0] syntax element is parsed if the conditions described below are met, whereas if the conditions described below are not met, lfnst_idx[x0][y0] is not present in the current coding unit and lfnst_idx[x0][y0] is set to 0.
[0125] In other words, if the conditions described in the first to fourth embodiments, including the lfnst_idx[x0][y0] syntax element parsing conditions described below, are satisfied, the encoder generates a bitstream including the lfnst_idx[x0][y0] syntax element for the current coding unit. On the other hand, if the conditions described below are not satisfied, the bitstream generated by the encoder does not include the lfnst_idx[x0][y0] syntax element for the current coding unit, and lfnst_idx[x0][y0] is set to 0. A decoder receiving such a bitstream parses the lfnst_idx[x0][y0] syntax element based on the conditions described below.
[0126] Parsing conditions for lfnst_idx[x0][y0] syntax element
[0127] i) Min(lfnWidth,lfnHeight)>=4
[0128] The first condition concerns the size of the block. If the width and height of the block are each 4 pixels or more, the decoder parses the lfnst_idx[x0][y0] syntax element.
[0129] In more detail, the decoder checks the block size conditions for which the secondary transform can be applied. The variables SubWidthC and SubHeightC are set according to the color format and indicate the ratio of the width and height of the chroma component to the width and height of the luma component of the picture, respectively. For example, a 4:2:0 color format image has a structure including one chroma sample per four luma samples, so SubWidthC and SubHeightC are both set to 2. As another example, a 4:4:4 color format image has a structure including one chroma sample per luma sample, so SubWidthC and SubHeightC are both set to 1. The number of horizontal samples of the current block, lfnWidth, and the number of vertical samples, lfnHeight, are set based on SubWidthC and SubHeightC. If treeType is DUAL_TREE_CHROMA, the coding unit includes only chroma components, so the number of horizontal samples of the chroma coding block is equal to the value obtained by dividing cbwidth, which is the width of the luma coding block, by SubWidthC. Similarly, the number of samples in the vertical direction of a chroma coding block is equal to the height of the luma coding block, cbHeight, divided by SubHeightC. If treeType is SINGLE_TREE or DUAL_TREE_LUMA, the coding unit contains a luma component, so lfnWidth and lfnHeight are set to cbwidth and cbHeight, respectively. Since the minimum condition for a block to which a degree 22 transform can be applied is 4x4, lfnst_idx[x0][y0] is parsed if Min(lfnWidth,lfnHeight)>=4 is satisfied.
[0130] ii) sps_lfnst_enabled_flag==1
[0131] The second condition relates to a flag value indicating whether a secondary transform is activated or applicable. If the value of the flag (sps_lfnst_enabled_flag) indicating whether a secondary transform is activated or applicable is set to 1, the decoder parses the lfnst_idx[x0][y0] syntax element.
[0132] In detail, the secondary transform is indicated by the higher level syntax RBSP. At least one of the SPS, PPS, VPS, tile group header, and slice header includes a flag having a 1-bit size indicating whether the secondary transform is activated and applicable. If sps_lfnst_enabled_flag is 1, it indicates that the lfnst_idx[x0][y0] syntax element is present in the coding unit syntax. If sps_lfnst_enabled_flag is 0, it indicates that the lfnst_idx[x0][y0] syntax element is not present in the coding unit syntax.
[0133] iii)CuPredMode[x0][y0]==MODE_INTRA
[0134] The third condition concerns the prediction mode: the secondary transform is only applied to intra-predicted blocks, so if the current block is an intra-predicted block, the decoder parses the lfnst_idx[x0][y0] syntax element.
[0135] iv)IntraSubPartitionsSplitType==ISP_NO_SPLIT
[0136] The fourth condition relates to whether the ISP prediction method is applied or not. If ISP is not applied to the current block, the decoder parses the lfnst_idx[x0][y0] syntax element.
[0137] In detail, as described with reference to FIG. 11, when the current CU is divided into a number of transform units smaller than the CU size, the secondary transform is not applied to the divided transform units. In this case, lfnst_idx[x0][y0], which is a syntax element related to the secondary transform, is not parsed and is set to 0. When the current CU is divided into a number of transform units whose CU size is smaller than the transform tree, the case where ISP prediction is applied to the current coding unit is included. The ISP prediction method is a prediction method for dividing the transform tree into a number of transform units smaller than the CU size according to a preset division method when intra prediction is applied to the current coding unit. An ISP prediction mode is indicated at the coding unit level, and a variable IntraSubPartitionsSplitType is set based on the ISP prediction mode. In this case, if IntraSubPartitionsSplitType is ISP_NO_SPLIT, it indicates that ISP is not applied to the current block. Although the secondary transform is indicated at the coding unit level, the actual secondary transform is applied at the transform unit level. Therefore, when the transform tree is divided into a number of transform units, it is inefficient to apply the same secondary transform kernel to all the divided transform units. In addition, due to the characteristics of intra prediction that generates prediction samples at the transform unit level, when the transform tree is divided into multiple transform units, the prediction accuracy is higher than when it is not divided. Therefore, if the transform tree is divided into multiple transform units, the energy of the residual signal is likely to be efficiently compressed even if secondary transform is not applied to the divided multiple transform units. In addition, if the size of the current CU is larger than the size of the luma maximum transform block (MaxTbSizeY) (i.e., cbWidth>MaxTbSizeY||cbHeight>MaxTbSizeY), the transform tree is divided into multiple transform units smaller than the CU size. Although not shown in FIG. 15, secondary transform is not applied when the size of the current CU is larger than the luma maximum transform block size (MaxTbSizeY).Therefore, the fourth condition may be expressed as IntraSubPartitionsSplitType==ISP_NO_SPLIT&&cbWidth<=MaxTbSizeY&&cbHeight<=MaxTbSize, where MaxTbSizeY is a natural number expressed in the form of a power of 2. MaxTbSizeY may be included and indicated in a higher level syntax RBSP such as SPS, PPS, slice header, or tile group header, or the encoder and decoder may use the same preset value. For example, the preset value may be 64 (2^6).
[0138] v)!intra_mip_flag[x0][y0]
[0139] The fifth condition relates to the intra prediction method. If MIP (Matrix based Intra Prediction) is not applied to the prediction of the current coding unit, the decoder parses the lfnst_idx[x0][y0] syntax element.
[0140] In detail, MIP is used as one method of intra prediction, and whether MIP is applied is indicated by intra_mip_flag[x0][y0] at the coding unit level. If intra_mip_flag[x0][y0] is 1, it indicates that MIP is applied to the prediction of the current coding unit, and prediction is performed by multiplying the reconstructed samples around the current block by a preset matrix. When MIP is applied, it shows a different residual signal property from general intra prediction that performs directional or non-directional prediction, so that secondary transformation may not be applied to the transform block when MIP is applied.
[0141] vi)numSigCoeff>((treeType==SIGNLE_TREE)?2:1)
[0142] The sixth condition concerns the treeType and coefficients.
[0143] In particular, if treeType is SINGLE_TREE, then if the value of the variable numSigCoeff is greater than 2, then a quadratic transform is applied to the current block and the decoder parses the lfnst_idx[x0][y0] syntax element.
[0144] When treeType is DUAL_TREE_LUMA or DUAL_TREE_CHROMA, if the value of the variable numSigCoeff is greater than 1, a secondary transform is applied to the current block, and lfnst_idx[x0][y0] is parsed. Here, numSigCoeff means a variable indicating the number of significant coefficients present in the current coding unit. If numSigCoeff is smaller than a critical value, efficient coding may not be performed even if a secondary transform is applied to the current block. This is because if the number of significant coefficients is small, the overhead of signaling lfnst_idx[x0][y0] is relatively large compared to the bits required for coefficient coding. Hereinafter, the significant coefficient means a coefficient that is not 0. Hereinafter, the significant coefficient in the present invention means a coefficient that is not 0 as described above.
[0145] vii) numZeroOutSigCoeff==0
[0146] The seventh condition relates to the effectiveness coefficients present at a particular location.
[0147] In more detail, if a secondary transform is applied to the current block, the transform coefficient quantized by the decoder is always 0 at a particular position. Therefore, if a non-zero (quantized) coefficient exists at a particular position, it means that a secondary transform is not applied to the current block, and whether or not lfnst_idx[x0][y0] is parsed is determined according to the number of significant coefficients at the particular position. For example, if numZeroOutSigCoeff is not 0, it means that a significant coefficient exists at the particular position, so lfnst_idx[x0][y0] is not parsed and is set to 0. On the other hand, if numZeroOutSigCoeff is 0, it means that no significant coefficient exists at the particular position, so lfnst_idx[x0][y0] is parsed.
[0148] FIG. 16 is a diagram illustrating a residual_coding syntax structure according to one embodiment of the present invention.
[0149] The residual_coding syntax structure is a syntax structure related to quantization coefficients and receives x0, y0, log2TbWidth, and log2TbHeight as inputs. At this time, x0 and y0 represent the left-upper coordinates (x0, y0) of the conversion block, log2TbWidth is the value obtained by taking the base-2 logarithm of the width of the conversion block, and log2TbHeight is the value obtained by taking the base-2 logarithm of the height of the conversion block. The number within the conversion block is coded in sub-block units, and the values of the coefficients within each sub-block are determined based on various syntax elements including sig_coeff_flag. At this time, the coefficients in sub-block units may be expressed as a Coefficient Group (CG). sig_coeff_flag[xC][yC] indicates whether the coefficient value at the (xC, yC) position within the current block is 0. If sig_coeff_flag[xC][yC] is 1, it indicates that the coefficient value at the corresponding position is not 0, and if sig_coeff_flag[xC][yC] is 0, it indicates that the coefficient value at the corresponding position is 0. In residual_coding, the x-coordinate value and y-coordinate value of the last significant coefficient in the scan order are indicated. Based on the x-coordinate value and y-coordinate value of the last significant coefficient in the scan order, the index (lastSubBlock) of the sub-block including the last significant coefficient in the scan order is determined. The index of the sub-block is also indexed based on the scan order. The scan order is the upper-right diagonal scan order described in FIG. 13. In the coefficient coding in sub-block units, the indices xC and yC indicating the coefficient position (coordinate value) are determined based on the left-upper coordinates (xS<<log2SbW, yS<<log2SbH) of the sub-block and the upper-right diagonal scan order (DiagScanOrder). At this time, xS and yS indicate the horizontal index and vertical index, respectively. log2SbW and log2SbH are the values obtained by taking the base-2 logarithm of the width and height of the sub-block, respectively.
[0150] If the value of sig_coeff_flag[xC][yC] is 1 (i.e., the number of positions (xC,yC) is not 0), and if transform skip is not applied to the current block (i.e., !transform_skip_flag[x0][y0]), numSigCoeff is counted. Since the quadratic transform may not be applied when transform skip is applied, numSigCoeff used for parsing lfnst_idx[x0][y0] counts the number of valid coefficients of the block to which transform skip is not applied.
[0151] Also, as described in FIG. 15, when a secondary transform is applied to a transform block, no significant coefficients exist in a particular region in the transform block. Therefore, a numZeroOutSigCoeff counter counts the number of significant coefficients (numZeroOutSigCoeff) existing in a particular region, and lfnst_idx[x0][y0] is not parsed unless numZeroOutSigCoeff is 0. In more detail, when a secondary transform is applied to a transform block, the region in which significant coefficients cannot exist is determined by the size of the transform block.
[0152] For example, in order to apply a quadratic transform, if the size of the transform block is 4x4 (i.e., log2TbWidth==2&& log2TbHeight==2), the transform block is divided into index [0,7] and index [8,15] regions in the scan order, and valid coefficients exist in the [0,7] region, but no valid coefficients can exist in the [8,15] region. The 4x4 transform block includes one sub-block. Thus, when the size of the transform block is 4x4, if the scan position is 8 or greater and the sub-block index is 0 (i.e., n>=8&&i==0), the number of valid coefficients is counted. In this case, the scan order is the upper right diagonal scan order.
[0153] As another example, in order for a quadratic transform to be applied, if the size of the transform block is 8x8 (i.e., log2TbWidth==3&& log2TbHeight==3), a valid coefficient exists only in the first sub-block of the transform block, and no valid coefficient exists in the remaining sub-blocks (e.g., the second and third sub-blocks). Even in the first sub-block, a valid coefficient exists in the index [0,7] range in the scan order, but no valid coefficient exists in the index [8,15] range. Thus, if the size of the transform block is 8x8, the number of valid coefficients is counted if the scan position in the first sub-block is 8 or more (i.e., n>=8&&i==0) or if the scan position exists in the remaining sub-blocks other than the first sub-block (e.g., exists in the second and third sub-blocks, i==1||i==2).
[0154] Finally, if the size of the transform block is greater than 8x8, only the first subblock in the transform block has a significant coefficient, and the remaining subblocks (e.g., the second and third subblocks) cannot have a significant coefficient. Therefore, if the subblock is the second or third subblock (i.e., i==1||i==2), the number of significant coefficients is counted. The numZeroOutSigCoeff counter, like the numSigCoeff counter, counts the number of significant coefficients only when sig_coeff_flag[xC][yC] is 1 and transform_skip_flag[x0][y0] is 0. In this case, the subblocks are indexed according to the upper right diagonal scan order described in FIG. 13.
[0155] In other words, if a non-zero coefficient exists in an area (specific area) where no significant coefficients can exist, it means that a secondary transformation has not been performed, so the significant coefficients are counted to check whether or not there is a non-zero coefficient in the specific area.
[0156] FIG. 17 illustrates a method for indicating a secondary transform at a coding unit level according to an embodiment of the present invention.
[0157] As described in FIG. 15 and FIG. 16, whether or not a secondary transform is applied is indicated by the lfnst_idx[x0][y0] syntax element at the coding unit level, and two significant coefficient counters (i.e., numSigCoeff counter and numZeroOutSigCoeff counter) are required to parse lfnst_idx[x0][y0]. In particular, in the case of numSigCoeff, since the numSigCoeff counter should count the number of significant coefficients present in the entire area of the coding unit, the throughput of coefficient coding may be reduced. Therefore, a method is required to reduce the number of counters or not use counters.
[0158] The secondary conversion instruction method shown in Fig. 17 is a method of parsing lfnst_idx[x0][y0] regardless of numSigCoeff. In other words, if all of the conditions i), ii), iii), iv), and v) described in Fig. 15 are satisfied (all are true), the decoder parses lfnst_idx[x0][y0]. Also, since the value of numSigCoeff is not referenced, the operation of the numSigCoeff counter described in Fig. 16 is not performed.
[0159] Hereinafter, this specification will describe a method of instructing secondary transform based on the position information of the last effective coefficient in the scan order. As in the case where the number of effective coefficients is small, if the position (scan index) of the last effective coefficient in the scan order is small, the coding efficiency by secondary transform is low. Therefore, it is necessary to efficiently instruct secondary transform based on the position information of the last effective coefficient in the scan order without using a counter.
[0160] (First Example)
[0161] FIG. 18 illustrates a method for indicating secondary transformation at a coding unit level according to one embodiment of the present invention. FIG.
[0162] FIG. 18 is a diagram illustrating a method of parsing lfnst_idx[x0][y0] using position information of the last significant coefficient in the scan order obtained by residual_coding instead of the numSigCoeff counter.
[0163] According to FIG. 18, since the numSigCoeff counter is not used, the numSigCoeff value does not need to be initialized, and lfnLastScanPos, which is a variable related to the position of the position information of the last significant coefficient in the scan order, is initialized to 1. If the value of lfnLastScanPos is 1, it indicates that the position (scan index) of the last significant coefficient in the scan order is smaller than the critical value or all transform coefficients in the block are 0. On the other hand, if the value of lfnLastScanPos is 0, it indicates that there is one or more significant coefficients in the block and the position (scan index) of the last significant coefficient in the scan order is equal to or greater than the critical value. Therefore, if the value of lfnLastScanPos is 1, lfnst_idx[x0][y0] is not parsed, and if the value of lfnLastScanPos is 0, lfnst_idx[x0][y0] is parsed. In addition, lfnst_idx[x0][y0] may be parsed if the value of lfnLastScanPos is 0 and all of the conditions i), ii), iii), iv), v), and vii) described in Figure 15 are satisfied (all are true).
[0164] In other words, if there is at least one significant coefficient in the current block and the position (scan index) of the last significant coefficient in the scan order is equal to or greater than a threshold value, lfnst_idx[x0][y0] is parsed. In this case, the threshold value is an integer equal to or greater than 0, as described below. For example, assuming that the threshold value is 1, the position (scan index) of the last significant coefficient in the scan order is equal to or greater than the threshold value means that the significant coefficient is present at a position other than the upper left end of the block. In other words, lfnst_idx[x0][y0] is parsed only in the remaining cases except when the significant coefficient is not present in the current block or is present only at the upper left end of the current block, that is, only when the significant coefficient is present at a position other than the upper left end of the current block. The presence of a significant coefficient at a position other than the upper left end of the current block may be expressed as 'LfnstDConly==0'. The upper left end of a block described in the present invention may mean that the vertical coordinate value is (0,0), or may mean the first position according to a preset scan order (e.g., diagonal order from the upper right), or may be referred to as DC.
[0165] FIG. 19 is a diagram illustrating a residual_coding syntax structure according to an embodiment of the present invention.
[0166] FIG. 19 shows the residual_coding syntax structure according to FIG. 18 described above. In the residual_coding, syntax elements related to the x-coordinate and y-coordinate of the last significant coefficient in the scan order are parsed, and LastSignificantCoeffX and LastSignificantCoeffY variables are set. LastSignificantCoeffX indicates the x-coordinate of the last significant coefficient in the scan order, and LastSignificantCoeffY indicates the y-coordinate of the last significant coefficient in the scan order. Based on LastSignificantCoeffX and LastSignificantCoeffY, the LastScanPos variable, which is the scan index of the last significant coefficient in the scan order, and the index (lastSubBlock) of the subblock including the last significant coefficient are determined. In this case, as described in FIG. 16, when the secondary transform is applied to the current block, the significant coefficient exists only in the first subblock. In other words, if the significant coefficient exists only in the first subblock, the secondary transform is applied.
[0167] For example, in the 4×4 block in FIG. 14(a), if LastSignificantCoeffX is 2 and LastSignificantCoeffY is 3, LastScanPos is determined to be 13. Since the 4×4 block is composed of one subblock, the index (lastSubBlock) of the subblock containing the last significant coefficient is determined to be 0. As another example, the 8×8 block in FIG. 14(b) is divided into 4×4 subblocks. In detail, in FIG. 14(b), the 4×4 block corresponding to x coordinates 0 to 3 and y coordinates 0 to 3 is set to the first subblock, the 4×4 block corresponding to x coordinates 0 to 3 and y coordinates 4 to 37 is set to the second subblock, the 4×4 block corresponding to x coordinates 4 to 7 and y coordinates 0 to 34 is set to the third subblock, and the 4×4 block corresponding to x coordinates 4 to 7 and y coordinates 4 to 37 is set to the fourth subblock. In this case, the first subblock is indexed with index 0, the second subblock with index 1, the third subblock with index 2, and the fourth subblock with index 3. The subblocks are indexed in the upper right diagonal scan order described in FIG. 13. In this case, if LastScanPos is 2 and LastScanPos is 3, lastScanPos is determined to be 13. Since lastScanPos is 13, the subblock containing lastScanPos13 is the first subblock (i.e., subblock index 0), and the index of the subblock containing the last significant coefficient (lastSubBlock) is determined to be 0.
[0168] Based on the above-mentioned lastScanPos, lfnstLastScanPos is determined. In detail, if the width and height of the transform block are 4 or more and transform skip is not applied to the transform block, lfnstLastScanPos is set as shown in the following formula 1. In other words, if log2TbWidth>=2, log2TbHeight>=2, and transform_skip_flag[x0][y0] is 0, lfnstLastScanPos is set as shown in the following formula 1. In this case, if transform_skip_flag[x0][y0] is 0, it means that transform skip is not applied to the current transform block. In detail, the flag transform_skip_flag[x0][y0] described in this specification indicates whether a primary transform and a secondary transform are applied to the transform block. For example, if the value of transform_skip_flag[x0][y0] is 1, it indicates that no linear transform or secondary transform is applied to the transform block (i.e., transform skip is applied), and if the value of transform_skip_flag[x0][y0] is 0, it indicates that no linear transform or secondary transform is applied to the transform block (i.e., transform skip is not applied).
[0169]
number
[0170] As mentioned above, the initialization value of lfnstLastScanPos is set to 1.
[0171] In Equation 1, cIdx indicates a variable representing a color component of the current transform block. For example, if cIdx is 0, it indicates that the transform block processed by residual_coding is a luma Y component. If cIdx is 1, it indicates that the transform block processed by residual_coding is a chroma Cb component, and if cIdx is 2, it indicates that the transform block processed is a chroma Cr component. lfnstLastScanPosTh[cIdx], which is a threshold value for lastScanPos, is set to a different value according to the color component.
[0172] According to Equation 1, if lfnstLastScanPos of a line is 1 and lastScanPos is smaller than lfnstLastScanPosTh[cIdx], lfnstLastScanPos is updated to 1. On the other hand, if lfnstLastScanPos of a line is 0 or lastScanPos is equal to or larger than lfnstLastScanPosTh[cIdx], lfnstLastScanPos is updated to 0. In other words, if lastScanPos of all transform blocks included in a coding unit are smaller than a threshold value or the number of all transform blocks is 0, lfnstLastScanPos is determined to be 1, and lfnst_idx[x0][y0] is set to 0 without being parsed according to the lfnst_idx[x0][y0] parsing condition of FIG. 18. When lfnst_idx[x0][y0] is not parsed and is set to 0, it indicates that secondary transformation is not applied to the current block. On the other hand, if the LastScanPos of any one of the transform blocks included in the coding unit is equal to or greater than the threshold value, lfnstLastScanPos is determined to be 0, and if all of the conditions i), ii), iii), iv), v), and vii) described in FIG. 15 are satisfied (all are true), the decoder parses lfnst_idx[x0][y0]. The decoder parses lfnst_idx[x0][y0] to determine whether secondary transformation is applied to the current block, and if secondary transformation is applied, it determines / determines the transform kernel to be used for secondary transformation.
[0173] lfnstLastScanPosTh[cIdx] in Equation 1 is a preset integer value equal to or greater than 0, and both the encoder and the decoder use the same value. Also, the same threshold value may be used for all color components. In this case, lfnstLastScanPos is set as in Equation 2 below. A coding unit described in this specification is composed of a plurality of coding blocks, and there is a transform block corresponding to each coding block. The transform block is a transform block having luminance and chrominance components. More specifically, it is a Y transform block, a Cb transform block, and a Cr transform block. In this case, whether or not to parse lfnst_idx[x0][y0] described in this specification is determined for each transform block corresponding to each coding block. In other words, if any one of the transform blocks of the Y transform block, the Cb transform block, and the Cr transform block satisfies the conditions described in this specification, lfnst_idx[x0][y0] is parsed.
[0174]
number
[0175] lfnstLastScanPosTh is a preset integer value equal to or greater than 0, and both the encoder and the decoder use the same value. For example, lfnstLastScanPosTh may be 1. That is, if lastScanPos is equal to or greater than 1, lfnstLastScanPos is updated to 0, and lfnst_idx[x0][y0] is parsed. In this case, since the threshold value (lfnstLastScanPosTh) is an integer value, if lastScanPos is equal to or greater than 1, this has the same meaning as if lastScanPos is greater than 0. Although the case where the threshold value is 1 has been described as an example of the present invention, the present invention is not limited thereto.
[0176] In other words, whether or not to parse lfnst_idx[x0][y0] is determined based on lastScanPos. More specifically, if the secondary transform is applied as described above, the last valid coefficient in scan order is only present in the first subblock of the transform block. Therefore, if the index (lastSubBlock) of the subblock (where the index indicated by lastScanPos is located) containing the last valid coefficient in scan order is 0, the width of the transform block is 4 or more (log2TbWidth>=2), the height of the transform block is 4 or more (log2TbHeight>=2), transform_skip_flag[x0][y0] is 0 (transform skip is not applied), and LastScanPos is greater than 0 (LastScanPos is 1 or greater), lfnst_idx[x0][y0] is parsed. This can be expressed as Equation 3 below.
[0177]
number
[0178] On the other hand, in the above-described first embodiment, the numSigCoeff counter is not used for parsing lfnst_idx[x0][y0], so the number of valid coefficients (numSigCoeff) is not counted.
[0179] (Second Example)
[0180] FIG. 20 is a diagram illustrating a residual_coding syntax structure according to another embodiment of the present invention.
[0181] FIG. 20 is a diagram showing a method of setting a threshold value for LastScanPos according to treeType when residual_coding is further input to FIG. 19 and a treeType variable is further input.
[0182] If the width and height of the transform block are 4 or more and transform skip is not applied to the transform block, lfnstLastScanPos is set as shown in the following formula 4. In other words, if log2TbWidth>=2, log2TbHeight>=2, and transform_skip_flag[x0][y0] is 0, lfnstLastScanPos is set as shown in the following formula 4. In this case, if transform_skip_flag[x0][y0] is 0, it means that transform skip is not applied to the current transform block.
[0183]
number
[0184] In Formula 4, lfnstLastScanPosTh means a threshold value for lastScanPos, and its value is set according to treeType. If treeType is SINGLE_TREE, DUAL_TREE_LUMA, or DUAL_TREE_CHROMA, lfnstLastScanPosTh is set to val1, val2, or val3, respectively. If lfnstLastScanPos of a line is 1 and lastScanPos is smaller than lfnstLastScanPosTh, lfnstLastScanPos is updated to 1. On the other hand, if lfnstLastScanPos of a line is 0 or lastScanPos is equal to or larger than lfnstLastScanPosTh, lfnstLastScanPos is updated to 0.
[0185] As a result, in Equation 4, if the lastScanPos of all the transform blocks included in the coding unit are smaller than the threshold value or the number of all the transform blocks is 0, lfnstLastScanPos is determined to be 1, and lfnst_idx[x0][y0] is not parsed and is set to 0 according to the lfnst_idx[x0][y0] parsing condition of FIG. 18. This indicates that the secondary transform is not applied to the current block. Meanwhile, if the LastScanPos of any one of the transform blocks included in the coding unit is equal to or greater than the threshold value, lfnstLastScanPos is determined to be 0, and if all of i), ii), iii), iv), v), and vii) described in FIG. 15 are satisfied (all are true), the decoder parses lfnst_idx[x0][y0]. The decoder parses lfnst_idx[x0][y0] to determine whether a secondary transform is applied to the current block, and if so, determines the transform kernel to be used for the secondary transform.
[0186] val1, val2, and val3 are preset integer values equal to or greater than 0, and both the encoder and decoder use the same value. If treeType is SINGLE_TREE, both luma and chroma components are included, so val1, which is the value of lfnstLastScanPosTh, may be expressed as the sum of val2 and val3.
[0187] In the second embodiment, the numSigCoeff counter is not used for parsing lfnst_idx[x0][y0], so the number of valid coefficients (numSigCoeff) is not counted.
[0188] (Third Example)
[0189] FIG. 21 is a diagram illustrating a method for indicating a secondary transformation at a coding unit level according to another embodiment of the present invention.
[0190] According to FIG. 21, lfnst_idx[x0][y0] is parsed using the position information of the last significant coefficient in the scan order obtained by residual_coding, instead of the numSigCoeff counter.
[0191] Since the numSigCoeff counter is not used, numSigCoeff does not need to be initialized, and lfnLastScanPos, which is a variable related to the position of the position information of the last effective coefficient in the scan order, is initialized to 0. The lfnstLastScanPos variable in FIG. 21 is a value obtained by adding lastScanPos of the transform block included in the coding unit. At this time, if lfnLastScanPos is greater than the critical value and all of the conditions i), ii), iii), iv), v), and vii) described in FIG. 15 are satisfied (all are true), the decoder parses lfnst_idx[x0][y0]. The decoder parses lfnst_idx[x0][y0] to check whether a secondary transform is applied to the current block, and if a secondary transform is applied, checks / determines the transform kernel to be used for the secondary transform. On the other hand, if lfnLastScanPos is less than the critical value, lfnst_idx[x0][y0] is not parsed and is set to 0. This indicates that no quadratic transformation is applied.
[0192] The threshold is set according to the treeType. If the treeType is SINGLE_TREE, DUAL_TREE_LUMA, or DUAL_TREE_CHROMA, the threshold is set to Th1, Th2, or Th3, respectively. Th1, Th2, and Th3 are preset integer values equal to or greater than 0, and both the encoder and the decoder use the same value. If the treeType is SINGLE_TREE, both luma and chroma components are included, so the threshold Th1 may be expressed as the sum of Th2 and Th3.
[0193] FIG. 22 is a diagram illustrating a residual_coding syntax structure according to another embodiment of the present invention.
[0194] Figure 22 shows the residual_coding syntax structure according to Figure 21 above, and if the width and height of the transform block are 4 or more and transform skip is not applied to the transform block, lfnstLastScanPos is set as shown in the following Formula 5. In other words, if log2TbWidth>=2, log2TbHeight>=2, and transform_skip_flag[x0][y0] is 0, lfnstLastScanPos is set as shown in the following Formula 5. In this case, if transform_skip_flag[x0][y0] is 0, it means that transform skip is not applied to the current transform block.
[0195]
number
[0196] In Equation 5, lfnLastScanPos is the sum of all lastScanPos of the transform blocks included in the coding unit, and as described in FIG. 21, lfnLastScanPos is compared with a threshold value to determine whether or not lfnst_idx[x0][y0] should be parsed.
[0197] In the third embodiment, the numSigCoeff counter is not used for parsing lfnst_idx[x0][y0], so the number of valid coefficients (numSigCoeff) is not counted.
[0198] Meanwhile, the coding unit includes transform units divided by a transform tree with the same size as the coding unit as the root node. In this case, the transform unit includes a transform block for each color component. If the secondary transform is specified at the coding unit level, residual coding is performed on all transform blocks included in the coding unit, and then lfnst_idx[x0][y0] is parsed based on coefficient information. In another embodiment, the secondary transform may be specified at the transform unit level. If the secondary transform is specified at the transform unit level, each transform unit included in the coding unit uses a different lfnst_idx[x0][y0]. Thus, the encoder can find an optimal lfnst_idx[x0][y0] for each transform unit, and the coding efficiency can be further improved. Also, if the secondary transform is specified at the coding unit level and the coding unit includes four transform units, residual coding should be processed on all transform blocks included in the four transform units in order to parse lfnst_idx[x0][y0]. That is, even if the decoder obtains the transform coefficients through residual coding for the first transform unit, the decoder cannot perform inverse transform for the first transform unit because it cannot obtain the lfnst_idx[x0][y0] value, which not only increases the buffer size of the decoder but also may cause excessive delay in the decoder.
[0199] The first to third embodiments described in Figures 18 to 22 can also be applied to the case where the secondary transform is specified at the transform unit level. If the secondary transform is specified at the coding unit level, the first to third embodiments determine whether or not to parse lfnst_idx[x0][y0] based on the position of the last significant coefficient in the scan order of the transform block included in the coding unit. If the secondary transform is specified at the transform unit level, the first to third embodiments determine whether or not to parse lfnst_idx[x0][y0] based on the position of the last significant coefficient in the scan order of the transform block included in the transform unit.
[0200] Hereinafter, the specific manner in which the secondary transform is indicated at the transform unit level will be described in this specification.
[0201] FIG. 23 is a diagram illustrating a method for indicating a secondary transform at the transform unit level according to an embodiment of the present invention.
[0202] According to FIG. 12, lfnst_idx[x0][y0] is parsed using the position information of the last significant coefficient in the scan order obtained by residual_coding, instead of the numSigCoeff counter.
[0203] First, before performing residual_coding, lfnLastScanPos, which is a variable related to the position of the last significant coefficient in the scan order, is initialized to 1. If the lfnLastScanPos variable is 1, it indicates that the position (scan index) of the last significant coefficient in the scan order for all transform blocks included in the transform unit is smaller than a threshold value or all transform coefficients in the block are 0. If the lfnLastScanPos variable is 0, it indicates that one or more significant coefficients exist in one or more transform blocks included in the transform unit and the position (scan index) of the last significant coefficient in the scan order is equal to or greater than a threshold value. According to the first embodiment described above, if lfnLastScanPos, which is set based on the position of the last significant coefficient in the scan order of the transform block, is 0 and all of the conditions i), ii), iii), iv), v), and vi) described below are satisfied (all are true), the decoder parses lfnst_idx[x0][y0].
[0204] Parsing conditions for lfnst_idx[x0][y0] syntax element
[0205] i) Min(lfnWidth,lfnHeight)>=4
[0206] The first condition concerns the size of the block. If the width and height of the block are each 4 pixels or more, the decoder parses the lfnst_idx[x0][y0] syntax element.
[0207] In more detail, the decoder checks the block size conditions for which the secondary transform can be applied. The variables SubWidthC and SubHeightC are set according to the color format and indicate the ratio of the width and height of the chroma component to the width and height of the luma component of the picture, respectively. For example, a 4:2:0 color format image has a structure including one chroma sample per four luma samples, so SubWidthC and SubHeightC are both set to 2. As another example, a 4:4:4 color format image has a structure including one chroma sample per luma sample, so SubWidthC and SubHeightC are both set to 1. The number of horizontal samples of the current block, lfnWidth, and the number of vertical samples, lfnHeight, are set based on SubWidthC and SubHeightC. If treeType is DUAL_TREE_CHROMA, the transform unit includes only chroma components, so the number of horizontal samples of the chroma transform block is equal to the value obtained by dividing tbwidth, which is the width of the luma transform block, by SubWidthC. Similarly, the number of samples in the vertical direction of a chroma transform block is equal to the height of the luma transform block, tbHeight, divided by SubHeightC. If treeType is SINGLE_TREE or DUAL_TREE_LUMA, the transform unit contains a luma component, so lfnWidth and lfnHeight are set to tbwidth and tbHeight, respectively. The minimum condition for a block to which a quadratic transform can be applied is 4x4, so if Min(lfnWidth,lfnHeight)>=4 is satisfied, lfnst_idx[x0][y0] is parsed.
[0208] ii) sps_lfn_enabled_flag==1
[0209] The second condition relates to a flag value indicating whether a secondary transform is activated or applicable. If the value of the flag (sps_lfnst_enabled_flag) indicating whether a secondary transform is activated or applicable is set to 1, the decoder parses lfnst_idx[x0][y0].
[0210] In detail, the secondary transform is indicated by the higher level syntax RBSP. At least one of the SPS, PPS, VPS, tile group header, and slice header includes a flag having a 1-bit size indicating whether the secondary transform is activated and applicable. If sps_lfnst_enabled_flag is 1, it indicates that the lfnst_idx[x0][y0] syntax element is present in the transform unit syntax. If sps_lfnst_enabled_flag is 0, it indicates that the lfnst_idx[x0][y0] syntax element is not present in the transform unit syntax.
[0211] iii)CuPredMode[x0][y0]==MODE_INTRA
[0212] The third condition is related to the prediction mode: the secondary transform is only applied to intra-predicted blocks, so if the current block is an intra-predicted block, the decoder parses lfnst_idx[x0][y0].
[0213] iv)IntraSubPartitionsSplitType==ISP_NO_SPLIT
[0214] The fourth condition relates to whether the ISP prediction method is applied or not. If ISP is not applied to the current block, the decoder parses the lfnst_idx[x0][y0] syntax element.
[0215] In more detail, as described with reference to FIG. 11, when a current CU is divided into a number of transform units smaller than the CU size, secondary transform is not applied to the divided transform units. In this case, lfnst_idx[x0][y0], which is a syntax element related to secondary transform, is not parsed and is set to 0. When a current CU is divided into a number of transform units whose CU size is smaller than the transform tree, the case where ISP prediction is applied to the current coding unit is included. The ISP prediction method is a prediction method for dividing a transform tree into a number of transform units smaller than the CU size according to a preset division method when intra prediction is applied to the current coding unit. An ISP prediction mode is indicated at a coding unit level, and a variable IntraSubPartitionsSplitType variable is set based on the ISP prediction mode. In this case, if IntraSubPartitionsSplitType is ISP_NO_SPLIT, it indicates that ISP is not applied to the current block. Due to the characteristics of intra prediction that generates a prediction sample at a transform unit level, when a transform tree is divided into a number of transform units, prediction accuracy is higher than when the transform tree is not divided. Therefore, even if a secondary transform is not applied to the multiple divided transform units, the energy of the residual signal is likely to be efficiently compressed.
[0216] v)!intra_mip_flag[x0][y0]
[0217] The fifth condition relates to the intra prediction method. If matrix based intra prediction (MIP) is not applied to the current coding unit, the decoder parses the lfnst_idx[x0][y0] syntax element.
[0218] In detail, MIP is used as one method of intra prediction, and whether MIP is applied is indicated by intra_mip_flag[x0][y0] at the coding unit level. If intra_mip_flag[x0][y0] is 1, it indicates that MIP is applied to the prediction of the current coding unit, and prediction is performed by multiplying the reconstructed samples around the current block by a preset matrix. When MIP is applied, it shows a different residual signal property from general intra prediction that performs directional or non-directional prediction, so that secondary transformation may not be applied to the transform block when MIP is applied.
[0219] vi) numZeroOutSigCoeff==0
[0220] The sixth condition relates to the effectiveness coefficient present at a particular location.
[0221] In more detail, if a secondary transform is applied to the current block, the transform coefficients quantized by the decoder are always 0 at a particular position. Therefore, if a non-zero quantized coefficient exists at a particular position, it means that a secondary transform has not been applied, and lfnst_idx[x0][y0] is parsed according to the number of significant coefficients at the particular position. For example, if numZeroOutSigCoeff is not 0, it means that a significant coefficient exists at the particular position, so lfnst_idx[x0][y0] is not parsed and is set to 0. On the other hand, if numZeroOutSigCoeff is 0, it means that no significant coefficient exists at the particular position, so lfnst_idx[x0][y0] is parsed.
[0222] If it is indicated at the transform unit level whether or not secondary transform is applied to the current block according to the first embodiment described above, the residual_coding method described in FIG. 19 is followed. According to Equation 1 for determining lfnLastScanPos described in FIG. 19, if the lastScanPos of all transform blocks included in the transform unit are less than the threshold value or the number of all transform blocks is 0, lfnLastScanPos is determined to be 1, and lfnst_idx[x0][y0] is set to 0 without being parsed. This indicates that secondary transform is not applied to the current block. On the other hand, if the LastScanPos of any one of the transform blocks included in the transform unit is equal to or greater than the threshold value, lfnstLastScanPos is determined to be 0, and if all of the conditions i), ii), iii), iv), v), and vi) described in FIG. 23 are satisfied (all are true), the decoder parses lfnst_idx[x0][y0]. The decoder parses lfnst_idx[x0][y0] to determine whether a secondary transform is applied to the current block, and if so, determines the transform kernel to be used for the secondary transform.
[0223] If whether or not secondary transformation is applied is indicated at the transform unit level according to the second embodiment described above, the transform unit syntax structure described in FIG. 23 is applied, and the residual_coding method described in FIG. 20 is used. According to Equation 4 for determining lfnLastScanPos described in FIG. 20, if the lastScanPos of all transform blocks included in the transform unit are less than the threshold value or the number of all transform blocks is 0, lfnLastScanPos is determined to be 1, and lfnst_idx[x0][y0] is set to 0 without being parsed. This indicates that secondary transformation is not applied to the current block. On the other hand, if the LastScanPos of any one of the transform blocks included in the transform unit is equal to or greater than the threshold value, lfnstLastScanPos is determined to be 0, and if all of the conditions i), ii), iii), iv), v), and vi) described in FIG. 23 are satisfied (all are true), the decoder parses lfnst_idx[x0][y0]. The decoder parses lfnst_idx[x0][y0] to determine whether a secondary transform is applied to the current block, and if so, determines the transform kernel to be used for the secondary transform.
[0224] FIG. 24 is a diagram illustrating a method for indicating a secondary transform at a transform unit level according to another embodiment of the present invention.
[0225] According to the above-mentioned third embodiment, lfnst_idx[x0][y0] is parsed using the position information of the last significant coefficient in the scan order obtained by residual_coding, instead of the numSigCoeff counter.
[0226] Before performing residual_coding, the variable lfnLastScanPos, which is related to the position of the last significant coefficient in the scan order, is initialized to 0. The variable lfnLastScanPos is a value obtained by adding the lastScanPos of the transform block included in the transform unit. At this time, if lfnLastScanPos is greater than the critical value and all of the conditions i), ii), iii), iv), v), and vii) described in FIG. 23 are satisfied (all are true), the decoder parses lfnst_idx[x0][y0]. The decoder parses lfnst_idx[x0][y0] to check whether a secondary transform is applied to the current block, and if a secondary transform is applied, checks / determines the transform kernel to be used for the secondary transform. On the other hand, if lfnLastScanPos is less than the critical value, lfnst_idx[x0][y0] is not parsed and is set to 0. This indicates that a secondary transform is not applied.
[0227] The threshold is set according to the treeType. If the treeType is SINGLE_TREE, DUAL_TREE_LUMA, or DUAL_TREE_CHROMA, the threshold is set to Th1, Th2, or Th3, respectively. Th1, Th2, and Th3 are preset integer values equal to or greater than 0, and both the encoder and the decoder use the same value. If the treeType is SINGLE_TREE, both luma and chroma components are included, so the threshold Th1 may be expressed as the sum of Th2 and Th3.
[0228] If whether or not secondary transform is applied is indicated at the transform unit level based on the third embodiment described above, the residual_coding method described in Figure 22 is used. According to Equation 5 for determining lfnLastScanPos described in Figure 22, the variable lfnLastScanPos is set to a value obtained by adding up all lastScanPos of the transform blocks included in the transform unit. Then, lfnLastScanPos is compared with a threshold value to determine whether or not lfnst_idx[x0][y0] should be parsed.
[0229] On the other hand, if a secondary transform is indicated at the transform unit level, there is a possibility that there is a high correlation between the transform units included in the coding unit. This is because the prediction method is determined at the coding unit level. Therefore, lfnst_idx[x0][y0] is signaled only in the first transform unit included in the coding unit, and the signaled lfnst_idx[x0][y0] is shared with the remaining transform units. That is, only when subTuIndex indicating the index of the transform unit is 0, lfnst_idx[x0][y0] may be parsed using the above-mentioned first to third embodiments. If subTuIndex is greater than 0, the corresponding transform unit does not parse lfnst_idx[x0][y0], and uses the value of lfnst_idx[x0][y0] of the first transform unit shared.
[0230] On the other hand, a counter is used to count the significant coefficients, but the decoder decides whether to parse lfnst_idx[x0][y0] by considering only the significant coefficients in the left-top sub-block of the transform block, in order to reduce the amount of calculation.
[0231] Meanwhile, if the secondary transform is instructed at the transform unit level, the delay time of the decoder is reduced compared to when it is instructed at the coding unit level, but other delay times may occur. For example, even if the secondary transform is instructed at the transform unit level, the secondary transform is instructed after all the coding of the luma transform coefficient, the Cb transform coefficient, and the Cr transform coefficient is completed. Thus, even if all the coding (processing) of the luma transform coefficient is completed, the inverse transform process for the luma transform coefficient is performed after the coding (processing) of the Cb transform coefficient and the Cr transform coefficient is completed. This causes other delay times in the decoder.
[0232] Hereinafter, in this specification, a method for instructing a secondary transformation that can minimize the delay time of a decoder will be described.
[0233] (Fourth Example)
[0234] As an example of a method of indicating a secondary transform that can minimize the delay time of a decoder, the secondary transform is indicated at a transform unit level, but a syntax element lfnst_idx[x0][y0] related to the secondary transform is parsed before coding the luma transform coefficient. Thus, the decoder can perform an inverse transform process on the luma transform coefficient immediately after the coding of the luma transform coefficient is completed without waiting for the Cb transform coefficient and the Cr transform coefficient. Similarly, the decoder can perform an inverse transform process on the Cb transform coefficient immediately after the coding of the Cb transform coefficient is completed without waiting for the coding of the Cr transform coefficient. Such a method of indicating a secondary transform can minimize the delay time of a decoder and solve a pipeline problem.
[0235] FIG. 25 is a diagram illustrating a coding unit syntax according to one embodiment of the present invention.
[0236] As shown in Figure 25, since the secondary transform is indicated at the transform unit level, the syntax for the secondary transform, lfnst_idx[x0][y0], is not parsed at the coding unit level, but at the transform unit level which is split by the transform_tree.
[0237] FIG. 26 illustrates a method for indicating a secondary transform at a transform unit level according to another embodiment of the present invention.
[0238] Referring to FIG. 26, the secondary transform is indicated at the transform unit level, and lfnst_idx[x0][y0], which is a syntax element related to the secondary transform, is parsed first before luma and chroma transform coefficient coding (residual_coding). For example, if lfnst_idx[x0][y0] is parsed before obtaining transform coefficients, inverse transforms for Y, Cb, and Cr transform coefficients are processed immediately after coefficient coding for each color component Y, Cb, and Cr is completed. For example, inverse transforms for luma (Y) transform coefficients are performed immediately after transform coefficient coding for the Y component is completed. Similarly, inverse transforms for Cb transform coefficients are performed immediately after transform coefficient coding for the Cb component (residual_coding) is completed, and inverse transforms for Cr transform coefficients are performed immediately after transform coefficient coding for the Cr component (residual_coding) is completed.
[0239] If lfnst_idx[x0][y0] is parsed after transform coefficient coding (residual_coding) for Y, Cb, and Cr, even if transform coefficient coding (residual_coding) for Y is completed, inverse transform for Y transform coefficient is not performed / processed unless transform coefficient coding (residual_coding) for Cb, Cr is completed / processed. Therefore, even if transform coefficient coding (residual_coding) for Y is completed, the decoder cannot perform inverse transform for Y transform coefficient until transform coefficient coding (residual_coding) for other components (Cb, Cr) is completed, which causes unnecessary delay. However, as described above, if lfnst_idx[x0][y0] is parsed before transform coefficient coding (residual_coding), inverse transform for each transform coefficient of each color component is performed immediately after transform coefficient coding (residual_coding) for each color component is completed, so that the delay of the decoder is minimized.
[0240] In the transform_unit() syntax structure, tu_cbf_luma[x0][y0], tu_cbf_cb[x0][y0], tu_cbf_cr[x0][y0], transform_skip_flag[x0][y0], etc. are parsed.
[0241] In detail, tu_cbf_luma[x0][y0] is an element indicating whether the current luma transform block includes one or more non-zero transform coefficients. When tu_cbf_luma[x0][y0] is 1, it indicates that the current luma transform block includes one or more non-zero transform coefficients. When tu_cbf_luma[x0][y0] is 0, it indicates that all transform coefficients of the current luma transform block are zero. tu_cbf_cb[x0][y0] is an element indicating whether the current chroma Cb transform block includes one or more non-zero transform coefficients. When tu_cbf_cb[x0][y0] is 1, it indicates that the current chroma Cb transform block includes one or more non-zero transform coefficients. When tu_cbf_cb[x0][y0] is 0, it indicates that all transform coefficients of the current chroma Cb transform block are zero. tu_cbf_cr[x0][y0] is an element that indicates whether the current chroma Cr transform block contains one or more non-zero transform coefficients. When tu_cbf_cr[x0][y0] is 1, it indicates that the current chroma Cr transform block contains one or more non-zero transform coefficients. When tu_cbf_cr[x0][y0] is 0, it indicates that all transform coefficients of the current chroma Cr transform block are zero. transform_skip_flag[x0][y0] is a syntax element related to transform skip. When transform_skip_flag[x0][y0] is 1, it indicates that inverse transform is not applied to the current luma transform block. When transform_skip_flag[x0][y0] is 0, it indicates that whether inverse transform is applied to the current luma transform block is determined by other syntax elements.
[0242] As one example of a method for indicating a secondary transform according to Figure 26, the syntax element lfnst_idx[x0][y0] for the secondary transform is parsed based on the position of the last significant coefficient in the scan order, rather than based on the number of non-zero transform coefficients.
[0243] First, the lfnLastScanPos variable is initialized to 1. As described in Figure 23, the variable lfnLastScanPos indicates the position information of the last significant coefficient in the scan order of the transform block included in the current transform unit. In detail, if lfnLastScanPos is 1, it indicates that the position (scan index) of the last significant coefficient in the scan order for all transform blocks included in the transform unit is smaller than a critical value or all transform coefficients in the blocks are 0. If lfnLastScanPos is 0, it indicates that for one or more transform blocks included in the transform unit, there is one or more significant coefficients in the block and the position (scan index) of the last significant coefficient in the scan order is greater than or equal to the critical value.
[0244] Next, the variable numZeroOutSigCoeff is initialized and set to 0. If a secondary transform is applied to a transform block, the last significant coefficient in the scan order cannot exist. Therefore, the variable numZeroOutSigCoeff indicates whether a significant coefficient exists at a specific position, and whether a secondary transform is applied is confirmed based on the existence of a significant coefficient. For example, it is assumed that only up to 16 significant coefficients are allowed if a secondary transform is applied to a transform block. In a transform block of 4×4 or 8×8 size, a significant coefficient may exist in the index [0,7] range in the scan order (up to 8 non-zero transform coefficients are allowed). Meanwhile, in a transform block having a size other than 4×4 or 8×8, a significant coefficient may exist in the index [0,15] range in the scan order (up to 16 non-zero transform coefficients are allowed). Therefore, if the position (scan index) of the last significant coefficient in the scan order exists outside the range where the significant coefficients can exist, the decoder can automatically recognize that a secondary transform is not applied to the current transform block.
[0245] Whether or not to parse lfnst_idx[x0][y0], which is a syntax element related to secondary transform, is determined before coefficient coding (residual_coding) based on the position (scan index) of the last significant coefficient in the scan order. Therefore, the decoder processes information about the position of the last significant coefficient in the scan order before coefficient coding (residual_coding).
[0246] In detail, if the current luma transform block contains one or more non-zero significant coefficients (tu_cbf_luma[x0][y0] == 1) and transform skip is not applied to the current luma transform block (transform_skip_flag[x0][y0] == 0), last_significant_pos, a syntax structure related to the position of the last significant coefficient in the luma scanning order, is processed.
[0247] If the value of tu_cbf_luma[x0][y0] is 0 (tu_cbf_luma[x0][y0]=0), it indicates that all coefficients of the corresponding transform block are 0, which indicates that coefficient coding (residual_coding) is not performed. Therefore, there is no need to perform processing related to the position information of the last significant coefficient in the scan order.
[0248] If transform_skip_flag[x0][y0] is 1, it indicates that the inverse transform is not applied to the current luma transform block, and therefore, coefficient coding (residual_coding) is performed without being based on the position information of the last significant coefficient in the scan order.
[0249] If the current chroma Cb transform block includes at least one significant coefficient (tu_cbf_cb[x0][y0]==1), the syntax structure last_significant_pos, which is related to the position of the last non-zero coefficient in the scan order of the current chroma Cb transform block, is processed. The syntax structure last_significant_pos receives as input (x0,y0), which is the left-top coordinate of the transform block, the base 2 logarithm of the width of the transform block, the base 2 logarithm of the height of the transform block, and cIdx, which is a variable indicating which color component the transform block is. For example, cIdx=0 indicates a luma Y transform block, cIdx=1 indicates a chroma Cb transform block, and cIdx=2 indicates a chroma Cr transform block. If the value of tu_cbf_cb[x0][y0] is 0 (tu_cbf_cb[x0][y0]==0), it indicates that all coefficients of the corresponding transform block are 0. This means that coefficient coding (residual_coding) is not performed, so there is no need to process the position information of the last coefficient that is not 0 in the scan order.
[0250] Meanwhile, if the current luma Cr transform block includes at least one significant coefficient (tu_cbf_cr[x0][y0]==1), tu_joint_cbcr_residual[x0][y0], which is a syntax element that indicates whether chroma Cb and Cr are represented as one residual signal, is parsed before processing last_significant_pos. For example, if tu_joint_cbcr_residual[x0][y0] is 1, coefficient coding (residual_coding) for Cr is not processed and the residual signal for Cr is derived from the restored residual signal of Cb. On the other hand, if tu_joint_cbcr_residual[x0][y0] is 0, coefficient coding (residual_coding) for Cr is performed according to the value of tu_cbf_cr[x0][y0]. If the current chroma Cr transform block contains at least one significant coefficient (tu_cbf_cr[x0][y0] == 1), the syntax structure last_significant_pos related to the position of the last significant coefficient in the scan order of chroma Cr is processed. If the value of tu_cbf_cbr[x0][y0] is 0 (tu_cbf_cr[x0][y0] == 0), it indicates that all coefficients of the chroma Cr transform block are 0. This means that coefficient coding (residual_coding) is not performed, so there is no need to process the position information of the last coefficient that is not 0 in the scan order.
[0251] The last_significant_pos process for each color component is performed to obtain the position (scan index) of the last significant coefficient in the scan order for each color component, and the lfnLastScanPos and numZeroOutSigCoeff values are updated based on that.
[0252] Then, if all of the conditions i), ii), iii), iv), v), vi), and vii) described below are satisfied (all are true), the decoder parses lfnst_idx[x0][y0] before coefficient coding (residual_coding).
[0253] Parsing conditions for the lfnst_idx[x0][y0] syntax element before coefficient coding (residual_coding)
[0254] i) Min(lfnWidth,lfnHeight)>=4
[0255] The first condition concerns the size of the block. If the width and height of the block are each 4 pixels or more, the decoder parses the lfnst_idx[x0][y0] syntax element.
[0256] In more detail, the decoder checks the block size conditions for which the secondary transform can be applied. The variables SubWidthC and SubHeightC are set according to the color format and indicate the ratio of the width and height of the chroma component to the width and height of the luma component of the picture, respectively. For example, a 4:2:0 color format image has a structure including one chroma sample per four luma samples, so SubWidthC and SubHeightC are both set to 2. As another example, a 4:4:4 color format image has a structure including one chroma sample per luma sample, so SubWidthC and SubHeightC are both set to 1. The number of horizontal samples of the current block, lfnWidth, and the number of vertical samples, lfnHeight, are set based on SubWidthC and SubHeightC. If treeType is DUAL_TREE_CHROMA, the transform unit includes only chroma components, so the number of horizontal samples of the chroma transform block is equal to the value obtained by dividing tbwidth, which is the width of the luma transform block, by SubWidthC. Similarly, the number of samples in the vertical direction of a chroma transform block is equal to the height of the luma transform block, tbHeight, divided by SubHeightC. If treeType is SINGLE_TREE or DUAL_TREE_LUMA, the transform unit contains a luma component, so lfnWidth and lfnHeight are set to tbwidth and tbHeight, respectively. The minimum condition for a block to which a quadratic transform can be applied is 4x4, so if Min(lfnWidth,lfnHeight)>=4 is satisfied, lfnst_idx[x0][y0] is parsed.
[0257] ii) sps_lfnst_enabled_flag==1
[0258] The second condition is a flag value indicating whether the secondary transformation is activated or applicable. sps_lfnst_enabled_flag ) is set to 1, the decoder parses lfnst_idx[x0][y0].
[0259] In detail, the secondary transform is indicated by the higher level syntax RBSP. At least one of the SPS, PPS, VPS, tile group header, and slice header includes a flag having a 1-bit size indicating whether the secondary transform is activated and applicable. If sps_lfnst_enabled_flag is 1, it indicates that the lfnst_idx[x0][y0] syntax element is present in the transform unit syntax, and if sps_lfnst_enabled_flag is 0, it indicates that the lfnst_idx[x0][y0] syntax element is not present in the transform unit syntax.
[0260] iii)CuPredMode[x0][y0]==MODE_INTRA
[0261] The third condition is related to the prediction mode: the secondary transform is only applied to intra-predicted blocks, so if the current block is an intra-predicted block, the decoder parses lfnst_idx[x0][y0].
[0262] iv)IntraSubPartitionsSplitType==ISP_NO_SPLIT
[0263] The fourth condition relates to whether the ISP prediction method is applied or not. If ISP is not applied to the current block, the decoder parses the lfnst_idx[x0][y0] syntax element.
[0264] In more detail, as described with reference to FIG. 11, when the current CU is divided into a number of transform units smaller than the CU size, the secondary transform is not applied to the divided transform units. In this case, lfnst_idx[x0][y0], which is a syntax element related to the secondary transform, is not parsed and is set to 0. When the current CU is divided into a number of transform units whose CU size is smaller than the transform tree, the case where ISP prediction is applied to the current coding unit is included. The ISP prediction method is a prediction method for dividing the transform tree into a number of transform units smaller than the CU size according to a preset division method when intra prediction is applied to the current coding unit. An ISP prediction mode is indicated at the coding unit level, and the variable IntraSubPartitionsSplitType is set based on the ISP prediction mode. If IntraSubPartitionsSplitType is ISP_NO_SPLIT, it indicates that ISP is not applied to the current block. Due to the characteristics of intra prediction that generates a prediction sample at the transform unit level, when the transform tree is divided into a number of transform units, the prediction accuracy is higher than when it is not divided. Therefore, even if a secondary transform is not applied to the multiple divided transform units, the energy of the residual signal is likely to be efficiently compressed.
[0265] v)!intra_mip_flag[x0][y0]
[0266] The fifth condition relates to the intra prediction method. If matrix based intra prediction (MIP) is not applied to the current coding unit, the decoder parses the lfnst_idx[x0][y0] syntax element.
[0267] In detail, MIP is used as one method of intra prediction, and whether MIP is applied is indicated by intra_mip_flag[x0][y0] at the coding unit level. If intra_mip_flag[x0][y0] is 1, it indicates that MIP is applied to the prediction of the current coding unit, and prediction is performed by multiplying the reconstructed samples around the current block by a preset matrix. When MIP is applied, it shows a different residual signal property from general intra prediction that performs directional or non-directional prediction, so that secondary transformation may not be applied to the transform block when MIP is applied.
[0268] vi) lfnLastScanPos==0 The sixth condition relates to the last significant coefficient in the scan order of the transform block.
[0269] In detail, if the position information (scan index) of the last significant coefficient in the scan order of the transform block included in the current transform unit is smaller than a preset critical value, the gain in coding efficiency obtained by the secondary transform may be small. Therefore, in such a case, the encoder is likely not to apply secondary transform to the transform block (lfnst_idx[x0][y0] is 0), and therefore, it is considered that the overhead is large for the encoder to signal lfnst_idx[x0][y0]. Therefore, lfnst_idx[x0][y0] is parsed only if the position (scan index) of the last significant coefficient in the scan order of at least one transform block included in the transform unit is equal to or larger than a preset critical value.
[0270] In other words, as described above, the threshold is an integer equal to or greater than 0. For example, assuming that the threshold is 1, the position (scan index) of the last significant coefficient in the scan order being equal to or greater than the threshold means that the significant coefficient is present at a position other than the upper left corner (scan index 0, DC) of the block. In this case, the position of the last significant coefficient in the scan order of the transform block being equal to or greater than the threshold may be expressed as "lfnLastScanPos==".
[0271] vii) numZeroOutSigCoeff==0
[0272] The seventh condition relates to the effectiveness coefficient present at a particular location.
[0273] In more detail, if a secondary transform is applied to the current block, a significant coefficient cannot exist at a particular position on the scan position. That is, the numZeroOutSigCoeff variable indicates whether or not a non-zero transform coefficient exists at a particular position. For example, assume that only a maximum of 16 significant coefficients are allowed if a secondary transform is applied to the current block. A significant coefficient may exist in the index [0,7] range on the scan order for a transform block of 4×4 or 8×8 size (maximum of 8 non-zero transform coefficients allowed). Meanwhile, a significant coefficient may exist in the index [0,15] range on the scan order for a transform block of a size other than 4×4 or 8×8 (maximum of 16 non-zero transform coefficients allowed). Therefore, if the position (scan index) of the last significant coefficient on the scan order is outside the range where the significant coefficients can exist, the decoder can automatically recognize that a secondary transform is not applied to the current block. Therefore, if numZeroOutSigCoeff>0, this means that a secondary transform is not applied to the current block, and lfnst_idx[x0][y0] is set to 0 without parsing.
[0274] In other words, if numZeroOutSigCoeff is not 0, it means that a valid coefficient exists at the specific position, so lfnst_idx[x0][y0] is not parsed and is set to 0. On the other hand, if numZeroOutSigCoeff is 0, it means that no valid coefficient exists at the specific position, so lfnst_idx[x0][y0] is parsed.
[0275] If all of the above conditions i) to vii) are true, lfnst_idx[x0][y0] is parsed, otherwise lfnst_idx[x0][y0] is not parsed and is set to 0.
[0276] FIG. 27 is a diagram showing a syntax structure regarding the position of the last significant coefficient in the scan order according to an embodiment of the present invention.
[0277] Looking at Figure 27, the last_significant_pos syntax structure is The last_significant_pos syntax structure includes position information of the last significant coefficient in the scan order for each color component Y, Cb, and Cr transform block. The last_significant_pos syntax structure receives as input (x0, y0) which is the left-top coordinate of the transform block, log2TbWidth which is the logarithm of the width of the transform block to the base 2, log2TbHeight which is the logarithm of the height of the transform block to the base 2, and cIdx which indicates which color component the transform block is. If cIdx is 0, it indicates a luma transform block, if cIdx is 1, it indicates a chroma Cb transform block, and if cIdx is 2, it indicates a chroma Cr transform block.
[0278] In the last_significant_pos syntax structure, a syntax element related to the position information of the last significant coefficient in the scan order is parsed. More specifically, a syntax element related to the x-coordinate value and the y-coordinate value of the last significant coefficient in the scan order is parsed. At this time, each coordinate value is divided into prefix information and suffix information and specified. The decoder sets a LastSignificantCoeffX variable, which is the x-coordinate of the last significant coefficient in the scan order, based on the prefix information and suffix information for the x-coordinate. Similarly, the decoder sets a LastSignificantCoeffY variable, which is the y-coordinate of the last significant coefficient in the scan order, based on the prefix information and suffix information for the y-coordinate. As shown in FIG. 27, the decoder sets lastScanPos, which is the scan index of the last significant coefficient in the scan order, based on LastSignificantCoeffX, LastSignificantCoeffY, and DiagScanOrder in the do{}while() structure. In addition, the decoder updates numZeroOutSigCoeff and lfnstLastScanPos, which are variables used in the parsing condition of lfnst_idx[x0][y0], which is a syntax element related to the secondary transform, based on lastScanPos.
[0279] If a secondary transform is applied to the current block, a significant coefficient cannot exist at a particular position on the scan position. The numZeroOutSigCoeff variable indicates whether a non-zero transform coefficient exists at such a position. For example, assume that only a maximum of 16 significant coefficients are allowed if a secondary transform is applied to the current block. For a transform block of size 4×4 or 8×8, a significant coefficient can exist in the index [0,7] range on the scan order (maximum of 8 non-zero transform coefficients are allowed). On the other hand, for a transform block having a size other than 4×4 or 8×8, a significant coefficient can exist in the index [0,15] range on the scan order (maximum of 16 non-zero transform coefficients are allowed). Therefore, if the position of the last significant coefficient on the scan order (scan index) exists outside the range where the significant coefficients can exist, the decoder can automatically recognize that a secondary transform is not applied to the current block. The minimum size of a block to which a secondary transform can be applied is 4×4, and if a transform skip is applied (transform_skip_flag[x0][y0]==1), the secondary transform is not applied. Therefore, numZeroOutSigCoeff is updated for transform blocks whose width is 4 or more (log2TbWidth>=2), whose height is 4 or more (log2TbHeight>=2), and whose transform skip is not applied (transform_skip_flag[x0][y0]==0). If a quadratic transform is applied, for transform blocks of size 4x4 or 8x8, non-zero transform coefficients can exist only in the scan order index [0,7] range. Therefore, if the transform block is 4x4 or 8x8, ((log2TbWidth==2||log2TbHeight==3)&&(log2TbWidth==log2TbHeight)), and lastScanPos is greater than 7 (lastScanPos>7), numZeroOutSigCoeff is increased by 1. For transform blocks other than those of 4x4 and 8x8 size, where quadratic transforms can be applied, non-zero transform coefficients can exist only in the range of scan order indices [0,15]. Therefore, if lastScanPos is greater than 15 (lastScanPos>15), numZeroOutSigCoeff is increased by 1.
[0280] The decoder determines lfnstLastScanPos based on lastScanPos. More specifically, if the width and height of the transform block are 4 or more and transform skip is not applied to the transform block, lfnstLastScanPos is set as shown in the following Formula 6. In other words, if log2TbWidth>=2, log2TbHeight>=2, and transform_skip_flag[x0][y0] is 0, lfnstLastScanPos is set as shown in the following Formula 1. In this case, if transform_skip_flag[x0][y0] is 0, it means that transform skip is not applied to the current transform block.
[0281]
number
[0282] As mentioned above, the initialization value of lfnstLastScanPos is set to 1.
[0283] In Equation 6, cIdx represents a variable that indicates the color components of the current transform block as described above.
[0284] According to formula 6, if the lfnstLastScanPos of a line is 1 and lastScanPos is less than lfnstLastScanPosTh[cIdx], lfnstLastScanPos is updated to 1. On the other hand, if the lfnstLastScanPos of a line is 0 or lastScanPos is greater than or equal to lfnstLastScanPosTh[cIdx], lfnstLastScanPos is updated to 0.
[0285] In other words, if the lastScanPos of all transform blocks included in the transform unit is smaller than the critical value or the number of all transform blocks is 0, lfnstLastScanPos is determined to be 1, and lfnst_idx[x0][y0] is not parsed and is set to 0 according to the lfnst_idx[x0][y0] parsing condition in FIG. 26. This indicates that secondary transform is not applied to the current block. On the other hand, if the LastScanPos of any one of the transform blocks included in the transform unit is equal to or greater than the critical value, lfnstLastScanPos is determined to be 0, and if all of the conditions i), ii), iii), iv), v), and vii) described in FIG. 26 are satisfied (true), the decoder parses lfnst_idx[x0][y0]. The decoder parses lfnst_idx[x0][y0] to determine whether a secondary transform is applied to the current block, and if a secondary transform is applied to the current block, it determines / determines the transform kernel to be used for the secondary transform.
[0286] In Equation 6, lfnstLastScanPosTh[cIdx] is a preset integer value equal to or greater than 0, and both the encoder and the decoder use the same value. Also, all color components may use the same threshold value. In this case, lfnstLastScanPos is set as in Equation 7.
[0287]
number
[0288] lfnstLastScanPosTh is a preset integer value equal to or greater than 0, and both the encoder and decoder use the same value. For example, lfnstLastScanPosTh may be 1. That is, if lastScanPos is equal to or greater than 1, lfnstLastScanPos is updated to 0, and lfnst_idx[x0][y0] is parsed. In this case, since the threshold value (lfnstLastScanPosTh) is an integer value, if lastScanPos is equal to or greater than 1, this has the same meaning as if lastScanPos is greater than 0. Although the case where all color components have the same threshold value of 1 has been described in FIG. 27, the present invention is not limited thereto.
[0289] FIG. 28 is a diagram illustrating a residual_coding syntax structure according to an embodiment of the present invention.
[0290] Referring to FIG. 28, position information of the last significant coefficient in the scan order is indicated in the eye of coefficient coding (residual_coding). Therefore, the syntax structure of coefficient coding (residual_coding) may not include a syntax structure related to position information of the last significant coefficient in the scan order. For example, the position information of the last significant coefficient in the scan order is a prefix, a suffix for the x-coordinate, and a prefix, a suffix for the y-coordinate of the last significant coefficient in the scan order. Looking at the coefficient coding (residual_coding) syntax structure of FIG. 28, coefficient coding (residual_coding) is performed based on LastSignificantCoeffX and LastSignificantCoeffY, which are the x-coordinate and y-coordinate of the last significant coefficient in the scan order determined before coefficient coding (residual_coding).
[0291] The method of indicating a secondary transformation according to the fourth embodiment does not use a numSigCoeff counter. Therefore, even if the coefficient at the (xC, yC) position is a valid coefficient (sig_coeff_flag[xC][yC]==1), numSigCoeff is not updated. In other words, the method of indicating a secondary transformation according to the fourth embodiment does not use a counter for valid coefficients. Also, according to the method of indicating a secondary transformation according to the fourth embodiment, since the numZeroOutSigCoeff variable is set based on lastScanPos, a counter based on sig_coeff_flag does not need to be used in coefficient coding (residual_coding).
[0292] FIG. 29 is a flow chart illustrating a method of processing a video signal according to an embodiment of the present invention.
[0293] In the following, a video signal processing method and apparatus based on the embodiments described with reference to FIGS. 15 to 28 will be described.
[0294] The video signal decoding device includes a processor that performs the video signal processing method described in FIG.
[0295] First, the processor receives a bitstream including syntax elements related to a secondary transformation of a coding unit.
[0296] The processor checks whether one or more preset conditions are satisfied, and if the one or more preset conditions are satisfied, parses the syntax elements related to the secondary transform of the coding unit (S2910, S2920). On the other hand, if the one or more preset conditions are not satisfied, the processor does not parse the syntax elements related to the secondary transform of the coding unit (S2930). In this case, the value of the syntax element related to the secondary transform is set to 0.
[0297] The syntax element related to the secondary transformation of the coding unit described in Figure 29 is lfnst_idx[x0][y0], which is a syntax element indicating whether or not the secondary transformation of the transform block included in the current coding unit described in Figures 5 to 28 is applied.
[0298] The processor parses syntax elements related to a secondary transformation of the coding unit via step S2920, and determines whether the secondary transformation is applied to a transform block included in the coding unit based on the parsed syntax elements (S2940).
[0299] In this case, if the secondary transform has been applied to the transform block, the processor performs a secondary inverse transform based on one or more coefficients of a first sub-block, which is one of one or more sub-blocks constituting the transform block, and identifies one or more inverse transform coefficients for the first sub-block (S2950).
[0300] The processor then performs a first inverse transform based on the one or more inverse transform coefficients obtained in step S2950, and identifies residual samples for the transform block at step S2960.
[0301] The secondary transform is a low-band non-separable transform (LFNST), and the transform block is a block to which a primary transform is applied, which is separated into a vertical transform and a horizontal transform. In this case, the primary inverse transform is an inverse transform of the primary transform, and the secondary inverse transform is an inverse transform of the secondary transform.
[0302] The syntax element related to a secondary transform of the coding unit includes information indicating whether the secondary transform is applied to the coding unit and information indicating a transform kernel used for the secondary transform.
[0303] The first sub-block is the first sub-block according to a preset scan order. In this case, the index of the first sub-block is 0.
[0304] A first condition of the one or more preset conditions is that an index value indicating a position of a first coefficient among the one or more coefficients of the first sub-block is greater than a preset threshold value. In this case, the first coefficient is the last significant coefficient according to a preset scan order, and the significant coefficient means a coefficient that is not 0. The preset threshold value is 0. The preset scan order is the upper right diagonal scan order described with reference to FIGS. 13 and 14.
[0305] A second condition of the one or more preset conditions is that the width and height of the transformation block are equal to or greater than 4 pixels.
[0306] A third condition among the one or more preset conditions is when a transform skip flag value included in the bitstream is not a specific value, in which case, if the transform skip flag value has a specific value, the transform skip flag indicates that the primary transform and the secondary transform are not applied to the transform block.
[0307] A fourth condition among the one or more preset conditions is that at least one of the one or more coefficients of the first sub-block is not 0 and the at least one coefficient is present at a position other than the first position according to a preset scan order, where the first position according to the preset scan order means a position whose horizontal and vertical coordinate values are (0,0) as described above, or means a first position according to a preset scan order (e.g., upper right diagonal order).
[0308] The coding unit is composed of a plurality of coding blocks, and if at least one of the transformation blocks corresponding to each of the plurality of coding blocks satisfies one or more of the preset conditions, syntax elements related to the secondary transformation are parsed.
[0309] On the other hand, if a syntax element relating to a secondary transform is not parsed or is set to 0 (S2930), or if it is determined in step S2940 that the secondary transform is not applied to the transform block included in the coding unit, the processor performs a primary inverse transform based on one or more coefficients of the transform block to obtain a residual sample for the transform block (S2970).
[0310] In this case, the above-mentioned primary inverse transform and secondary inverse transform are inverse transforms of the primary transform and secondary transform, respectively.
[0311] The video signal processing method performed in the video signal decoding apparatus described with reference to FIG. 29, or a method similar thereto, is performed in the video signal encoding apparatus.
[0312] The video signal encoding device includes a processor for encoding the video signal.
[0313] In this case, the processor performs a primary transform on residual samples of a block included in a coding unit to obtain a plurality of primary transform coefficients for the block, performs a secondary transform based on one or more of the plurality of primary transform coefficients to obtain one or more secondary transform coefficients for a first sub-block, which is one of the sub-blocks constituting the block, and encodes information on the one or more secondary transform coefficients and syntax elements related to the secondary transform of the coding unit to obtain a bitstream.
[0314] The secondary transform may be a low-band non-separable transform (LFNST), and the primary transform may be separated into a horizontal transform and a vertical transform.
[0315] Also, the syntax element related to the secondary transform is coded if it satisfies one or more predefined conditions. The syntax element related to the secondary transform includes information indicating whether the secondary transform is applied to the coding unit and information indicating a transform kernel used for the secondary transform. In this case, the syntax element related to the secondary transform is lfnst_idx[x0][y0], which is the syntax element described in Figures 15 to 28.
[0316] The first sub-block is the first sub-block in a preset scan order, and has an index of 0.
[0317] A first condition among the one or more preset conditions is that an index value indicating a position of a first coefficient among the one or more secondary transform coefficients is greater than a preset critical value. In this case, the first coefficient is the last significant coefficient according to a preset scan order, and the significant coefficient means a coefficient that is not 0. The preset critical value is 0. The preset scan order is the upper right diagonal scan order described with reference to FIGS. 13 and 14.
[0318] A second condition among the one or more preset conditions is that the width and height of the primary transformation block are equal to or greater than 4 pixels.
[0319] A third condition among the one or more preset conditions is when a transform skip flag value included in the bitstream is not a specific value, in which case, if the transform skip flag value has a specific value, the transform skip flag indicates that the first transform and the second transform are not applied to the block.
[0320] A fourth condition among the one or more preset conditions is that at least one of the one or more secondary transform coefficients is not 0, and the one or more coefficients are present at a position other than the first position according to a preset scan order. In this case, the first position according to the preset scan order means a position whose horizontal and vertical coordinate values are (0,0) as described above, or means the first position according to a preset scan order (e.g., upper right diagonal order).
[0321] The coding unit is composed of a plurality of coding blocks, and if at least one of the (transformed) blocks included in the coding unit corresponding to each of the plurality of coding blocks satisfies one or more of the preset conditions, the syntax element related to the secondary transform is coded.
[0322] The video signal encoding device may also include a video signal decoding processor that performs the video signal processing method described in FIG.
[0323] As described above, the bitstream includes syntax elements related to the secondary transformation of the coding units described in Figures 15 to 29. In this case, the bitstream is stored in a non-transitory computer readable medium. Meanwhile, if one or more of the above-mentioned preset conditions are not satisfied, the video signal encoding device does not include the syntax elements related to the secondary transformation in the bitstream or sets the syntax elements related to the secondary transformation to 0. The bitstream is decoded by the video signal decoding device described in Figure 29 or encoded by the above-mentioned video signal encoding device.
[0324] A method of encoding such a bitstream includes, for example, a process of performing a primary transform on residual samples of a block included in a coding unit to obtain a plurality of primary transform coefficients for the block, performing a secondary transform based on one or more of the plurality of primary transform coefficients to obtain one or more secondary transform coefficients for a first sub-block, which is one of the sub-blocks constituting the block, and encoding information on the one or more secondary transform coefficients and syntax elements related to the secondary transform of the coding unit.
[0325] Obtaining a coefficient as described herein means obtaining a pixel / block for the coefficient, and obtaining a residual sample means obtaining a residual signal / pixel / block for the residual sample.
[0326] The above-described embodiments of the present invention may be implemented in a variety of ways, for example, in hardware, firmware, software, or a combination thereof.
[0327] In the case of a hardware implementation, the methods according to the embodiments of the present invention may be implemented using one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSDPs), Programmable Logic Devices (PDLs), Field Programmable Gate Arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, etc.
[0328] In the case of implementation by firmware or software, the method according to the embodiment of the present invention may be implemented in the form of modules, procedures, or functions performing the functions or operations described above. The software code is stored in a memory and implemented by a processor. The memory may be located inside or outside the processor and may exchange data with the processor by various means known in the art.
[0329] Some embodiments may be embodied in the form of a recording medium including computer executable instructions such as program modules executed by a computer. Computer readable media is any available medium that can be accessed by a computer, including both volatile and non-volatile media, and both separate and non-separate media. Computer readable media also includes both storage media and communication media. Computer storage media includes both volatile and non-volatile media, and both separate and non-separate media embodied in any method or technology for storage of information such as computer readable instructions, data structures, program modules, or other data. Communication media typically includes computer readable instructions, data structures, or other data in a modulated data signal such as a program module, or other transmission mechanism, and includes any information delivery media.
[0330] The above description of the present invention is for illustrative purposes only, and those skilled in the art will understand that the present invention can be easily modified into other specific forms without changing the technical idea or essential features of the present invention. Therefore, it should be understood that the above-described embodiments are illustrative in all respects and are not limiting. For example, each component described as a single type may be implemented in a distributed form, and each component described as a distributed form may be implemented in a combined form.
[0331] The scope of the present invention is indicated by the claims set forth below rather than by the above detailed description, and all modifications and variations derived from the meaning and scope of the claims and their equivalent concepts should be interpreted as being included in the scope of the present invention. [Explanation of symbols]
[0332] 110 Conversion unit 115 Quantization section 120 Inverse quantization section 125 Reverse conversion unit 130 Filtering section 150 Prediction Department 152 Intra prediction unit 154 Inter Prediction Unit 154a Motion Estimation Unit 154b Motion compensation section 160 Entropy coding unit 210 Entropy Decoding Unit 220 Inverse quantization section 225 Inverse conversion unit 230 Filtering section 250 Prediction Department 252 Intra prediction unit 254 Inter Prediction Unit
Claims
1. 1. A video decoding method performed by an apparatus, comprising: obtaining syntax information for a secondary transformation of the coding unit; determining whether the secondary transform is applied to a transform block included in the coding unit based on the acquired syntax information; if the secondary transform is applied to the transform block, obtaining one or more inverse transform coefficients based on the secondary transform; and obtaining a residual sample for the transform block based on the one or more inverse transform coefficients, The secondary transform is a low frequency non-separable transform (LFNST), the syntax information indicates whether the secondary transform is applied to the coding unit and which transform kernel is used for the secondary transform; The syntax information is obtained if one or more conditions are met, Among the one or more conditions, a condition is that an index of a last significant coefficient in a sub-block according to a preset scan order is greater than a preset value; The sub-block is a sub-block having a sub-block index 0 according to a preset scan order in the transform block.
2. The video decoding method of claim 1 , wherein the default value is zero.
3. The video decoding method of claim 1 , wherein the preset scan order is an up-right diagonal scan order.
4. indices of the one or more inverse transform coefficients are determined based on the preset scan order; an index of a first coefficient among the one or more inverse transform coefficients is 0; 4. The video decoding method of claim 3, wherein the last significant coefficient is a non-zero coefficient.
5. The method of claim 1 , wherein the residual samples are obtained by performing an inverse transform of a primary transform based on the one or more inverse transform coefficients.
6. In a video encoding device, the video encoding device comprises: at least one processor; The at least one processor configured to generate a bitstream using the encoding method; The encoding method comprises: encoding syntax information related to a secondary transformation of the coding unit; obtaining one or more first-order transform coefficients based on residual samples for a transform block included in the coding unit; if the secondary transform is applied to the transform block according to the syntax information, obtaining one or more secondary transform coefficients using the one or more primary transform coefficients based on the secondary transform; and encoding the one or more secondary transform coefficients, The secondary transform is a low frequency non-separable transform (LFNST), the syntax information indicates whether the secondary transform is applied to the coding unit and which transform kernel is used for the secondary transform; the syntax information is encoded if one or more conditions are met; Among the one or more conditions, a condition is that an index of a last significant coefficient in a sub-block according to a preset scan order is greater than a preset value; The sub-block is a sub-block having sub-block index 0 according to a preset scan order in the transform block.
7. The video encoding apparatus of claim 6 , wherein the default value is zero.
8. The video encoding apparatus of claim 6 , wherein the preset scan order is an up-right diagonal scan order.
9. indices of the one or more primary transform coefficients are determined based on the preset scan order; An index of a first coefficient among the one or more primary transform coefficients is 0; 9. The video encoding apparatus of claim 8, wherein the last significant coefficient is a non-zero coefficient.
10. The video encoding device of claim 6, wherein the one or more primary transform coefficients are obtained by performing a primary transform based on the residual samples.
11. A method for storing a bitstream, comprising: The bitstream is generated by an encoding method, The encoding method comprises: encoding syntax information related to a secondary transformation of the coding unit; obtaining one or more first-order transform coefficients based on residual samples for a transform block included in the coding unit; if the secondary transform is applied to the transform block according to the syntax information, obtaining one or more secondary transform coefficients using the one or more primary transform coefficients based on the secondary transform; and encoding the one or more secondary transform coefficients, The secondary transform is a low frequency non-separable transform (LFNST), the syntax information indicates whether the secondary transform is applied to the coding unit and which transform kernel is used for the secondary transform; the syntax information is encoded if one or more conditions are met; Among the one or more conditions, a condition is that an index of a last significant coefficient in a sub-block according to a preset scan order is greater than a preset value; The sub-block is a sub-block having a sub-block index 0 according to a preset scan order in the transform block.
Citation Information
Patent Citations
Binarizing secondary transform index
US20170324643A1
Method and device for coding residual signal in video coding system
US20180288409A1
Transform method in image coding system and apparatus for same
US20200177889A1
