Method and apparatus for video encoding in which transform kernel and inverse transform kernel are selected according to prediction mode

Abstract

The present embodiment discloses a method and apparatus for video encoding in which a transform kernel and an inverse transform kernel are selected according to a prediction mode, in the present embodiment, a video decoding apparatus acquires an inverse quantized transform coefficient from a transform block of a current block and decodes non-separable transform information; here, the inseparable transform information indicates whether an inseparable inverse transform is applied, the inseparable inverse transform representing an inseparable primary inverse transform or an inseparable secondary inverse transform; when performing an inseparable inverse transform according to the inseparable transform information, the video decoding device derives an intra prediction mode of the transform block according to the prediction mode of the current block; a video decoding device determines an inseparable inverse transform kernel corresponding to an inseparable primary inverse transform kernel or an inseparable secondary inverse transform kernel according to the size of a transform block and an intra prediction mode.

Description

TECHNICAL FIELD

[0001] The disclosure relates to a video encoding method and apparatus selecting a transform kernel and an inverse transform kernel according to a prediction mode. BACKGROUND

[0002] The following description is provided only for complete understanding of the invention, and does not constitute prior art.

[0003] Since video data has more data amount than voice data or still image data, etc., a large amount of hardware resources including a memory are required to directly store or transmit the video data without compression processing.

[0004] Therefore, when storing or transmitting video data, it is common to compress the video data using an encoder and store or transmit the compressed video data, and a decoder receives the compressed video data, decompresses and plays it. Such a video compression technique includes H.264 / AVC, High Efficiency Video Coding (HEVC), and Versatile Video Coding (VVC), etc., and VVC improves coding efficiency by more than about 30% compared to HEVC.

[0005] However, as the size, resolution, and frame rate of video gradually increase, the amount of data to be encoded also increases, and thus a new compression technique having higher coding efficiency than the existing compression technique and having more excellent quality improvement effect is required.

[0006] Low-Frequency Non-Separable Transform (LFNST) technology applies a secondary transform to a low-frequency region among transform coefficients generated according to a primary transform of a Transform Unit (TU). At the encoding level, the LFNST technology applies a secondary transform to L low-frequency primary transform coefficients among W×H primary transform coefficients to generate K (here, K≤L) secondary transform coefficients. Here, the size of the transform kernel of the LFNST is L×K. On the other hand, the type of the transform kernel can be determined according to an intra prediction mode of the current TU, the size of the TU, and an LFNST index (lfnst_idx). At this time, as the intra prediction mode, a directional prediction mode, a DC mode, and a Planar mode are used.

[0007] Non-separable primary transform (NSPT) technology is a technology in which, instead of performing vertical and horizontal direction transforms on a residual signal of a block predicted by an intra prediction mode, a non-separable primary transform is performed using a pre-learned transform kernel. The NSPT technology uses non-separable primary transform kernels of different sizes according to the size of a residual block. For example, when the size of a transform block is 4x4, a 16x16 kernel is applied to generate transform coefficients of 4x4 size. The generated transform coefficients are arranged in the transform block in a scan order agreed in advance. When the number of transform coefficients generated after the transform is less than the number of pixels of the transform block, the remaining area in which the transform coefficients are not arranged is padded with 0.

[0008] For the non-separable primary transform, the transform kernel can be defined according to the shape of a block and an intra prediction mode. For example, using the symmetry property of a square block, for a square block, there are 35 transform kernel sets according to the intra prediction mode. The non-separable primary transform can be applied to a luma component. In addition, when the non-separable primary transform is applied, a secondary transform (i.e., LFNST) can not be performed.

[0009] As described above, the LFNST technology and the NSPT technology use an intra prediction mode in selecting a transform kernel. Therefore, in order to improve video encoding efficiency and improve video quality, there is a need for a comprehensive use scheme of selecting a transform kernel according to a prediction mode. SUMMARY

[0010] Technical Problem The present disclosure aims to provide a video encoding method and apparatus capable of effectively selecting a transform kernel and an inverse transform kernel according to a prediction mode of a current block to perform a transform and an inverse transform of the current block.

[0011] Means for Solving Technical Problem According to an embodiment of the present disclosure, a method of reconstructing a current block performed by a video decoding apparatus is provided, including the steps of: obtaining inverse-quantized transform coefficients of a transform block of the current block; decoding non-separable transform information from a bitstream, wherein the non-separable transform information indicates whether a non-separable inverse transform is applied, and the non-separable inverse transform represents a non-separable primary inverse transform or a non-separable secondary inverse transform; and determining whether the non-separable transform information indicates that the non-separable inverse transform is performed, and when the non-separable transform information indicates that the non-separable inverse transform is performed, including the steps of: deriving an intra prediction mode of the transform block according to a prediction mode of the current block; and determining a non-separable inverse transform kernel corresponding to a non-separable primary inverse transform kernel or a non-separable secondary inverse transform kernel according to the size of the transform block and the intra prediction mode.

[0012] According to another embodiment of the disclosure, there is provided a method of encoding a current block performed by a video encoding apparatus, the method comprising: obtaining a residual signal for a transform block of the current block; deriving an intra prediction mode of the transform block according to a prediction mode of the current block; determining a non-separable transform kernel corresponding to a non-separable primary transform kernel or a non-separable secondary transform kernel according to a size of the transform block and the intra prediction mode; applying the non-separable primary transform to the residual signal to generate first primary transform coefficients; determining a pair of primary transform kernels in a vertical and a horizontal direction for the transform block; applying the pair of primary transform kernels to the residual signal to generate second primary transform coefficients; and applying the non-separable secondary transform kernel to the second primary transform coefficients to generate secondary transform coefficients.

[0013] According to another embodiment of the disclosure, there is provided a computer-readable recording medium storing a bitstream generated by a video encoding method, the video encoding method comprising: a step of obtaining a residual signal for a transform block of a current block; a step of deriving an intra prediction mode of the transform block according to a prediction mode of the current block; a step of determining a non-separable transform kernel corresponding to a non-separable primary transform kernel or a non-separable secondary inverse transform kernel according to a size of the transform block and the intra prediction mode; a step of applying the non-separable primary transform kernel to the residual signal to generate first primary transform coefficients; a step of explicitly or implicitly determining a pair of primary transform kernels in a vertical and a horizontal direction for the transform block; a step of applying the pair of primary transform kernels to the residual signal to generate second primary transform coefficients; and a step of applying the non-separable secondary transform kernel to the second primary transform coefficients to generate secondary transform coefficients.

[0014] Effects of Invention As described above, according to the present embodiment, by providing a video encoding method and apparatus that effectively selects transform and inverse transform kernels according to a prediction mode of a current block to perform transform and inverse transform of the current block, it is possible to achieve the effects of improving video encoding efficiency and improving video quality. BRIEF DESCRIPTION OF DRAWINGS

[0015] FIG. 1 is an exemplary block diagram of a video encoding apparatus that can be used to implement the techniques of the disclosure.

[0016] Figure 2 is a diagram for explaining a method of partitioning a block using a QuadTree plus BinaryTree TernaryTree (QTBTTT) structure.

[0017] Figure 3a and Figure 3bis a diagram showing a plurality of intra prediction modes including a wide angle intra prediction mode.

[0018] Figure 4 is a diagram showing the surrounding blocks of a current block.

[0019] Figure 5 is an exemplary block diagram of a video decoding apparatus capable of implementing the disclosed technology.

[0020] FIGS. 6a and Figure 6b is a diagram showing a matrix-weighted intra prediction (MIP) technique.

[0021] Figure 7 is a detailed block diagram of a part of a video decoding apparatus based on an embodiment of the disclosure.

[0022] Figure 8 is a diagram showing a method of determining an inverse transform kernel based on an embodiment of the disclosure.

[0023] Figure 9 is a diagram showing a method of determining a prediction mode of a current TU based on an embodiment of the disclosure.

[0024] Figure 10 is a diagram showing a method of determining a prediction mode of a current TU based on another embodiment of the disclosure.

[0025] Figure 11 is a diagram showing a directional filter based on another embodiment of the disclosure.

[0026] Figure 12 is a flowchart showing a method of performing a transform on a transform block by a video encoding apparatus based on an embodiment of the disclosure.

[0027] Figure 13 is a flowchart showing a method of performing an inverse transform on a transform block by a video decoding apparatus based on an embodiment of the disclosure. DETAILED DESCRIPTION

[0028] Embodiments of the disclosure will be described in detail by way of example with reference to the attached drawings. Note that, when assigning reference numerals to the components in the drawings, the same components are denoted by the same reference numerals, even if they are shown in different drawings. Furthermore, for the purpose of facilitating the understanding of the embodiments of the disclosure, detailed description of the related known functions or constitutions will be omitted when such an omission does not cause any confusion.

[0029] FIG. 1 is an exemplary block diagram of a video encoding apparatus that can be used to implement the disclosed technology. Hereinafter, the video encoding apparatus and its subcomponents will be described with reference to the diagram of FIG. 1.

[0030] The video encoding apparatus can include a picture partitioning section 110, a prediction section 120, a subtracter 130, a transform section 140, a quantization section 145, a reordering section 150, an entropy encoding section 155, an inverse quantization section 160, an inverse transform section 165, an adder 170, a loop filter section 180, and a memory 190.

[0031] The components of the video encoding apparatus can be implemented by hardware or software, or a combination of hardware and software. In addition, the functions of the components can be implemented by software and executed by a microprocessor.

[0032] One video (video) is composed of one or more sequences including a plurality of pictures. Each picture is partitioned into a plurality of regions, and encoding is performed for each region. For example, one picture is partitioned into one or more tiles and / or slices. Here, one or more tiles can be defined as a tile group. Each tile or slice is partitioned into one or more coding tree units (CTUs). And each CTU is partitioned into one or more coding units (CUs) by tree structure. Information applied to each CU is encoded as the syntax of the CU, while information commonly applied to a plurality of CUs included in one CTU is encoded as the syntax of the CTU. In addition, information commonly applied to all blocks within one slice is encoded as the syntax of the slice header, while information applied to all blocks constituting one or more pictures is encoded in a picture parameter set (PPS) or a picture header. Also, information commonly referred to by a plurality of pictures is encoded in a sequence parameter set (SPS). And information commonly referred to by one or more SPSs is encoded in a video parameter set (VPS). In addition, information commonly applied to one tile or tile group can also be encoded as the syntax of the tile or tile group header. The syntax included in the SPS, PPS, slice header, tile or tile group header can be referred to as high-level syntax.

[0033] The picture partitioning section 110 determines the size of the CTU. Information on the size of the CTU (CTU size) is encoded as the syntax of the SPS or PPS and transmitted to the video decoding apparatus.

[0034] The picture division unit 110 divides each picture (picture) constituting an image into a plurality of CTUs having a predetermined size, and then divides the CTUs recursively using a tree structure. A leaf node in the tree structure becomes a basic unit of encoding, that is, a CU.

[0035] As the tree structure, there can be a QuadTree (QT) in which a parent node is divided into four child nodes of the same size, a BinaryTree (BT) in which a parent node is divided into two child nodes, a TernaryTree (TT) in which a parent node is divided into three child nodes in a ratio of 1:2:1, or a structure in which two or more of these QT structure, BT structure, and TT structure are mixed. For example, a QuadTree plus BinaryTree (QTBT) structure or a QuadTree plus BinaryTree TernaryTree (QTBTTT) structure can be used. Here, the BTTT can be collectively referred to as a Multiple-Type Tree (MTT).

[0036] Figure 2 is a diagram for explaining a method of dividing a block using the QTBTTT structure.

[0037] As shown in Figure 2 , a CTU can be first divided into a QT structure. The QuadTree division can be repeated until the size of a splitting block reaches the minimum block size of a leaf node allowed in the QT (MinQTSize). A first flag (QT_split_flag) indicating whether each node of the QT structure is divided into four nodes of a lower layer is encoded by the entropy encoding unit 155 and signaled to the video decoding apparatus. When the leaf node of the QT is not larger than the maximum block size of a root node allowed in the BT (MaxBTSize), the division can be further performed by either of the BT structure or the TT structure. In the BT structure and / or the TT structure, there can be a plurality of division directions. For example, there can be two directions in which a block of a corresponding node is divided horizontally and vertically. As shown in Figure 2 , when MTT division is started, a second flag (mtt_split_flag) indicating whether a node is divided, and a flag further indicating a division direction (vertical or horizontal) and / or a flag indicating a division type (binary or ternary) when divided are encoded by the entropy encoding unit 155 and signaled to the video decoding apparatus.

[0038] Alternatively, a CU split flag (split_cu_flag) indicating whether the node is split or not can be encoded before a first flag (QT_split_flag) indicating whether each node is split into four nodes of a lower layer. When the CU split flag (split_cu_flag) value indicates that the node is not split, the block of the corresponding node becomes a leaf node in the split tree structure and becomes a CU (coding unit) that is a basic unit of encoding. When the CU split flag (split_cu_flag) value indicates that the node is split, the video encoding apparatus encodes from the first flag in the above-described manner.

[0039] As another example of the tree structure, when QTBT is used, there can be two types of splitting the block of the corresponding node into two blocks of the same size in the horizontal direction (i.e., symmetric horizontal splitting) and in the vertical direction (i.e., symmetric vertical splitting). A split flag (split_flag) indicating whether each node of the BT structure is split into blocks of a lower layer and split type information indicating the split type are encoded by the entropy encoding section 155 and transmitted to the video decoding apparatus. On the other hand, there can also be a type of splitting the block of the corresponding node into two blocks in an asymmetric form. The asymmetric form can include a form of splitting the block of the corresponding node into two rectangular blocks having a size ratio of 1:3, or can include a form of splitting the block of the corresponding node in a diagonal direction.

[0040] According to the QTBT or QTBTTT splitting of the CTU, the CU can have various sizes. Hereinafter, a block corresponding to a CU to be encoded or to be decoded (i.e., a leaf node of the QTBTTT) will be referred to as a "current block". When the QTBTTT splitting is adopted, the shape of the current block can be not only a square but also a rectangle.

[0041] The prediction section 120 predicts the current block to generate a prediction block. The prediction section 120 includes an intra prediction section 122 and an inter prediction section 124.

[0042] Generally, each current block within a picture can be separately prediction encoded. Generally, the prediction of the current block can be performed using an intra prediction technique (using data from the picture including the current block) or an inter prediction technique (using data from a picture that has been encoded before the picture including the current block). The inter prediction includes single prediction and bi-prediction.

[0043] The intra-frame prediction unit 122 uses pixels (reference pixels) located around the current block in the current image, including the current block, to predict pixels within the current block. Depending on the prediction direction, multiple intra-frame prediction modes exist. For example, such as... Figure 3a As shown, multiple intra-frame prediction modes can include two non-directional modes (Planar mode and DC mode) and 65 directional modes. The surrounding pixels to be used and the calculation formulas are defined differently for each prediction mode.

[0044] To perform effective directional prediction for the current rectangular block, additional methods can be used. Figure 3b The dashed arrows in the diagram illustrate several directional modes (67 to 80, -1 to -14 intra-prediction modes). These can be referred to as "wide angle intra-prediction modes". Figure 3b The middle arrow indicates the corresponding reference sample used in the prediction, not the prediction direction. The prediction direction is opposite to the direction indicated by the arrow. Wide-angle intra-frame prediction mode is a mode that performs prediction in the opposite direction of a specific directional mode without transmitting additional bits when the current block is rectangular. In this case, based on the ratio of the width to the height of the current rectangular block, a subset of wide-angle intra-frame prediction modes that can be used for the current block can be determined. For example, wide-angle intra-frame prediction modes with an angle less than 45 degrees (67 to 80 intra-frame prediction modes) are available when the current block is a rectangle with a height less than its width; while wide-angle intra-frame prediction modes with an angle greater than -135 degrees (-1 to -14 intra-frame prediction modes) are available when the current block is a rectangle with a width greater than its height.

[0045] The intra prediction unit 122 is capable of determining the intra prediction mode for encoding the current block. In some examples, the intra prediction unit 122 can use multiple intra prediction modes to encode the current block, and can also select a suitable intra prediction mode to use from the tested modes. For example, the intra prediction unit 122 uses rate-distortion analysis for multiple tested intra prediction modes to calculate rate-distortion values, and selects the intra prediction mode with the best rate-distortion characteristics from the multiple tested modes.

[0046] The intra-prediction unit 122 selects one intra-prediction mode from a plurality of intra-prediction modes and uses the surrounding pixels (reference pixels) determined according to the selected intra-prediction mode and the calculation formula to predict the current block. Information about the selected intra-prediction mode is encoded by the entropy coding unit 155 and transmitted to the video decoding device.

[0047] The inter-frame prediction unit 124 generates a prediction block for the current block using a motion compensation process. The inter-frame prediction unit 124 searches for the most similar block to the current block within reference images that have been encoded and decoded prior to the current image, and generates a prediction block for the current block using the searched block. Furthermore, it generates a motion vector (MV) corresponding to the displacement between the current block in the current image and the prediction block in the reference image. Typically, motion estimation is performed on the luma component, and the motion vector calculated based on the luma component is used for both the luma and chroma components. Information about the reference image used for predicting the current block, and motion information including information about the motion vector, are encoded by the entropy coding unit 155 and transmitted to the video decoding device.

[0048] To improve prediction accuracy, the inter-frame prediction unit 124 can also perform interpolation on a reference image or reference block. That is, subsamples between two consecutive integer samples are obtained by interpolation using filter coefficients applied to multiple consecutive integer samples, including those two integer samples. If a process of searching for the most similar block to the current block is performed on the interpolated reference image, the motion vector can be represented with precision down to decimal units, rather than integer sample units. The precision or resolution of the motion vector can be set differently depending on the target region to be encoded, such as a slice, tile, CTU, CU, etc. When applying the Adaptive Motion Vector Resolution (AMVR) as described above, information about the motion vector resolution applied to each target region should be sent via a signal for each target region. For example, when the target region is a CU, information about the motion vector resolution applied to each CU is sent via a signal. The information about the motion vector resolution can be information representing the precision of the differential motion vector, which will be described later.

[0049] On the other hand, the inter-frame prediction unit 124 can perform inter-frame prediction using bi-prediction. In the case of bi-prediction, two reference images and two motion vectors representing the positions of the most similar blocks to the current block within each reference image are used. The inter-frame prediction unit 124 selects a first reference image and a second reference image from reference image list 0 (RefPicList0) and reference image list 1 (RefPicList1), respectively, and searches for blocks similar to the current block within each reference image to generate a first reference block and a second reference block. Furthermore, the first reference block and the second reference block are averaged or weighted averaged to generate a predicted block for the current block. Information about the two reference images used for predicting the current block and motion information including information about the two motion vectors are transmitted to the entropy coding unit 155. Here, reference image list 0 can consist of images in the reconstructed images that are displayed before the current image, and reference image list 1 can consist of images in the reconstructed images that are displayed after the current image. However, this is not a strict limitation. Reconstructed images that are displayed after the current image can be additionally included in reference image list 0, and conversely, reconstructed images that are displayed before the current image can also be additionally included in reference image list 1.

[0050] Various methods can be used to minimize the number of bits required to encode motion information.

[0051] For example, if the reference image and motion vector of the current block are the same as those of surrounding blocks, the motion information of the current block can be transmitted to the video decoding device by encoding the information that can identify the surrounding blocks. This method is called "merge mode".

[0052] In the merge mode, the inter-frame prediction unit 124 selects a predetermined number of merge candidate blocks (hereinafter referred to as "merge candidates") from the surrounding blocks of the current block.

[0053] like Figure 4 As shown, the surrounding blocks used to derive merging candidates can be all or part of the left block (A0), lower left block (A1), upper block (B0), upper right block (B1), and upper left block (B2) adjacent to the current block in the current image. In addition to using the current image where the current block is located, blocks located in a reference image (which may be the same as or different from the reference image used for predicting the current block) can also be used as merging candidates. For example, blocks located in the same position as the current block (co-located block) or adjacent to that block in the reference image can be additionally used as merging candidates. When the number of merging candidates selected by the method described above is less than a preset number, a 0 vector is added to the merging candidates.

[0054] The inter-frame prediction unit 124 uses these peripheral blocks to construct a merge list including a predetermined number of merge candidates. From the multiple merge candidates included in the merge list, it selects a merge candidate to be used as motion information for the current block and generates merge index information to identify the selected candidate. The generated merge index information is encoded by the entropy coding unit 155 and transmitted to the video decoding device.

[0055] Merge skip mode is a special case of merging mode. After quantization, when the transform coefficients used for entropy coding are close to zero, only the peripheral block selection information is transmitted instead of the residual signal. Merge skip mode can achieve relatively high coding efficiency in images with little motion, still images, and screen content images.

[0056] Hereinafter, merge mode and merge skip mode will be collectively referred to as merge / skip mode.

[0057] Another method for encoding motion information is the AMVP (Advanced Motion Vector Prediction) mode.

[0058] In AMVP mode, the inter-frame prediction unit 124 uses multiple peripheral blocks of the current block to derive multiple predicted motion vector candidates for the motion vector of the current block. As the peripheral blocks used to derive the predicted motion vector candidates, multiple predicted motion vector candidates can be derived using... Figure 4 The diagram shows all or part of the left block (A0), lower left block (A1), upper block (B0), upper right block (B1), and upper left block (B2) adjacent to the current block within the current image. Alternatively, blocks located within a reference image (which may be the same as or different from the reference image used for predicting the current block) can be used as peripheral blocks for deriving motion vector candidates, instead of using the current image containing the current block. For example, a block located at the same position as the current block within the reference image (co-located block) or a block adjacent to that block at the same position can be used. Using the method described above, if the number of motion vector candidates is less than a preset number, a zero vector is added to the motion vector candidate list.

[0059] The inter-frame prediction unit 124 uses the motion vectors of these peripheral blocks to derive candidate motion vectors for prediction, and uses these candidate motion vectors to determine the predicted motion vector for the current block. Furthermore, it subtracts the predicted motion vector from the motion vector of the current block to calculate the differential motion vector.

[0060] The predicted motion vectors can be obtained by applying a predefined function (e.g., median, mean, etc.) to the predicted motion vector candidates. The video decoding device is also aware of this predefined function. Furthermore, the surrounding blocks used to derive the predicted motion vector candidates are already encoded and decoded blocks, so the video decoding device is also aware of the motion vectors of these surrounding blocks. Therefore, the video encoding device does not need to encode the information used to identify the predicted motion vector candidates. Thus, what is encoded at this time is information about the differential motion vectors and information about the reference image used to predict the current block.

[0061] On the other hand, the predicted motion vector can also be determined by selecting one from multiple predicted motion vector candidates. In this case, along with information about the differential motion vector and information about the reference image used to predict the current block, additional information for identifying the selected predicted motion vector candidate is encoded.

[0062] Subtractor 130 subtracts the prediction block generated by intra-frame prediction unit 122 or inter-frame prediction unit 124 from the current block to generate a residual block.

[0063] The transformation unit 140 transforms the residual signal within the residual block, which has pixel values ​​in the spatial domain, into transform coefficients in the frequency domain. The transformation unit 140 can use the entire size of the residual block as a transformation unit to transform the residual signal within the residual block, or it can divide the residual block into multiple sub-blocks and use these sub-blocks as transformation units. Alternatively, it can divide the block into two sub-blocks: a transformation region and a non-transformation region, and only use the transformation region sub-block as a transformation unit to transform the residual signal. Here, the transformation region sub-block can be one of two rectangular blocks with a 1:1 size ratio based on the horizontal (or vertical) axis. In this case, a flag indicating that only the sub-block is transformed (cu_sbt_flag), directional (vertical / horizontal) information (cu_sbt_horizontal_flag), and / or position information (cu_sbt_pos_flag) are encoded by the entropy encoding unit 155 and sent as signals to the video decoding device. In addition, the size of the transformed region sub-blocks can have a 1:3 size ratio based on the horizontal axis (or vertical axis). In this case, the flag for dividing the corresponding segment (cu_sbt_quad_flag) is additionally encoded by the entropy coding unit 155 and sent to the video decoding device as a signal.

[0064] On the other hand, the transform unit 140 can perform transforms on the residual block in both the horizontal and vertical directions. Various types of transform functions or transform matrices can be used for the transform. For example, the transform function pairs used for horizontal and vertical transforms can be defined as a Multiple Transform Set (MTS). The transform unit 140 can select the transform function pair with the highest transform efficiency from the MTS and perform transforms on the residual block in both the horizontal and vertical directions. Information (mts_idx) about the transform function pair selected from the MTS is encoded by the entropy coding unit 155 and sent as a signal to the video decoding device.

[0065] The quantization unit 145 quantizes the transform coefficients output from the transform unit 140 using quantization parameters and outputs the quantized transform coefficients to the entropy encoding unit 155. For a specific block or frame, the quantization unit 145 can directly quantize the relevant residual block without transformation. The quantization unit 145 can also apply different quantization coefficients (scaling values) to the transform coefficients based on their positions within the transform block. The quantization matrix applied to the quantized transform coefficients arranged in two dimensions can be encoded and sent as a signal to the video decoding device.

[0066] The reordering unit 150 is capable of reordering the coefficient values ​​of the quantized residual values.

[0067] The reordering unit 150 can transform a two-dimensional coefficient array into a one-dimensional coefficient sequence using coefficient scanning. For example, in the reordering unit 150, a zig-zag scan or a diagonal scan can be used to scan from DC coefficients to coefficients in the high-frequency region, and output a one-dimensional coefficient sequence. Depending on the size of the transform unit and the intra-frame prediction mode, a vertical scan that scans the two-dimensional coefficient array in the column direction or a horizontal scan that scans the two-dimensional block shape coefficients in the row direction can be used instead of a zig-zag scan. That is, the scanning method to be used can be determined from zig-zag scan, diagonal scan, vertical scan, and horizontal scan based on the size of the transform unit and the intra-frame prediction mode.

[0068] The entropy coding unit 155 uses various coding methods such as Context-based Adaptive Binary Arithmetic Code (CABAC) and Exponential Golomb coding to encode the sequence of multiple one-dimensional quantized transform coefficients output from the reordering unit 150, thereby generating a bit stream.

[0069] Furthermore, the entropy coding unit 155 encodes information related to block segmentation, such as CTU size, CU segmentation flag, QT segmentation flag, MTT segmentation type, and MTT segmentation direction, so that the video decoding device can segment blocks in the same way as the video encoding device. Additionally, the entropy coding unit 155 encodes information indicating whether the current block is encoded using intra-frame prediction or inter-frame prediction, and encodes intra-frame prediction information (i.e., information about the intra-frame prediction mode) or inter-frame prediction information (motion information encoding mode (merging mode or AMVP mode), merging index in the merging mode case, reference image index and differential motion vector information in the AMVP mode case) according to the prediction type. Furthermore, the entropy coding unit 155 encodes information related to quantization, i.e., information about quantization parameters and information about the quantization matrix.

[0070] The inverse quantization unit 160 performs inverse quantization on the quantized transform coefficients output from the quantization unit 145 to generate transform coefficients. The inverse transform unit 165 transforms the transform coefficients output from the inverse quantization unit 160 from the frequency domain to the spatial domain to reconstruct the residual block.

[0071] Adder 170 adds the reconstructed residual block to the prediction block generated by prediction unit 120, thereby reconstructing the current block. When performing intra-frame prediction for the next sequential block, the pixels in the reconstructed current block are used as reference pixels.

[0072] The loop filter unit 180 performs filtering on the reconstructed pixels to reduce blocking artifacts, ringing artifacts, and blurring artifacts caused by block-based prediction and transform / quantization. The loop filter unit 180, as an in-loop filter, can include all or part of the deblocking filter 182, the Sample Adaptive Offset (SAO) filter 184, and the Adaptive Loop Filter (ALF) 186.

[0073] Deblocking filter 182 performs filtering on the boundaries between reconstructed blocks to remove blocking artifacts caused by the encoding / decoding of block units. SAO filter 184 and ALF 186 perform additional filtering on the deblocked image. SAO filter 184 and ALF 186 are filters used to compensate for differences between reconstructed pixels and original pixels caused by lossy coding. SAO filter 184 applies offset on a CTU basis, thereby improving not only subjective image quality but also coding efficiency. In contrast, ALF 186 performs filtering on a block-by-block basis, applying different filters by dividing the edges and varying degrees of change of the corresponding blocks to compensate for distortion. Information about the filter coefficients used in the ALF can be encoded and sent as a signal to the video decoding device.

[0074] The reconstructed blocks filtered by deblocking filter 182, SAO filter 184, and ALF 186 are stored in memory 190. When all blocks in an image have been reconstructed, the reconstructed image can be used as a reference image for inter-frame prediction of blocks in subsequent images to be encoded.

[0075] Video encoding devices can store the bit stream of encoded video data on a non-transitory recording medium or transmit it to a video decoding device via a communication network.

[0076] Figure 5 This is an exemplary block diagram of a video decoding apparatus capable of implementing the technology disclosed herein. Hereinafter, reference is made to... Figure 5 The video decoding device and its subordinate components are described.

[0077] The video decoding device may include an entropy decoding unit 510, a reordering unit 515, an inverse quantization unit 520, an inverse transform unit 530, a prediction unit 540, an adder 550, a loop filter unit 560, and a memory 570.

[0078] Similar to the video encoding device in Figure 1, the components of the video decoding device can be implemented in hardware or software, or a combination of both. Furthermore, the functions of each component can also be implemented in software, with a microprocessor executing the corresponding software functions for each component.

[0079] The entropy decoding unit 510 decodes the bitstream generated by the video encoding device to extract information related to block segmentation, thereby determining the current block to be decoded and extracting prediction information, residual signal information, etc. required to reconstruct the current block.

[0080] The entropy decoding unit 510 extracts information about the size of the CTU from the Sequence Parameter Set (SPS) or the Picture Parameter Set (PPS) to determine the size of the CTU, and segments the image into CTUs of the determined size. Furthermore, the CTU is identified as the highest level (i.e., the root node) of the tree structure, and segmentation information about the CTU is extracted, thereby utilizing the tree structure to segment the CTU.

[0081] For example, when using the QTBTT structure to segment the CTU, the first flag (QT_split_flag) related to the QT split is extracted to divide each node into four lower-level nodes. Then, for the node corresponding to a leaf node of the QT, the second flag (mtt_split_flag) related to the MTT split and the splitting direction (vertical / horizontal) and / or splitting type (binary / ternary) information are extracted, and the corresponding leaf node is segmented using the MTT structure. Thus, each node below the leaf node of the QT is recursively segmented using either a BT or TT structure.

[0082] As another example, when using the QTBTTT structure to split the CTU, the CU split flag (split_cu_flag) indicating whether the CU should be split is first extracted. If the corresponding block is split, the first flag (QT_split_flag) can also be extracted. During the splitting process, each node can undergo zero or more repeated QT splits followed by zero or more repeated MTT splits. For example, the CTU can be directly split using MTT, or conversely, only multiple QT splits can be performed.

[0083] As another example, when segmenting a CTU using a QTBT structure, the first flag (QT_split_flag) related to the QT split is extracted, and each node is split into four lower-level nodes. Furthermore, for nodes corresponding to leaf nodes of the QT, a split flag (split_flag) indicating whether they can be further split using BT and the split direction information are extracted.

[0084] On the other hand, when the entropy decoding unit 510 uses tree structure segmentation to determine the current block to be decoded, it extracts information about the prediction type, indicating whether the current block undergoes intra-frame prediction or inter-frame prediction. When the prediction type information indicates intra-frame prediction, the entropy decoding unit 510 extracts the syntax elements of the intra-frame prediction information (intra-frame prediction mode) for the current block. When the prediction type information indicates inter-frame prediction, the entropy decoding unit 510 extracts the syntax elements of the inter-frame prediction information, namely, information representing the motion vector and the reference image referenced by that motion vector.

[0085] In addition, the entropy decoding unit 510 extracts information related to quantization, as well as information about the quantized transform coefficients of the current block as information about the residual signal.

[0086] The reordering unit 515 can reorder the sequence of one-dimensional quantized transform coefficients entropy decoded in the entropy decoding unit 510 into a two-dimensional coefficient array (i.e., a block) in the reverse order of coefficient scanning performed by the video encoding device.

[0087] The inverse quantization unit 520 performs inverse quantization on the quantized transform coefficients using quantization parameters. The inverse quantization unit 520 can apply different quantization coefficients (scaling values) to the two-dimensional array of quantized transform coefficients. The inverse quantization unit 520 can perform inverse quantization by applying a matrix of quantization coefficients (scaling values) from the video encoding device to the two-dimensional array of quantized transform coefficients.

[0088] The inverse transform unit 530 inversely transforms the inversely quantized transform coefficients from the frequency domain to the spatial domain to reconstruct the residual signal, thereby generating the residual block of the current block.

[0089] In addition, when performing inverse transformation only on a portion of the transform block (sub-block), the inverse transformation unit 530 extracts a flag (cu_sbt_flag) indicating that only the sub-block of the transform block is transformed, the directionality (vertical / horizontal) information (cu_sbt_horizontal_flag) of the sub-block, and / or the position information (cu_sbt_pos_flag) of the sub-block. The residual signal is reconstructed by inversely transforming the transform coefficients of the corresponding sub-block from the frequency domain to the spatial domain. For the regions that have not undergone inverse transformation, the residual signal is filled with "0" values, thereby generating the final residual block of the current block.

[0090] In addition, when applying MTS, the inverse transform unit 530 uses the MTS information (mts_idx) sent from the video encoding device to determine the transform function or transform matrix to be applied in the horizontal and vertical directions respectively, and performs inverse transform on the transform coefficients in the transform block in the horizontal and vertical directions using the determined transform function.

[0091] The prediction unit 540 may include an intra-prediction unit 542 and an inter-prediction unit 544. The intra-prediction unit 542 is activated when the prediction type of the current block is intra-prediction, and the inter-prediction unit 544 is activated when the prediction type of the current block is inter-prediction.

[0092] The intra prediction unit 542 determines the intra prediction mode of the current block from multiple intra prediction modes based on the grammatical elements of the intra prediction mode extracted from the entropy decoding unit 510, and predicts the current block using reference pixels around the current block based on the intra prediction mode.

[0093] The inter-frame prediction unit 544 uses the syntax elements of the inter-frame prediction mode extracted from the entropy decoding unit 510 to determine the motion vector of the current block and the reference picture to which the motion vector is referenced, and uses the motion vector and the reference picture to predict the current block.

[0094] Adder 550 adds the residual block output by inverse transform unit 530 to the prediction block output by inter-frame prediction unit 544 or intra-frame prediction unit 542, thereby reconstructing the current block. The pixels in the reconstructed current block will be used as reference pixels when performing intra-frame prediction on the block to be decoded later.

[0095] The loop filtering unit 560, acting as an in-loop filter, can include a deblocking filter 562, a SAO filter 564, and an ALF 566. The deblocking filter 562 performs deblocking filtering on the boundaries between reconstructed blocks to remove blocking artifacts caused by the decoding of block units. The SAO filter 564 and ALF 566 perform additional filtering on the reconstructed blocks after deblocking to compensate for differences between reconstructed pixels and original pixels caused by lossy coding. The filter coefficients of the ALF are determined using information about the filter coefficients decoded from the bitstream.

[0096] The reconstructed blocks filtered by deblocking filter 562, SAO filter 564, and ALF 566 are stored in memory 570. When all blocks in an image are reconstructed, the reconstructed image is used as a reference image for inter-frame prediction of blocks in subsequent images to be encoded.

[0097] This embodiment relates to the encoding and decoding of images (videos) as described above. More specifically, it provides a video coding method and apparatus that efficiently selects transform kernels and inverse transform kernels based on the prediction mode of the current block to perform the transform and inverse transform of the current block.

[0098] The following embodiments can be executed using the transformation unit 140 and the inverse transformation unit 165 within a video encoding apparatus. Alternatively, they can be executed using the inverse transformation unit 530 within a video decoding apparatus.

[0099] When encoding the current block, the video encoding apparatus can generate signaling information relevant to this embodiment from a rate-distortion optimization perspective. The video encoding apparatus can then transmit the signaling information to the video decoding apparatus after encoding it using the entropy encoding unit 155. The video decoding apparatus can use the entropy decoding unit 510 to decode the signaling information related to the decoding of the current block from the bitstream.

[0100] In the following description, the term "target block" may be used in the same sense as the current block or coding unit (CU). Alternatively, "target block" may also refer to a portion of a coding unit.

[0101] Additionally, a flag value of "true" indicates that the flag is set to 1. Conversely, a flag value of "false" indicates that the flag is set to 0.

[0102] I. Transformation Techniques - Main Transformation Techniques As mentioned above, for efficient video compression, quantization or scaling can be applied to the residual signals remaining after prediction using various prediction techniques. In this case, based on the importance of the perceptual visual information contained in the residual signals, a transformation technique can be applied to group the residual signals to one side according to their frequency components before scaling. However, for non-natural signals such as screen contents, this frequency-based transformation technique may be inefficient. In such cases, the transformation technique can be omitted, and scaling can be performed only, or encoding / decoding can be performed without applying scaling.

[0103] In HEVC, DCT-II is used as the transform kernel (hereinafter referred to interchangeably with transform type) for transforming the residual signal when applying transforms. However, to apply more suitable transform techniques based on the diversity of residual signal characteristics, Multiple Transform Selection (MTS) can be used. MTS determines one or two optimal transform types from a variety of transform types and then transforms the block according to the determined transform type. For example, in VVC, as shown in Table 1, in addition to DCT-II, two other transform types, DCT-VIII and DST-VII, are added, thus enabling more diverse transformations of the residual signal.

[0104] [Table 1] Here, the basis functions constitute the transformation matrix that defines each transformation type. DCT-II, DCT-VIII, and DST-VII are then used interchangeably with DCT2, DCT8, and DST7, respectively.

[0105] On the other hand, the flag used to determine whether to use MTS can be controlled at the block unit level. Alternatively, the use of MTS can also be controlled using an activation flag at the higher-level SPS level.

[0106] When MTS is activated in SPS, a CU-level flag indicating whether MTS is applied can be displayed. Here, MTS can be applied to the luma component. The CU-level flag can be displayed when the width and height of the Transform Block (TB) are both less than or equal to 32 pixels, and the Coded Block Flag (CBF), indicating the presence of non-zero values ​​among multiple transform coefficient levels, is true.

[0107] When the CU level flag is 0, DCT2 is used as the core in both the horizontal and vertical directions. Conversely, when the CU level flag is not 0, MTS is applied. MTS can be used in both explicit and implicit modes.

[0108] In explicit MTS, the kernel used for TB is explicitly transferred. Typically, the index of the transformed kernel can be transferred. For example, as shown in Table 2, mts_idx can be defined as the index of the kernel.

[0109] [Table 2] Here, trTypeHor and trTypeVer represent the transformation type in the horizontal direction and the transformation type in the vertical direction, respectively. Additionally, 0 represents DCT2, 1 represents DST7, and 2 represents DCT8.

[0110] On the other hand, in implicit MTS, such as in the case of intra-block, the transformation type can be implicitly determined even if no signal is explicitly transmitted. In VVC, as shown in Equation 1, the transformation types in the horizontal and vertical directions can be implicitly determined.

[0111] [Formula 1] Here, nTbW and nTbH represent the horizontal and vertical lengths of the transform block, respectively.

[0112] As an example, when applying a specific coding technique, either explicit or implicit MTS can be used. For instance, in the case of Matrix-weighted Intra Prediction (MIP), explicit intra MTS can be used. In the case of Intra Sub-Partitions (ISP) mode, implicit MTS is used, with DST7 or DCT2 as the transform type.

[0113] On the other hand, when the transform block includes at least one non-DC coefficient, mts_idx is signaled. That is, mts_idx is signaled when the position of the last valid coefficient based on the scan order is greater than 0. Conversely, when the transform block includes only one non-DC coefficient, the signaling of mts_idx is omitted, and mts_idx=0 is derived, thus applying DCT2 as the transform kernel.

[0114] The first binary bit of the signaled mts_idx indicates whether mts_idx is greater than 0. When mts_idx is greater than 0 (i.e., mts_idx indicates one of 1 to 4), an additional 2 bits of fixed-length code are signaled to indicate the signaled mts_idx among the four candidates.

[0115] On the other hand, in the Enhanced Compression Model (ECM) software, which is a next-generation technology, the number and types of MTS kernels have been increased, and DST7, DCT8, DCT5, DST4, DST1 and identity transform have been added.

[0116] II. Low-frequency Non-separable Transform (LFNST) During intra-frame prediction, LFNST applies a secondary transform to the low-frequency regions of the transform coefficients generated from the primary transform of the Transform Unit (TU). At the coding level, LFNST applies a secondary transform to L low-frequency primary transform coefficients out of W×H primary transform coefficients to generate K (here, K≤L) secondary transform coefficients. Here, the transform kernel size of LFNST is L×K. That is, after representing the L low-frequency primary transform coefficients out of W×H primary transform coefficients as a 1×L vector, LFNST applies an L×K transform kernel to generate a 1×K vector. Subsequently, for quantization and other subsequent processes, LFNST represents the 1×K vector as a two-dimensional arrangement of the low-frequency regions.

[0117] Compared to the main transform that applies separate transform kernels in the horizontal and vertical directions, the LFNST technique performs a non-separable transform on a one-dimensional vector.

[0118] On the other hand, the type of transform kernel can be determined based on the intra-prediction mode of the current TU, the size of the TU, and the LFNST index (lfnst_idx). For example, in VVC, the transform kernel set can be determined based on the intra-prediction mode (IntraPredMode) of the current TU, as shown in Table 3.

[0119] [Table 3] Here, IntraPredMode follows Figure 3b Examples. Additionally, lfnstTrSetIdx is the index indicating the kernel set. In Table 3, IntraPredMode values ​​of 81, 82, and 83 represent the prediction modes for the Cross-component Linear Model (CCLM).

[0120] Two types of kernels are defined for each kernel set (lfnstTrSetIdx). The LFNST index indicates which of the two kernels to select. An LFNST index of 0 indicates that LFNST is not performed; an LFNST index of 1 or 2 applies different LFNST kernels within the same kernel set. An additional kernel set exists based on the TU size, resulting in a total of 4 × 2 × 2 = 16 transform kernels. The LFNST kernel sizes are defined as 16 × 16 and 16 × 48. Furthermore, the kernel size can be adjusted according to the TU size as shown in Table 4.

[0121] [Table 4] In ECM, transform kernel sets can be defined based on the shape of the block and the intra-prediction mode. For example, taking advantage of the symmetry of square blocks, 35 transform kernel sets can be defined for square blocks based on the intra-prediction mode.

[0122] On the other hand, as shown in the example of Figure 6a, when performing Matrix-weighted Intra Prediction (MIP) on the current TU in VVC, the video decoding device maps the intra prediction mode of the current TU to a Planar mode and then determines the transform kernel based on the mapped prediction mode. MIP technology can be simplified as shown in Figure 6a. In the example of Figure 6a, "MIP boundary downsampling" refers to the process of downsampling a specific number of pixels in the boundary pixels of the current block. "Matrix-Vector multiplication" refers to the process of multiplying the downsampled pixels (i.e., vectors) with a preset matrix to generate a reduced prediction block. "MIP prediction upsampling" refers to the process of upsampling the reduced prediction block to generate the final prediction block.

[0123] Furthermore, when performing MIP prediction in the Enhanced Compression Model (ECM) as a next-generation technology, video decoding devices such as Figure 6b As illustrated in the example, Decoder-side Intra Mode Derivation (DIMD) is applied to the block predicted by MIP (i.e., the output of "Matrix-Vector Multiplication") to obtain the directional prediction mode. The video decoding device maps the intra prediction mode of the current TU to the obtained directional prediction mode and then determines the transform kernel based on the mapped prediction mode. DIMD calculates the gradients of each sample in the current block's neighboring samples, generates a Histogram of Gradients (HoG) from the calculated gradients, and derives the prediction mode for intra prediction of the current block based on the generated HoG.

[0124] Additionally, during Intra Template Matching Prediction (IntraTMP) execution, the video decoding unit applies DIMD to the prediction blocks to obtain directional prediction modes. After mapping the intra prediction mode of the current TU to the obtained directional prediction modes, the video decoding unit determines the transform kernel based on the mapped prediction modes.

[0125] On the other hand, in VVC, if the DCT2 / DCT2 transform kernel is applied as the primary transform to the intra-predicted TU, then the LFNST technique can be applied as the secondary transform.

[0126] In ECM, when the block size in the TU of intra-frame predictive coding is small (e.g., 4×4, 4×8, 8×4, 4×16, 16×4, 8×8, 8×16, 16×8), the non-separable master transform technique can be applied. Furthermore, when the block size is large and the DCT2 / DCT2 transform kernel is applied as the master transform to the block, the LFNST technique can be applied.

[0127] III. Non-separable Primary Transform (NSPT) The non-separable master transform technique does not perform vertical and horizontal transformations on the residual signals of blocks predicted by intra-frame prediction modes. Instead, it performs the non-separable master transform using a pre-learned transform kernel. As described above, when DCT-2 is used as the separable master transform and LFNST is used as the non-separable quadratic transform, the NSPT technique can be applied to residual blocks of predetermined sizes (e.g., 4×4, 4×8, 8×4, 4×16, 16×4, 8×8, 8×16, 16×8).

[0128] The NSPT technique uses non-separable master transform kernels of different sizes depending on the size of the residual block. For example, when the transform block size is 4×4, a 16×16 kernel is applied to generate 4×4 transform coefficients. As another example, when the transform block size is 4×8, a 32×20 kernel is applied to generate 20 transform coefficients. The generated transform coefficients are arranged in the transform block according to a predefined scan order. When the number of transformed coefficients generated after the transform is less than the number of pixels in the transform block, the remaining areas without transformed coefficients are filled with 0.

[0129] Similar to the LFNST technique, the NSPT technique also defines kernel sets and allows selection of a kernel from among multiple kernels within each kernel set via NSPT indices. As described above, when DCT-2 is used as the separable master transform and LFNST is used as the non-separable quadratic transform, NSPT is applied to transform blocks of a predetermined size. That is, the separable master transform and LFNST are replaced by the non-separable master transform, and the LFNST index can be reused as the NSPT index.

[0130] For an inseparable master transform, the transform kernel can be defined based on the block shape, intra-prediction mode, and NSPT index (nspt_idx). For example, taking advantage of the symmetry of a square block, there are 35 transform kernel sets for a square block based on the intra-prediction mode. In this case, the NSPT index (i.e., the LFNST index) can be used to indicate one of the multiple candidate kernels within the transform kernel set.

[0131] The non-separable principal transform can be applied to the luminance component. Furthermore, when applying the non-separable principal transform, a secondary transform (i.e., LFNST) can be avoided.

[0132] The following embodiments are mainly described with regard to video decoding devices, but they can also be implemented in the same or similar way in video encoding devices.

[0133] IV. Embodiments based on this disclosure Figure 7 This is a detailed block diagram of a portion of a video decoding apparatus based on an embodiment of the present disclosure.

[0134] The video decoding device based on this embodiment determines prediction and transform units, and uses the determined prediction techniques and prediction modes to perform prediction and inverse transform on the current block corresponding to the determined units, ultimately generating a reconstructed block of the current block. For example... Figure 7 The content shown can be executed by the entropy decoding unit 510, inverse quantization unit 520, inverse transform unit 530, prediction unit 540, and adder 550 of the video decoding device. On the other hand, with Figure 7 The same operations shown can be performed by the inverse quantization unit 160, inverse transform unit 165, image segmentation unit 110, prediction unit 120, and adder 170 of the video encoding apparatus. At this time, the video decoding apparatus can use the encoded information parsed from the bitstream, while the video encoding apparatus, from the perspective of minimizing rate-distortion, can use the encoded information set by the upper layer. Hereinafter, for ease of explanation, this embodiment will be described focusing on the video decoding apparatus.

[0135] like Figure 5 As shown in the example, the prediction unit 540 includes an intra-frame prediction unit 542 and an inter-frame prediction unit 544 according to the prediction technique, but as... Figure 7 As shown, the prediction unit 540 may include all or part of the prediction unit determination unit 702, the prediction technology determination unit 704, the prediction mode determination unit 706, and the prediction execution unit 708.

[0136] When the input video's color format is YUV (YUV420, YUV411, YUV422, YUV444, etc.), the video decoding device can perform prediction and reconstruction of the chrominance component after predicting and reconstructing the luminance component. That is, the luminance and chrominance components can be... Figure 7 The components shown are reconstructed sequentially. Here, in the case of YUV format, the color format represents the correspondence between pixels of the luminance component and pixels of the chrominance component.

[0137] The operation of each component within the prediction unit 540 is described below. The prediction unit determination unit 702 determines the prediction unit (PU). The prediction technique determination unit 704 determines the prediction technique for the prediction unit (e.g., intra-frame prediction, inter-frame prediction, or Intra Block Copy (IBC) mode, palette mode, etc.). The prediction mode determination unit 706 determines a detailed prediction mode for the prediction technique. The prediction execution unit 708 generates a prediction block for the current block based on the determined prediction mode.

[0138] The prediction unit determined in the prediction unit determination unit 702 can be the current block or one of the multiple sub-blocks after the current block is divided. In this case, the prediction unit for the chroma component, depending on the color format, can be the same size as the prediction unit for the luminance component. Alternatively, prediction can be performed on the chroma component prediction unit after the prediction units for the luminance and chroma components are determined separately.

[0139] The prediction technique determination unit 704 determines the prediction technique for the prediction unit. As described above, the prediction technique can be one of inter-frame prediction, intra-frame prediction, IBC mode, and palette mode. In this case, the prediction technique for the chroma component can be determined to be the same as the prediction technique for the corresponding luma component, without the need for additional signaling transmission and parsing of information.

[0140] As an example, when the prediction technique for the current block is not intra-frame prediction, the video decoding device parses 1-bit flag information. For instance, if the parsed flag indicates a skip mode, the video decoding device determines the prediction mode for the current block to be either inter-frame prediction merge mode or IBC merge mode. The video decoding device can omit the inverse transform process and use the predicted signal as the reconstructed signal.

[0141] Conversely, when the parsed flag does not indicate the application of the Skip mode to the current block, the prediction technique determination unit 704 parses a series of 1-bit flags and is able to determine the prediction technique of the current block as one of techniques such as inter-frame prediction, intra-frame prediction, IBC mode, palette mode, etc.

[0142] For example, when no skip is applied to the current block and the prediction technique is determined to be inter-frame prediction or IBC mode, the video decoding device parses a 1-bit flag. Based on the parsed flag, the prediction mode of the current block can be determined to be either merge mode or Advanced Motion Vector Prediction (AMVP) mode.

[0143] The prediction model determination unit 706 determines a detailed prediction model for the prediction technology.

[0144] As an example, when the prediction technique is inter-frame prediction, the mode for generating the prediction block based on the parsed motion information can be the prediction mode of the current block. In this case, the video decoding device uses one or more motion compensations based on the parsed motion information to generate a prediction signal. The video decoding device weights and sums the multiple prediction signals to generate the prediction block for the current block.

[0145] As another example, when the prediction technique is inter-frame prediction, the geometric partitioning mode used to generate the prediction block based on the resolved motion information can be the prediction mode for the current block. In this case, the video decoding device divides the current block into more than one sub-region based on the geometric partitioning. For each sub-region, the video decoding device generates a prediction signal using more than one motion compensation determined based on the resolved motion information. The video decoding device then performs a weighted sum of the multiple prediction signals to generate the prediction block for the current block.

[0146] As an example, when the prediction technique is intra-frame prediction, the mode used to generate the predicted signal can be the prediction mode of the current block, using a unidirectional prediction mode, a Planar mode, or a DC mode. In this case, the Planar mode can be a horizontal Planar mode, a vertical Planar mode, or a regular Planar mode. When performing linear interpolation to generate the predicted signal, the Planar mode uses left and top reference samples, the horizontal Planar mode uses left and top-right reference samples, and the vertical Planar mode uses top and bottom-left reference samples.

[0147] As another example, when the prediction technique is intra-frame prediction, the geometric segmentation mode used to generate the prediction block based on the intra-frame prediction mode determined by geometric segmentation can be the prediction mode of the current block. In this case, the video decoding device divides the current block into more than one sub-region according to the geometric segmentation. The video decoding device uses an intra-frame prediction mode (directional prediction mode, Planar mode, or DC mode, etc.) to generate a prediction signal for each sub-region. The video decoding device performs a weighted sum of multiple prediction signals to generate the prediction block for the current block.

[0148] As another example, when the prediction technique is intra-frame prediction, the MIP mode using a matrix predefined according to an agreement between the video encoding and decoding devices can be the prediction mode for the current block. In this case, the video encoding device can send the matrix itself as a signal to the video decoding device. Alternatively, the video encoding device can send the index of the indicator matrix as a signal to the video decoding device.

[0149] As another example, when the prediction technique is intra-frame prediction, template-matching-based prediction can be the prediction pattern of the current block. In this case, the video decoding device defines the reconstructed region surrounding the current block as a template and performs template matching in the surrounding region of the current block to generate the prediction signal.

[0150] The prediction execution unit 708 generates a prediction block for the current block based on the determined prediction technology and prediction mode.

[0151] The following describes the operations related to inverse transformation and reconstruction.

[0152] Regarding the inverse transform, the entropy decoding unit 510 decodes the transform coefficients. When a quadratic transform is applied, the entropy decoding unit 510 decodes the quantized quadratic transform coefficients. When a quadratic transform is not applied, the entropy decoding unit 510 decodes the quantized main transform coefficients.

[0153] The inverse quantization unit 520 performs inverse quantization on the quantization transform coefficients decoded by the current transform block to generate an inverse quantized signal.

[0154] The inverse transform unit 530 performs an inverse transform on the TU represented by the inverse quantization signal to generate a residual signal.

[0155] like Figure 7 As shown in the example, the inverse transform unit 530 includes all or part of the inverse transform unit determining unit 710, the inverse transform core determining unit 712, and the inverse transform execution unit 714. The inverse transform unit determining unit 710 determines the transform unit (TU) for the inverse quantization signal of the current block, the inverse transform core determining unit 712 determines the inverse transform core, and the inverse transform execution unit 714 performs an inverse transform on the transform unit represented by the inverse quantization signal to generate a residual signal.

[0156] Adder 550 adds the predicted block to the residual signal to generate a reconstructed block. The reconstructed block is then applied to an in-loop filter. The filtered reconstructed block is stored in the Decoded Picture Buffer (DPB) in memory 570 and can be used for the prediction of other blocks later.

[0157] From now on, TU and the transform block will be used interchangeably.

[0158] The operation of each component within the inverter unit 530 will be explained below.

[0159] The transformation unit determined in the inverse transformation unit determination unit 710 can be a TU, or a sub-block of a TU after being divided into multiple sub-blocks.

[0160] The inverse transform kernel determination unit 712 determines separable main inverse transform kernels in the vertical and horizontal directions and / or non-separable secondary inverse transform kernels. Alternatively, the inverse transform kernel determination unit 712 determines non-separable main inverse transform kernels.

[0161] The inverse transform execution unit 714 uses the determined inverse transform kernel to perform an inverse transform on the inverse quantized transform coefficients to generate a residual signal.

[0162] Figure 8 This is a schematic diagram illustrating a method for determining the inverse transform kernel based on an embodiment of the present disclosure.

[0163] Inverse transform kernel determination unit 712 according to Figure 8 The example determines the type of inverse transform kernel for the current transform block.

[0164] On the other hand, whether the inseparable inverse transformation is executed can be determined based on the signaling transmission / parsing of the corresponding flag or index. Here, whether the inseparable inverse transformation is executed indicates whether the inseparable primary inverse transformation or the inseparable secondary inverse transformation is executed. In this case, whether the inseparable primary inverse transformation is executed can be implicitly determined based on whether the inseparable secondary inverse transformation is executed and the current size of the TU.

[0165] The inverse transform kernel determination unit 712 can determine the separable main inverse transform kernel as follows.

[0166] For example, when performing a separable master-inverse transform on the current TU, the inverse transform kernel determination unit 712 determines the vertical kernel and the horizontal kernel of the current TU based on the prediction technique, prediction mode, aspect ratio, size, index (hereinafter referred to as MTS_idx) of the current block.

[0167] At this point, when the current block is predicted using the intra-prediction mode (hereinafter referred to as the intra-prediction mode) and the mts_idx is parsed to explicitly determine the separable master and inverse transform kernels in the vertical and horizontal directions, the type of kernel represented by MTS_idx may differ depending on the current TU size and the intra-prediction mode. That is, for the combination of the current TU size (e.g., 4×4, 4×8, 8×4, 4×16, 16×4, ...) and the current block's intra-prediction mode, the type of master and inverse transform kernels in the vertical and horizontal directions represented by each MTS_idx is defined according to the convention between the video encoding device and the video decoding device. Through the parsing of mts_idx, the inverse transform kernel determination unit 712 determines the final master and inverse transform kernels in the vertical and horizontal directions.

[0168] When the vertical and horizontal separable master-inverse transform kernels are explicitly determined by predicting the current block using the prediction mode within the inter-frame prediction technique (hereinafter referred to as the inter-frame prediction mode) and resolving mtx_idx, the type of kernel represented by MTS_idx may differ depending on the size of the current TU.

[0169] As another example, when applying an inseparable second-order inverse transform to the current TU, the inverse transform kernel determination unit 712 implicitly sets the separable master inverse transform kernel to a fixed kernel (e.g., vertical and horizontal kernels: DCT2, DCT2). In this case, the fixed kernel can adaptively change according to the size and aspect ratio of the current block.

[0170] The inverse transform kernel determination unit 712 determines the non-separable principal inverse transform kernel as follows.

[0171] As one example, the non-separable master-inverse transform can be applied to blocks that have performed intra-frame prediction. As another example, the non-separable master-inverse transform can be applied to blocks that have performed inter-frame prediction.

[0172] As an example, when a preset kernel is used as a separable master transform and LFNST is used as a non-separable quadratic transform, a non-separable master transform technique can be applied to a transform block of a preset size. In this case, the separable master transform and LFNST are replaced by the non-separable master transform.

[0173] When applying the non-separable master-inverse transform to the current TU, the inverse transform kernel determination unit 712 determines the non-separable master-inverse transform kernel based on the size of the current TU and the intra-frame prediction mode.

[0174] As another example, the type of kernel represented by lfnst_idx may differ depending on more than one of the following: index (e.g., nspt_idx (i.e., lfnst_idx)), intra-prediction mode of the current block, aspect ratio of the current TU, and size of the current TU.

[0175] Furthermore, the use of the inseparable primary inverse transform is limited when using the inseparable quadratic inverse transform. Specifically, when applying the inseparable quadratic inverse transform (i.e., when the parsed inseparable quadratic inverse transform index indicates the use of the inseparable quadratic inverse transform), the inverse transform kernel determination unit 712 does not parse the index related to the inseparable primary inverse transform. At this time, the inverse transform kernel determination unit 712 parses the bitstream related to the separable primary inverse transform or transform skip. Therefore, the parsing of the inseparable primary transform index depends on whether the inseparable quadratic inverse transform is performed. According to an embodiment, when the inseparable primary inverse transform index is parsed first, the parsing of the inseparable quadratic inverse transform can depend on the index value of the inseparable primary inverse transform.

[0176] The inverse transform kernel determination unit 712 determines the non-separable quadratic inverse transform kernel as follows.

[0177] As an example, the inseparable quadratic inverse transform is applied to a block that has undergone intra-frame prediction.

[0178] As another example, when an inseparable second inverse transform is applied to the current TU, the inverse transform kernel determination unit 712 determines the inseparable second inverse transform kernel based on the size of the current TU, the intra-frame prediction mode, and the separable main inverse transform kernel.

[0179] As another example, the type of kernel represented by lfnst_idx may vary depending on the index (e.g., lfnst_idx), the intra-prediction mode of the current block, the aspect ratio of the current TU, and the size of the current TU.

[0180] The following explains the determination of the intra-prediction mode of the current TU used in the inverse transform kernel determination process.

[0181] As an example, when the current TU is square, the symmetry of the square block is utilized. In this case, the intra-frame prediction mode of the current TU used in the inverse transform kernel determination process can be in the diagonal mode of the upper left corner (…). Figure 3b The example (34 modes) is divided into n sets centered on the model. For example, in the case of a square block, as shown in Table 5, it can be divided into four mode sets based on the intra-frame prediction mode.

[0182] Furthermore, when divided into four pattern sets as shown in Table 5, pattern m and pattern j (=68-m) are included in the same pattern set. Therefore, the inverse transform kernel determination unit 712 can use the same kernel set for both pattern m and pattern j.

[0183] [Table 5] The index of the pattern set according to Table 5 can be used as the index of the kernel set.

[0184] As another example, when the current TU is rectangular, the intra-prediction modes of the current TU used in the inverse transform kernel determination process can be divided into k sets. For example, in the case of rectangular blocks, as shown in Table 6, they can be divided into 6 mode sets according to the intra-prediction modes.

[0185] [Table 6] When the pattern set is divided as shown in Table 6 and an inseparable inverse transform is applied to the current TU, a transform block of size A×B predicted in mode q can be inversely transformed using the same kernel as a transform block of size B×A predicted in mode p (=68-q). The index indicating the pattern set based on Table 6 can be used as an index indicating the kernel set.

[0186] As another example, transform kernel sets can be defined based on the shape of the block and the intra-prediction mode. For instance, taking advantage of the symmetry of square blocks, 35 transform kernel sets can be defined for square blocks based on the intra-prediction mode. These 35 transform kernel sets can be defined as shown in Table 7.

[0187] [Table 7] To indicate one of the multiple candidates within the transform kernel set shown in Tables 5, 6, or 7, lfnst_idx can be used.

[0188] Wide-angle intra-prediction (WAIP) mode has less diverse residual characteristics compared to other intra-prediction modes, thus allowing the use of fewer sets of inseparable transforms and / or kernels. Alternatively, in the case of WAIP mode, the inseparable principal inverse transform can be applied by mapping to any kernel (e.g., mapping to the transform set of an adjacent intra-prediction mode). This can lead to the following benefits: improved coding efficiency, reduced memory consumption for storing the inseparable principal transform kernel, and reduced overhead from signaling the syntax elements / additional information of the inseparable principal transform.

[0189] As described above, when the prediction mode of the current block is directional or non-directional, the video decoding device sets the intra-frame prediction mode of the transform block to the prediction mode of the current block.

[0190] On the other hand, when predicting the current block in MIP mode, the intra-prediction mode of the current TU used in the inverse transform kernel determination process can be partitioned by one of the n sets based on Table 5 / Table 6, or by a separate set not listed in Table 5 / Table 6.

[0191] As an example, when predicting the current block in MIP mode, the matrices used in the prediction can correspond to more than one directional or non-directional mode (e.g., a regular Planar mode) according to the agreement between the video encoding and decoding devices. When a matrix corresponds to a single directional or non-directional mode, the intra-prediction mode of the current TU used in the inverse transform kernel determination process is set to the corresponding mode, and the inverse transform kernel can be partitioned by including a set of corresponding modes. When a matrix corresponds to multiple directional or multiple non-directional modes, the video decoding device resolves indices or flags. In this case, the intra-prediction mode of the current TU is set to the prediction mode corresponding to the index or flag, and the inverse transform kernel can be partitioned by including a set of prediction modes.

[0192] As another example, when predicting the current block in MIP mode, the intra-prediction mode of the current TU used in the inverse transform kernel determination process can be divided into directional or non-directional prediction modes derived from the intermediate stage prediction signals or the final prediction signals generated during the MIP mode prediction process. For example, the video decoding device applies DIMD technology to the intermediate stage prediction signals or the final prediction signals generated during the MIP mode prediction process. The video decoding device generates a Histogram of Oriented Gradients (HoG) for the prediction signals or the final prediction signals, and extracts the directional prediction mode corresponding to the most accumulated directionality based on the generated HoG. At this time, the intra-prediction mode of the current TU is set to the extracted directional prediction mode, and the inverse transform kernel can be divided by a set including the extracted directional prediction mode. Alternatively, when the accumulated directions are all less than a preset threshold, the intra-prediction mode of the current TU can be set to a non-directional prediction mode. Figure 6b In the example, the intermediate prediction signal generated during the prediction process based on the MIP mode is the output of "Matrix-Vector Multiplication", while the final prediction signal is the output of "MIP Prediction Upsampling".

[0193] On the other hand, when predicting the current block in a geometric segmentation mode, as an example, the intra-frame prediction mode of the current TU used in the inverse transform kernel determination process can be segmented according to the size of the current block, the aspect ratio of the current block, and / or the directional prediction mode based on the direction of the geometric segmentation mode.

[0194] As another example, the intra-prediction mode of the current TU can be derived from the prediction signals of the current block generated by weighted summation. For instance, the video decoding device generates HoGs of multiple prediction signals for the current block and extracts the directional prediction mode corresponding to the direction with the most accumulated values ​​based on the generated HoGs. In this case, the intra-prediction mode of the current TU is set to the extracted directional prediction mode, and the inverse transform kernel can be partitioned by a set including the extracted directional prediction mode. Alternatively, when the accumulated directions are all less than a preset threshold, the intra-prediction mode of the current TU can be set to a non-directional prediction mode.

[0195] As another example, among the multiple predicted signals of the current block generated by weighted summation based on the geometric segmentation pattern, such as Figure 9As illustrated in the example, the intra-frame prediction mode of the current TU can be derived from a region where a weighted sum has been performed, or from a portion of a sub-region within a region where a weighted sum has been performed. For instance, the video decoding device generates a HoG of the corresponding prediction signal and extracts the directional prediction mode corresponding to the direction with the highest accumulation based on the generated HoG. In this case, the intra-frame prediction mode of the current TU is set to the extracted directional prediction mode, and the inverse transform kernel can be partitioned by a set including this directional prediction mode.

[0196] As another example, for the prediction signal of the current block generated by weighted summation based on the geometric segmentation pattern, the intra-prediction pattern of the current TU can be derived in the surrounding region of the geometric segmentation boundary. In this case, the size of the surrounding region of the geometric segmentation boundary used in the derivation process can be a fixed region based on the size of the current block.

[0197] As another example, it illustrates that... Figure 10 The example shows the case where the current block is predicted in geometric segmentation mode, and more than one of sub-regions 1 and sub-region 2 is predicted in intra-frame prediction mode.

[0198] When predicting a sub-region in intra-prediction mode, the intra-prediction mode of the current TU is set to the intra-prediction mode of that sub-region, and the inverse transform kernel can be partitioned by a set that includes the intra-prediction mode of that sub-region.

[0199] When only one sub-region is predicted in intra-prediction mode, and the width of that sub-region is larger than the width of other sub-regions, the intra-prediction mode of the current TU is set to the intra-prediction mode of that sub-region, and the inverse transform kernel can be partitioned by a set including the intra-prediction mode of that sub-region.

[0200] When both regions are predicted in intra-prediction mode, the intra-prediction mode of the current TU is set to the intra-prediction mode of the larger sub-region among the two sub-regions. The inverse transform kernel can be divided by the set of intra-prediction modes that include the sub-region.

[0201] When prediction is performed in both regions using intra-prediction mode, a combination of the geometric segmentation mode of the current block and the intra-prediction modes of each sub-region can be used. The intra-prediction mode based on this combination can be defined as a lookup table according to an agreement between the video encoding and decoding devices. The intra-prediction mode of the current TU used in the inverse transform kernel determination process is set as a combination-based intra-prediction mode, and the inverse transform kernel can be partitioned by including a set of combination-based intra-prediction modes. Furthermore, the lookup table based on this combination may differ depending on the size and aspect ratio of the current block.

[0202] The following describes the use of cross-component prediction in intra-frame prediction of chroma blocks. Models such as the Cross-component Linear Model (CCLM), Convolutional Cross-component Model (CCCM), Multi-mode Linear Model (MMLM), Multi-directional Linear Model (MDLM), Gradient Linear Model (GLM), and Gradient and Location-based Convolutional Cross-component Model (GL-CCCM) are cross-component prediction methods that utilize the correlation between chroma and luminance components.

[0203] As an example, the intra-prediction mode of the center position of the corresponding (co-located) luma block (hereinafter referred to as the corresponding luma block) is mapped to the intra-prediction mode of the current block. At this time, the intra-prediction mode of the current TU used in the inverse transform kernel determination process is set to the mapped intra-prediction mode, and the inverse transform kernel can be partitioned by including the set of mapped intra-prediction modes.

[0204] As another example, the intra-frame prediction mode of the current TU can be set to a directional or non-directional prediction mode derived from the intermediate or final prediction signals generated during the prediction process of the current block. In this case, the inverse transform kernel can be partitioned by including a set of derived directional or non-directional prediction modes.

[0205] As another example, when using such Figure 11 When the reconstructed signal generated by the directional filter shown in the example is used in the prediction process of the current block, the intra-prediction mode of the current TU is set to the directional prediction mode corresponding to the directional filter used, and the inverse transform kernel can be partitioned by including a set of corresponding directional prediction modes.

[0206] As another example, it is possible to use a separate inverse transform kernel for chroma blocks, distinct from luminance blocks.

[0207] As another example, based on the inverse transform information of the corresponding luma block or the inverse transform kernel information of the corresponding luma block, the video decoding device is able to parse some or all of the syntax elements related to the inverse transform of the chroma block.

[0208] The following describes the case where an inseparable second-order inverse transform is applied to the current TU. The inverse transform execution unit 714 uses a scan order agreed upon between the video encoding and decoding devices to perform one-dimensional packing of the t inverse-quantized transform coefficients, thereby constructing a one-dimensional vector ( The inverse transformation execution unit 714 uses the non-separable quadratic inverse transformation kernel determined by the inverse transformation kernel determination unit 712 to generate a one-dimensional vector of size r (t <= r). The inverse transformation execution unit 714 utilizes the agreed-upon scanning order between the video encoding device and the video decoding device to transform a one-dimensional vector ( The values ​​are mapped to the pixel values ​​of the current TU, thereby generating the inversely quantized master transform coefficients. At this point, the regions within the TU that have not undergone mapping are filled with 0.

[0209] On the other hand, the scanning order agreed upon between the video encoding device and the video decoding device can be changed according to the current TU size and the number of inverse quantization transform coefficients.

[0210] The following describes the application of separable master-inverse transform to the current TU. The inverse transform execution unit 714 applies the master transform coefficients generated from the execution of the second inverse transform to the separable master-inverse transform kernels in the vertical and horizontal directions respectively to perform the master-inverse transform, thereby generating the residual signal of the current block. As described above, the separable master-inverse transform kernels in the vertical and horizontal directions are determined by the inverse transform kernel determination unit 712.

[0211] The following describes the case where an inseparable master-inverse transform is performed on the current TU. The inverse transform execution unit 714 uses a scan order agreed upon between the video encoding and decoding devices to perform one-dimensional packing of the w inverse-quantized transform coefficients, thereby constructing a one-dimensional vector ( The inverse transformation execution unit 714 uses the non-separable master inverse transformation kernel determined by the inverse transformation kernel determination unit 712 to generate a one-dimensional vector of size z (w<=z). The inverse transformation execution unit 714 utilizes the agreed-upon scanning order between the video encoding device and the video decoding device to transform a one-dimensional vector ( The pixel values ​​of the current TU are mapped to the current TU, thus generating a residual signal. At this time, the areas within the TU that have not undergone mapping are filled with 0.

[0212] On the other hand, the scanning order agreed upon between the video encoding device and the video decoding device can be changed according to the current TU size and the number of inverse quantization transform coefficients.

[0213] The following describes the methods for performing transformations and inverse transformations on the current block's transform block.

[0214] Figure 12This is a flowchart illustrating a method for transforming transform blocks using a video encoding apparatus based on an embodiment of the present disclosure.

[0215] The video encoding device acquires the residual signal for the transform block of the current block (S1200).

[0216] The video encoding device derives the intra-frame prediction mode of the transform block based on the prediction mode of the current block (S1202).

[0217] When the prediction mode of the current block is directional or non-directional, the video coding device sets the intra-frame prediction mode of the transform block to directional or non-directional.

[0218] When the prediction mode of the current block is MIP mode, the video coding apparatus acquires a preset directional mode or a preset non-directional mode corresponding to the matrix used in MIP mode. The video coding apparatus sets the intra-frame prediction mode of the transform block to the preset directional mode or the preset non-directional mode.

[0219] When the prediction mode of the current block is geometric segmentation mode, the video coding device generates the prediction signal of the current block based on the geometric segmentation mode, and derives a directional or non-directional prediction mode based on all or part of multiple prediction signals. The video coding device sets the intra-frame prediction mode of the transform block to the derived directional or non-directional prediction mode.

[0220] The video coding device determines the non-separable transform core corresponding to the non-separable primary transform core or the non-separable secondary transform core based on the transform block size and the intra-frame prediction mode (S1204).

[0221] The video coding apparatus selects a mode set, including intra-prediction modes of the transform block, from a preset set of multiple modes. The video coding apparatus then selects a kernel set based on the transform block size and the selected mode set.

[0222] When the kernel set includes a transform kernel candidate, the video coding device will identify that transform kernel candidate as an indivisible transform kernel.

[0223] When the kernel set includes multiple transform kernel candidates, from a rate-distortion optimization perspective, the video coding apparatus can determine one of the multiple transform kernel candidates as an inseparable transform kernel. Then, the video coding apparatus can encode the information indicating the determined candidate.

[0224] The video encoding device applies a non-separable master transform kernel to the residual signal to generate the first master transform coefficients (S1206).

[0225] The video encoding device explicitly or implicitly determines the main transform pair in the vertical and horizontal directions for the transform block (S1208).

[0226] The video encoding device applies the master transform check to the residual signal to generate the second master transform coefficients (S1210).

[0227] The video encoding device applies an inseparable quadratic transform kernel to the second master transform coefficients to generate the second master transform coefficients (S1212). When generating the second master transform coefficients, the implicitly generated master transform kernel can be applied to the residual signal to generate the second master transform coefficients.

[0228] The video encoding device determines the inseparable transform information (S1214). Here, the inseparable transform information indicates whether an inseparable transform is applied. In this case, the inseparable transform represents either an inseparable primary transform or an inseparable secondary transform.

[0229] Inseparable transformation information can be a flag or index indicating whether an inseparable transformation is applied. When the inseparable transformation information includes a flag, it can be included in the inseparable transformation information after additionally determining the index of one of the multiple candidates within the transformation kernel set. When the inseparable transformation information includes a flag and the flag's value is true, the inseparable transformation is applied; when the flag's value is false, the inseparable transformation is not applied. When the inseparable transformation information is an index and the index's value is 0, the inseparable transformation is not applied. When the inseparable transformation information is an index and the index's value is 1 or greater, the inseparable transformation is applied, and this index indicates one of the multiple candidates within the transformation kernel set.

[0230] The video coding apparatus determines the inseparable transform information based on multiple second principal transform coefficients and multiple quadratic transform coefficients. This inseparable transform information indicates whether to apply the inseparable quadratic transform. From a rate-distortion optimization perspective, the video coding apparatus can determine the inseparable transform information. For example, when the quadratic transform coefficients are optimal, the video coding apparatus can set the inseparable transform information to apply the inseparable quadratic transform. Conversely, when the second principal transform coefficients are optimal, the video coding apparatus can set the inseparable transform information to not apply the inseparable quadratic transform.

[0231] As described above, under specific conditions (DCT-2 is used as a separable principal transform, LFNST is used as a non-separable quadratic transform, and the transform block has a preset size), a non-separable principal transform can be applied. Therefore, when the above specific conditions are met, the non-separable transform information can represent the application of the non-separable principal transform.

[0232] The video encoding device encodes the inseparable transformation information (S1216).

[0233] The video encoding device encodes the first main transform coefficient, the second main transform coefficient, or the second transform coefficient based on the inseparable transform information (S1218). When the inseparable transform information indicates the application of an inseparable quadratic transform, the video encoding device encodes the quadratic transform coefficients. At this time, if the specific conditions described above are met, the inseparable transform information indicates the application of an inseparable primary transform, and therefore the video encoding device encodes the first primary transform coefficients. Conversely, when the inseparable transform information does not indicate the application of an inseparable transform, the video encoding device encodes the second primary transform coefficients.

[0234] Figure 13 This is a flowchart illustrating a method for performing inverse transformation of a transform block using a video decoding apparatus based on an embodiment of the present disclosure.

[0235] The video decoding device obtains the inverse quantized transform coefficients for the transform block of the current block (S1300).

[0236] The video decoding device decodes the inseparable transform information from the bitstream (S1302). Here, the inseparable transform information indicates whether an inseparable inverse transform is applied. In this case, the inseparable inverse transform represents either an inseparable primary inverse transform or an inseparable secondary inverse transform.

[0237] Inseparable transform information can be a flag or index indicating whether an inverse inseparable transform is applied. When the inseparable transform information includes a flag, it can additionally include an index indicating one of the candidates within the transform kernel set. The index can be decoded from the bitstream. When the inseparable transform information includes a flag and the flag's value is true, the inverse inseparable transform is applied; when the flag's value is false, the inverse inseparable transform is not applied. When the inseparable transform information is an index and the index's value is 0, the inverse inseparable transform is not applied. When the inseparable transform information is an index and the index's value is 1 or greater, the inverse inseparable transform is applied, and the index indicates one of the candidates within the transform kernel set.

[0238] The video decoding device confirms whether the inseparable transformation information indicates that an inseparable inverse transformation should be performed (S1304).

[0239] When performing an inseparable inverse transform (S1304 "Yes"), the video decoding device performs the following steps.

[0240] The video decoding device derives the intra-frame prediction mode of the transform block based on the prediction mode of the current block (S1306).

[0241] When the prediction mode of the current block is directional or non-directional, the video decoding device sets the intra-frame prediction mode of the transform block to directional or non-directional.

[0242] When the prediction mode of the current block is MIP mode, the video decoding device acquires a preset directional mode or a preset non-directional mode corresponding to the matrix used in MIP mode. The video decoding device sets the intra-frame prediction mode of the transform block to the preset directional mode or the preset non-directional mode.

[0243] When the prediction mode of the current block is geometric segmentation mode, the video decoding device generates the prediction signal of the current block according to the geometric segmentation mode, and derives a directional or non-directional prediction mode based on all or part of multiple prediction signals. The video decoding device sets the intra-frame prediction mode of the transform block to the derived directional or non-directional prediction mode.

[0244] The video decoding device determines the non-separable inverse transform core corresponding to the non-separable primary inverse transform core or the non-separable secondary inverse transform core based on the size of the transform block and the intra-frame prediction mode (S1308).

[0245] The video decoding device selects a mode set, including intra-prediction modes of the transform block, from a preset set of multiple modes. The video decoding device then selects a kernel set based on the transform block size and the selected mode set.

[0246] When the kernel set includes an inverse transform kernel candidate, the video decoding device determines that inverse transform kernel candidate as an inseparable inverse transform kernel.

[0247] Alternatively, when the kernel set includes multiple inverse transform kernel candidates, the video decoding device can determine the candidate indicated by the inseparable transform information as the inseparable inverse transform kernel from among the multiple inverse transform kernel candidates.

[0248] The video decoding device uses an inseparable inverse transform kernel to generate residual signals from the transform coefficients (S1310).

[0249] The video decoding device applies a non-separable inverse quadratic transform kernel to the transform coefficients to generate the master transform coefficients. The video decoding device implicitly acquires the master and inverse transform kernel pairs in the vertical and horizontal directions for each transform block. The video decoding device applies the master and inverse transform kernel pairs in the vertical and horizontal directions to the master transform coefficients to generate the residual signal.

[0250] When the specific conditions mentioned above are met (DCT-2 is used as a separable master transform, LFNST is used as a non-separable second transform, and the transform block has a preset size), the video decoding device applies a non-separable master-inverse transform kernel to the transform coefficients to generate residual signals.

[0251] When the inseparable inverse transform is not performed (No in S1304), the video decoding device performs the following steps.

[0252] The video decoding device implicitly or explicitly acquires the master and inverse transform pairs in the vertical and horizontal directions for the transform blocks (S1320).

[0253] The video decoding device applies a vertical and horizontal master-inverse transform check to the transform coefficients to generate a residual signal (S1322).

[0254] Although the flowcharts / timing diagrams in this specification describe the processes as being executed sequentially, this is merely an illustrative representation of the technical concept of the embodiments of this disclosure. In other words, those skilled in the art can modify and adapt the order of the processes described in the flowcharts / timing diagrams, or execute more than one process within each process, without departing from the essential characteristics of the embodiments of this disclosure. Therefore, the flowcharts / timing diagrams are not limited to a time sequence order.

[0255] It should be understood that the exemplary embodiments described above can be implemented in many different ways. The functions or methods described in one or more examples can be implemented by hardware, software, firmware, or any combination thereof. It should be understood that functional components described in this specification are labeled "...unit" to particularly emphasize their implementation independence.

[0256] On the other hand, the various functions or methods described in this embodiment can also be implemented by one or more instructions stored in a processor-readable and executable non-transitory recording medium. Non-transitory recording media include all kinds of storage devices that store data in a form readable by a computer system. For example, non-transitory recording media include erasable programmable read-only memory (EPROM), flash memory drives, optical disc drives, magnetic hard disk drives, solid-state drives (SSDs), and other recording media.

[0257] The above description is merely an exemplary depiction of the technical concept of this embodiment. Those skilled in the art can make various modifications and variations without departing from the essential characteristics of this embodiment. Therefore, the embodiments disclosed herein are intended to illustrate the technical concept and not to limit it. Any interpretation based on these embodiments should not constitute a limitation on the scope of the technical concept of this embodiment. The scope of protection of this embodiment should be interpreted according to the claims, and all technical concepts within their equivalent scope should be interpreted as being included within the scope of the rights of this embodiment.

[0258] (Explanation of reference numerals in the attached image) 530: Inverse Transformation Unit 540: Forecasting Department 702: Prediction Unit Determination Section 704: Prediction Technology Determination Department 706: Prediction Model Determination Department 710: Inverse Transformation Unit Determination Section 712: Inverse Transform Kernel Determinism 714: Inverse Transformation Execution Unit.

[0259] Cross-reference to related applications This patent application claims priority to Korean Patent Application No. 10-2023-0050448, filed on April 18, 2023, and Korean Patent Application No. 10-2024-0047746, filed on April 9, 2024, the contents of which are incorporated herein by reference.

Claims

1. A method for reconstructing a current block executed by a video decoding device, characterized in that, Includes the following steps: Obtain the inverse-quantized transform coefficients for the transform block of the current block; Decoding non-separable transform information from a bitstream, wherein the non-separable transform information indicates whether a non-separable inverse transform is applied, and the non-separable inverse transform represents a non-separable primary inverse transform or a non-separable secondary inverse transform; and Confirm whether the inseparable transformation information indicates the execution of the inseparable inverse transformation. When the inseparable transformation information indicates that the inseparable inverse transformation should be performed, the following steps are included: Based on the prediction mode of the current block, derive the intra-prediction mode of the transform block; and Based on the size of the transform block and the intra-frame prediction mode, determine the non-separable inverse transform kernel corresponding to the non-separable primary inverse transform kernel or the non-separable secondary inverse transform kernel.

2. The method according to claim 1, characterized in that, The inseparable transformation information is a flag or index indicating whether the inseparable inverse transformation is applied.

3. The method according to claim 1, characterized in that, When the inseparable transformation information indicates that the inseparable inverse transformation should be performed, the following steps are also included: The inseparable master-inverse transform kernel is applied to the transform coefficients to generate the residual signal.

4. The method according to claim 1, characterized in that, When the inseparable transformation information indicates that the inseparable inverse transformation should be performed, the following steps are also included: The inseparable second-order inverse transform kernel is applied to the transform coefficients to generate the master transform coefficients; The transformation block implicitly acquires the master and inverse transformation kernels in the vertical and horizontal directions; and The main transform coefficients are checked by both vertical and horizontal inverse main transforms to generate residual signals.

5. The method according to claim 1, characterized in that, When the inseparable transformation information does not indicate the execution of the inseparable inverse transformation, the following steps are also included: Obtain the main and inverse transform pairs in the vertical and horizontal directions for the transform block; and The transformation coefficients are checked by applying the main and inverse transformations in the vertical and horizontal directions to generate residual signals.

6. The method according to claim 1, characterized in that, The steps for determining the inseparable inverse transform kernel include: Select a set of modes including the intra-prediction modes of the transform block from a preset set of multiple modes; and The kernel set is selected based on the size of the transform block and the selected mode set.

7. The method according to claim 6, characterized in that, In the step of determining the inseparable inverse transform kernel If the kernel set includes multiple inverse transform kernel candidates, then the candidate indicated by the inseparable transform information among the multiple inverse transform kernel candidates is determined as the inseparable inverse transform kernel.

8. The method according to claim 1, characterized in that, When the prediction mode is matrix-weighted intra prediction (MIP) mode, the steps for deriving the intra prediction mode of the transform block include: Obtain a preset directional pattern or a preset non-directional pattern corresponding to the matrix used in the MIP mode; and Set the intra-frame prediction mode of the transform block to the preset directional mode or the preset non-directional mode.

9. The method according to claim 1, characterized in that, When the prediction mode is MIP mode, the steps for deriving the intra-prediction mode of the transform block include: Generate the intermediate stage prediction signal or the final prediction signal of the current block according to the MIP mode; A histogram of oriented gradients (HoG) is generated based on the prediction signals from the intermediate stages or the final prediction signal, and a directional prediction pattern corresponding to the direction with the highest accumulation is derived from the HoG; and Set the intra-frame prediction mode of the transform block to the derived directional prediction mode.

10. The method according to claim 1, characterized in that, When the prediction mode is a geometric segmentation mode, the steps for deriving the intra-frame prediction mode of the transform block include: Generate a prediction signal for the current block based on the geometric segmentation pattern; The steps include generating a HoG based on the predicted signal, deriving the directional prediction pattern corresponding to the direction with the highest accumulation based on the HoG, and... Set the intra-frame prediction mode of the transform block to the derived directional prediction mode.

11. The method according to claim 1, characterized in that, In the step of deriving the intra-prediction mode of the transform block, If the prediction mode is a geometric segmentation mode and one of the sub-regions segmented according to the geometric segmentation mode uses intra-prediction, then the intra-prediction mode of the transform block is set to the intra-prediction mode of the sub-region.

12. The method according to claim 1, characterized in that, In the step of deriving the intra-prediction mode of the transform block, If the prediction mode is a geometric segmentation mode and multiple sub-regions segmented according to the geometric segmentation mode all use intra-prediction, then the intra-prediction mode of the transform block is set to the intra-prediction mode of the sub-region with the largest area.

13. A method for encoding a current block performed by a video encoding apparatus, characterized in that, Includes the following steps: Obtain the residual signal for the transformed block of the current block; The intra-frame prediction mode of the transform block is derived based on the prediction mode of the current block; Based on the size of the transform block and the intra-frame prediction mode, determine the inseparable transform kernel corresponding to the inseparable primary transform kernel or the inseparable secondary transform kernel; The non-separable master transform kernel is applied to the residual signal to generate the first master transform coefficients; For the aforementioned transformation block, determine the main transformation pairs in the vertical and horizontal directions; The master transform check is applied to the residual signal to generate the second master transform coefficients; as well as The inseparable quadratic transform kernel is applied to the second master transform coefficients to generate quadratic transform coefficients.

14. The method according to claim 13, characterized in that, It also includes the following steps: The inseparable transformation information is determined based on the second principal transform coefficients and the second-order transform coefficients, wherein the inseparable transformation information indicates whether an inseparable transform is applied, the inseparable transform represents an inseparable second-order transform, and an inseparable principal transform is indicated when a preset condition is met; and The inseparable transformation information is encoded.

15. The method according to claim 14, characterized in that, The inseparable transformation information is a flag or index indicating whether the inseparable transformation is applied.

16. The method according to claim 14, characterized in that, It also includes the following steps: The first primary transform coefficient, the second primary transform coefficient, or the secondary transform coefficient are encoded based on the inseparable transform information.

17. A computer-readable recording medium storing a bitstream generated by a video coding method, characterized in that, The video encoding method includes the following steps: Obtain the residual signal for the transformed block of the current block; The intra-frame prediction mode of the transform block is derived based on the prediction mode of the current block; Based on the size of the transform block and the intra-frame prediction mode, determine the inseparable transform kernel corresponding to the inseparable main transform kernel or the inseparable second inverse transform kernel; The non-separable master transform kernel is applied to the residual signal to generate the first master transform coefficients; The main transformation pairs in the vertical and horizontal directions are determined explicitly or implicitly for the transformation block; The master transform check is applied to the residual signal to generate the second master transform coefficients; as well as The inseparable quadratic transform kernel is applied to the second master transform coefficients to generate quadratic transform coefficients.

Images