Methods for adaptive correction of inter-component prediction signals in inter-frame prediction

By utilizing the correlation between luminance and chrominance components in video coding to correct the chrominance block prediction signal, the coding efficiency and quality issues during inter-frame prediction are resolved, resulting in more efficient video coding and better video quality.

CN123125867APending Publication Date: 2026-11-17HYUNDAI MOTOR CO LTD +2
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
CN202580029569.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2024-04-22
Filing Date
2025-04-16
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

Existing video coding technologies struggle to efficiently correct the predicted signals of chroma blocks during inter-frame prediction, resulting in insufficient coding efficiency and video quality.

Method used

By obtaining the correlation filter coefficients between the luminance and chrominance components, a chrominance correction signal is generated, and a weighted sum is performed using weights to correct the prediction signal of the current chrominance block in order to generate a reconstructed block.

Benefits of technology

It improves video coding efficiency and enhances video quality, especially in chroma block reconstruction during inter-frame prediction.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123125867A_ABST
    Figure CN123125867A_ABST
Patent Text Reader

Abstract

This embodiment provides a method for adaptively correcting inter-component prediction signals in inter-frame prediction. In this embodiment, a video decoding device determines the inverse transform unit of the current chroma block and obtains the chroma prediction signal of the inverse transform unit and the luminance prediction signal of the corresponding luma block. The video decoding device obtains filter coefficients representing the correlation between the luminance prediction signal and the chroma prediction signal, and generates the chroma correction signal of the inverse transform unit by applying the filter coefficients to the luminance reconstruction signal of the corresponding luma block. The video decoding device obtains weights and performs a weighted summation on the chroma prediction signal and the chroma correction signal based on the weights to generate the final chroma prediction signal of the inverse transform unit.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This disclosure relates to a video coding method and apparatus that performs transform or inverse transform in a geometric segmentation mode. Background Technology

[0002] The statements in this section are provided only as background information in connection with this disclosure and do not necessarily constitute prior art.

[0003] Assuming that unprocessed or uncompressed video data is much larger than audio or still image data, storing or transmitting video data in its raw form requires significant hardware resources (including memory).

[0004] Therefore, when storing or transmitting video data, encoders are typically used to compress the video data for storage or transmission, and decoders receive the compressed video data and perform decompression and reconstruction. Such video compression techniques include H.264 / AVC, High Efficiency Video Coding (HEVC), and Versatile Video Coding (VVC), which improves the coding efficiency of HEVC by approximately 30% or more.

[0005] However, with the continuous increase in video size, resolution, and frame rate, and the corresponding increase in the amount of data to be encoded, new compression techniques are needed that are more efficient in encoding and more effective in improving video quality than existing compression techniques. In particular, when predicting the current block based on inter-frame prediction, an efficient method for performing chroma component prediction is required. Summary of the Invention

[0006] Technical issues

[0007] This disclosure aims to provide a video coding method and apparatus, which, when predicting the current chroma block based on inter-frame prediction, corrects the prediction signal of the current chroma block based on the relationship between the corresponding luminance component and chrominance component, and adds the corrected prediction signal to the residual signal, thereby generating a reconstructed block of the current chroma block.

[0008] Technical solution

[0009] At least one aspect of this disclosure provides a method for reconstructing a current chroma block, performed by a video decoding apparatus. The method includes: determining an inverse transform unit for the current chroma block, and acquiring a chroma prediction signal of the inverse transform unit and a luminance prediction signal of a corresponding luminance block. Here, the corresponding luminance block is a luminance region located corresponding to the inverse transform unit of the current chroma block. The method further includes: acquiring filter coefficients representing the correlation between the luminance prediction signal and the chroma prediction signal. The method further includes: generating a chroma correction signal for the inverse transform unit by applying the filter coefficients to the luminance reconstruction signal of the corresponding luminance block. The method further includes: acquiring weights, and performing a weighted summation of the chroma prediction signal and the chroma correction signal based on the weights to generate a final chroma prediction signal for the inverse transform unit.

[0010] Another aspect of this disclosure provides a method for encoding a current block, performed by a video encoding apparatus. The method includes: determining an inverse transform unit for the current chroma block, and acquiring a chroma prediction signal of the inverse transform unit and a luminance prediction signal of a corresponding luma block. Here, the corresponding luma block is a luma region located corresponding to the inverse transform unit of the current chroma block. The method further includes: acquiring filter coefficients representing the correlation between the luminance prediction signal and the chroma prediction signal. The method further includes: generating a chroma correction signal for the inverse transform unit by applying the filter coefficients to a luminance reconstruction signal of the corresponding luma block. The method further includes: acquiring weights, and performing a weighted summation of the chroma prediction signal and the chroma correction signal based on the weights to generate a final chroma prediction signal for the inverse transform unit.

[0011] Another aspect of this disclosure provides a method for providing video data to a video decoding apparatus. The method includes: encoding the video data into a bitstream, and transmitting the bitstream to the video decoding apparatus. Encoding the video data includes: determining the inverse transform unit of a current chroma block, and acquiring a chroma prediction signal of the inverse transform unit and a luminance prediction signal of a corresponding luma block. Here, the corresponding luma block is a luma region located corresponding to the inverse transform unit of the current chroma block. Encoding the video data further includes: acquiring filter coefficients representing the correlation between the luminance prediction signal and the chroma prediction signal, and generating a chroma correction signal of the inverse transform unit by applying the filter coefficients to a luminance reconstruction signal of the corresponding luma block. Encoding the video data further includes: acquiring weights, and performing a weighted summation of the chroma prediction signal and the chroma correction signal based on the weights to generate a final chroma prediction signal of the inverse transform unit.

[0012] Beneficial effects

[0013] As described above, this disclosure provides a video coding method and apparatus. When predicting the current chroma block based on inter-frame prediction, the predicted signal of the current chroma block is corrected based on the relationship between the corresponding luminance and chrominance components, and the corrected predicted signal is added to the residual signal to generate a reconstructed block of the current chroma block. Therefore, this video coding method and apparatus improve video coding efficiency and enhance video quality. Attached Figure Description

[0014] Figure 1 This is a block diagram of a video encoding apparatus that can implement the technology disclosed herein.

[0015] Figure 2 This demonstrates a method for partitioning blocks using a quadtree plus binary tree ternary tree (QTBTTT) structure.

[0016] Figure 3a and Figure 3b It shows a variety of intra-frame prediction modes, including the wide-angle intra-frame prediction mode.

[0017] Figure 4 The adjacent blocks of the current block are shown.

[0018] Figure 5 This is a block diagram of a video decoding apparatus that can implement the technology disclosed herein.

[0019] Figure 6 This is a block diagram showing a portion of a video decoding apparatus according to embodiments of the present disclosure.

[0020] Figure 7 This is an exemplary view illustrating the correction of the prediction signal for the chromaticity component according to an embodiment of the present disclosure.

[0021] Figure 8 This is an exemplary view showing the positions of samples of chromaticity components and the corresponding positions of luminance samples according to embodiments of the present disclosure.

[0022] Figure 9 This is an exemplary view showing the positions of adjacent blocks according to embodiments of the present disclosure.

[0023] Figure 10 This is an exemplary view showing corresponding brightness blocks according to embodiments of the present disclosure.

[0024] Figure 11 This is an exemplary view illustrating the process of determining the inverse transform kernel according to embodiments of the present disclosure.

[0025] Figure 12 This is an exemplary view illustrating the process of determining the inverse transform kernel according to another embodiment of this disclosure.

[0026] Figure 13This is a flowchart illustrating a method for encoding a current chroma block using a video encoding apparatus according to an embodiment of the present disclosure.

[0027] Figure 14 This is a flowchart illustrating a method for reconstructing the current chroma block using a video decoding apparatus according to an embodiment of the present disclosure. Detailed Implementation

[0028] In the following description, some embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In the following description, the same reference numerals indicate the same elements, although these elements are shown in different drawings. Furthermore, in the following description of some embodiments, detailed descriptions of relevant known components and functions may be omitted for clarity and brevity when it is thought to obscure the subject matter of the present disclosure.

[0029] Figure 1 This is a block diagram of a video encoding apparatus that can implement the technology disclosed herein. Referring below, Figure 1 The illustration depicts a video encoding device and its components.

[0030] The encoding device may include an image segmenter 110, a predictor 120, a subtractor 130, a transformer 140, a quantizer 145, a reordering unit 150, an entropy encoder 155, an inverse quantizer 160, an inverse transformer 165, an adder 170, a loop filter unit 180, and a memory 190.

[0031] Each component of the encoding device can be implemented as hardware or software, or as a combination of hardware and software. Furthermore, the function of each component can be implemented as software, and a microprocessor can be implemented to execute the software function corresponding to each component.

[0032] A video consists of one or more sequences of images. Each image is divided into multiple regions, and encoding is performed on each region. For example, an image is divided into one or more tiles and / or stripes. Here, one or more tiles can be defined as a tile group. Each tile and / or strip is divided into one or more coding tree units (CTUs). Furthermore, each CTU is divided into one or more coding units (CUs) through a tree structure. Information applied to each coding unit (CU) is encoded as the CU syntax, while common information applied to the CUs included in a CTU is encoded as the CTU syntax. Furthermore, common information applied to all blocks in a strip is encoded as the stripe header syntax, and information applied to all blocks constituting one or more images is encoded as a picture parameter set (PPS) or picture header. Furthermore, information commonly referenced by multiple images is encoded as a sequence parameter set (SPS). Furthermore, information commonly referenced by one or more SPSs is encoded as a video parameter set (VPS). Additionally, common information applied to a tile or tile group can also be encoded as a tile or tile group header syntax. The syntax included in SPS, PPS, strip headers, blocks, or block group headers can be referred to as advanced syntax.

[0033] Image splitter 110 determines the size of the coding tree unit (CTU). Information about the CTU size is encoded into SPS or PPS syntax and passed to the video decoding device.

[0034] Image segmenter 110 segments each image constituting the video into multiple coding tree units (CTUs) of predetermined size, and then recursively segments the CTUs using a tree structure. The leaf nodes in the tree structure become coding units (CUs), which are the basic units of coding.

[0035] A tree structure can be a quadtree (QT), where a top-level node (or parent node) is divided into four bottom-level nodes (or child nodes) of the same size. A tree structure can also be a binary tree (BT), where a top-level node is divided into two bottom-level nodes. A tree structure can also be a ternary tree (TT), where a top-level node is divided into three bottom-level nodes in a 1:2:1 ratio. A tree structure can also be a combination of two or more structures from the QT, BT, and TT structures. For example, a quadtree plus binary tree (QTBT) structure can be used, or a quadtree plus binary ternary tree (QTBTTT) structure can be used. Here, adding a binary ternary tree (BTTT) to the tree structure is called a multi-type tree (MTT).

[0036] Figure 2 This is a diagram used to describe the method of segmenting blocks using the QTBTTT structure.

[0037] like Figure 2As shown, the CTU can first be segmented into a QT structure. Quadtree segmentation can be performed recursively until the size of the segmented block reaches the minimum block size (MinQTSize) of the leaf node allowed in the QT. A first flag (QT_split_flag) indicating whether each node of the QT structure is segmented into the four nodes below is encoded by the entropy encoder 155 and signaled to the video decoding device. When the leaf node of the QT is not larger than the maximum block size (MaxBTSize) of the root node allowed in the BT, the leaf node can be further segmented into at least one of the BT structure and the TT structure. Multiple segmentation directions can exist in the BT structure and / or the TT structure. For example, there can be two directions, namely, the direction in which the block of the corresponding node is horizontally segmented and the direction in which the block of the corresponding node is vertically segmented. Figure 2 As shown, when MTT splitting begins, a second flag (mtt_split_flag) indicating whether a node is split, an additional flag indicating the splitting direction (vertical or horizontal), and / or a flag indicating the splitting type (binary or ternary) when a node is split are encoded by the entropy encoder 155 and signaled to the video decoding device.

[0038] Alternatively, before encoding the first flag (QT_split_flag) indicating whether each node is split into the four lower-level nodes, a CU flag (split_cu_flag) indicating whether the node is split can also be encoded. When the value of the CU flag (split_cu_flag) indicates that each node is not split, the block of the corresponding node becomes a leaf node in the split tree structure and becomes a CU, which is the basic unit of encoding. When the value of the CU flag (split_cu_flag) indicates that each node is split, the video encoding device first begins encoding the first flag using the above scheme.

[0039] When QTBT is used as another example of a tree structure, two types can exist: the type where the corresponding node's block is horizontally split into two blocks of the same size (i.e., symmetrical horizontal split), and the type where the corresponding node's block is vertically split into two blocks of the same size (i.e., symmetrical vertical split). A flag (split_flag) indicating whether each node in the BT structure is split into lower-level blocks and split type information indicating the split type are encoded by the entropy encoder 155 and transmitted to the video decoding device. Additionally, there can be a type where the corresponding node's block is split into two blocks that are asymmetrically divided. Asymmetrical forms can include the form where the corresponding node's block is split into two rectangular blocks with a size ratio of 1:3, or it can also include the form where the corresponding node's block is split along a diagonal direction.

[0040] The CU can have various sizes depending on the QTBT or QTBTTT segmentation from the CTU. In the following text, the block to be encoded or decoded corresponding to the CU (i.e., the leaf node of the QTBTTT) is referred to as the "current block". Due to the QTBTTT segmentation, the current block can be rectangular in shape in addition to being square.

[0041] Predictor 120 predicts the current block to generate a prediction block. Predictor 120 includes an intra-frame predictor 122 and an inter-frame predictor 124.

[0042] Typically, each current block in an image can be predicted and encoded. This prediction can usually be performed using either intra-frame prediction (using data from the image that includes the current block) or inter-frame prediction (using data from an image encoded before the image that includes the current block). Inter-frame prediction includes both one-way and two-way prediction.

[0043] Intra-predictor 122 predicts pixels in the current block by using pixels (reference pixels) located at adjacent positions in the current image, including the current block. Multiple intra-prediction modes exist depending on the prediction direction. For example, such as... Figure 3a As shown, multiple intra-frame prediction modes can include two non-directional modes, including Planar mode and DC mode, and can include 65 directional modes. Depending on the prediction mode, the neighboring pixels to be used and the arithmetic equations are defined differently.

[0044] To perform effective orientation prediction for the current block with a rectangular shape, additional methods can be used. Figure 3b The directional modes indicated by the dashed arrows (intra-prediction modes #67 to #80, #-1 to #-14) are shown in the center. These directional modes can be referred to as "wide-angle intra-prediction modes." Figure 3b In the diagram, the arrow indicates the corresponding reference sample used for prediction, not the prediction direction. The prediction direction is opposite to the direction indicated by the arrow. When the current block has a rectangular shape, the wide-angle intra-frame prediction mode is a mode that performs prediction in the opposite direction to the specific direction mode without additional bit transmission. In this case, some wide-angle intra-frame prediction modes available for the current block can be determined by the ratio of the width to the height of the current block with a rectangular shape. For example, when the current block has a rectangular shape with a height less than its width, wide-angle intra-frame prediction modes with an angle less than 45 degrees (intra-frame prediction modes #67 to #80) are available. When the current block has a rectangular shape with a width greater than its height, wide-angle intra-frame prediction modes with an angle greater than -135 degrees are available.

[0045] Intra-predictor 122 can determine the intra-prediction to be used for encoding the current block. In some examples, intra-predictor 122 can encode the current block using multiple intra-prediction modes, and can also select the appropriate intra-prediction mode to use from the tested modes. For example, intra-predictor 122 can calculate rate-distortion values ​​using rate-distortion analysis for multiple tested intra-prediction modes, and can also select the intra-prediction mode with the best rate-distortion characteristics from the tested modes.

[0046] Intra-predictor 122 selects one intra-prediction mode from multiple intra-prediction modes and predicts the current block using neighboring pixels (reference pixels) determined according to the selected intra-prediction mode and an arithmetic equation. Information about the selected intra-prediction mode is encoded by entropy encoder 155 and transmitted to the video decoding device.

[0047] Inter-frame predictor 124 generates a predicted block for the current block using a motion compensation process. Inter-frame predictor 124 searches for the most similar block to the current block in a reference image that was encoded and decoded earlier than the current image, and generates a predicted block for the current block using the searched block. Furthermore, a motion vector (MV) corresponding to the displacement between the current block in the current image and the predicted block in the reference image is generated. Typically, motion estimation is performed on the luma component, and the motion vector calculated based on the luma component is used for both the luma and chroma components. Motion information, including information about the reference image and information about the motion vector used to predict the current block, is encoded by entropy encoder 155 and transmitted to the video decoding device.

[0048] The inter-frame predictor 124 can also perform interpolation on a reference picture or reference block to improve prediction accuracy. In other words, it interpolates subsamples between two consecutive integer samples by applying filter coefficients to multiple consecutive integer samples, including two integer samples. When performing a process of searching for the most similar block to the current block on the interpolated reference picture, the motion vector can be represented with fractional-unit precision, rather than integer-sample-unit precision. The precision or resolution of the motion vector can be set differently for each target region to be encoded (e.g., stripe, tile, CTU, CU, etc.). When this adaptive motion vector resolution (AMVR) is applied, information about the motion vector resolution to be applied to each target region should be signaled for each target region. For example, when the target region is a CU, information about the motion vector resolution applied to each CU is signaled. The motion vector resolution information can be information representing the precision of the motion vector interpolation, which will be described below.

[0049] Simultaneously, the inter-frame predictor 124 can perform inter-frame prediction using bidirectional prediction. In the case of bidirectional prediction, two reference images and two motion vectors representing the location of the block most similar to the current block in each reference image are used. The inter-frame predictor 124 selects a first reference image and a second reference image from reference image list 0 (RefPicList0) and reference image list 1 (RefPicList1), respectively. The inter-frame predictor 124 also searches for the block most similar to the current block in each reference image to generate a first reference block and a second reference block. Furthermore, the predicted block for the current block is generated by averaging or weighted averaging the first and second reference blocks. In addition, motion information including information about the two reference images used to predict the current block and information about the two motion vectors is transmitted to the entropy encoder 155. Here, reference image list 0 may consist of images from the pre-reconstructed images that are preceding the current image in display order, and reference image list 1 may consist of images from the pre-reconstructed images that are following the current image in display order. However, although not particularly limited to this, pre-reconstructed images that are following the current image in display order may also be additionally included in reference image list 0. Conversely, pre-reconstructed images preceding the current image may also be additionally included in the reference image list 1.

[0050] Various methods can be used to minimize the number of bits consumed in encoding motion information.

[0051] For example, when the reference image and motion vector of the current block are the same as those of the adjacent blocks, the encoder can identify the information of the adjacent blocks to transmit the motion information of the current block to the video decoding device. This method is called merging mode.

[0052] In merge mode, the inter-frame predictor 124 selects a predetermined number of merge candidate blocks (hereinafter referred to as "merge candidates") from the neighboring blocks of the current block.

[0053] As adjacent blocks used to derive merge candidates, such as Figure 4 As shown, all or part of the left block A0, lower left block A1, upper block B0, upper right block B1, and upper left block B2 adjacent to the current block in the current image can be used. Furthermore, blocks located in a reference image different from the current image (which may be the same as or different from the reference image used to predict the current block) can also be used as merge candidates. For example, blocks co-located with or adjacent to the current block in the reference image can be additionally used as merge candidates. If the number of merge candidates selected by the above method is less than a preset number, a zero vector is added to the merge candidates.

[0054] The inter-frame predictor 124 configures a merge list including a predetermined number of merge candidates by using neighboring blocks. It selects merge candidates from the merge list that will be used as motion information for the current block and generates merge index information to identify the selected candidates. The generated merge index information is encoded by the entropy encoder 155 and transmitted to the video decoding device.

[0055] The merge skip mode is a special case of the merge mode. After quantization, when all transform coefficients used for entropy coding are close to zero, only the adjacent block selection information is transmitted, and the residual signal is not transmitted. By using the merge skip mode, relatively high coding efficiency can be achieved for images with slight motion, still images, and screen content images.

[0056] In the following text, merge mode and merge skip mode are collectively referred to as merge / skip mode.

[0057] Another method for encoding motion information is the Advanced Motion Vector Prediction (AMVP) pattern.

[0058] In AMVP mode, the inter-frame predictor 124 derives motion vector predictor candidates for the current block's motion vectors by using neighboring blocks of the current block. As neighboring blocks used to derive the motion vector predictor candidates, [the following can be used]. Figure 4 The current image shown includes all or part of the left block A0, lower left block A1, upper block B0, upper right block B1, and upper left block B2 adjacent to the current block. Additionally, blocks located in a reference image (which may be the same as or different from the reference image used to predict the current block) that are different from the current block can also be used as neighboring blocks for deriving motion vector predictor candidates. For example, blocks in the same position as the current block or adjacent to the same-position block in the reference image can be used. If the number of motion vector candidates selected by the above method is less than a preset number, a zero vector is added to the motion vector candidates.

[0059] The inter-frame predictor 124 derives motion vector predictor candidates using motion vectors from neighboring blocks, and determines the motion vector predictor for the current block's motion vector using these candidates. Furthermore, the motion vector difference is calculated by subtracting the motion vector predictor from the motion vector of the current block.

[0060] Motion vector predictors can be obtained by applying predefined functions (e.g., center value and average value calculations) to motion vector predictor candidates. In this case, the video decoding device is also aware of the predefined functions. Furthermore, since the neighboring blocks used to derive the motion vector predictor candidates are blocks that have already been encoded and decoded, the video decoding device may also already know the motion vectors of the neighboring blocks. Therefore, the video encoding device does not need to encode information for identifying motion vector predictor candidates. Thus, in this case, information about the motion vector differences and information about the reference image used to predict the current block are encoded.

[0061] Alternatively, a motion vector predictor can be determined by selecting any of the motion vector predictor candidates. In this case, the information used to identify the selected motion vector predictor candidate is additionally encoded along with the information on the motion vector difference and the information on the reference image used to predict the current block.

[0062] Subtractor 130 generates a residual block by subtracting the prediction block generated by intra-predictor 122 or inter-predictor 124 from the current block.

[0063] Transformer 140 transforms the residual signal in the residual block, which has pixel values ​​in the spatial domain, into transform coefficients in the frequency domain. Transformer 140 can use the total size of the residual block as a transform unit to transform the residual signal in the residual block, or it can divide the residual block into multiple sub-blocks and use these sub-blocks as transform units to perform the transformation. Alternatively, the residual block can be divided into two sub-blocks, one as a transform region and the other as a non-transform region, so that only the transform region sub-block is used as a transform unit to transform the residual signal. Here, the transform region sub-block can be one of two rectangular blocks with a 1:1 size ratio based on the horizontal (or vertical) axis. In this case, a flag (cu_sbt_flag) indicates that only this sub-block is transformed, and orientation (vertical / horizontal) information (cu_sbt_horizontal_flag) and / or position information (cu_sbt_pos_flag) are encoded by entropy encoder 155 and signaled to the video decoding device. Furthermore, the size of the transform region sub-block can have a 1:3 size ratio based on the horizontal (or vertical) axis. In this case, the flag (cu_sbt_quad_flag) corresponding to the segmentation is additionally encoded by the entropy encoder 155 and signaled to the video decoding device.

[0064] Simultaneously, the transformer 140 can perform transformations on the residual block in both the horizontal and vertical directions. Various types of transformation functions or transformation matrices can be used for the transformation. For example, a pair of transformation functions for horizontal and vertical transformations can be defined as a multiple transform set (MTS). The transformer 140 can select the transformation function pair with the highest transformation efficiency from the MTS and can transform the residual block in each of the horizontal and vertical directions. The information of the transformation function pair in the MTS (mts_idx) is encoded by the entropy encoder 155 and signaled to the video decoding device.

[0065] Quantizer 145 uses quantization parameters to quantize the transform coefficients output from transformer 140 and outputs the quantized transform coefficients to entropy encoder 155. Quantizer 145 can also directly quantize the associated residual block for any block or frame without performing a transform. Quantizer 145 can also apply different quantization coefficients (scaling values) based on the position of the transform coefficients within the transform block. The quantization matrix applied to the quantized transform coefficients arranged in a 2D array can be encoded and signaled to the video decoding device.

[0066] The reordering unit 150 can reorder the coefficient values ​​of the quantized residual values.

[0067] The reordering unit 150 can transform a 2D coefficient array into a 1D coefficient sequence by using coefficient scanning. For example, the reordering unit 150 can output a 1D coefficient sequence by scanning the DC coefficients to high-frequency region coefficients using a zigzag scan or a diagonal scan. Depending on the size of the transform unit and the intra-frame prediction mode, a vertical scan that scans the 2D coefficient array along the column direction and a horizontal scan that scans the 2D block-type coefficients along the row direction can also be used instead of a zigzag scan. In other words, the scanning method to be used can be determined from zigzag scan, diagonal scan, vertical scan, and horizontal scan, depending on the size of the transform unit and the intra-frame prediction mode.

[0068] The entropy encoder 155 generates a bitstream by encoding the 1D quantized transform coefficient sequence output from the reordering unit 150 using various encoding schemes, including context-based adaptive binary arithmetic coding (CABAC) and exponential Columbus coding.

[0069] Furthermore, the entropy encoder 155 encodes block-segmentation related information, such as CTU size, CTU flag, QT flag, MTT segmentation type, and MTT segmentation direction, to enable the video decoding device to segment blocks in the same way as the video encoding device. Additionally, the entropy encoder 155 encodes information indicating whether the current block is encoded using intra-frame prediction or inter-frame prediction. The entropy encoder 155 encodes intra-frame prediction information (i.e., information about the intra-frame prediction mode) or inter-frame prediction information (the merge index in merge mode, and information about the reference picture index and motion vector difference in AMVP mode) according to the prediction type. Furthermore, the entropy encoder 155 encodes quantization-related information, namely information about quantization parameters and information about the quantization matrix.

[0070] Inverse quantizer 160 inverse quantizes the quantized transform coefficients output from quantizer 145 to generate transform coefficients. Inverse transformer 165 transforms the transform coefficients output from inverse quantizer 160 from the frequency domain to the spatial domain to reconstruct the residual block.

[0071] Adder 170 adds the reconstructed residual block to the predicted block generated by predictor 120 to reconstruct the current block. Pixels in the reconstructed current block can be used as reference pixels when performing intra-frame prediction for the next sequential block.

[0072] The loop filter unit 180 performs filtering on the reconstructed pixels to reduce block artifacts, ringing artifacts, blur artifacts, etc., that occur due to block-based prediction and transform / quantization. The loop filter unit 180, as an in-loop filter, may include all or some of the following: a deblocking filter 182, a sample adaptive offset (SAO) filter 184, and an adaptive loop filter (ALF) 186.

[0073] Deblocking filter 182 filters the boundaries between reconstructed blocks to remove block artifacts caused by block unit encoding / decoding, while SAO filter 184 and ALF 186 perform additional filtering on the deblocked video. SAO filter 184 and ALF 186 are filters used to compensate for differences between reconstructed and original pixels caused by lossy coding. SAO filter 184 applies an offset in units of CTU to enhance subjective image quality and coding efficiency. On the other hand, ALF 186 performs block unit filtering and compensates for distortion by applying different filters based on the boundaries and degree of variation of the corresponding blocks. Information about the filter coefficients used for ALF can be encoded and signaled to the video decoding device.

[0074] The reconstructed blocks, filtered by deblocking filter 182, SAO filter 184, and ALF 186, are stored in memory 190. When all blocks in an image are reconstructed, the reconstructed image can be used as a reference image for inter-frame prediction of blocks in subsequent images to be encoded.

[0075] Video encoding devices can store the bit stream of encoded video data in a non-transitory storage medium or transmit the bit stream to a video decoding device via a communication network.

[0076] Figure 5 This is a functional block diagram of a video decoding apparatus capable of implementing the technology of this disclosure. Referring below... Figure 5 This describes a video decoding device and its components.

[0077] The video decoding device may include an entropy decoder 510, a reordering unit 515, an inverse quantizer 520, an inverse transformer 530, a predictor 540, an adder 550, a loop filter unit 560, and a memory 570.

[0078] and Figure 1 Similar to video encoding devices, each component of a video decoding device can be implemented as hardware or software, or a combination of both. Furthermore, the function of each component can be implemented as software, and a microprocessor can be implemented to execute the software function corresponding to each component.

[0079] The entropy decoder 510 extracts information related to block segmentation by decoding the bitstream generated by the video encoding device to determine the current block to be decoded, and extracts the prediction information and information about the residual signal needed to reconstruct the current block.

[0080] The entropy decoder 510 determines the size of the CTU by extracting information about the CTU size from the sequence parameter set (SPS) or picture parameter set (PPS), and segments the image into CTUs with the determined size. Furthermore, the CTU is identified as the highest level of the tree structure, i.e., the root node, and segmentation information for the CTU can be extracted to segment the CTU using the tree structure.

[0081] For example, when using the QTBTTT structure to segment the CTU, the first flag (QT_split_flag) related to the QT segmentation is first extracted to split each node into four lower-level nodes. Furthermore, for the nodes corresponding to the leaf nodes of the QT, the second flag (mtt_split_flag), segmentation direction (vertical / horizontal), and / or segmentation type (binary / ternary) related to the MTT segmentation are extracted to segment the corresponding leaf nodes into an MTT structure. As a result, each node below the leaf node of the QT is recursively segmented into a BT or TT structure.

[0082] As another example, when splitting a CTU using a QTBTTT structure, the CU flag (split_cu_flag) indicating whether a CU has been split is extracted. When the corresponding block is split, the first flag (QT_split_flag) can also be extracted. During the splitting process, for each node, recursive MTT splits may occur zero or more times after zero or more recursive QT splits. For example, for a CTU, MTT splits may occur immediately, or conversely, only multiple QT splits may occur.

[0083] As another example, when using a QTBT structure to segment a CTU, the first flag (QT_split_flag) associated with the segmentation of the QT is extracted to split each node into the four nodes of the lower layer. Additionally, flags (split_flag) indicating whether the node corresponding to a leaf node of the QT should be further segmented into BTs and segmentation direction information are extracted.

[0084] Simultaneously, when the entropy decoder 510 determines the current block to be decoded using tree-structured segmentation, it extracts information indicating whether the current block is predicted intra-frame or inter-frame. When the prediction type information indicates intra-frame prediction, the entropy decoder 510 extracts the syntax elements of the intra-frame prediction information (intra-frame prediction mode) for the current block. When the prediction type information indicates inter-frame prediction, the entropy decoder 510 extracts information representing the syntax elements of the inter-frame prediction information, namely, the motion vector and the reference image to which the motion vector is referenced.

[0085] In addition, the entropy decoder 510 extracts quantization-related information and extracts the information of the quantization transform coefficients of the current block as information of the residual signal.

[0086] The reordering unit 515 can reorder the 1D quantized transform coefficient sequence entropy decoded by the entropy decoder 510 into a 2D coefficient array (i.e., a block) in the reverse order of coefficient scanning performed by the video encoding device.

[0087] The inverse quantizer 520 performs inverse quantization on the quantized transform coefficients and uses quantization parameters to perform inverse quantization on the quantized transform coefficients. The inverse quantizer 520 can also apply different quantization coefficients (scaling values) to the quantized transform coefficients arranged in 2D. The inverse quantizer 520 can perform inverse quantization by applying a matrix of quantization coefficients (scaling values) from the video encoding device to a 2D array of quantized transform coefficients.

[0088] The inverse transformer 530 reconstructs the residual signal by inversely transforming the inversely quantized transform coefficients from the frequency domain to the spatial domain, thereby generating the residual block of the current block.

[0089] Furthermore, when the inverse transformer 530 performs an inverse transform on a portion (sub-block) of the transform block, it extracts a flag (cu_sbt_flag) indicating that only the transform block is being transformed, the orientation (vertical / horizontal) information (cu_sbt_horizontal_flag) of the sub-block, and / or the position information (cu_sbt_pos_flag) of the sub-block. The inverse transformer 530 also inversely transforms the transform coefficients of the corresponding sub-block from the frequency domain to the spatial domain to reconstruct the residual signal, and fills the untransformed regions with values ​​of "0" as the residual signal to generate the final residual block for the current block.

[0090] Furthermore, when applying MTS, the inverse transformer 530 uses the MTS information (mts_idx) signaled from the video encoding device to determine the transform function or transform matrix to be applied in each direction, both horizontally and vertically. The inverse transformer 530 also performs an inverse transform on the transform coefficients in the transform block in both the horizontal and vertical directions using the determined transform function.

[0091] Predictor 540 may include intra-predictor 542 and inter-predictor 544. Intra-predictor 542 is activated when the prediction type of the current block is intra-prediction, and inter-predictor 544 is activated when the prediction type of the current block is inter-prediction.

[0092] Intra-predictor 542 determines the intra-prediction mode for the current block from the syntax elements of the intra-prediction modes extracted by the self-entropy decoder 510, among multiple intra-prediction modes. Intra-predictor 542 also uses neighboring reference pixels of the current block to predict the current block based on the intra-prediction mode.

[0093] The inter-frame predictor 544 uses the syntax elements of the inter-frame prediction mode extracted from the entropy decoder 510 to determine the motion vector of the current block and the reference picture to which the motion vector is referenced, and uses the motion vector and the reference picture to predict the current block.

[0094] Adder 550 reconstructs the current block by adding the residual block output from inverse transform 530 to the prediction block output from inter-frame predictor 544 or intra-frame predictor 542. The pixels in the reconstructed current block are used as reference pixels when performing intra-frame prediction on the block to be decoded later.

[0095] The loop filter unit 560, serving as an in-loop filter, may include a deblocking filter 562, a SAO filter 564, and an ALF 566. The deblocking filter 562 performs deblocking filtering on the boundaries between reconstructed blocks to remove blocking artifacts caused by block unit decoding. The SAO filter 564 and ALF 566 perform additional filtering on the reconstructed blocks after deblocking to compensate for differences between reconstructed pixels and original pixels caused by lossy encoding. The filter coefficients of the ALF are determined using information from the filter coefficients decoded from the bitstream.

[0096] The reconstructed blocks, filtered by deblocking filter 562, SAO filter 564, and ALF 566, are stored in memory 570. When all blocks in an image are reconstructed, the reconstructed image can be used as a reference image for inter-frame prediction of blocks in subsequent images to be encoded.

[0097] This disclosure relates, in some embodiments, to the encoding and decoding of video images as described above. More specifically, this disclosure provides a video encoding method and apparatus for, when predicting a current chroma block based on inter-frame prediction, correcting the prediction signal of the current chroma block based on the relationship between the corresponding luminance component and chrominance component, and adding the corrected prediction signal to the residual signal, thereby generating a reconstructed block of the current chroma block.

[0098] The following implementation can be performed by the predictor 120 in the video encoding apparatus. The following implementation can also be performed by the predictor 540 in the video decoding apparatus.

[0099] When encoding the current block, the video encoding apparatus can generate signaling information associated with this embodiment in terms of optimizing rate distortion. The video encoding apparatus can encode this signaling information using the entropy encoder 155 and can transmit the encoded signaling information to the video decoding apparatus. The video decoding apparatus can decode the signaling information associated with the decoding of the current block from the bitstream using the entropy decoder 510.

[0100] In the following description, the term "target block" may be used interchangeably with the current block or coding unit (CU), or may refer to a region of a coding unit.

[0101] Furthermore, a flag value of true indicates that the flag is set to 1. Conversely, a flag value of false indicates that the flag is set to 0.

[0102] Decoder side includes Figure 1 The video encoding apparatus shown includes all or a portion of the inverse quantizer 160, inverse converter 165, predictor 120, adder 170, loop filter unit 180, and memory 190. Alternatively, the decoder side includes... Figure 5The video decoding apparatus shown includes all or some of the following components: inverse quantizer 520, inverse transformer 530, predictor 540, adder 550, loop filter unit 560, and memory 570. For a series of decoding processes, the decoder side of the video encoding apparatus and the decoder side of the video decoding apparatus perform the same operations. The video encoding apparatus determines information related to the operation on the decoder side and signals this determined information to the video decoding apparatus. The video decoding apparatus can decode the signaled information and operate the decoder side based on the decoded information.

[0103] I-1. Transformation Techniques – Main Transformation Techniques

[0104] As mentioned above, for efficient video compression, quantization or scaling can be additionally applied to the residual signal after prediction using various prediction techniques. In this case, transformation techniques can be applied based on the importance of the inherent perceptual visual information in the residual signal to concentrate the residual signal to one side according to its frequency components, after which scaling can be performed. However, when the content (e.g., screen content) is not a natural signal, such frequency-based transformation techniques may be inefficient. In this case, the transformation technique can be omitted, and only scaling can be performed, or encoding / decoding can be performed without applying scaling.

[0105] In HEVC, when applying transforms, DCT-II is used as the transform kernel for transforming the residual signal (hereafter, it is used interchangeably with transform type). However, to apply more suitable transform techniques based on the diversity of residual signal characteristics, Multiple Transform Selection (MTS) can be used. MTS determines the optimal transform type from one or two of several transform types and then transforms the block according to the determined transform type. For example, in VVC, as shown in Table 1, two additional transform types, DCT-VIII and DST-VII, are added in addition to DCT-II, allowing for various ways to transform the residual signal.

[0106] Table 1

[0107] Here, the basis functions constitute the transformation matrix that defines each type of transformation. In the following text, DCT-II, DCT-VIII, and DST-VII are used interchangeably with DCT2, DCT8, and DST7, respectively.

[0108] Meanwhile, the flag used to determine whether MTS is used can be controlled at the block level. Furthermore, whether MTS is used can be controlled using an activation flag at a higher SPS level.

[0109] When MTS is activated in SPS, a CU-level flag indicating whether MTS is applied can be displayed. Here, MTS can be applied to the luma component. The CU-level flag can be displayed when the width and height of the transform block (TB) are both less than or equal to 32 pixels, and the Code Block Flag (CBF), indicating whether there are non-zero values ​​in the transform coefficient level, is true.

[0110] When the CU level flag is 0, DCT2 is used as the nucleus in both the horizontal and vertical directions. Conversely, if the CU level flag is not 0, MTS is applied. For MTS, two schemes can be used: explicit MTS and implicit MTS.

[0111] In an explicit MTS, the kernel used for the TB is explicitly transferred. Typically, the index of the transformation kernel can be transferred. For example, as shown in Table 2, the kernel index mts_idx can be defined.

[0112] Table 2

[0113] Here, trTypeHor and trTypeVer represent the horizontal and vertical transform types, respectively. Furthermore, 0 represents DCT2, 1 represents DST7, and 2 represents DCT8.

[0114] Furthermore, in implicit MTS, for example in the case of intra-block, the transform type can be implicitly determined even without explicitly signaling the MTS. In VVC, as shown in Equation 1, the transform types in the horizontal and vertical directions can be implicitly determined.

[0115] [Formula 1]

[0116] trTypeHor = ( nTbW >= 4 && nTbW <= 16 ) ? DST7 : DCT2

[0117] trTypeVer = ( nTbH >= 4 && nTbH <= 16 ) ? DST7 : DCT2

[0118] Here, nTbW and nTbH represent the width and height of the transform block, respectively.

[0119] Meanwhile, in the next-generation technology-enhanced compression model (ECM) software, the number and types of multiple transform selection (MTS) kernels have been increased, and DST7, DCT8, DCT5, DST4, DST1 and identity transforms have been added.

[0120] I-2. Low-Frequency Non-Separable Transform (LFNST)

[0121] During intra-frame prediction, the LFNST technique applies a quadratic transform to the low-frequency regions of the transform coefficients generated from the main transforms of the transform unit (TU). In terms of coding, the LFNST technique applies a quadratic transform to L of the W×H main transform coefficients, representing the low-frequency main transform coefficients, to generate K (where K≤L) quadratic transform coefficients. Here, the transform kernel of LFNST has a size of L×K. That is, the LFNST technique represents the L low-frequency main transform coefficients out of the W×H main transform coefficients as a 1×L vector, and then generates a 1×K vector by applying an L×K transform kernel. Subsequently, the LFNST technique represents the 1×K vector as a two-dimensional array in the low-frequency regions for subsequent processing (e.g., quantization).

[0122] Compared to the main transform, which applies separate transform kernels in the horizontal and vertical directions, the LFNST technique performs an inseparable transform of a one-dimensional vector.

[0123] Meanwhile, the type of transform kernel can be determined based on the current TU's intra-prediction mode, the TU's size, and the LFNST index (lfnst_idx). For example, in VVC, the transform kernel set can be determined based on the current TU's intra-prediction mode (IntraPredMode) as shown in Table 3.

[0124] Table 3

[0125] Here, IntraPredMode follows Figure 3b Examples are provided. Furthermore, lfnstTrSetIdx is an index representing the kernel set. In Table 3, IntraPredMode values ​​of 81, 82, or 83 indicate the cross-component linear model (CCLM) prediction mode.

[0126] Two types of kernels are defined for each kernel set (lfnstTrSetIdx). The selection of which kernel to use is indicated by the LFNST index. When the LFNST index is 0, LFNST is not performed, and when the LFNST index is 1 or 2, different LFNST kernels within the same kernel set are applied. Since there is an additional kernel set depending on the size of the TU, there are a total of 4 × 2 × 2 = 16 transform kernels. The kernel sizes for LFNST are defined as 16 × 16 and 16 × 48. Furthermore, as shown in Table 4, the kernel size can be adjusted according to the size of the TU.

[0127] Table 4

[0128] In ECM, transform kernel sets can be defined based on the shape of the block and the intra-prediction mode. For example, by taking advantage of the symmetry of square blocks, 35 transform kernel sets are defined for square blocks based on the intra-prediction mode.

[0129] Meanwhile, in VVC, when performing Matrix Weighted Intra Prediction (MIP) on the current TU, the video decoding device maps the intra prediction mode of the current TU to a planar mode, and then determines the transform kernel based on the mapped prediction mode. MIP technology includes the processes of MIP boundary downsampling, matrix-vector multiplication, and MIP prediction upsampling. "MIP boundary downsampling" refers to the process of downsampling a specific number of pixels in the boundary pixels of the current block. "Matrix-vector multiplication" refers to the process of generating a reduced prediction block by multiplying the downsampled pixels (i.e., vectors) by a predetermined matrix. "MIP prediction upsampling" refers to the process of generating the final prediction block by upsampling the reduced prediction block.

[0130] Furthermore, when performing matrix-based intra-frame prediction (MIP) in the Enhanced Compression Model (ECM) as a next-generation technology, the video decoding device obtains a directional prediction mode by applying decoder-side intra-frame mode derivation (DIMD) to the MIP prediction block (i.e., the output of "matrix-vector multiplication"). The video decoding device maps the intra-frame prediction mode of the current TU to the obtained directional prediction mode and then determines the transform kernel based on the mapped prediction mode. The DIMD technique computes the gradient of each sample for the neighboring samples of the current block, generates a gradient histogram (HoG) based on the computed gradients, and derives the prediction mode for intra-frame prediction of the current block based on the generated HoG.

[0131] Meanwhile, in VVC, when the DCT2 / DCT2 transform kernel is used as the main transform for intra-frame prediction TU, the LFNST technique can be used as the secondary transform.

[0132] In ECM, when the block size in the TU based on intra-frame predictive coding is small (e.g., 4×4, 4×8, 8×4, 4×16, 16×4, 8×8, 8×16, 16×8), the non-separable master transform technique can be applied. Furthermore, when the block size is large and the DCT2 / DCT2 transform kernel is applied as the master transform to that block, the LFNST technique can be applied.

[0133] I-4 Non-Separable Master Transform (NSPT)

[0134] The non-separable master transform technique uses a pre-learned transform kernel to perform the non-separable master transform, instead of performing vertical and horizontal transforms on the residual signal of the block predicted by the intra-frame prediction mode.

[0135] The NSPT technique uses non-separable master transform kernels of different sizes depending on the size of the residual block. For example, when the transform block size is 4×4, a 16×16 kernel is applied to generate 4×4 transform coefficients. As another example, when the transform block size is 4×8, a 32×20 kernel is applied to generate 20 transform coefficients. The generated transform coefficients are arranged in the transform block according to a predetermined scan order. When the number of transformed coefficients generated after transformation is less than the number of pixels in the transform block, the remaining areas where no transform coefficients are placed are filled with 0s.

[0136] For an inseparable master transform, the transform kernel can be defined based on the block shape, intra-prediction mode, and NSPT index (nspt_idx). For example, by leveraging the symmetry of a square block, there are 35 transform kernel sets for a square block based on the intra-prediction mode. In this case, the NSPT index (i.e., the LFNST index) can be used to indicate one of the candidate kernels within the transform kernel set.

[0137] The non-separable principal transform can be applied to the luminance component. Furthermore, when applying the non-separable principal transform, a secondary transform (i.e., LFNST) can be omitted.

[0138] In the following text, "prediction block", "prediction unit block", "decoding block" and "block to be decoded" are used interchangeably.

[0139] In the following text, "luminance component block" and "luminance block" are used interchangeably.

[0140] In the following text, "corresponding brightness block", "corresponding position brightness block", "corresponding brightness area" and "corresponding position brightness area" can be used interchangeably.

[0141] In the following text, "chroma component block" and "chroma block" are used interchangeably.

[0142] The following embodiments are described with an emphasis on a video decoding apparatus, but they can be implemented in the same or similar manner in a video encoding apparatus. Alternatively, the following embodiments are described with an emphasis on the decoder side of a video decoding apparatus, but they can be implemented in the same or similar manner on the decoder side of a video encoding apparatus.

[0143] II. Embodiments according to this disclosure

[0144] Figure 6 This is a detailed block diagram illustrating a portion of a video decoding apparatus according to an embodiment of the present disclosure.

[0145] The video decoding apparatus according to this embodiment can determine prediction units and transformation units, and for the current block corresponding to the determined units, perform prediction and inverse transformation using the determined prediction techniques and prediction modes, thereby ultimately generating a reconstructed block of the current block. Figure 6The example shown can be performed by the inverse transformer 530, predictor 540, and adder 550 of a video decoding device. Meanwhile, with Figure 6 The same operations shown can be performed by the inverse transformer 165, image segmenter 110, predictor 120, and adder 170 of the video encoding apparatus. In this case, the video decoding apparatus uses the encoded information parsed from the bitstream, while the video encoding apparatus can use encoded information set from a higher level for rate-distortion optimization. For convenience, this embodiment will be described in focus on the video decoding apparatus in the following description.

[0146] like Figure 5 For example, predictor 540 includes an intra-frame predictor 542 and an inter-frame predictor 544 depending on the prediction technique; however, as Figure 6 As shown, the predictor 540 may include all or a portion of the prediction unit determiner 602, prediction technique determiner 604, prediction mode determiner 606, and prediction executor 608.

[0147] Prediction Unit Determiner 602 determines the prediction unit (PU). Prediction Technique Determiner 604 determines the prediction technique for the prediction unit (e.g., intra-frame prediction, inter-frame prediction, or intra-block copy (IBC) mode, palette mode, a hybrid intra-frame and inter-frame prediction technique, etc.). Prediction Mode Determiner 606 determines a detailed prediction mode for the prediction technique. Prediction Executor 608 generates the prediction block for the current block based on the determined prediction mode.

[0148] The inverse transformer 530 includes all or a portion of the following: an inverse transformer unit determiner 610, an inverse transformer kernel determiner 612, and an inverse transformer executor 614. The inverse transformer unit determiner 610 determines the transformer unit of the inverse-quantized signal (i.e., the inverse-quantized transformer coefficients) of the current block. The inverse transformer kernel determiner 612 determines the inverse transformer kernel, and the inverse transformer executor 614 generates residual samples by performing an inverse transformer on the transformer unit represented by the inverse-quantized transformer coefficients.

[0149] In the following text, the transform unit (TU) for encoding can be used interchangeably with the transform block. The inverse transform unit and inverse transform block for decoding correspond to the TU and the transform block, respectively. Accordingly, the inverse transform unit and inverse transform block can be used interchangeably with the TU and the transform block.

[0150] The transform unit indicates the unit that determines whether to perform an inverse transform on the transform coefficients and transmits information about the inverse transform. The video decoding apparatus determines the core to be applied to the transform unit and performs an inverse transform on the transform coefficients based on the determined core. At least one core can be determined for a transform unit. According to an embodiment, an Nth-order inverse transform can be applied, wherein the size of the transform unit and the inverse transform core can be different in each order. For example, for the current transform unit block (hereinafter referred to as the transform block or current transform block), when applying an Nth-order inverse transform, the size of the core used for each order of inverse transform can be different. That is, the video decoding apparatus can apply an inverse transform to some input transform coefficients.

[0151] Adder 550 generates a reconstructed block by adding the predicted block and the residual sample. The reconstructed block can be stored in memory and subsequently used for the prediction of another block.

[0152] When the input video's color format is YUV (YUV420, YUV411, YUV422, YUV444, etc.), the video decoding device can perform prediction and reconstruction of the luminance component, and then it can perform prediction and reconstruction of the chrominance component. That is, it can be... Figure 6 The components shown sequentially reconstruct the luminance and chrominance components. Simultaneously, when the input video's color format is RGB, the video encoding device can perform a color format conversion from RGB to YUV, and then encode the converted video. Here, in the case of YUV format, the color format indicates the correspondence between pixels of the luminance component and pixels of the chrominance component.

[0153] The prediction unit determiner 602 determines the size and shape of the prediction target block and can use all direct or indirect information transmitted from the video encoding device. The prediction unit determiner 602 can use direct information related to the size and shape of the current block and can utilize information that may affect the determination of the size and shape of the current block, such as the number of segments, depth, segment shape, segment direction, size information related to the smallest segment block, and segmentation information / prediction mode of previously decoded neighboring blocks.

[0154] The prediction unit determined by prediction unit determiner 602 can be the current block, one of the sub-blocks into which the current block is divided, a group of pixels, or a single pixel. The prediction unit can include size and shape information for performing predictions of the chroma and luminance components.

[0155] Prediction units can be determined dependently or independently for the chroma and luminance components. Dependent determination means that the prediction units for the luminance or chroma components are not determined for each component individually; rather, once a prediction unit is determined for one component, units for the other component or components are determined with corresponding sizes and shapes. In this case, one component can correspond to one or more of the luminance and chroma components. That is, the luminance component can be determined based on information from the chroma component. As an example, the prediction unit for the chroma component can have a size corresponding to the prediction unit for the luminance component according to the color format. When dependent determination is performed, information about the prediction unit for the other component corresponding to one component can be omitted; that is, information about the component determined dependently. Independent determination means that the prediction units for the luminance and chroma components are determined separately. When determined independently, information about the prediction unit for each component can be signaled separately.

[0156] The prediction technique determiner 604 determines the prediction technique of the prediction unit. As described above, the prediction technique can be one of inter-frame prediction, intra-frame prediction, IBC mode, palette mode, or a combination of intra-frame and inter-frame prediction techniques. In this case, the prediction technique for the chroma component can be determined to be the same as the prediction technique for the corresponding luma component, without the need to send a signal and parse separate information. According to an embodiment, a technique that combines intra-frame and inter-frame prediction can be included in the inter-frame prediction.

[0157] As an example, when the prediction technique for the current block is not intra-frame prediction, the video decoding device parses 1 bit of flag information. For instance, when the parsed flag indicates skip mode, the video decoding device determines the prediction mode for the current block to be either inter-frame prediction merging mode or IBC merging mode. In skip mode, the video decoding device can use the prediction signal as the reconstruction signal while omitting the inverse transform process (i.e., not parsing the residual signal). Here, skipping can be a method of not transmitting motion information (e.g., motion vectors, reference pictures, reference picture lists, etc.) or transmitting motion information using only at least one syntax information. Alternatively, skipping can be a method of not transmitting the residual signal of the current block.

[0158] On the other hand, if the parsed flags do not indicate the skip mode of the current block, the prediction technique determiner 604 can parse a series of 1-bit flags to determine the prediction technique of the current block as one of techniques such as inter-frame prediction, intra-frame prediction, IBC mode, or palette mode.

[0159] For example, when no skip is applied to the current block and the prediction technique is determined to be inter-frame prediction or IBC mode, the video decoding device parses a 1-bit flag. The prediction mode for the current block can be determined as either regular merge mode or advanced motion vector prediction (AMVP) mode based on the parsed flag.

[0160] The prediction mode determiner 606 combines prediction techniques to determine the detailed prediction mode of the current prediction unit block (hereinafter referred to as the current block).

[0161] As an example, when the prediction technique for the current block is inter-frame prediction, the prediction mode determiner 606 can determine either a regular merge mode or an AMVP mode as the prediction mode for the current block. In the regular merge mode or AMVP mode, the video decoding device generates prediction blocks based on the resolved motion information according to one or more motion compensations, and generates the final prediction signal for the current block by performing a weighted summation of the multiple generated prediction blocks. In this case, one or more of the prediction blocks may include signals from the intra-frame decoding region containing the current block.

[0162] As another example, when the prediction technique for the current block is inter-frame prediction and intra-frame prediction samples are used for the final predicted block of the current block, the adjacent pre-reconstructed regions of the current block are defined as templates, and these templates can be used to derive the intra-frame prediction mode. In this case, the template can also include regions that are not adjacent to the current block. Subsequently, the derived intra-frame prediction mode can be used to generate the final predicted block of the current block.

[0163] As another example, when the prediction technique for the current block is inter-frame prediction, the prediction mode determiner 606 can determine a prediction mode based on geometric segmentation (hereinafter, geometric segmentation mode (GPM)) as the prediction mode for the current block. In GPM, the video decoding device divides the current block into two or more sub-regions according to geometric segmentation, generates prediction blocks based on resolved motion information and the prediction mode information of the current block, and generates the final prediction signal for the current block by performing a weighted summation of the generated multiple prediction blocks. According to the implementation, when inter-frame prediction is performed and the geometric segmentation mode is determined, at least one sub-block within the current block can be predicted based on inter-frame prediction.

[0164] Meanwhile, when the prediction technique for the current block is intra-frame prediction, the prediction mode for the current block can be generated based on at least one of the following: single-direction prediction mode, planar mode (horizontal plane, vertical plane, or regular plane), DC mode, extrapolation intra-frame prediction (EIP) prediction mode, matrix-based prediction mode (e.g., MIP), and prediction mode based on inter-component correlation (such as inter-component linear model (CCLM), convolutional inter-component model (CCCM), gradient linear model (GLM), etc.).

[0165] As an example, when the prediction technique for the current block is intra-frame prediction, the prediction mode for the current block can be a matrix-based intra-frame prediction mode (e.g., matrix-based intra-frame prediction (MIP)). The MIP mode can generate the prediction block for the current block based on the matrix by signaling / parsing and / or deriving the matrix index, or by signaling / parsing and / or deriving the matrix, based on an agreement between the video encoding device and the video decoding device.

[0166] As another example, when the prediction technique for the current block is intra-frame prediction, the prediction mode for the current block can be intra-frame template matching prediction (intra-frame TMP) mode. Intra-frame TMP mode defines the pre-reconstructed region surrounding the current block as a template and performs template matching in the pre-reconstructed adjacent regions of the current block to generate the predicted block. The template can also include regions that are not adjacent to the current block.

[0167] As another example, when the prediction technique for the current block is intra-frame prediction, the prediction mode for the current block can be an intra-frame geometric segmentation-based prediction mode. The intra-frame geometric segmentation-based prediction mode divides the current block into one or more sub-regions according to geometric segmentation, generates prediction blocks for each region using intra-frame prediction modes including different directional prediction modes, planar modes, DC modes, etc., and generates the prediction block for the current block by weighting the individual prediction blocks. According to the implementation, when intra-frame prediction is used and a geometric segmentation-based prediction mode is determined, at least one sub-block among all sub-blocks within the current block can be predicted based on intra-frame prediction.

[0168] As another example, when the current block is a chroma block and the prediction technique for the current block is intra-frame prediction, the current block can be predicted according to Direct Mode (DM). DM mode can perform the current prediction of the chroma block based on a prediction method applied to the luma block at the position corresponding to the current chroma block, or based on a prediction method predefined between the video encoding and video decoding devices.

[0169] As another example, linear and / or nonlinear models can be used to calculate the relationship between neighboring pre-reconstructed chromaticity samples of the current chromaticity component and neighboring pre-reconstructed luminance samples of the luminance region corresponding to the location of the current chromaticity block, and a predicted block of the current chromaticity block can be generated based on one or more calculated models.

[0170] As another example, there can be various methods for deriving the intra-prediction mode of the current block. Among these methods, one or more can be used based on signal notification / resolution of flags and / or indices.

[0171] Simultaneously, when the prediction technique for the current block is intra-block copy (IBC) prediction, a predicted block can be obtained from the pre-reconstructed region within the frame, including the current block, using one or more block vectors, and a final predicted block can be generated based on the predicted block. The video encoding device can signal the block vector information, and the video decoding device can parse the block vector information. For example, the video decoding device can configure a block vector candidate list according to the position and search order defined by an agreement between the video encoding and decoding devices, and can parse information such as indexes. The video decoding device can obtain the final block vector information by refining the initial block vector information obtained based on the parsed information using methods such as template matching.

[0172] As an example, when the prediction technique for the current block is Intra-Block Copy (IBC) prediction, the prediction mode for the current block can be a prediction mode based on IBC geometric segmentation (hereinafter, IBC geometric segmentation mode). The IBC geometric segmentation mode can divide the current block into one or more sub-regions based on geometric segmentation, generate prediction blocks for each region using different block vectors, and generate the final prediction block for the current block by performing a weighted summation of the prediction blocks. According to the implementation, when IBC prediction is used and the geometric segmentation mode is determined, at least one of all sub-blocks within the current block can be predicted based on IBC prediction.

[0173] As another example, when the current block is a chroma block and the prediction technique for the current block is Intra-Block Copy (IBC) prediction, if the luma block at the corresponding position is pre-reconstructed during the configuration of the block vector candidate list or the acquisition of block vectors, the block vector information of the luma block at the corresponding position can be used. For example, when the block segmentation structures of the luma and chroma components are the same, information is generated by scaling the block vector information of the corresponding luma block according to the color format, and the scaled information can be used as the block vector and / or block vector candidate for the current chroma block. On the other hand, when the block segmentation structures of the luma and chroma components are different, one or more block vectors can be obtained according to a predefined position and order within the corresponding luma region. Information is generated by scaling the block vectors obtained according to the color format, and the scaled information can be used as the block vector and / or block vector candidate for the current chroma block.

[0174] Meanwhile, when the prediction technique for the current block is a hybrid of intra-frame and inter-frame prediction, the pre-reconstructed adjacent regions of the current block can be defined as templates. Intra-frame prediction patterns can be derived using some or all of the template's pixel information, or intra-frame prediction blocks can be generated based on template matching. Predictive blocks for the current block can be derived from the derived intra-frame prediction patterns, or prediction blocks can be generated based on block vector information sent from the video encoding device. The final prediction block can be generated by mixing prediction blocks / intra-frame or inter-frame prediction blocks generated based on template matching and / or block vectors using methods such as weighted summation.

[0175] The template can include regions adjacent to the current block and regions not adjacent to the current block. Non-adjacent regions can be areas within a certain pixel line distance from the current block. When a non-adjacent region is used as the template, information regarding whether to use the region and / or distance information can be transmitted from the video encoding device to the video decoding device. For example, the distance information can be defined according to an agreement between the video encoding and decoding devices, and the transmission of this information can be omitted. When defined according to an agreement, this value can be fixed as a specific constant value, or it can be variably determined based on the horizontal or vertical pixel length of the prediction unit, the area of ​​the block, aspect ratio, etc. Subsequently, the final prediction block for the current block can be generated using the derived intra-frame prediction mode.

[0176] The prediction executor 608 generates the final prediction block of the current decoded block (hereinafter referred to as the current block) based on the determined prediction technique and prediction mode.

[0177] As an example, after predicting and reconstructing the luma component block, prediction and reconstruction can be performed on the corresponding chroma component block. Here, a block can be a coding unit (CU), a coding tree unit (CTU), a set of one or more CUs, and / or a set of one or more CTUs.

[0178] As an example, the prediction executor 608 performs a prediction according to the prediction pattern determined as described above to generate a prediction block for the current block, and the adder 550 adds the prediction block of the current block to the residual sample (i.e., the residual block) to generate a reconstruction block.

[0179] Regarding the inverse transform, the entropy decoder 510 decodes the transform coefficients. When a quadratic transform is applied, the entropy decoder 510 decodes the quantized quadratic transform coefficients. When no quadratic transform is applied, the entropy decoder 510 decodes the quantized main transform coefficients. The entropy decoder 510 parses information such as the quantization method and quantization parameter information.

[0180] The inverse quantizer 520 performs inverse quantization on the decoded quantized transform coefficients based on information such as quantization method and quantization parameter information to generate inverse quantized transform coefficients.

[0181] The inverse transformer 530 generates residual samples by performing an inverse transform on the TU represented by the inverse quantization transform coefficients. For example... Figure 6 As shown, the inverse converter 530 may include an inverse converter unit determiner 610, an inverse converter core determiner 612, and an inverse converter actuator 614. Additionally, the inverse converter 530 may include a prediction signal corrector 609.

[0182] As an example, the inverse transform unit determiner 610 can determine a TU or a sub-block obtained by dividing a TU into multiple sub-blocks as the target of the inverse transform. For example, the TU can be the entire current block of the predicted target, or it can be a partial region (e.g., a sub-region or sub-block) of the current block.

[0183] The prediction signal corrector 609 can perform prediction signal correction within a defined inverse transform unit. The prediction signal correction can be applied only to inverse transform units of specific components (e.g., to the luma component, to the chroma component, etc.). The prediction signal correction can be applied to inverse transform units predicted according to a specific prediction technique (e.g., to inter-frame prediction techniques, to inter-frame prediction and IBC prediction, to intra-frame prediction, etc.). For example, when applied to a chroma component block (hereinafter used interchangeably with chroma block), prediction signal correction can be applied to the luma block at the corresponding position (hereinafter used interchangeably with the corresponding luma block) when a non-zero residual value exists.

[0184] As an example, signaling notification / resolution based on flag / index information can indicate whether the prediction signal should be corrected for the current inverse transform unit. When the correction of the prediction signal is applied to the chromaticity component, a flag / index can be signaled / resolved to indicate whether correction should be performed on the Cb and Cr components. Alternatively, a flag / index indicating whether to perform an operation on each of the Cb and Cr components can be signaled / resolved.

[0185] Figure 7 This is an exemplary diagram illustrating the prediction signal correction of the chromaticity component according to an embodiment of the present disclosure.

[0186] As an example, when the correction of the predicted signal is applied to the current inverse transform unit, such as... Figure 7 As shown, the video decoding device can perform prediction signal correction on the inverse transform unit of the chroma component. For example... Figure 7 For example, to correct the prediction signal, the prediction signal and residual signal of the luminance component at the position corresponding to the current chroma transformation block can be used. M and P can represent the weights of the prediction signal associated with the chroma component, respectively, and N and Q can represent the weights of the corrected prediction signal associated with the chroma component, respectively. According to the implementation, Figure 7 Some processes in the block diagram shown may be omitted, added, or changed.

[0187] As an example, when correcting the prediction signal with respect to the inverse transform unit of the current chroma component (hereinafter referred to as "chroma component"), the video decoding device can use the prediction signal of the corresponding luminance component at the corresponding position, as well as the correlation between the luminance component and the chroma component based on the reconstructed signal. The video decoding device can model the correlation between the prediction signal of the inverse transform unit of the chroma component and the prediction signal of the luminance component at the corresponding position as shown in Equation 2, and can use, for example, Gaussian elimination to calculate the filter coefficients c0 to c7 that define the correlation.

[0188] [Formula 2]

[0189] In Equation 2, L0 to L5 represent the luminance prediction signals at the corresponding positions, and C represents the prediction signal of the inverse transform unit of the chrominance component. c6 represents the filter coefficients associated with the nonlinear term, and c7 represents the filter coefficients associated with the bias B.

[0190] According to the implementation method, depending on the color format of the input video, the position of each sample in the prediction signal of the inverse chroma component transformation unit may be different from the position of the sample in the corresponding luminance prediction block. Figure 8 This shows the location of the chromaticity component samples and the corresponding luminance sample locations. Figure 8 In the examples, example (a) applies to YUV4:2:0 format, example (b) applies to YUV4:2:2 format, and example (c) applies to YUV4:4:4 format. About Figure 8 For each of the scenarios shown, the filter coefficients can be derived using Equation 2. Depending on the color format of the input image, in Equation 2, L0 to L5, sample values ​​that do not correspond to those in the luminance blocks can be set to 0. For example, in the case of YUV4:2:2 format, L3 = L4 = L5 = 0.

[0191] As an example, after deriving the filter coefficients as described above, the video decoding device can apply the filter coefficients to the reconstructed luminance signal at the corresponding location to perform correction of the predicted signal, as shown in Equation 3. Based on this correction, the video decoding device can generate a correction signal C' for the inverse transform unit of the chroma component.

[0192] [Formula 3]

[0193] In Equation 3, c0 to c7 represent the filter coefficients derived in Equation 2 based on the correlation between the chroma prediction signal and the luminance prediction signal at the corresponding location. Depending on the color format of the input image, regarding, for example... Figure 8 The positions shown, L0 to L5, represent the luminance prediction signals at the corresponding positions. After correcting the prediction signals of the inverse transform unit for the chrominance components, the video decoding device can generate the final prediction signal for the inverse transform unit for the chrominance components by performing a weighted summation of the corrected prediction signal and the uncorrected prediction signal, as shown in Equation 4. In this case, the weights used for the weighted summation can be different for the Cb and Cr components.

[0194] [Formula 4]

[0195] In Formula 4, n and u can be integers of 1 or greater. M and N can be integers satisfying M + N = 2. n The integers P and Q are 0 or greater, and P and Q can be integers satisfying P + Q = 2. u The integers 0 or greater. The following description of the weights of the pre-corrected and post-corrected predicted signals refers to the case where M and P are the same and N and Q are the same. When M and P are different and N and Q are different, this means that weighted summations can be performed separately for the Cb and Cr components.

[0196] As an example, regarding weights M and N, the signaling can notify / parse the values ​​or information about those values. Alternatively, the signaling can notify / parse information about a weight table defined according to an agreement between the video encoding and decoding devices (e.g., index information indicating the table, index information indicating elements within the table, etc.).

[0197] As an example, when the residual signal for the corresponding luma block is absent, the weights can be implicitly determined as N=0 and M=1. For instance, regarding the residual signal for the corresponding luma block, when the CBF (encoded block flag, indicating the luma CBF or root CBF) is 0, it can indicate that the residual signal is absent. When the residual signal is absent, the flag / index indicating whether to adaptively correct the inter-component prediction signal in inter-frame prediction can be used without signaling notification / parsing.

[0198] As an example, a video decoding device can implicitly determine weights based on whether the predicted signals of neighboring blocks of the inverse transform unit for the chroma component are corrected.

[0199] As an example, the inverse transform unit for the chromaticity components can be described as follows: Figure 9 The example determines the position of adjacent blocks based on the agreement between the video encoding and decoding devices. Besides... Figure 9Beyond the examples above, non-adjacent peripheral blocks can also be utilized. Weights can be implicitly determined based on the number of neighboring blocks among those associated with the inverse transform unit of the current chroma component that have undergone prediction signal correction. For example, when prediction signals are corrected in Y neighboring blocks out of T blocks, (M:N) can be implicitly determined as (1:7). The 1:7 weight mentioned above is one of various implementations, indicating that other weights can be determined. In this case, the weights based on whether the prediction signals of neighboring blocks have been corrected can be determined differently depending on information about the residual signals of the corresponding luma block (e.g., whether the residual signals exist). When no block among the T blocks has undergone prediction signal correction, the weights can be implicitly determined based on information about the residual signals of the corresponding luma block (e.g., whether the residual signals exist). The above examples indicate that the weight N can be set to a larger value as more blocks among the neighboring blocks of the inverse transform unit of the chroma component apply prediction signal correction.

[0200] As another example, the merging candidate list can be configured based on whether the predicted signals of neighboring blocks of the inverse transform unit for the chroma component are corrected and the weights used. The merging candidate list can be configured in a predetermined order according to an agreement between the video encoding and decoding devices. Information added to the merging candidate list can be as follows: For example, only information about blocks whose predicted signals have been corrected can be added to the list. Additionally, at least one of the following information can be added to the list: filter coefficients (c0 to c7), weights (M, N), etc. When information from neighboring blocks is inherited and applied to the current block, a trusted command notifies / resolves the merging index. In this case, both filter coefficients and weights can be inherited. Alternatively, only filter coefficients or weights can be inherited. When inheriting filter coefficients, the weights of the filters from neighboring blocks can be applied to the current transform block as is. The video decoding device can set the inherited weights to initial values, correct the initial values ​​based on the correlation between the predicted signals of the inverse transform unit for the chroma component and the predicted signals of the corresponding luminance components, and use the corrected values ​​for weighted summation.

[0201] As another example, for the inverse transform unit of the chroma component, the weights can be implicitly determined based on the prediction technique / prediction mode of the corresponding luma block. When the block partitioning structure is "single-tree," the block partitioning structure of the chroma component is the same as that of the luma component. The weights can be determined differently depending on whether the prediction technique for the luma block at the corresponding position is inter-frame prediction, IBC prediction, or intra-frame prediction. For example, in the case of intra-frame prediction, the weights can be implicitly determined as (M:N)=(3:1); in the case of inter-frame prediction, as (M:N)=(1:3); and in the case of IBC prediction, as (M:N)=(1:3).

[0202] As another example, when the block partitioning structure is "dual-tree," the block partitioning structure for the chroma component can be determined separately from that for the luma component. In this case, the partitioning structures for the luma and chroma blocks can be the same until a specific partitioning depth is reached. Information regarding the specific partitioning depth can be a fixed value determined by an agreement between the video encoding and decoding devices, or it can be signaled / parsed for each specific unit such as a CTU or stripe. Figure 10 As shown in the example, the video decoding device can identify the prediction technique and / or prediction mode for blocks A, B, C, D, and E of the corresponding luma blocks, and determine the weights based on the prediction technique / prediction mode of the blocks. For example, in the five positions, when the number of blocks applying intra-frame prediction is larger, the weights can be implicitly determined as (M:N) = (3:1), while when the number of blocks applying inter-frame prediction and / or IBC prediction is larger, the weights can be implicitly determined as (M:N) = (1:3).

[0203] As an example, as one of the factors in determining the weights, the quantization parameter (QP) of the corresponding luma block can be used for the inverse transform unit of the chrominance component. When the residual signal of the corresponding luma block exists and inverse quantization is performed using a high QP, the value of the weight N can be determined to be larger. Here, a high QP can indicate that the QP of the luma block at the corresponding position is greater than a certain threshold. When the segmentation structure is a single tree, the QP of the corresponding luma block can be identified. When the segmentation structure is a dual tree, a high QP can be identified by identifying the average or maximum QP of the luma block at the corresponding position and comparing that QP with a predetermined threshold. A high QP can indicate that the difference between the QP of the transform unit block of the chrominance component and the QP of the corresponding luma block is greater than a predetermined threshold. In this case, only the case where the QP of the corresponding luma block is higher can be considered. The QP of the luma block at the same position can be determined using one of the examples above.

[0204] As an example, for the chroma component transformation unit, when the residual signal of the corresponding luma block exists, the video decoding device can implicitly determine the weights based on the method of performing the inverse transform. The weights can be determined differently depending on whether a non-separable inverse transform or a separate inverse transform is performed. For example, when a non-separable quadratic inverse transform is performed on the corresponding luma block, (M:N) can be implicitly determined as (7:1). When a non-separable quadratic inverse transform is performed, (M:N) can be determined as (3:1), and when a non-separable inverse transform is not performed, (M:N) can be implicitly determined as (1:3). In the above example, each weight is one implementation method, and different weights can be implicitly determined based on the above conditions.

[0205] As an example, for the transformation unit of the chroma component, the video decoding device can implicitly determine the weights based on the information of the residual signal of the corresponding luma block.

[0206] For example, a video decoding device can implicitly determine the weights based on the number of residual samples with a value of 0 in the residual samples of the corresponding luminance block. When the number of residual samples with a value of 0 is at least half of the samples in the corresponding luminance block, (M:N) can be implicitly determined as (3:1). Regarding the residual samples in the corresponding luminance block, when the variance value is higher than a predetermined threshold, (M:N) can be implicitly determined as (1:3). When the maximum absolute value of the residual samples in the luminance block at the corresponding position is higher than a certain threshold, (M:N) can be implicitly determined as (1:3).

[0207] In the above-described embodiments related to weight determination, the value of M:N is an example based on each condition, and different weights can be implicitly determined based on the conditions described above. The video decoding apparatus can use at least one of the above embodiments to determine the weights. For example, two or more embodiments can be applied to a single transformation unit.

[0208] exist Figure 6 In the example, the prediction signal corrector 609 is included in the inverter 530, but as another example, it can be included in the predictor 540 or the adder 550. When included in the predictor 540, the prediction signal corrector 609 can correct the prediction signal based on TU.

[0209] The inverse transform kernel determiner 612 can determine separable vertical and horizontal primary inverse transform kernels and / or non-separable secondary inverse transform kernels, or it can determine non-separable primary inverse transform kernels. The inverse transform actuator 614 can use the determined inverse transform kernels to perform an inverse transform on the inverse-quantized transform coefficients.

[0210] As an example, whether to perform an inseparable inverse transform can be determined based on signaling notification / parsing of flags or indices. Here, whether to perform an inseparable inverse transform indicates whether to perform an inseparable primary inverse transform or an inseparable secondary inverse transform. Alternatively, whether to perform an inseparable inverse transform can be implicitly determined based on the size of the current transform block. The current transform block represents a TU or a sub-block transform unit. A sub-block transform unit can be a transform unit divided from a TU or a transform unit corresponding to a sub-region of the current block.

[0211] Figure 11 This is an exemplary view illustrating the process of determining the inverse transform kernel according to embodiments of the present disclosure.

[0212] Inverse transform kernel determiner 612 can be based on Figure 11 The example determines the type of the inverse transform kernel for the current transform block. Figure 11 The order shown can be changed, and some steps can be omitted.

[0213] As an example, whether to perform a non-separable inverse transform and a non-separable master inverse transform for the current transform block can be determined based on signaling notification / parsing of flags and / or indices. Alternatively, whether to perform a non-separable inverse transform and a non-separable master inverse transform can be implicitly determined based on the size of the current inverse transform unit.

[0214] Figure 12 This is an exemplary view illustrating the process of determining the inverse transform kernel according to another embodiment of this disclosure.

[0215] As another example, the inverse transform kernel determiner 612 can be based on Figure 12 The example determines the type of the inverse transform kernel for the current transform block. Figure 12 The order shown can be changed, and some steps can be omitted.

[0216] As an example, whether to perform an inseparable inverse transformation for the current transform block can be determined based on signaling notification / parsing of flags and / or indexes. According to the implementation, when performing an inseparable inverse transformation, it can be determined whether to perform an inseparable primary inverse transformation or an inseparable secondary inverse transformation based on the block size. For example, when the block size is N×4 or 4×N (16≥N≥4), or 8×N or N×8 (16≥N≥8), an inseparable primary inverse transformation can be performed on the corresponding block, and an inseparable secondary inverse transformation can be performed on blocks with different sizes.

[0217] The method for determining the separable master-inverse transform kernel is described below.

[0218] As an example, when performing a separable master-inverse transform on the current transform block, the vertical and horizontal kernels can be determined by defining a table according to an agreement between the video encoding and decoding units and signaling / parsing the index of that table. For instance, different tables can be configured based on prediction combinations for sub-regions (e.g., inter-frame / inter-frame prediction, inter-frame / intra-frame prediction, IBC / intra-frame prediction, intra-frame / intra-frame TMP prediction, etc.). Here, as a prediction combination, combinations of inter-frame prediction, intra-frame prediction, IBC prediction, and intra-frame TMP prediction can be considered.

[0219] As another example, when performing a non-separable second-order inverse transform on the current transform block, an implicitly fixed kernel (e.g., DCT2, DCT2 as a kernel for both vertical and horizontal directions) can be used as a separable master inverse transform kernel. The fixed kernel can adaptively change according to the size / aspect ratio of the current transform block.

[0220] The following describes a method for determining the inseparable principal-inverse transform kernel.

[0221] As an example, whether to perform an inseparable master-inverse transformation can be implicitly determined based on the size of the current transform block.

[0222] As an example, when performing an inseparable master-inverse transform on the current transform block, the inseparable master-inverse transform kernel can be determined based on the size of the current transform block and the prediction mode of each sub-region.

[0223] As another example, the inseparable master-inverse transform can be applied only to blocks where all sub-regions are intra-predicted.

[0224] As another example, the video decoding device signals / parses flags and / or indices indicating the inseparable master-inverse transform core in relation to the current transform block. The video decoding device can determine the inverse transform core indicated by the parsed index / flag as the inseparable master-inverse transform core of the current transform block based on the size of the current transform block, the prediction mode of each sub-region of the current block, etc.

[0225] In this case, the type of inverse transform kernel available can be configured differently depending on the agreement between the video coding unit and the video decoding unit, based on the prediction combination of the sub-regions of the current block (e.g., inter-frame / inter-frame prediction, inter-frame / intra-frame prediction, IBC / intra-frame prediction, intra-frame / intra-frame TMP prediction, etc.). Here, as a prediction combination, combinations of inter-frame prediction, intra-frame prediction, IBC prediction, and intra-frame TMP prediction can be considered.

[0226] As another example, when the prediction mode of each sub-region of the current block is not included in intra-frame prediction (e.g., such as directional intra-frame prediction), the video decoding device can derive at least one directionality in the entire or a portion of the prediction block corresponding to the current transform block, and can use a non-separable master-inverse transform kernel corresponding to one of the at least one derived directionality. Alternatively, when the prediction mode of each sub-region of the current block is included in intra-frame prediction (e.g., directional intra-frame prediction), the video decoding device can derive at least one directionality in the entire or a portion of the prediction block corresponding to the current transform block, and can use a non-separable master-inverse transform kernel corresponding to one of the at least one derived directionality. For example, a directionality similar to the intra-frame prediction mode of each sub-region of the current block can be used among the derived directionities. The video decoding device applies a Sobel filter to the region in the prediction block corresponding to the current transform block to compute vertical / horizontal gradients and magnitudes for each predetermined unit. The video decoding device can configure a gradient histogram (HoG) by the predetermined vertical / horizontal gradients and magnitudes for each predetermined unit, and can derive the directionality based on the HoG. Here, the Sobel filter stands for edge detection filter, and the predetermined unit can be, for example, a pixel or a group of pixels.

[0227] As another example, when there are N types of available non-separable primary inverse transform kernels, for a transform block within the current block to which the geometric partitioning mode is applied, the number of types of available non-separable primary inverse transform kernels may be K (K<N). Among the K kernels, information about the kernel to be used (for example, index information of one kernel among the K kernels) may be signaled / parsed to determine the kernel for the primary inverse transform.

[0228] As another example, for the current transform block, when a non-separable secondary inverse transform is used, the use of the non-separable primary inverse transform is restricted. According to an embodiment, when the non-separable primary inverse transform index is parsed first, whether to parse the non-separable secondary inverse transform may depend on the index value of the non-separable primary inverse transform.

[0229] Hereinafter, a method for determining a non-separable secondary inverse transform kernel is described.

[0230] As an example, whether to perform the non-separable secondary inverse transform may be implicitly determined according to the size of the current transform block.

[0231] As an example, when the non-separable secondary inverse transform is performed on the current transform block, the non-separable secondary inverse transform kernel may be determined based on the size of the current transform block and the prediction mode of each sub-region.

[0232] As another example, the non-separable secondary inverse transform may be applied only to blocks for which all sub-regions are intra predicted.

[0233] As another example, the video decoding apparatus may signal / parse a flag and / or an index indicating the non-separable secondary inverse transform kernel for the current transform block. The video decoding apparatus may determine the inverse transform kernel indicated by the parsed index / flag as the non-separable secondary inverse transform kernel for the current transform block, based on the size of the current transform block, the prediction mode of each sub-region of the current block, and the like.

[0234] In this case, based on the prediction combination of the sub-regions of the current block (for example, inter / inter prediction, inter / intra prediction, IBC / intra prediction, intra / intra TMP prediction, etc.), the types of available inverse transform kernels may be differently configured according to the agreement between the video encoding apparatus and the video decoding apparatus. Here, as the prediction combination, combinations among inter prediction, intra prediction, IBC prediction, and intra TMP prediction may be considered.

[0235] As another example, when the prediction mode of each sub-region of the current block is not included in intra prediction (for example, directional intra prediction), the video decoding apparatus may derive at least one directionality in all or part of the prediction block corresponding to the current transform block, and may use a non-separable quadratic inverse transform kernel corresponding to one of the derived at least one directionality. Alternatively, when the prediction mode of each sub-region of the current block is included in intra prediction (for example, such as directional intra prediction), the video decoding apparatus may derive at least one directionality in all or part of the prediction block corresponding to the current transform block, and may use a non-separable quadratic inverse transform kernel corresponding to one of the derived at least one directionality. For example, a directionality similar to the intra prediction mode of each sub-region of the current block among the derived directionalities may be used. The video decoding apparatus applies a Sobel filter to the region in the prediction block corresponding to the current transform block, so as to calculate vertical / horizontal gradients and magnitude values for each predetermined unit. The video decoding apparatus may configure a Histogram of Oriented Gradients (HoG) based on the vertical / horizontal gradients and magnitude values predetermined for each unit, and may derive directionality based on the HoG. Here, the Sobel filter denotes an edge detection filter, and the predetermined unit may be, for example, a pixel or a group of pixels.

[0236] As another example, when the number of types of available non-separable quadratic inverse transform kernels is N, for a transform block within the current block to which the geometric partitioning mode is applied, the number of types of available non-separable quadratic inverse transform kernels may be K (K<N). Among the K kernels, information about the kernel to be used (for example, index information of one of the K kernels) may be signaled / parsed to determine the kernel for the quadratic inverse transform.

[0237] As another example, regarding the current transform block, when a non-separable quadratic inverse transform is performed, a separable vertical and horizontal inverse transform kernel may be used to perform the primary inverse transform. As the kernel for the primary inverse transform, a fixed kernel (for example, DCT2, DCT2 as the vertical and horizontal direction kernel) may be used based on the type of the quadratic inverse transform kernel according to the agreement between the video encoding apparatus and the video decoding apparatus.

[0238] Hereinafter, using Figure 13 and Figure 14 examples describe a method for performing prediction signal correction on a chroma component.

[0239] Figure 13 is a flowchart illustrating a method for encoding a current chroma block by a video encoding apparatus according to an embodiment of the present disclosure.

[0240] The video encoding apparatus determines the inverse transform unit of the current chroma block (S1300).

[0241] The video encoding device obtains the chroma prediction signal of the inverse transform unit and the luminance prediction signal of the corresponding luminance block (S1302). The corresponding luminance block can represent the luminance region that is located in relation to the inverse transform unit of the current chroma block.

[0242] A video coding device can generate a residual signal by subtracting the predicted signal of the current block from the signal of the current block. The device can then generate quantization transform coefficients by applying transform / quantization to the residual signal, and generate a bitstream by encoding these coefficients. At the decoder side, the device can reconstruct the residual signal by applying inverse quantization / inverse transform to the quantization transform coefficients.

[0243] The video encoding device can obtain the chroma residual signal of the inverse transform unit for the current chroma block and the luminance residual signal of the corresponding luminance block at the decoder side. The video encoding device can generate a luminance reconstruction signal by adding the luminance residual signal and the luminance prediction signal.

[0244] The video coding apparatus receives a flag indicating that the chroma prediction signal of the inverse transform unit should be corrected (S1304). The video coding apparatus may obtain the flag from a higher level. Alternatively, the video coding apparatus may determine the value of the flag for coding efficiency optimization.

[0245] In addition, the video encoding device can encode the aforementioned flags.

[0246] Video encoding device inspection mark (S1306).

[0247] When the aforementioned flag indicates that the chroma prediction signal is to be corrected ("Yes" in S1306), the video encoding device may perform the following steps.

[0248] The video encoding device obtains filter coefficients (S1308) representing the correlation between the luminance prediction signal and the chrominance prediction signal.

[0249] As an example, a video coding device can calculate filter coefficients representing correlation based on luminance prediction signals and chrominance prediction signals.

[0250] As another example, a video coding apparatus can generate a merged candidate list that includes candidate blocks for which chroma prediction signal correction has been performed. The merged candidate list can include filter coefficients of the chroma prediction signal used to correct the candidate blocks, serving as information about the candidate blocks for which chroma prediction signal correction has been performed. The video coding apparatus can determine and encode a merge index, for example, for coding efficiency optimization. The video coding apparatus can use the merge index to extract candidate blocks from the merged candidate list and can inherit the filter coefficients of the candidate blocks.

[0251] The video encoding device generates the chroma correction signal of the inverse transform unit by applying filter coefficients to the luminance reconstruction signal of the corresponding luminance block (S1310).

[0252] The video encoding device obtains weights for weighted summation of the chroma prediction signal and the chroma correction signal (S1312).

[0253] As an example, a video coding apparatus can determine the index of an element in a weight table, for instance, in order to optimize coding efficiency. The video coding apparatus can then extract weights from the table based on these indexes.

[0254] As another example, the merge candidate list can also include weights as information about candidate blocks for which chroma prediction signal correction has been performed. The video coding device can extract candidate blocks from the merge candidate list using the merge index and can inherit the weights of the candidate blocks.

[0255] As another example, a video coding device can implicitly determine weights based on the number of blocks in the adjacent blocks associated with the inverse transform unit that have undergone chroma prediction signal correction.

[0256] As another example, a video encoding device can implicitly determine weights based on the block segmentation structure of the current chroma block and the prediction technique of the corresponding luma block.

[0257] As another example, video encoding devices can implicitly determine weights based on the magnitude of the quantization parameters of the corresponding luma blocks.

[0258] As another example, a video coding device can implicitly determine weights based on whether the inverse transform unit applies a non-separable primary inverse transform or a non-separable secondary inverse transform to the corresponding luminance block.

[0259] The video encoding device performs a weighted summation of the chroma prediction signal and the chroma correction signal based on the weights (S1314).

[0260] By performing a weighted summation of the chroma prediction signal and the chroma correction signal, the video coding apparatus can generate the final chroma prediction signal for the inverse transform unit. The video coding apparatus can then generate a reconstructed signal for the inverse transform unit by adding the chroma residual signal and the final chroma prediction signal.

[0261] When the aforementioned flag does not indicate that the chroma prediction signal should be corrected ("No" in S1306), the video encoding apparatus may omit the process of correcting the chroma prediction signal. The video encoding apparatus can generate a reconstructed signal about the inverse transform unit by adding the chroma residual signal and the chroma prediction signal.

[0262] Figure 14 This is a flowchart illustrating a method for a video decoding apparatus according to an embodiment of the present disclosure to reconstruct a current chroma block.

[0263] The video decoding device determines the inverse transform unit of the current chroma block (S1400).

[0264] The video decoding device obtains the chroma prediction signal of the inverse transform unit and the luminance prediction signal of the corresponding luminance block (S1402). The corresponding luminance block can represent the luminance region that corresponds to the inverse transform unit of the current chroma block in terms of position.

[0265] Video decoding devices can decode quantization transform coefficients from a bitstream. The video decoding device can then reconstruct the residual signal by applying inverse quantization / inverse transform to the quantization transform coefficients.

[0266] The video decoding device can obtain the chroma residual signal and luminance residual signal of the corresponding luminance block from the inverse transform unit of the current chroma block. The video decoding device can generate a luminance reconstruction signal by adding the luminance residual signal and the luminance prediction signal.

[0267] The video decoding device decodes the flag indicating the correction of the chroma prediction signal of the inverse transform unit (S1404). The video decoding device can decode the above flag from the bitstream.

[0268] Video decoding device check mark (S1406).

[0269] When the aforementioned flag indicates that the chroma prediction signal is to be corrected ("Yes" in S1406), the video decoding device may perform the following steps.

[0270] The video decoding device obtains filter coefficients (S1408) representing the correlation between the luminance prediction signal and the chrominance prediction signal.

[0271] As an example, a video decoding device can calculate filter coefficients representing correlation based on luminance prediction signals and chrominance prediction signals.

[0272] As another example, the video decoding apparatus can generate a merged candidate list that includes candidate blocks for which chroma prediction signal correction has been performed. The merged candidate list can include filter coefficients of the chroma prediction signal used to correct the candidate blocks, serving as information about the candidate blocks for which chroma prediction signal correction has been performed. The video decoding apparatus can decode a merge index from the bitstream. The video decoding apparatus can use the merge index to extract candidate blocks from the merged candidate list and can inherit the filter coefficients of the candidate blocks.

[0273] The video decoding device generates the chroma correction signal of the inverse transform unit by applying filter coefficients to the luminance reconstruction signal of the corresponding luminance block (S1410).

[0274] The video decoding device obtains weights for weighted summation of the chroma prediction signal and the chroma correction signal (S1412).

[0275] As an example, a video decoding device can obtain an index from the bitstream indicating an element in a weight table. The video decoding device can then extract weights from the table based on the index.

[0276] As another example, the merged candidate list can also include weights as information about candidate blocks for which chroma prediction signal correction has been performed. The video decoding device can use the merge index to extract candidate blocks from the merged candidate list and can inherit the weights of the candidate blocks.

[0277] As another example, a video decoding device can implicitly determine weights based on the number of blocks in the adjacent blocks associated with the inverse transform unit that have undergone chroma prediction signal correction.

[0278] As another example, a video decoding device can implicitly determine weights based on the block segmentation structure of the current chroma block and the prediction technique of the corresponding luma block.

[0279] As another example, a video decoding device can implicitly determine weights based on the magnitude of the quantization parameters of the corresponding luma blocks.

[0280] As another example, a video decoding device can implicitly determine weights based on whether the inverse transform unit applies an inseparable primary inverse transform or an inseparable secondary inverse transform to the corresponding luminance block.

[0281] The video decoding device performs a weighted summation of the chroma prediction signal and the chroma correction signal based on the weights (S1414).

[0282] By performing a weighted summation of the chroma prediction signal and the chroma correction signal, the video decoding device can generate the final chroma prediction signal for the inverse transform unit. The video decoding device can then generate a reconstructed signal for the inverse transform unit by adding the chroma residual signal and the final chroma prediction signal.

[0283] When the aforementioned flag does not indicate that the chroma prediction signal should be corrected ("No" in S1406), the video decoding device may omit the process of correcting the chroma prediction signal. The video decoding device can generate a reconstructed signal about the inverse transform unit by adding the chroma residual signal and the chroma prediction signal.

[0284] Although the steps in the corresponding flowcharts are described as being executed sequentially, these steps are merely examples of the technical ideas of some embodiments of this disclosure. Therefore, those skilled in the art to which this disclosure pertains can perform these steps by changing the sequence described in the corresponding figures or by executing two or more steps in parallel. Thus, the steps in the corresponding flowcharts are not limited to the temporal order shown.

[0285] It should be understood that the above description presents illustrative embodiments that can be implemented in various other ways. The functionality described in some embodiments may be implemented by hardware, software, firmware, and / or combinations thereof. It should also be understood that the functional components described in this disclosure are designated as "...unit" to strongly emphasize their potential for independent implementation.

[0286] Furthermore, the various methods or functions described in some embodiments can be implemented as instructions stored in one or more processor-readable and executable non-transitory recording media. Non-transitory recording media can include various types of recording devices, such as those storing data in a computer system-readable form. For example, non-transitory recording media can include storage media such as erasable programmable read-only memory (EPROM), flash memory drives, optical disk drives, magnetic hard disk drives, and solid-state drives (SSDs).

[0287] Although embodiments of this disclosure have been described for illustrative purposes, those skilled in the art to which this disclosure pertains will understand that various modifications, additions, and substitutions can be made without departing from the spirit and scope of this disclosure. Therefore, embodiments of this disclosure have been described for the sake of brevity and clarity. The scope of the technical concept of the embodiments of this disclosure is not limited by the accompanying drawings. Therefore, those skilled in the art to which this disclosure pertains should understand that the scope of this disclosure should not be limited to the embodiments explicitly described above, but rather to the claims and their equivalents.

[0288] Cross-references to related applications

[0289] This application claims the benefit of Korean Patent Application No. 10-2024-0053611, filed on April 22, 2024, the entire contents of which are incorporated herein by reference.

Claims

1. A method for reconstructing a current chroma block, performed by a video decoding device, the method comprising: Determine the inverse transform unit of the current chroma block; Obtain the chromaticity prediction signal of the inverse transform unit and the luminance prediction signal of the corresponding luminance block, wherein the corresponding luminance block is the luminance region that is located in relation to the inverse transform unit of the current chromaticity block; Obtain the filter coefficients representing the correlation between the luminance prediction signal and the chrominance prediction signal; The chromaticity correction signal of the inverse transform unit is generated by applying the filter coefficients to the luminance reconstruction signal of the corresponding luminance block; Obtaining weights; and The chromaticity prediction signal and the chromaticity correction signal are weighted and summed based on the weights to generate the final chromaticity prediction signal of the inverse transform unit.

2. The method according to claim 1, further comprising: A flag indicating that the chroma prediction signal of the inverse transform unit should be corrected; as well as Check the aforementioned mark. When the flag indicates that the chromaticity prediction signal should be corrected, the calculation of the filter coefficients is performed.

3. The method according to claim 1, wherein, Obtaining the filter coefficients includes: Based on the luminance prediction signal and the chrominance prediction signal, filter coefficients representing the correlation are calculated.

4. The method according to claim 1, wherein, Obtaining the filter coefficients includes: Generate a merged candidate list, wherein the merged candidate list includes filter coefficients for correcting the chroma prediction signal of the candidate block, as information on the candidate block for which chroma prediction signal correction has been performed; Decode merge index; and The candidate blocks in the merge candidate list are extracted using the merge index, and the filter coefficients of the candidate blocks are inherited.

5. The method according to claim 1, wherein, Obtaining the weights includes: The index of the element in the decoding indicator table; and The weight is extracted from the table based on the index.

6. The method according to claim 1, wherein, Obtaining the weights includes: The weights are implicitly determined based on the number of adjacent blocks associated with the inverse transform unit that have already undergone chromaticity prediction signal correction.

7. The method according to claim 1, wherein, Obtaining the weights includes: The weights are implicitly determined based on the block segmentation structure of the current chroma block and the prediction technique of the corresponding luminance block.

8. The method according to claim 1, wherein, Obtaining the weights includes: The weights are implicitly determined based on the magnitude of the quantization parameters of the corresponding luminance blocks.

9. The method according to claim 1, wherein, Obtaining the weights includes: The weights are implicitly determined based on whether an inseparable inverse transform is applied to the corresponding luminance block associated with the inverse transform unit.

10. A method for encoding a current block, performed by a video encoding apparatus, the method comprising: Determine the inverse transform unit of the current chroma block; Obtain the chromaticity prediction signal of the inverse transform unit and the luminance prediction signal of the corresponding luminance block, wherein the corresponding luminance block is the luminance region that is located in relation to the inverse transform unit of the current chromaticity block; Obtain the filter coefficients representing the correlation between the luminance prediction signal and the chrominance prediction signal; The chromaticity correction signal of the inverse transform unit is generated by applying the filter coefficients to the luminance reconstruction signal of the corresponding luminance block; Obtaining weights; and The chromaticity prediction signal and the chromaticity correction signal are weighted and summed based on the weights to generate the final chromaticity prediction signal of the inverse transform unit.

11. The method of claim 10, further comprising: Obtain a flag indicating that the chromaticity prediction signal of the inverse transform unit should be corrected; Encode the aforementioned flag; as well as Check the aforementioned mark. When the flag indicates that the chromaticity prediction signal should be corrected, the calculation of the filter coefficients is performed.

12. The method according to claim 10, wherein, Obtaining the filter coefficients includes: Based on the luminance prediction signal and the chrominance prediction signal, filter coefficients representing the correlation are calculated.

13. The method according to claim 10, wherein, Obtaining the filter coefficients includes: Generate a merged candidate list, wherein the merged candidate list includes filter coefficients for correcting the chroma prediction signal of the candidate block, as information on the candidate block for which chroma prediction signal correction has been performed; Determine which indexes to merge; Encode the merged index; and The candidate blocks in the merge candidate list are extracted using the merge index, and the filter coefficients of the candidate blocks are inherited.

14. The method of claim 10, wherein, Obtaining the weights includes: Determine the index of the element in the indicator table; and The weight is extracted from the table based on the index.

15. The method according to claim 10, wherein, Obtaining the weights includes: The weights are implicitly determined based on whether an inseparable inverse transform is applied to the corresponding luminance block associated with the inverse transform unit.

16. A method for providing video data to a video decoding device, the method comprising: The video data is encoded into a bitstream; as well as The bitstream is sent to the video decoding device. Encoding the video data includes: Determine the inverse transform unit of the current chroma block; Obtain the chromaticity prediction signal of the inverse transform unit and the luminance prediction signal of the corresponding luminance block, wherein the corresponding luminance block is the luminance region that is located in relation to the inverse transform unit of the current chromaticity block; Obtain the filter coefficients representing the correlation between the luminance prediction signal and the chrominance prediction signal; The chromaticity correction signal of the inverse transform unit is generated by applying the filter coefficients to the luminance reconstruction signal of the corresponding luminance block; Obtaining weights; and The chromaticity prediction signal and the chromaticity correction signal are weighted and summed based on the weights to generate the final chromaticity prediction signal of the inverse transform unit.

Citation Information

Patent Citations

  • Compositions and methods for treating bone injuries

    KR1020240053611A