Method and apparatus for chroma predictor correction based on cross-component variation

Abstract

The present embodiment provides a method and an apparatus for chroma predictor correction based on a cross-component variation. In the present embodiment, an image decoding device acquires a block vector of a current block on the basis of a corresponding luma area of a current chroma block, wherein the current block includes a current luma block and the current chroma block. The image decoding device configures an area indicated by the block vector as a reference block, and configures a reference chroma block as a chroma predictor of the current chroma block, wherein the reference block includes a reference luma block and the reference chroma block. The image decoding device estimates parameters representing the correlation between a luma variation and a chroma variation between a template of the current block and a template of the reference block. The image decoding device calculates a correction value on the basis of the estimated parameters, and corrects the chroma predictor on the basis of the correction value.

Description

Method and device for correcting chroma predictor based on variation of cross components

[0001] The present disclosure relates to a video coding method and apparatus using chroma predictor correction based on the amount of variation in cross components.

[0002] The content described below merely provides background information related to the present invention and does not constitute prior art.

[0003] Since video data has a large amount of data compared to voice data or still image data, it requires a lot of hardware resources, including memory, to store or transmit it without processing for compression.

[0004] Therefore, when storing or transmitting video data, the encoder compresses the video data and stores or transmits it, and the decoder receives the compressed video data, decompresses it, and plays it back. These video compression technologies include H.264 / AVC, HEVC (High Efficiency Video Coding), and VVC (Versatile Video Coding), which improves encoding efficiency by about 30% compared to HEVC.

[0005] However, as the size, resolution, and frame rate of images are gradually increasing, and the amount of data that needs to be encoded is also increasing, a new compression technology that has better encoding efficiency and better image quality improvement than existing compression technologies is required.

[0006] When performing intra prediction by predicting pixel values ​​of the current picture based on pixel information within the same picture, the optimal mode among multiple intra prediction modes can be selected according to the characteristics of the image. The encoder selects one of the multiple intra prediction modes, encodes the target block using the selected prediction mode, and transmits information about the selected prediction mode to the decoder.

[0007] Intra prediction can generally use prediction modes that use information surrounding the current block and prediction modes that use information far from the current block. As for modes that use information surrounding the current block, there are angular modes and non-angular modes, as shown in Figure 3b. For directional modes, there are 65 angular modes based on VVC, and for non-directional modes, there are 2 non-directional modes (Planar, DC) based on VVC. Modes that perform prediction using information far from the current block include the Intra Block Copy (IBC) mode and the Intra Template Matching Prediction (IntraTMP) mode, which use block vector (BV) information. Both modes perform prediction of the current block by setting a reference block within the same frame as a predictor. In the IBC mode, the encoder selects a BV for optimal prediction and signals the BV information to the decoder. The decoder sets the region indicated by the BV information as a reference block. In the IntraTMP mode, without receiving BV information, the decoder derives an optimal BV based on the Sum of Absolute Difference (SAD) cost of adjacent templates, and sets a reference block based on the derived BV.

[0008] Meanwhile, if there is a brightness change between the luma region corresponding to the current block and the luma region corresponding to the reference block, the relationship between the luma component and the chroma component in the reference block may differ from the relationship between the luma component and the chroma component in the current block. This may contradict the assumption that the correlation between the luma component and the chroma component is maintained in the current block when setting the reference block for the current chroma block. Therefore, a method for responding to the brightness change between the luma region corresponding to the current block and the luma region corresponding to the reference block needs to be considered.

[0009] The present disclosure provides a video coding method and device for calculating luma changes and chroma changes between a current block and a reference block when making predictions using a reference block within the same frame, modeling the correlation between the luma changes and the chroma changes, and correcting a chroma predictor based on the modeled correlation.

[0010] According to an embodiment of the present disclosure, a method for restoring a current chroma block, performed by a video decoding device, is provided, comprising: obtaining a block vector of a current block based on a corresponding luma region of the current chroma block, wherein the current block includes a current luma block and the current chroma block; configuring a region indicated by the block vector as a reference block and setting the reference chroma block as a chroma predictor of the current chroma block, wherein the reference block includes a reference luma block and the reference chroma block; estimating parameters expressing a correlation between a luma variation and a chroma variation between a template of the current block and a template of the reference block; and calculating a correction value based on the estimated parameters and correcting the chroma predictor based on the correction value.

[0011] According to another embodiment of the present disclosure, a method for encoding a current chroma block, performed by a video encoding device, is provided, comprising: obtaining a block vector of a current block based on a corresponding luma region of the current chroma block, wherein the current block includes a current luma block and the current chroma block; configuring a region indicated by the block vector as a reference block and setting the reference chroma block as a chroma predictor of the current chroma block, wherein the reference block includes a reference luma block and the reference chroma block; estimating parameters expressing a correlation between a luma variation and a chroma variation between a template of the current block and a template of the reference block; and calculating a correction value based on the estimated parameters and correcting the chroma predictor based on the correction value.

[0012] According to another embodiment of the present disclosure, a method for providing video data to a video decoding device is provided, comprising: encoding the video data into a bitstream; and transmitting the bitstream to the video decoding device, wherein the encoding the video data comprises: obtaining a block vector of a current block based on a corresponding luma region of a current chroma block, wherein the current block includes a current luma block and the current chroma block; configuring a region indicated by the block vector as a reference block and setting the reference chroma block as a chroma predictor of the current chroma block, wherein the reference block includes a reference luma block and the reference chroma block; estimating parameters expressing a correlation between a luma variation and a chroma variation between a template of the current block and a template of the reference block; and calculating a correction value based on the estimated parameters and correcting the chroma predictor based on the correction value.

[0013] As described above, according to the present embodiment, when predicting using a reference block within the same frame, a video coding method and device are provided that calculates a luma change amount and a chroma change amount between a current block and a reference block, models a correlation between the luma change amount and the chroma change amount, and corrects a chroma predictor based on the modeled correlation, thereby making it possible to improve objective video encoding efficiency and subjective video quality.

[0014] FIG. 1 is an exemplary block diagram of an image encoding device capable of implementing the techniques of the present disclosure.

[0015] Figure 2 is a drawing for explaining a method of dividing a block using the QTBTTT (QuadTree plus BinaryTree TernaryTree) structure.

[0016] FIGS. 3A and 3B are diagrams illustrating multiple intra prediction modes, including wide-angle intra prediction modes.

[0017] Figure 4 is an example diagram of the surrounding blocks of the current block.

[0018] FIG. 5 is an exemplary block diagram of an image decoding device capable of implementing the techniques of the present disclosure.

[0019] FIG. 6 is a flowchart illustrating a method by which an image decoding device restores a current chroma block according to one embodiment of the present disclosure.

[0020] FIG. 7 is an exemplary diagram showing preset locations within a corresponding luma area according to one embodiment of the present disclosure.

[0021] FIG. 8 is an exemplary diagram illustrating modeling of correlation and correction of a chroma predictor according to one embodiment of the present disclosure.

[0022] FIG. 9 is a flowchart illustrating a method by which a video decoding device restores a current chroma block according to another embodiment of the present disclosure.

[0023] FIG. 10 is an exemplary diagram illustrating modeling of correlation and correction of a chroma predictor according to another embodiment of the present disclosure.

[0024] FIG. 11 is a flowchart illustrating a method for a video decoding device to restore a current chroma block according to another embodiment of the present disclosure.

[0025] FIG. 12 is an exemplary diagram illustrating modeling of correlation and correction of chroma predictor according to another embodiment of the present disclosure.

[0026] FIG. 13 is a flowchart illustrating a method by which a video decoding device restores a current chroma block according to another embodiment of the present disclosure.

[0027] FIG. 14 is an exemplary diagram illustrating modeling of correlation and correction of chroma predictor according to another embodiment of the present disclosure.

[0028] FIG. 15 is an exemplary diagram showing a configuration of a plurality of reference blocks according to one embodiment of the present disclosure.

[0029] FIG. 16 is an exemplary diagram showing a configuration of multiple reference blocks according to another embodiment of the present disclosure.

[0030] FIG. 17 is an exemplary diagram illustrating modeling of correlation and correction of chroma predictor according to another embodiment of the present disclosure.

[0031] Hereinafter, embodiments of the present invention will be described in detail with reference to exemplary drawings. When designating components in each drawing, it should be noted that, where possible, identical components are given the same reference numerals, even if they appear in different drawings. Furthermore, in describing the present embodiments, detailed descriptions of related known structures or functions will be omitted if they are deemed to obscure the gist of the present embodiments.

[0032] FIG. 1 is an exemplary block diagram of an image encoding device capable of implementing the techniques of the present disclosure. Hereinafter, the image encoding device and its subcomponents will be described with reference to the illustration in FIG. 1.

[0033] The video encoding device may be configured to include a picture segmentation unit (110), a prediction unit (120), a subtractor (130), a transformation unit (140), a quantization unit (145), a reordering unit (150), an entropy encoding unit (155), an inverse quantization unit (160), an inverse transformation unit (165), an adder (170), a loop filter unit (180), and a memory (190).

[0034] Each component of the video encoding device may be implemented in hardware, software, or a combination of hardware and software. Furthermore, the functions of each component may be implemented in software, with a microprocessor executing the software functions corresponding to each component.

[0035] A single image (video) is composed of one or more sequences containing multiple pictures. Each picture is divided into multiple regions, and encoding is performed for each region. For example, a single picture is divided into one or more tiles and / or slices. Here, one or more tiles can be defined as a tile group. Each tile or slice is divided into one or more Coding Tree Units (CTUs). Each CTU is then divided into one or more Coding Units (CUs) by a tree structure. Information applied to each CU is encoded as the syntax of the CU, and information commonly applied to CUs included in a CTU is encoded as the syntax of the CTU. In addition, information commonly applied to all blocks within a single slice is encoded as the syntax of the slice header, and information applied to all blocks constituting one or more pictures is encoded in the Picture Parameter Set (PPS) or the picture header. Furthermore, information commonly referenced by multiple pictures is encoded in a Sequence Parameter Set (SPS). And, information commonly referenced by one or more SPS is encoded in a Video Parameter Set (VPS). In addition, information commonly applied to one tile or tile group may be encoded as syntax of a tile or tile group header. Syntaxes included in an SPS, PPS, slice header, tile or tile group header may be referred to as high level syntax.

[0036] The picture segmentation unit (110) determines the size of the CTU. Information about the size of the CTU (CTU size) is encoded as the syntax of SPS or PPS and transmitted to the image decoding device.

[0037] The picture segmentation unit (110) divides each picture constituting an image into a plurality of CTUs having a predetermined size, and then recursively divides the CTUs using a tree structure. A leaf node in the tree structure becomes a CU, which is a basic unit of encoding.

[0038] The tree structure may be a QuadTree (QT) in which an upper node (or parent node) is divided into four lower nodes (or child nodes) of the same size, a BinaryTree (BT) in which an upper node is divided into two lower nodes, or a TernaryTree (TT) in which an upper node is divided into three lower nodes in a 1:2:1 ratio, or a structure that mixes two or more of the QT structures, BT structures, and TT structures. For example, a QTBT (QuadTree plus BinaryTree) structure may be used, or a QTBTTT (QuadTree plus BinaryTree TernaryTree) structure may be used. Here, BTTT may be combined and referred to as a MTT (Multiple-Type Tree).

[0039] Figure 2 is a drawing for explaining a method of dividing a block using the QTBTTT structure.

[0040] As illustrated in FIG. 2, a CTU may first be split into a QT structure. The quadtree splitting may be repeated until the size of the splitting block reaches the minimum block size (MinQTSize) of the leaf node allowed in the QT. A first flag (QT_split_flag) indicating whether each node of the QT structure is split into four nodes of the lower layer is encoded by the entropy encoding unit (155) and signaled to the image decoding device. If the leaf node of the QT is not larger than the maximum block size (MaxBTSize) of the root node allowed in the BT, it may be further split into one or more of the BT structure or the TT structure. There may be multiple splitting directions in the BT structure and / or the TT structure. For example, there may be two directions in which the block of the corresponding node is split horizontally and two directions in which the block is split vertically. As illustrated in FIG. 2, when MTT splitting begins, a second flag (mtt_split_flag) indicating whether nodes have been split, and if splitting has occurred, a flag indicating the splitting direction (vertical or horizontal) and / or a flag indicating the splitting type (Binary or Ternary) are encoded by the entropy encoding unit (155) and signaled to the image decoding device.

[0041] Alternatively, before encoding the first flag (QT_split_flag) indicating whether each node is split into four nodes of a lower layer, a CU split flag (split_cu_flag) indicating whether the node is split may be encoded. If the CU split flag (split_cu_flag) value indicates that the node is not split, the block of the corresponding node becomes a leaf node in the split tree structure and becomes a CU (coding unit), which is a basic unit of encoding. If the CU split flag (split_cu_flag) value indicates that the node is split, the video encoding device starts encoding from the first flag in the above-described manner.

[0042] As another example of a tree structure, when QTBT is used, there may be two types: a type that horizontally splits the block of the corresponding node into two blocks of the same size (i.e., symmetric horizontal splitting) and a type that vertically splits it (i.e., symmetric vertical splitting). A split flag (split_flag) indicating whether each node of the BT structure is split into blocks of a lower layer and split type information indicating the type of split are encoded by the entropy encoding unit (155) and transmitted to the image decoding device. Meanwhile, there may additionally be a type that splits the block of the corresponding node into two blocks of an asymmetrical shape. The asymmetric shape may include a shape that splits the block of the corresponding node into two rectangular blocks with a size ratio of 1:3, or a shape that splits the block of the corresponding node in a diagonal direction.

[0043] A CU can have various sizes depending on the QTBT or QTBTTT partitioning from the CTU. Hereinafter, the block corresponding to the CU to be encoded or decoded (i.e., the leaf node of the QTBTTT) is referred to as the "current block." Depending on the QTBTTT partitioning employed, the current block may be rectangular as well as square.

[0044] The prediction unit (120) predicts the current block and generates a prediction block. The prediction unit (120) includes an intra prediction unit (122) and an inter prediction unit (124).

[0045] In general, each current block within a picture can be predictively coded. Prediction of the current block can typically be performed using either intra-prediction (using data from the picture containing the current block) or inter-prediction (using data from a picture coded before the picture containing the current block). Inter-prediction encompasses both unidirectional and bidirectional prediction.

[0046] The intra prediction unit (122) predicts pixels within the current block using pixels (reference pixels) located around the current block within the current picture including the current block. There are multiple intra prediction modes depending on the prediction direction. For example, as shown in Fig. 3a, the multiple intra prediction modes may include two non-directional modes including the Planar mode and the DC mode, and 65 directional modes. The surrounding pixels to be used and the calculation formula are defined differently depending on each prediction mode.

[0047] For efficient directional prediction for a rectangular current block, directional modes (intra prediction modes 67 to 80 and -1 to -14) indicated by dotted arrows in Fig. 3b may be additionally used. These may be referred to as "wide-angle intra-prediction modes." In Fig. 3b, the arrows point to corresponding reference samples used for prediction, and do not indicate the prediction direction. The prediction direction is opposite to the direction indicated by the arrows. Wide-angle intra-prediction modes are modes that perform prediction in the opposite direction of a specific directional mode without additional bit transmission when the current block is rectangular. At this time, among the wide-angle intra-prediction modes, some wide-angle intra-prediction modes available for the current block may be determined based on the ratio of the width and height of the rectangular current block. For example, wide-angle intra prediction modes (intra prediction modes 67 to 80) having an angle less than 45 degrees are available when the current block is a rectangular shape whose height is smaller than its width, and wide-angle intra prediction modes (intra prediction modes -1 to -14) having an angle greater than -135 degrees are available when the current block is a rectangular shape whose width is larger than its height.

[0048] The intra prediction unit (122) can determine an intra prediction mode to be used to encode the current block. In some examples, the intra prediction unit (122) can encode the current block using multiple intra prediction modes and select an appropriate intra prediction mode to be used from the tested modes. For example, the intra prediction unit (122) can calculate bit-rate distortion values ​​using rate-distortion analysis for multiple tested intra prediction modes and select an intra prediction mode with the best bit-rate distortion characteristics among the tested modes.

[0049] The intra prediction unit (122) selects one intra prediction mode from among multiple intra prediction modes and predicts the current block using surrounding pixels (reference pixels) and an operation formula determined according to the selected intra prediction mode. Information about the selected intra prediction mode is encoded by the entropy encoding unit (155) and transmitted to the image decoding device.

[0050] The inter prediction unit (124) generates a prediction block for the current block using a motion compensation process. The inter prediction unit (124) searches for a block most similar to the current block within reference pictures that were encoded and decoded before the current picture, and generates a prediction block for the current block using the searched block. Then, a motion vector (MV) corresponding to the displacement between the current block within the current picture and the prediction block within the reference picture is generated. Generally, motion estimation is performed on the luma component, and the motion vector calculated based on the luma component is used for both the luma component and the chroma component. The motion information including information on the reference picture used to predict the current block and information on the motion vector is encoded by the entropy encoding unit (155) and transmitted to the image decoding device.

[0051] The inter prediction unit (124) may perform interpolation on a reference picture or a reference block to improve prediction accuracy. That is, subsamples between two consecutive integer samples are interpolated by applying filter coefficients to a plurality of consecutive integer samples including the two integer samples. When a process of searching for a block most similar to the current block is performed on the interpolated reference picture, the motion vector can be expressed up to a precision in decimal units rather than a precision in integer sample units. The precision or resolution of the motion vector can be set differently for each target region to be encoded, such as a slice, tile, CTU, CU, etc. When such adaptive motion vector resolution (AMVR) is applied, information on the motion vector resolution to be applied to each target region must be signaled for each target region. For example, when the target region is a CU, information on the motion vector resolution applied to each CU is signaled. Information on the motion vector resolution may be information indicating the precision of a differential motion vector, which will be described later.

[0052] Meanwhile, the inter prediction unit (124) can perform inter prediction using bi-prediction. In the case of bi-prediction, two reference pictures and two motion vectors indicating the block position most similar to the current block within each reference picture are used. The inter prediction unit (124) selects a first reference picture and a second reference picture from reference picture list 0 (RefPicList0) and reference picture list 1 (RefPicList1), respectively, and searches for a block similar to the current block within each reference picture to generate a first reference block and a second reference block. Then, the first reference block and the second reference block are averaged or weighted averaged to generate a prediction block for the current block. Then, motion information including information on two reference pictures used to predict the current block and information on two motion vectors is transmitted to the entropy encoding unit (155). Here, reference picture list 0 may be composed of pictures that are before the current picture in display order among the restored pictures, and reference picture list 1 may be composed of pictures that are after the current picture in display order among the restored pictures. However, this is not necessarily limited to this, and restored pictures that are after the current picture in display order may be additionally included in reference picture list 0, and conversely, restored pictures that are before the current picture may be additionally included in reference picture list 1.

[0053] Various methods can be used to minimize the number of bits required to encode motion information.

[0054] For example, if the reference picture and motion vector of the current block are identical to those of a neighboring block, the motion information of the current block can be transmitted to the image decoding device by encoding information that can identify the neighboring block. This method is called 'merge mode'.

[0055] In merge mode, the inter prediction unit (124) selects a predetermined number of merge candidate blocks (hereinafter referred to as 'merge candidates') from the surrounding blocks of the current block.

[0056] As the surrounding blocks for deriving merge candidates, all or part of the left block (A0), the lower left block (A1), the upper block (B0), the upper right block (B1), and the upper left block (B2) adjacent to the current block within the current picture may be used, as illustrated in FIG. 4. In addition, a block located within a reference picture (which may or may not be the same as the reference picture used to predict the current block) other than the current picture in which the current block is located may be used as a merge candidate. For example, a block co-located with the current block within the reference picture or blocks adjacent to the block at the co-located block may be additionally used as a merge candidate. If the number of merge candidates selected by the method described above is less than a preset number, a 0 vector is added to the merge candidates.

[0057] The inter prediction unit (124) uses these surrounding blocks to construct a merge list containing a predetermined number of merge candidates. Among the merge candidates included in the merge list, the merge candidate to be used as motion information of the current block is selected and merge index information for identifying the selected candidate is generated. The generated merge index information is encoded by the entropy encoding unit (155) and transmitted to the video decoding device.

[0058] Merge Skip mode is a special case of merge mode. After quantization, when all transform coefficients for entropy encoding are close to zero, only neighboring block selection information is transmitted without transmitting residual signals. By utilizing merge skip mode, relatively high encoding efficiency can be achieved for low-motion images, still images, and screen content images.

[0059] Hereinafter, merge mode and merge skip mode are collectively referred to as merge / skip mode.

[0060] Another method for encoding motion information is Advanced Motion Vector Prediction (AMVP) mode.

[0061] In AMVP mode, the inter prediction unit (124) derives predicted motion vector candidates for the motion vector of the current block using neighboring blocks of the current block. As neighboring blocks used to derive predicted motion vector candidates, all or some of the left block (A0), the lower left block (A1), the upper block (B0), the upper right block (B1), and the upper left block (B2) adjacent to the current block in the current picture as shown in FIG. 4 may be used. In addition, a block located in a reference picture (which may or may not be the same as the reference picture used to predict the current block) other than the current picture in which the current block is located may be used as the neighboring block used to derive predicted motion vector candidates. For example, a block co-located with the current block in the reference picture or blocks adjacent to the block in the co-located block may be used. If the number of motion vector candidates is less than a preset number by the method described above, a 0 vector is added to the motion vector candidates.

[0062] The inter prediction unit (124) derives predicted motion vector candidates using the motion vectors of these surrounding blocks, and determines a predicted motion vector for the motion vector of the current block using the predicted motion vector candidates. Then, the predicted motion vector is subtracted from the motion vector of the current block to produce a differential motion vector.

[0063] The predicted motion vector can be obtained by applying a predefined function (e.g., median, mean, etc.) to the predicted motion vector candidates. In this case, the image decoding device also knows the predefined function. In addition, since the surrounding blocks used to derive the predicted motion vector candidates are blocks that have already been encoded and decoded, the image decoding device also already knows the motion vectors of the surrounding blocks. Therefore, the image encoding device does not need to encode information to identify the predicted motion vector candidates. Therefore, in this case, information about the differential motion vector and information about the reference picture used to predict the current block are encoded.

[0064] Alternatively, the predicted motion vector can be determined by selecting one of the predicted motion vector candidates. In this case, information for identifying the selected predicted motion vector candidate is additionally encoded, along with information about the differential motion vector and the reference picture used to predict the current block.

[0065] The subtractor (130) subtracts the prediction block generated by the intra prediction unit (122) or inter prediction unit (124) from the current block to generate a residual block.

[0066] The transformation unit (140) transforms residual signals within a residual block having pixel values ​​in a spatial domain into transform coefficients in a frequency domain. The transformation unit (140) may transform the residual signals within the residual block using the entire size of the residual block as a transformation unit, or may divide the residual block into a plurality of sub-blocks and use the sub-blocks as transformation units to perform the transformation. Alternatively, the residual signals may be transformed using only the transformation domain sub-block as a transformation unit by dividing the sub-blocks into two sub-blocks, that is, a transformation domain and a non-transform domain. Here, the transformation domain sub-block may be one of two rectangular blocks having a size ratio of 1:1 with respect to the horizontal axis (or vertical axis). In this case, a flag (cu_sbt_flag) indicating that only a sub-block has been converted, directionality (vertical / horizontal) information (cu_sbt_horizontal_flag), and / or position information (cu_sbt_pos_flag) are encoded by the entropy encoding unit (155) and signaled to the image decoding device. In addition, the size of the conversion area sub-block may have a size ratio of 1:3 with respect to the horizontal axis (or vertical axis), and in this case, a flag (cu_sbt_quad_flag) distinguishing the corresponding division is additionally encoded by the entropy encoding unit (155) and signaled to the image decoding device.

[0067] Meanwhile, the transformation unit (140) can individually perform transformations on the residual block in the horizontal and vertical directions. For the transformation, various types of transformation functions or transformation matrices can be used. For example, a pair of transformation functions for horizontal transformation and vertical transformation can be defined as a Multiple Transform Set (MTS). The transformation unit (140) can select one transformation function pair with the best transformation efficiency among the MTS and transform the residual block in the horizontal and vertical directions, respectively. Information (mts_idx) on the transformation function pair selected among the MTS is encoded by the entropy encoding unit (155) and signaled to the image decoding device.

[0068] The quantization unit (145) quantizes the transform coefficients output from the transform unit (140) using quantization parameters and outputs the quantized transform coefficients to the entropy encoding unit (155). The quantization unit (145) may directly quantize a related residual block without transformation for a certain block or frame. The quantization unit (145) may also apply different quantization coefficients (scaling values) according to the positions of the transform coefficients within the transform block. The quantization matrix applied to the quantized transform coefficients arranged in two dimensions may be encoded and signaled to an image decoding device.

[0069] The rearrangement unit (150) can perform rearrangement of coefficient values ​​for quantized residual values.

[0070] The reordering unit (150) can change a two-dimensional coefficient array into a one-dimensional coefficient sequence by using coefficient scanning. For example, the reordering unit (150) can output a one-dimensional coefficient sequence by scanning from the DC coefficient to the coefficients of the high-frequency region by using a zig-zag scan or a diagonal scan. Depending on the size of the transformation unit and the intra prediction mode, a vertical scan that scans the two-dimensional coefficient array in the column direction or a horizontal scan that scans the two-dimensional block-shaped coefficients in the row direction may be used instead of the zig-zag scan. That is, depending on the size of the transformation unit and the intra prediction mode, the scanning method to be used may be determined among the zig-zag scan, the diagonal scan, the vertical scan, and the horizontal scan.

[0071] The entropy encoding unit (155) generates a bitstream by encoding a sequence of one-dimensional quantized transform coefficients output from the rearrangement unit (150) using various encoding methods such as CABAC (Context-based Adaptive Binary Arithmetic Code) and Exponential Golomb.

[0072] In addition, the entropy encoding unit (155) encodes information related to block division, such as CTU size, CU division flag, QT division flag, MTT division type, and MTT division direction, so that the image decoding device can divide the block in the same manner as the image encoding device. In addition, the entropy encoding unit (155) encodes information about the prediction type indicating whether the current block is encoded by intra prediction or inter prediction, and encodes intra prediction information (i.e., information about the intra prediction mode) or inter prediction information (information about the encoding mode of motion information (merge mode or AMVP mode), a merge index in the case of the merge mode, and a reference picture index and a differential motion vector in the case of the AMVP mode) according to the prediction type. In addition, the entropy encoding unit (155) encodes information related to quantization, that is, information about quantization parameters and information about a quantization matrix.

[0073] The inverse quantization unit (160) inversely quantizes the quantized transform coefficients output from the quantization unit (145) to generate transform coefficients. The inverse transform unit (165) transforms the transform coefficients output from the inverse quantization unit (160) from the frequency domain to the spatial domain to restore the residual block.

[0074] An adder (170) adds the restored residual block and the predicted block generated by the prediction unit (120) to restore the current block. The pixels within the restored current block are used as reference pixels when intra-predicting the next block.

[0075] The loop filter unit (180) performs filtering on restored pixels to reduce blocking artifacts, ringing artifacts, blurring artifacts, etc. that occur due to block-based prediction and transformation / quantization. The loop filter unit (180) may include all or part of a deblocking filter (182), a sample adaptive offset (SAO) filter (184), and an adaptive loop filter (ALF, 186) as an in-loop filter.

[0076] The deblocking filter (182) filters the boundaries between restored blocks to remove blocking artifacts caused by block-based encoding / decoding, and the SAO filter (184) and the ALF (186) perform additional filtering on the deblocking-filtered image. The SAO filter (184) and the ALF (186) are filters used to compensate for the differences between restored pixels and original pixels caused by lossy coding. The SAO filter (184) improves not only subjective image quality but also encoding efficiency by applying an offset in units of CTUs. In contrast, the ALF (186) performs block-based filtering, and compensates for distortion by applying different filters by distinguishing the edges and degrees of variation of the corresponding block. Information on filter coefficients to be used in the ALF can be encoded and signaled to an image decoding device.

[0077] The restored blocks filtered through the deblocking filter (182), SAO filter (184), and ALF (186) are stored in the memory (190). When all blocks within a picture are restored, the restored picture can be used as a reference picture for inter-predicting blocks within a picture to be encoded later.

[0078] The video encoding device can store the bitstream of encoded video data on a non-transitory storage medium or transmit it to the video decoding device using a communication network.

[0079] FIG. 5 is an exemplary block diagram of an image decoding device capable of implementing the techniques of the present disclosure. Hereinafter, the image decoding device and its subcomponents will be described with reference to FIG. 5.

[0080] The video decoding device may be configured to include an entropy decoding unit (510), a rearrangement unit (515), an inverse quantization unit (520), an inverse transformation unit (530), a prediction unit (540), an adder (550), a loop filter unit (560), and a memory (570).

[0081] Similar to the video encoding device of FIG. 1, each component of the video decoding device may be implemented in hardware, software, or a combination of hardware and software. Furthermore, the functions of each component may be implemented in software, with a microprocessor executing the software functions corresponding to each component.

[0082] The entropy decoding unit (510) decodes the bitstream generated by the image encoding device to extract information related to block division, thereby determining the current block to be decoded, and extracts prediction information, information on residual signals, etc. required to restore the current block.

[0083] The entropy decoding unit (510) extracts information about the CTU size from the Sequence Parameter Set (SPS) or the Picture Parameter Set (PPS), determines the size of the CTU, and divides the picture into CTUs of the determined size. Then, the CTU is determined as the top layer of the tree structure, i.e., the root node, and the CTU is divided using the tree structure by extracting division information about the CTU.

[0084] For example, when splitting a CTU using the QTBTTT structure, first, the first flag (QT_split_flag) related to the splitting of QT is extracted, and each node is split into four nodes of the lower layer. Then, for the nodes corresponding to the leaf nodes of QT, the second flag (mtt_split_flag) related to the splitting of MTT and the split direction (vertical / horizontal) and / or split type (binary / ternary) information are extracted, and the corresponding leaf nodes are split into the MTT structure. Accordingly, each node below the leaf nodes of QT are split recursively into the BT or TT structure.

[0085] As another example, when splitting a CTU using the QTBTTT structure, the CU split flag (split_cu_flag) indicating whether the CU is split is first extracted, and if the block is split, the first flag (QT_split_flag) may be extracted. During the splitting process, each node may undergo zero or more repeated QT splits followed by zero or more repeated MTT splits. For example, a CTU may undergo an MTT split right away, or conversely, may undergo only multiple QT splits.

[0086] As another example, when splitting a CTU using the QTBT structure, the first flag (QT_split_flag) related to the splitting of QT is extracted, and each node is split into four nodes of the lower layer. Furthermore, for nodes corresponding to leaf nodes of QT, a split flag (split_flag) indicating whether to further split into BTs and splitting direction information are extracted.

[0087] Meanwhile, when the entropy decoding unit (510) determines the current block to be decoded by using the division of the tree structure, it extracts information on the prediction type indicating whether the current block is intra-predicted or inter-predicted. If the prediction type information indicates intra-prediction, the entropy decoding unit (510) extracts syntax elements for intra-prediction information (intra-prediction mode) of the current block. If the prediction type information indicates inter-prediction, the entropy decoding unit (510) extracts syntax elements for inter-prediction information, i.e., information indicating a motion vector and a reference picture referenced by the motion vector.

[0088] Additionally, the entropy decoding unit (510) extracts information about the quantized transform coefficients of the current block as information related to quantization and information about residual signals.

[0089] The rearrangement unit (515) can change the sequence of one-dimensional quantized transform coefficients entropy-decoded in the entropy decoding unit (510) back into a two-dimensional coefficient array (i.e., block) in the reverse order of the coefficient scanning performed by the image encoding device.

[0090] The inverse quantization unit (520) inversely quantizes the quantized transform coefficients and inversely quantizes the quantized transform coefficients using the quantization parameters. The inverse quantization unit (520) may also apply different quantization coefficients (scaling values) to the quantized transform coefficients arranged in two dimensions. The inverse quantization unit (520) may perform inverse quantization by applying a matrix of quantized coefficients (scaling values) from an image encoding device to a two-dimensional array of quantized transform coefficients.

[0091] The inverse transform unit (530) inversely transforms the inverse quantized transform coefficients from the frequency domain to the spatial domain to restore residual signals, thereby generating a residual block for the current block.

[0092] In addition, when the inverse transform unit (530) inversely transforms only a portion of a transform block (sub-block), it extracts a flag (cu_sbt_flag) indicating that only a sub-block of the transform block has been transformed, directionality (vertical / horizontal) information (cu_sbt_horizontal_flag) of the sub-block, and / or position information (cu_sbt_pos_flag) of the sub-block, and inversely transforms the transform coefficients of the corresponding sub-block from the frequency domain to the spatial domain to restore residual signals, and fills “0” values ​​with residual signals for areas that have not been inversely transformed, thereby generating a final residual block for the current block.

[0093] In addition, when MTS is applied, the inverse transform unit (530) determines a transform function or a transform matrix to be applied in the horizontal and vertical directions using MTS information (mts_idx) signaled from the image encoding device, and performs inverse transform on the transform coefficients within the transform block in the horizontal and vertical directions using the determined transform function.

[0094] The prediction unit (540) may include an intra prediction unit (542) and an inter prediction unit (544). The intra prediction unit (542) is activated when the prediction type of the current block is intra prediction, and the inter prediction unit (544) is activated when the prediction type of the current block is inter prediction.

[0095] The intra prediction unit (542) determines the intra prediction mode of the current block among a plurality of intra prediction modes from the syntax elements for the intra prediction mode extracted from the entropy decoding unit (510), and predicts the current block using reference pixels around the current block according to the intra prediction mode.

[0096] The inter prediction unit (544) uses the syntax elements for the inter prediction mode extracted from the entropy decoding unit (510) to determine the motion vector of the current block and the reference picture referenced by the motion vector, and predicts the current block using the motion vector and the reference picture.

[0097] An adder (550) adds the residual block output from the inverse transform unit (530) and the predicted block output from the inter prediction unit (544) or the intra prediction unit (542) to restore the current block. The pixels within the restored current block are used as reference pixels when intra-predicting a block to be decoded later.

[0098] The loop filter unit (560) may include a deblocking filter (562), an SAO filter (564), and an ALF (566) as in-loop filters. The deblocking filter (562) deblocks the boundaries between restored blocks to remove blocking artifacts caused by block-by-block decoding. The SAO filter (564) and the ALF (566) perform additional filtering on restored blocks after deblocking filtering to compensate for differences between restored pixels and original pixels caused by lossy coding. The filter coefficients of the ALF are determined using information about filter coefficients decoded from the non-stream.

[0099] The restored blocks filtered through the deblocking filter (562), SAO filter (564), and ALF (566) are stored in the memory (570). When all blocks within a picture are restored, the restored picture is used as a reference picture for inter-predicting blocks within a picture to be encoded later.

[0100] The present embodiment relates to encoding and decoding of images (video) as described above. More specifically, a video coding method and device are provided, which, when predicting using a reference block within the same frame, calculates luma changes and chroma changes between a current block and a reference block, models the correlation between the luma changes and the chroma changes, and corrects a chroma predictor based on the modeled correlation.

[0101] The following embodiments may be performed by an intra prediction unit (122) within a video encoding apparatus. In addition, the following embodiments may be performed by an intra prediction unit (542) within a video decoding apparatus.

[0102] The video encoding device can generate signaling information related to the present embodiment in terms of rate distortion optimization in encoding the current block. The video encoding device can encode the signaling information using the entropy encoding unit (155) and then transmit it to the video decoding device. The video decoding device can decode the signaling information related to the decoding of the current block from the bitstream using the entropy decoding unit (510).

[0103] In the following description, the term "target block" may be used interchangeably with the current block or coding unit (CU). Alternatively, the term "target block" may also refer to a portion of a coding unit.

[0104] Also, a value of a flag being true indicates that the flag is set to 1. Also, a value of a flag being false indicates that the flag is set to 0.

[0105] I. Intra Block Copy (IBC) and IntraTMP

[0106] IBC performs intra prediction of the current block by copying a reference block within the same frame using a block vector (BV) to generate a prediction block of the current block.

[0107] A video encoding device performs block matching to derive an optimal block vector. Here, the block vector represents the displacement from the current block to the reference block. In order to increase encoding efficiency, the video encoding device may not transmit the block vector as is, but may divide it into a block vector predictor (BVP) and a block vector difference (BVD), encode the BVP and BVD, and then transmit them to the video decoding device.

[0108] In terms of utilizing block vectors, IBC possesses the characteristics of inter prediction. Therefore, IBC can be divided into IBC Merge / Skip mode and IBC AMVP mode.

[0109] In IBC merge / skip mode, a video encoding device constructs an IBC merge list. In order to optimize encoding efficiency, the video encoding device can select a block vector from among the candidates included in the IBC merge list and use the selected candidate as a block vector predictor (BVP). The video encoding device determines a merge index indicating the selected block vector. However, the video encoding device does not generate a BVD. The video encoding device encodes the merge index and transmits it to the video decoding device. The IBC merge list can be constructed by the video encoding device and the video decoding device in the same manner. The video decoding device can decode the merge index and then generate a block vector from the IBC merge list using the merge index.

[0110] The video encoding device uses the same block vector transmission method as the IBC merge mode in the case of the IBC skip mode, but does not transmit a residual block corresponding to the difference between the current block and the predicted block.

[0111] In IBC AMVP mode, in order to optimize encoding efficiency, the video encoding device determines a block vector and constructs an IBC AMVP list. The video encoding device determines a candidate index that designates one of the candidate block vectors included in the IBC AMVP list as a BVP. The video encoding device calculates the BVD, which is the difference between the BVP and the block vector. Thereafter, the video encoding device encodes the candidate index and BVD and transmits them to the video decoding device.

[0112] The video decoding device decodes the candidate index and BVD. The video decoding device can obtain the BVP indicated by the candidate index from the IBC AMVP list, and then restore the block vector by adding the BVP and BVD.

[0113] In the Enhanced Compression Model (ECM), which corresponds to the next-generation VVC (beyond VVC), the IntraTMP (Intra Template Matching Prediction) technology sets an L-shaped / left / top template around the current block. IntraTMP technology searches for the template most similar to the current template in the restoration area of ​​the current frame, and then uses a reference block adjacent to the searched template and having the same size as the current block as the prediction block for the current block. IntraTMP technology searches for similar templates based on a loss function, and SAD (Sum of Absolute Differences) can be used as the loss function.

[0114] The following embodiments are described with a focus on an image decoding device, but can be implemented identically or similarly in an image encoding device.

[0115] Hereinafter, linear models and linear functions representing inter-component correlations can be used interchangeably. Hereinafter, nonlinear models and nonlinear functions representing inter-component correlations can be used interchangeably.

[0116] II. Embodiments according to the present disclosure

[0117] If there is a brightness difference between the luma region corresponding to the current block and the luma region corresponding to the reference block (within the same frame), the relationship between the luma and chroma components in the reference block may differ from the relationship between the luma and chroma components in the current block. When setting a reference block for the current chroma block, this may contradict the assumption that the correlation between the luma and chroma components is maintained in the current block. In other words, optimal prediction may not be performed for the current block.

[0118] In addition, when information is acquired from the luma region corresponding to the current block to construct a reference block for the current chroma component and chroma prediction is performed using the constructed reference block, the correlation between the luma component and the chroma component of the current block may differ from the correlation between the luma component and the chroma component of the current block and the adjacent block. If there is an adjacent block for which prediction is performed based on a correlation different from that of the current block, a discontinuity may occur between predictors at the boundary between the current block and the adjacent block. In other words, depending on the discontinuity, the prediction for the current block may not be optimal.

[0119] To improve the encoding (or decoding) performance of a predictor using a reference block in a chroma channel, embodiments according to the present disclosure are proposed. The present disclosure can be applied to searching for a reference block within the same component region, i.e., within the same frame, and to prediction using the searched reference block. An image encoding / decoding device according to the present disclosure calculates a luma change amount and a chroma change amount between a current block and a reference block, models the correlation between the luma change amount and the chroma change amount, and corrects a chroma predictor based on the modeled correlation.

[0120] The following embodiments are described with a focus on an image decoding device, but can be implemented identically or similarly in an image encoding device.

[0121] According to the correction of the chroma predictor according to the present disclosure, the final chroma predictor can be expressed as in mathematical expression 1.

[0122]

[0123] In Equation 1, Predicted C represents the final chroma predictor. Reference-based chroma sample values, i.e., the chroma predictor is estimated from identical components of the reference block, and the compensating value is estimated based on non-identical components. At this time, the video decoding device can calculate the final chroma predictor based on the following steps.

[0124] 1. Configure the adjacent template of the current block and the adjacent template of the reference block.

[0125] 2. Estimate the parameters that define the correlation.

[0126] 3. Calculate the correction value of the chroma component using the estimated function.

[0127] 4. Correct the chroma predictor by adding the correction value and the chroma predictor.

[0128] Hereinafter, the current block includes the current luma block and the current chroma block. The reference block includes the reference luma block and the reference chroma block.

[0129] Hereinafter, examples for calculating correction values ​​according to the present disclosure are described.

[0130] Example 1: Modeling the correlation between the change in luma value and the change in chroma value using the surrounding area of ​​the current block and the surrounding area of ​​the reference block.

[0131] This implementation example relates to a method for constructing a reference block of a current block, and a method for modeling the correlation between luma value changes and chroma value changes using the surrounding area of ​​the current block and the surrounding area of ​​the reference block. Implementation Example 1-1 models the correlation using a single function and calibrates a chroma predictor based on the modeled correlation. Implementation Example 1-2 models the correlation using multiple functions and calibrates a chroma predictor using the modeled correlation.

[0132] Example 1-1: Modeling Correlation Using a Single Function

[0133] This implementation example relates to a method for constructing a reference block for a current block, and a method for using a single function to model the correlation between luma value changes and chroma value changes using the surrounding areas of the current block and the surrounding areas of the reference block. Examples of single functions for modeling correlation include linear functions and nonlinear functions.

[0134] As an example, the image decoding device can model the correlation using a linear function and calibrate the chroma predictor based on the modeled correlation.

[0135] FIG. 6 is a flowchart illustrating a method by which an image decoding device restores a current chroma block according to one embodiment of the present disclosure.

[0136] The video decoding device configures an adjacent template of the current block and an adjacent template of the reference block (S600).

[0137] The video decoding device configures a reference block as follows. The video decoding device can obtain block vector information from a reconstructed luma block (hereinafter, "corresponding luma block") corresponding to the current chroma block. As a method for obtaining the block vector information, there are a method of obtaining block vector information by searching five positions (TL (top left), TR (top right), C, BL (bottom left), BR (bottom right)) within the corresponding luma area (hereinafter, "corresponding luma area") as shown in FIG. 7, a method of searching only the center (C) position of the corresponding luma area, and a method of searching all pixels existing within the corresponding luma area. In order for a block vector to exist within the corresponding luma area, a block at the search position must be predicted according to the IBC mode, the IntraTMP mode, etc.

[0138] After acquiring BV information, the video decoding device configures the region indicated by the BV as a reference block. The video decoding device can utilize the reference chroma block as the current chroma predictor. The video decoding device can configure the adjacent templates of the reference block and the adjacent templates of the current block as regions for estimating the parameters of a linear function. For example, the adjacent templates may be the reconstructed pixels located above and / or to the left of the reference block / current block.

[0139] The image decoding device can perform the following steps to estimate α and β, which are parameters defining a linear model.

[0140] The video decoding device calculates the chroma variation between the template of the current block and the template of the reference block (S602). The chroma variation between the chroma template of the current block and the chroma template of the reference block is dC tem is defined as

[0141] The video decoding device calculates the luma variation between the template of the current block and the template of the reference block (S604). The luma variation between the luma template of the current block and the luma template of the reference block is dY tem is defined as

[0142] The image decoding device estimates the parameters of the linear model based on the changes (S606).

[0143] The video decoding device uses chroma and luma variations (dC tem , dY tem ) are used to estimate the parameters α and β of the linear model. In relation to the estimation of the parameters, the linear model expressed in mathematical expression 2 can be used.

[0144]

[0145] The image decoding device calculates a correction value based on the estimated parameters of the linear model, and corrects the current chroma predictor using the correction value (S608).

[0146] The video decoding device generates a compensating value for the current chroma predictor based on the parameters (α, β) of the linear model. The video decoding device can generate the final chroma predictor for the current block by compensating the current chroma predictor by adding the compensating value to the current chroma predictor. The compensation of the current chroma predictor can be expressed as shown in Equation 3.

[0147]

[0148] In mathematical expression 3, dY represents the amount of change between the current luma block and the reference luma block, and dC represents the amount of change between the current chroma block and the reference chroma block based on the linear model, i.e., the correction value. The video decoding device can generate the correction value by applying the linear model to the amount of change between the current luma block and the reference luma block.

[0149] The correction of the current chroma predictor described above can be illustrated as in Fig. 8. In Fig. 8, dC tem , dY tem depends on equation 2, and dC and dY depend on equation 3.

[0150] As another example, an image decoding device can model correlations using nonlinear functions and calibrate chroma predictors based on the modeled correlations. The image decoding device constructs the nonlinear function using spatial components, nonlinear terms, and bias terms. For example, if five spatial components, five nonlinear terms, and one bias term are used, an 11-tap convolution filter can be used.

[0151] FIG. 9 is a flowchart illustrating a method by which a video decoding device restores a current chroma block according to another embodiment of the present disclosure.

[0152] The video decoding device configures an adjacent template of the current block and an adjacent template of the reference block (S900). As described above (see step S600), the video decoding device can configure a reference block and an adjacent template of the current block and a reference block template.

[0153] The image decoding device defines the parameters p0, p1, ..., p that define the nonlinear model. l The following steps can be taken to estimate .

[0154] The video decoding device calculates the chroma variation between the template of the current block and the template of the reference block (S902). The chroma variation between the chroma template of the current block and the chroma template of the reference block is dC tem is defined as

[0155] The video decoding device calculates the luma variation between the template of the current block and the template of the reference block (S904). The luma variation between the luma template of the current block and the luma template of the reference block is dY tem is defined as

[0156] The image decoding device estimates the parameters of the nonlinear model based on the changes (S906).

[0157] The video decoding device uses chroma and luma variations (dC tem , dY tem ) based on the parameters of the nonlinear model, p0, p1, ..., p l is estimated. In relation to the estimation of parameters, the nonlinear model expressed in mathematical expression 4 can be used.

[0158]

[0159] In mathematical expression 4, dY tem is a non-square matrix with size n×(l+1), and dC tem is a non-square matrix (or column vector) with a size of n×1. Here, n represents the number of samples in the template for deriving the parameters of the nonlinear function, and (l+1) represents the number of parameters of the nonlinear function.

[0160] The image decoding device calculates a correction value based on the estimated parameters of the nonlinear model, and corrects the current chroma predictor using the correction value (S908).

[0161] The image decoding device is a nonlinear model with parameters (p0, p1, ..., p l ) to generate a correction value for the current chroma predictor. The video decoding device can generate a final chroma predictor of the current block by correcting the current chroma predictor by adding the correction value and the current chroma predictor. The correction of the current chroma predictor can be expressed as in mathematical expression 5.

[0162]

[0163] In mathematical expression 5, dY represents the amount of change between the current luma block and the reference luma block, and dC represents the amount of change between the current chroma block and the reference chroma block based on the nonlinear model, i.e., the correction value. The video decoding device can generate the correction value by applying the nonlinear model to the amount of change between the current luma block and the reference luma block.

[0164] The correction of the current chroma predictor described above can be illustrated as in Fig. 10. In Fig. 10, dC tem , dY tem depends on equation 4, and dC and dY depend on equation 5.

[0165] Example 1-2: Modeling Correlation Using Multiple Functions

[0166] This implementation example relates to a method for constructing a reference block for a current block, and a method for using multiple functions to model the correlation between luma value changes and chroma value changes using the surrounding area of ​​the current block and the surrounding area of ​​the reference block. Examples of functions for modeling the correlation include linear functions and nonlinear functions.

[0167] In Example 1-1, a single linear function is used to model correlations. However, when correlations with diverse characteristics exist, it may be difficult to estimate the optimal correction value based on Example 1-1. As an example of solving this problem, an image decoding device can model correlations using multiple linear functions and correct the chroma predictor based on the modeled correlations.

[0168] FIG. 11 is a flowchart illustrating a method for a video decoding device to restore a current chroma block according to another embodiment of the present disclosure.

[0169] The video decoding device configures an adjacent template of the current block and an adjacent template of the reference block (S1100). As described in Implementation Example 1-1, the video decoding device can configure a reference block and an adjacent template of the current block and a reference block template.

[0170] The image decoding device defines a number of linear models, which are parameters α k , β k The following steps can be performed to estimate . Here, k (1≤k≤K, K is a positive integer greater than or equal to 2) is an index representing a sample interval.

[0171] The image decoding device sets a threshold applied to the derivation of multiple linear models (S1102).

[0172] The threshold can be set based on sample values. That is, the threshold can be set based on the average / median of some / all of the upper left samples of the current block, the average / median of some / all of the upper left sample values ​​of the corresponding luma region, the average / median of some / all of the adjacent templates, etc. Based on the threshold, the image decoding device can determine K sample sections corresponding to multiple linear models.

[0173] The video decoding device calculates the chroma variation between the template of the current block and the template of the reference block (S1104). The chroma variation between the chroma template of the current block and the chroma template of the reference block is dC in the sample interval k. tem,k is defined as

[0174] The video decoding device calculates the luma variation between the template of the current block and the template of the reference block (S1106). The luma variation between the luma template of the current block and the luma template of the reference block is dY in the sample interval k. tem,k is defined as

[0175] The image decoding device estimates parameters of multiple linear models based on the changes (S1108).

[0176] The video decoding device uses chroma and luma variations (dC tem,k , dY tem,k ) based on the parameters of each linear model, α k , β k In relation to the estimation of parameters in the sample interval k, the linear model expressed in mathematical expression 6 can be used.

[0177]

[0178] The image decoding device calculates a correction value based on estimated parameters of a plurality of linear models, and corrects the current chroma predictor using the correction value (S1110).

[0179] The image decoding device uses a number of linear model parameters (α k , β k ) to generate a correction value for the current chroma predictor. The video decoding device can generate a final chroma predictor of the current block by correcting the current chroma predictor by adding the correction value and the current chroma predictor. The correction of the current chroma predictor can be expressed as in mathematical expression 7.

[0180]

[0181] In mathematical expression 7, dY represents the amount of change between the current luma block and the reference luma block, and dC represents the amount of change between the current chroma block and the reference chroma block based on the linear model in the sample interval k, i.e., the correction value. The video decoding device can generate the correction value by applying multiple linear models to the amount of change between the current luma block and the reference luma block.

[0182] The correction of the current chroma predictor described above can be illustrated as in Fig. 12. In Fig. 12, dC tem,k , dY tem,k depends on equation 6, and dC and dY depend on equation 7.

[0183] As another example, the image decoding device can model correlations using multiple nonlinear functions and calibrate the chroma predictor based on the modeled correlations.

[0184] FIG. 13 is a flowchart illustrating a method by which a video decoding device restores a current chroma block according to another embodiment of the present disclosure.

[0185] The video decoding device configures an adjacent template of the current block and an adjacent template of the reference block (S1300). As described in Implementation Example 1-1, the video decoding device can configure a reference block and an adjacent template of the current block and a reference block template.

[0186] The image decoding device defines a number of nonlinear models, which are parameters p 0,k , p 1,k , ..., p l,k The following steps can be performed to estimate . Here, k (1≤k≤K, K is a positive integer greater than or equal to 2) represents the number of sample intervals.

[0187] The image decoding device sets a threshold applied to the derivation of multiple nonlinear models (S1302).

[0188] As described above, the threshold can be set based on the sample value. The image decoding device can determine K sample intervals corresponding to multiple nonlinear models based on the threshold.

[0189] The video decoding device calculates the chroma variation between the template of the current block and the template of the reference block (S1304). The chroma variation between the chroma template of the current block and the chroma template of the reference block is dC in the sample interval k. tem,k is defined as

[0190] The video decoding device calculates the luma variation between the template of the current block and the template of the reference block (S1306). The luma variation between the luma template of the current block and the luma template of the reference block is dY in the sample interval k. tem,k is defined as

[0191] The image decoding device estimates parameters of multiple nonlinear models based on the changes (S1308).

[0192] The video decoding device uses chroma and luma variations (dC tem,k , dY tem,k ) based on the parameters of each nonlinear model, p 0,k , p 1,k , ..., p l,k In relation to the estimation of parameters in the sample interval k, the nonlinear model expressed in Equation 8 can be used.

[0193]

[0194] In mathematical expression 8, dY tem,k is a non-square matrix with size n×(l+1), and dC tem,k is a non-square matrix (or column vector) with a size of n×1. Here, n represents the number of samples in the template for deriving the parameters of the nonlinear function, and (l+1) represents the number of parameters of the nonlinear function.

[0195] The image decoding device calculates a correction value based on estimated parameters of a plurality of nonlinear models, and corrects the current chroma predictor using the correction value (S1310).

[0196] The image decoding device is a nonlinear model parameter (p 0,k , p 1,k , ..., p l,k) to generate a correction value for the current chroma predictor. The video decoding device can generate a final chroma predictor of the current block by correcting the current chroma predictor by adding the correction value and the current chroma predictor. The correction of the current chroma predictor can be expressed as in mathematical expression 9.

[0197]

[0198] In mathematical expression 9, dY represents the amount of change between the current luma block and the reference luma block, and dC represents the amount of change between the current chroma block and the reference chroma block based on the nonlinear model in the sample interval k, i.e., the correction value. The video decoding device can generate the correction value by applying multiple nonlinear models to the amount of change between the current luma block and the reference luma block.

[0199] The correction of the current chroma predictor described above can be illustrated as in Fig. 14. In Fig. 14, dC tem,k , dY tem,k depends on equation 8, and dC and dY depend on equation 9.

[0200] Implementation Example 2: Modeling the correlations between changes in luma values ​​and changes in chroma values ​​using the surrounding areas of the current block and the surrounding areas of multiple reference blocks.

[0201] The present implementation example relates to a method for configuring multiple reference blocks of a current block, and a method for modeling correlations between luma value changes and chroma value changes using the surrounding areas of the current block and the surrounding areas of multiple reference blocks.

[0202] In Example 1, the video decoding device configures one reference block, models the correlation between the luma value change and the chroma value change using the current block and the reference block, and corrects the current chroma predictor using the correlation. However, as shown in FIG. 15, when multiple reference blocks can be configured, the video decoding device can estimate a more accurate correction value based on the multiple reference blocks compared to using a single reference block. In addition, as shown in FIG. 16, when some of the reference blocks are invalid in modeling the correlation, the video decoding device can configure multiple reference blocks and estimate an optimal correction value based on the multiple reference blocks.

[0203] Hereinafter, Implementation Example 2-1 relates to a method for configuring multiple reference blocks, and Implementation Example 2-2 relates to a method for modeling correlations based on multiple reference blocks. After the correlations are modeled, the image decoding device can correct the chroma predictor based on the modeled correlations.

[0204] The correction of the chroma predictor according to Example 2 can be illustrated as in Fig. 17. In Fig. 17, dC tem,k , dY tem,k can be generated based on the template of the kth reference block and the template of the current block. Here, k (1 ≤ k ≤ K, K is a positive integer greater than or equal to 2) is an index indicating a reference block. dY represents the amount of change between the current luma block and the reference luma block, and dC represents the amount of change between the current chroma block and the reference chroma block, i.e., the correction value.

[0205] Implementation Example 2-1: How to Configure Multiple Reference Blocks

[0206] In this implementation example, the image decoding device configures a plurality of reference blocks for the current block. The image decoding device can configure the plurality of reference blocks using one or more of a method of configuring using block vector information, a method of configuring using motion vector information, and a method of configuring adjacent blocks as reference blocks.

[0207] As an example, an image decoding device can configure multiple reference blocks using block vector information. The image decoding device can search for block vector information within a corresponding luma region. For example, the image decoding device can search for block vectors as illustrated in FIG. 7. Additionally, the image decoding device can search for block vectors within a corresponding luma region according to the method illustrated in Example 1.

[0208] As another example, a video decoding device can construct multiple reference blocks using motion vector information. The video decoding device can search for motion vector information within the corresponding luma region. In this case, the method of searching for motion vectors can be replaced with a method of searching for block vectors.

[0209] As another example, an image decoding device can configure multiple reference blocks using adjacent blocks. The image decoding device can configure multiple reference blocks using adjacent blocks, such as the leftmost, topmost, and upper-leftmost blocks of the current block. In this case, if an adjacent block is an unrestored area, that area is not configured as a reference block.

[0210] Implementation Example 2-2: Modeling Correlations for Multiple Reference Blocks

[0211] In this implementation example, the image decoding device models correlations for multiple reference blocks. At this time, the multiple reference blocks can be configured according to Implementation Example 2-1.

[0212] As a method of modeling correlation, a method of modeling one correlation for all of a plurality of reference blocks or a method of modeling a correlation for each of a plurality of reference blocks can be used.

[0213] When modeling a single correlation for multiple reference blocks, the image decoding device can model the correlation using Example 1-1. For example, in FIG. 17, one model can represent a linear model or a nonlinear model.

[0214] When modeling correlations for each of multiple reference blocks, the video decoding device can model the correlations for each reference block using Implementation Example 1-1. For example, in FIG. 17, each model can represent a linear model or a nonlinear model. The video decoding device can model the final correlation by weighting the derived correlations. The video decoding device can model the final correlation by performing a weighted sum of the correlations using preset or signaled weights.

[0215] A video decoding device can decode quantized transform coefficients of a current chroma block from a bitstream and apply inverse quantization / inverse transformation to the quantized transform coefficients to generate a residual block. The video decoding device can reconstruct the current chroma block by adding the residual block and a corrected chroma predictor.

[0216] The above-described embodiments are described with a focus on an image decoding device, but can be implemented identically or similarly in an image encoding device.

[0217] The video encoding device can generate a residual block by subtracting a corrected chroma predictor from the current chroma block. The video encoding device can apply transformation / quantization to the residual block to generate transform coefficients and encode the generated transform coefficients.

[0218] For example, a video encoding device may obtain a flag indicating whether the current chroma predictor is corrected from a higher level. Alternatively, the video encoding device may determine a flag indicating whether the current chroma predictor is corrected from a rate-distortion optimization perspective. The video encoding device may encode a flag indicating whether the current chroma predictor is corrected.

[0219] Based on the aforementioned flag, the video encoding device can determine whether to correct the current chroma predictor. If the correction of the current chroma predictor is determined (e.g., if the aforementioned flag is true), the video encoding device can perform processes related to the correction of the current chroma predictor as described above. On the other hand, if the correction of the current chroma predictor is not determined (e.g., if the aforementioned flag is false), the video encoding device can omit the processes related to the correction of the current chroma predictor as described above.

[0220] Meanwhile, the video decoding device can decode a flag indicating whether to correct the current chroma predictor from the bitstream. Based on the aforementioned flag, the video decoding device can determine whether to correct the current chroma predictor. If the correction of the current chroma predictor is determined (e.g., if the aforementioned flag is true), the video decoding device can perform processes related to the correction of the current chroma predictor as described above. On the other hand, if the correction of the current chroma predictor is not determined (e.g., if the aforementioned flag is false), the video decoding device can omit processes related to the correction of the current chroma predictor as described above.

[0221] Although the flowchart / timing diagram of this specification describes each process as being executed sequentially, this is merely an illustrative description of the technical idea of ​​one embodiment of the present disclosure. In other words, a person of ordinary skill in the art to which one embodiment of the present disclosure belongs may modify and apply various modifications and variations by changing the order described in the flowchart / timing diagram without departing from the essential characteristics of one embodiment of the present disclosure, or by executing one or more of the processes in parallel. Therefore, the flowchart / timing diagram is not limited to a chronological order.

[0222] It should be understood that the exemplary embodiments described above can be implemented in many different ways. The functions or methods described in one or more examples can be implemented in hardware, software, firmware, or any combination thereof. It should be understood that the functional components described herein are labeled as "units" to further emphasize their implementation independence.

[0223] Meanwhile, the various functions or methods described in this embodiment may also be implemented as instructions stored on a non-transitory storage medium that can be read and executed by one or more processors. Non-transitory storage media include, for example, all types of storage devices that store data in a form readable by a computer system. For example, non-transitory storage media include storage media such as erasable programmable read-only memory (EPROM), flash drives, optical drives, magnetic hard drives, and solid-state drives (SSDs).

[0224] The above description is merely an example of the technical idea of ​​the present embodiment, and those skilled in the art will appreciate that various modifications and variations can be made without departing from the essential characteristics of the present embodiment. Therefore, the present embodiments are not intended to limit the technical idea of ​​the present embodiment, but rather to explain it, and the scope of the technical idea of ​​the present embodiment is not limited by these embodiments. The scope of protection of the present embodiment should be interpreted by the claims below, and all technical ideas within a scope equivalent thereto should be interpreted as being included in the scope of rights of the present embodiment.

[0225]

[0226]

[0227] CROSS-REFERENCE TO RELATED APPLICATION

[0228] This patent application claims priority to Korean patent application No. 10-2023-0187578, filed in Korea on December 20, 2023, and Korean patent application No. 10-2024-0172235, filed in Korea on November 27, 2024, the entire contents of which are incorporated herein by reference.

Claims

1. A method for restoring a current chroma block performed by a video decoding device, A step of obtaining a block vector of a current block based on a corresponding luma region of the current chroma block, wherein the current block includes a current luma block and the current chroma block; A step of configuring a region indicated by the above block vector as a reference block and setting the reference chroma block as a chroma predictor of the current chroma block, wherein the reference block includes a reference luma block and the reference chroma block; A step of estimating parameters expressing the correlation between the luma change amount and the chroma change amount between the template of the current block and the template of the reference block; and A step of calculating a correction value based on the estimated parameters and correcting the chroma predictor based on the correction value. A method comprising:

2. In paragraph 1, A step of decoding a flag indicating whether to correct the chroma predictor from the bitstream; and Further comprising a step of determining whether to correct the chroma predictor based on the flag; A method comprising the step of estimating parameters expressing the correlation when the correction of the above chroma predictor is determined.

3. In paragraph 1, The step of estimating the above parameters is: A step of calculating the chroma change amount between the template of the current block and the template of the reference block; A step of calculating a luma change amount between the template of the current block and the template of the reference block; and A step of estimating parameters expressing the correlation based on the above luma change amount and the above chroma change amount. A method including:

4. In paragraph 1, The above correlation is, A method represented by a linear model or a nonlinear model.

5. In paragraph 1, A step for setting a threshold applied to the derivation of multiple correlations; and A step of determining sample intervals corresponding to the plurality of correlations based on the above threshold. A method further comprising:

6. In paragraph 5, The step of estimating the above parameters is: A step of calculating the chroma change between the template of the current block and the template of the reference block in each sample interval; A step of calculating the luma change amount between the template of the current block and the template of the reference block in each of the above sample intervals; and A step of estimating parameters expressing correlation in each sample interval based on the above luma change amount and the above chroma change amount. A method comprising:

7. In paragraph 1, The step of calculating the above correction value is: a step of calculating a luma change amount between the current luma block and the reference luma block; and A step of generating the correction value from the luma variation based on the estimated parameters. A method comprising:

8. A method for encoding a current chroma block performed by a video encoding device, A step of obtaining a block vector of a current block based on a corresponding luma region of the current chroma block, wherein the current block includes a current luma block and the current chroma block; A step of configuring an area indicated by the above block vector as a reference block and setting the reference chroma block as a chroma predictor of the current chroma block, wherein the reference block includes a reference luma block and the reference chroma block; A step of estimating parameters expressing the correlation between the luma change amount and the chroma change amount between the template of the current block and the template of the reference block; and A step of calculating a correction value based on the estimated parameters and correcting the chroma predictor based on the correction value. A method comprising:

9. In paragraph 8, A step of obtaining a flag indicating whether the above chroma predictor is corrected; and Step of encoding the above flag A method further comprising:

10. In paragraph 8, Further comprising a step of determining whether to correct the chroma predictor based on the flag; A method comprising the step of estimating parameters expressing the correlation when the correction of the above chroma predictor is determined.

11. In paragraph 9, The step of estimating the above parameters is: A step of calculating the chroma change amount between the template of the current block and the template of the reference block; A step of calculating a luma change amount between the template of the current block and the template of the reference block; and A step of estimating parameters expressing the correlation based on the above luma change amount and the above chroma change amount. A method including:

12. In paragraph 8, The step of calculating the above correction value is: a step of calculating a luma change amount between the current luma block and the reference luma block; and A step of generating the correction value from the luma variation based on the estimated parameters. A method comprising:

13. A method for providing video data to a video decoding device, A step of encoding the above video data into a bitstream; and A step of transmitting the above bitstream to the image decoding device Including, The step of encoding the above video data is: A step of obtaining a block vector of a current block based on a corresponding luma region of a current chroma block, wherein the current block includes a current luma block and the current chroma block; A step of configuring a region indicated by the above block vector as a reference block and setting the reference chroma block as a chroma predictor of the current chroma block, wherein the reference block includes a reference luma block and the reference chroma block; A step of estimating parameters expressing the correlation between the luma change amount and the chroma change amount between the template of the current block and the template of the reference block; and A step of calculating a correction value based on the estimated parameters and correcting the chroma predictor based on the correction value. A method comprising:

Images