Method and device for video coding for adaptively reconstructing blending parameter list

Abstract

The present embodiment discloses a video coding method and device for adaptively reconstructing a blending parameter list. In the present embodiment, an image decoding device acquires, for the current block, one or more reference blocks and a blending parameter list, and decodes an index indicating one of elements within the blending parameter list. The image decoding device acquires a first flag indicating whether or not to reconstruct the blending parameter list. The image decoding device identifies the first flag, and when the first flag is true, reconstructs the blending parameter list on the basis of pre-reconstructed information of the current block by using a blending parameter list reconstruction method. The image decoding device extracts a weight from the reconstructed blending parameter list on the basis of the index, and generates a prediction block of the current block by performing weighted combination on the one or more reference blocks on the basis of the weight.

Description

Method and device for video coding adaptively reconstructing a blending parameter list

[0001] The present disclosure relates to a video coding method and apparatus for adaptively reconstructing and utilizing a blending parameter list.

[0002] The content described below merely provides background information related to the present invention and does not constitute prior art.

[0003] Since video data has a large amount of data compared to voice data or still image data, it requires a lot of hardware resources, including memory, to store or transmit it without processing for compression.

[0004] Therefore, when storing or transmitting video data, the encoder compresses the video data and stores or transmits it, and the decoder receives the compressed video data, decompresses it, and plays it back. These video compression technologies include H.264 / AVC, HEVC (High Efficiency Video Coding), and VVC (Versatile Video Coding), which improves encoding efficiency by about 30% compared to HEVC.

[0005] However, as the size, resolution, and frame rate of images are gradually increasing, and the amount of data that needs to be encoded is also increasing, a new compression technology with better encoding efficiency and better image quality is required than existing compression technologies. Weighted Prediction (WP) technology is a technology that corrects the brightness between pictures on a picture-by-picture basis by approximating the brightness change between different pictures with a linear relationship. WP technology is effective for compressing images with fade-ins or fade-outs and is used in HEVC and VVC. When WP technology is used, the encoder signals a flag indicating whether to use WP technology to the decoder on a picture-by-picture basis, and transmits weights and offsets for brightness compensation using the picture header or slice header to indicate the degree of brightness compensation. However, when WP technology is used, the weights and offsets signaled on a picture-by-picture or slice basis may not be optimal for some blocks.

[0006] Accordingly, VVC applies the BCW (Bi-prediction with CU-level Weight) technique, which can use different weights for reference blocks on a CU basis in bi-prediction. To apply the BCW technique, a weight list containing weight candidates is defined identically in the encoder and decoder. The encoder can convey the applied weight to the decoder by signaling an index (bcw_idx) indicating one candidate in the weight list. However, even with the BCW technique, the values ​​of the elements in the weight list used to signal bcw_idx may not always be optimal. In addition, the order of the weights corresponding to each bcw_idx may not always be optimal.

[0007] Therefore, a method for efficiently calculating the weights of reference blocks is needed when predicting the inter of the current block.

[0008] The present disclosure provides a video coding method and apparatus for reconstructing an optimal blending parameter list by modifying elements in a blending parameter list to be used for each reference block to optimal values ​​or arranging the order of elements based on restored information when generating a predictor of a current block using a single or multiple reference blocks in inter prediction.

[0009] An embodiment according to the present disclosure aims to provide a video coding method and device that uses at least one of a weight and an offset for each reference block when reconstructing a blending parameter list for a plurality of reference blocks.

[0010] One embodiment of the present disclosure is directed to providing a video coding method and apparatus that utilizes at least one of restored information, such as pixels, transform coefficients, etc., according to a preset method when using restored information to reconstruct a blending parameter list.

[0011] An embodiment according to the present disclosure aims to provide a video coding method and device that utilizes a signaled list element modification method or a predetermined restored information to reconstruct a blending parameter list.

[0012] One embodiment of the present disclosure aims to provide a video coding method and apparatus that utilizes a signaled list element sorting method or restored information of a surrounding block to reconstruct a blending parameter list.

[0013] One embodiment of the present disclosure is directed to providing a video coding method and apparatus for selecting a method for reconstructing the aforementioned blending parameter list according to information of a current block or a transmitted signal.

[0014] According to an embodiment of the present disclosure, a method for restoring a current block, performed by an image decoding device, comprises the steps of: obtaining one or more reference blocks and a blending parameter list for the current block; decoding an index indicating one of elements in the blending parameter list, wherein each element in the blending parameter list is used for weighted combining of one or more reference blocks; obtaining a first flag indicating whether to reconstruct the blending parameter list; and checking the first flag, wherein, if the first flag is true, the method further comprises: reconstructing the blending parameter list based on the restoration information of the current block using a blending parameter list reconstruction method; extracting a weight from the reconstructed blending parameter list based on the index; and generating a prediction block of the current block by weighted combining the one or more reference blocks based on the weight.

[0015] According to another embodiment of the present disclosure, a method for encoding a current block, performed by a video encoding device, is provided, the method further comprising: obtaining one or more reference blocks for the current block, a blending parameter list, and an index indicating one of elements in the blending parameter list, wherein each element in the blending parameter list is used for weighted combining of the one or more reference blocks; obtaining a first flag indicating whether to reconstruct the blending parameter list; and checking the first flag, wherein, when the first flag is true, the method further comprises: reconstructing the blending parameter list based on restoration information of the current block using a blending parameter list reconstruction method; extracting a weight from the reconstructed blending parameter list based on the index; and generating a prediction block of the current block by weighted combining the one or more reference blocks based on the weight.

[0016] According to another embodiment of the present disclosure, a method for providing video data to a video decoding device is provided, comprising: encoding the video data into a bitstream; and transmitting the bitstream to the video decoding device, wherein the encoding the video data further comprises: obtaining, for a current block, one or more reference blocks, a blending parameter list, and an index indicating one of elements in the blending parameter list, wherein each element in the blending parameter list is used for weighted combining of the one or more reference blocks; obtaining a first flag indicating whether to reconstruct the blending parameter list; and checking the first flag, and when the first flag is true, reconstructing the blending parameter list based on the restoration information of the current block using a blending parameter list reconstruction method; extracting a weight from the reconstructed blending parameter list based on the index; and generating a prediction block of the current block by weighted combining the one or more reference blocks based on the weight.

[0017] As described above, according to the present embodiment, when generating a predictor of a current block using a single or multiple reference blocks in inter prediction, a video coding method and device are provided for reconstructing an optimal blending parameter list by modifying elements in a blending parameter list to be used for each reference block to optimal values ​​or arranging the order of elements based on restored information, thereby making it possible to improve video encoding efficiency and enhance video quality.

[0018] FIG. 1 is an exemplary block diagram of an image encoding device capable of implementing the techniques of the present disclosure.

[0019] Figure 2 is a drawing for explaining a method of dividing a block using the QTBTTT (QuadTree plus BinaryTree TernaryTree) structure.

[0020] FIGS. 3A and 3B are diagrams illustrating multiple intra prediction modes, including wide-angle intra prediction modes.

[0021] Figure 4 is an example diagram of the surrounding blocks of the current block.

[0022] FIG. 5 is an exemplary block diagram of an image decoding device capable of implementing the techniques of the present disclosure.

[0023] FIG. 6 is an exemplary diagram showing the reconstruction of a blending parameter list according to one embodiment of the present disclosure.

[0024] FIG. 7 is an exemplary diagram showing an area for calculating a matching cost according to one embodiment of the present disclosure.

[0025] FIG. 8 is an exemplary diagram showing a predefined location around a current block according to one embodiment of the present disclosure.

[0026] FIG. 9 is an exemplary diagram showing the distribution of whether elements in a blending parameter list are modified, according to one embodiment of the present disclosure.

[0027] FIG. 10 is an exemplary diagram showing the distribution of whether elements in a blending parameter list are modified, according to another embodiment of the present disclosure.

[0028] FIG. 11 is an exemplary diagram showing elements in a blending parameter list of a surrounding block according to one embodiment of the present disclosure.

[0029] FIG. 12 is an exemplary diagram illustrating modification of elements in a blending parameter list according to one embodiment of the present disclosure.

[0030] FIG. 13 is an exemplary diagram illustrating modification of elements in a blending parameter list according to another embodiment of the present disclosure.

[0031] FIG. 14 is an exemplary diagram showing elements in a blending parameter list of a surrounding block according to another embodiment of the present disclosure.

[0032] FIG. 15 is an exemplary diagram illustrating modification of elements in a blending parameter list according to another embodiment of the present disclosure.

[0033] FIG. 16 is an exemplary diagram illustrating modification of elements in a blending parameter list according to another embodiment of the present disclosure.

[0034] FIG. 17 is an exemplary diagram illustrating modification of elements in a blending parameter list according to another embodiment of the present disclosure.

[0035] FIG. 18 is an exemplary diagram showing the reconstruction of a blending parameter list according to another embodiment of the present disclosure.

[0036] FIG. 19 is an exemplary diagram showing the sorting of a blending parameter list according to one embodiment of the present disclosure.

[0037] FIG. 20 is an exemplary diagram showing the sorting of a blending parameter list according to another embodiment of the present disclosure.

[0038] FIG. 21 is an exemplary diagram showing the sorting of a blending parameter list according to another embodiment of the present disclosure.

[0039] FIG. 22 is an exemplary diagram showing elements in a blending parameter list of a surrounding block according to another embodiment of the present disclosure.

[0040] FIG. 23 is an exemplary diagram showing the sorting of a blending parameter list according to another embodiment of the present disclosure.

[0041] FIG. 24 is an exemplary diagram showing the sorting of a blending parameter list according to another embodiment of the present disclosure.

[0042] FIG. 25 is a flowchart illustrating a method of encoding a current block performed by an image encoding device according to one embodiment of the present disclosure.

[0043] FIG. 26 is a flowchart illustrating a method for restoring a current block performed by an image decoding device according to one embodiment of the present disclosure.

[0044] Hereinafter, embodiments of the present invention will be described in detail with reference to exemplary drawings. When designating components in each drawing, it should be noted that, where possible, identical components are given the same reference numerals, even if they appear in different drawings. Furthermore, in describing the present embodiments, detailed descriptions of related known structures or functions will be omitted if they are deemed to obscure the gist of the present embodiments.

[0045] FIG. 1 is an exemplary block diagram of an image encoding device capable of implementing the techniques of the present disclosure. Hereinafter, the image encoding device and its subcomponents will be described with reference to the illustration in FIG. 1.

[0046] The video encoding device may be configured to include a picture segmentation unit (110), a prediction unit (120), a subtractor (130), a transformation unit (140), a quantization unit (145), a reordering unit (150), an entropy encoding unit (155), an inverse quantization unit (160), an inverse transformation unit (165), an adder (170), a loop filter unit (180), and a memory (190).

[0047] Each component of the video encoding device may be implemented in hardware, software, or a combination of hardware and software. Furthermore, the functions of each component may be implemented in software, with a microprocessor executing the software functions corresponding to each component.

[0048] A single image (video) is composed of one or more sequences containing multiple pictures. Each picture is divided into multiple regions, and encoding is performed for each region. For example, a single picture is divided into one or more tiles and / or slices. Here, one or more tiles can be defined as a tile group. Each tile or slice is divided into one or more Coding Tree Units (CTUs). Each CTU is then divided into one or more Coding Units (CUs) by a tree structure. Information applied to each CU is encoded as the syntax of the CU, and information commonly applied to CUs included in a CTU is encoded as the syntax of the CTU. In addition, information commonly applied to all blocks within a single slice is encoded as the syntax of the slice header, and information applied to all blocks constituting one or more pictures is encoded in the Picture Parameter Set (PPS) or the picture header. Furthermore, information commonly referenced by multiple pictures is encoded in a Sequence Parameter Set (SPS). And, information commonly referenced by one or more SPS is encoded in a Video Parameter Set (VPS). In addition, information commonly applied to one tile or tile group may be encoded as syntax of a tile or tile group header. Syntaxes included in an SPS, PPS, slice header, tile or tile group header may be referred to as high level syntax.

[0049] The picture segmentation unit (110) determines the size of the CTU. Information about the size of the CTU (CTU size) is encoded as the syntax of SPS or PPS and transmitted to the image decoding device.

[0050] The picture segmentation unit (110) divides each picture constituting an image into a plurality of CTUs having a predetermined size, and then recursively divides the CTUs using a tree structure. A leaf node in the tree structure becomes a CU, which is a basic unit of encoding.

[0051] The tree structure may be a QuadTree (QT) in which an upper node (or parent node) is divided into four lower nodes (or child nodes) of the same size, a BinaryTree (BT) in which an upper node is divided into two lower nodes, or a TernaryTree (TT) in which an upper node is divided into three lower nodes in a 1:2:1 ratio, or a structure that mixes two or more of the QT structures, BT structures, and TT structures. For example, a QTBT (QuadTree plus BinaryTree) structure may be used, or a QTBTTT (QuadTree plus BinaryTree TernaryTree) structure may be used. Here, BTTT may be combined and referred to as a MTT (Multiple-Type Tree).

[0052] Figure 2 is a drawing for explaining a method of dividing a block using the QTBTTT structure.

[0053] As illustrated in FIG. 2, a CTU may first be split into a QT structure. The quadtree splitting may be repeated until the size of the splitting block reaches the minimum block size (MinQTSize) of the leaf node allowed in the QT. A first flag (QT_split_flag) indicating whether each node of the QT structure is split into four nodes of the lower layer is encoded by the entropy encoding unit (155) and signaled to the image decoding device. If the leaf node of the QT is not larger than the maximum block size (MaxBTSize) of the root node allowed in the BT, it may be further split into one or more of the BT structure or the TT structure. There may be multiple splitting directions in the BT structure and / or the TT structure. For example, there may be two directions in which the block of the corresponding node is split horizontally and two directions in which the block is split vertically. As illustrated in FIG. 2, when MTT splitting begins, a second flag (mtt_split_flag) indicating whether nodes have been split, and if splitting has occurred, a flag indicating the splitting direction (vertical or horizontal) and / or a flag indicating the splitting type (Binary or Ternary) are encoded by the entropy encoding unit (155) and signaled to the image decoding device.

[0054] Alternatively, before encoding the first flag (QT_split_flag) indicating whether each node is split into four nodes of a lower layer, a CU split flag (split_cu_flag) indicating whether the node is split may be encoded. If the CU split flag (split_cu_flag) value indicates that the node is not split, the block of the corresponding node becomes a leaf node in the split tree structure and becomes a CU (coding unit), which is a basic unit of encoding. If the CU split flag (split_cu_flag) value indicates that the node is split, the video encoding device starts encoding from the first flag in the above-described manner.

[0055] As another example of a tree structure, when QTBT is used, there may be two types: a type that horizontally splits the block of the corresponding node into two blocks of the same size (i.e., symmetric horizontal splitting) and a type that vertically splits it (i.e., symmetric vertical splitting). A split flag (split_flag) indicating whether each node of the BT structure is split into blocks of a lower layer and split type information indicating the type of split are encoded by the entropy encoding unit (155) and transmitted to the image decoding device. Meanwhile, there may additionally be a type that splits the block of the corresponding node into two blocks of an asymmetrical shape. The asymmetric shape may include a shape that splits the block of the corresponding node into two rectangular blocks with a size ratio of 1:3, or a shape that splits the block of the corresponding node in a diagonal direction.

[0056] A CU can have various sizes depending on the QTBT or QTBTTT partitioning from the CTU. Hereinafter, the block corresponding to the CU to be encoded or decoded (i.e., the leaf node of the QTBTTT) is referred to as the "current block." Depending on the QTBTTT partitioning employed, the current block may be rectangular as well as square.

[0057] The prediction unit (120) predicts the current block and generates a prediction block. The prediction unit (120) includes an intra prediction unit (122) and an inter prediction unit (124).

[0058] In general, each current block within a picture can be predictively coded. Prediction of the current block can typically be performed using either intra-prediction (using data from the picture containing the current block) or inter-prediction (using data from a picture coded before the picture containing the current block). Inter-prediction encompasses both unidirectional and bidirectional prediction.

[0059] The intra prediction unit (122) predicts pixels within the current block using pixels (reference pixels) located around the current block within the current picture including the current block. There are multiple intra prediction modes depending on the prediction direction. For example, as shown in Fig. 3a, the multiple intra prediction modes may include two non-directional modes including the Planar mode and the DC mode, and 65 directional modes. The surrounding pixels to be used and the calculation formula are defined differently depending on each prediction mode.

[0060] For efficient directional prediction for a rectangular current block, directional modes (intra prediction modes 67 to 80 and -1 to -14) indicated by dotted arrows in Fig. 3b may be additionally used. These may be referred to as "wide-angle intra-prediction modes." In Fig. 3b, the arrows point to corresponding reference samples used for prediction, and do not indicate the prediction direction. The prediction direction is opposite to the direction indicated by the arrows. Wide-angle intra-prediction modes are modes that perform prediction in the opposite direction of a specific directional mode without additional bit transmission when the current block is rectangular. At this time, among the wide-angle intra-prediction modes, some wide-angle intra-prediction modes available for the current block may be determined based on the ratio of the width and height of the rectangular current block. For example, wide-angle intra prediction modes (intra prediction modes 67 to 80) having an angle less than 45 degrees are available when the current block is a rectangular shape whose height is smaller than its width, and wide-angle intra prediction modes (intra prediction modes -1 to -14) having an angle greater than -135 degrees are available when the current block is a rectangular shape whose width is larger than its height.

[0061] The intra prediction unit (122) can determine an intra prediction mode to be used to encode the current block. In some examples, the intra prediction unit (122) can encode the current block using multiple intra prediction modes and select an appropriate intra prediction mode to be used from the tested modes. For example, the intra prediction unit (122) can calculate bit-rate distortion values ​​using rate-distortion analysis for multiple tested intra prediction modes and select an intra prediction mode with the best bit-rate distortion characteristics among the tested modes.

[0062] The intra prediction unit (122) selects one intra prediction mode from among multiple intra prediction modes and predicts the current block using surrounding pixels (reference pixels) and an operation formula determined according to the selected intra prediction mode. Information about the selected intra prediction mode is encoded by the entropy encoding unit (155) and transmitted to the image decoding device.

[0063] The inter prediction unit (124) generates a prediction block for the current block using a motion compensation process. The inter prediction unit (124) searches for a block most similar to the current block within reference pictures that were encoded and decoded before the current picture, and generates a prediction block for the current block using the searched block. Then, a motion vector (MV) corresponding to the displacement between the current block within the current picture and the prediction block within the reference picture is generated. Generally, motion estimation is performed on the luma component, and the motion vector calculated based on the luma component is used for both the luma component and the chroma component. The motion information including information on the reference picture used to predict the current block and information on the motion vector is encoded by the entropy encoding unit (155) and transmitted to the image decoding device.

[0064] The inter prediction unit (124) may perform interpolation on a reference picture or a reference block to improve prediction accuracy. That is, subsamples between two consecutive integer samples are interpolated by applying filter coefficients to a plurality of consecutive integer samples including the two integer samples. When a process of searching for a block most similar to the current block is performed on the interpolated reference picture, the motion vector can be expressed up to a precision in decimal units rather than a precision in integer sample units. The precision or resolution of the motion vector can be set differently for each target region to be encoded, such as a slice, tile, CTU, CU, etc. When such adaptive motion vector resolution (AMVR) is applied, information on the motion vector resolution to be applied to each target region must be signaled for each target region. For example, when the target region is a CU, information on the motion vector resolution applied to each CU is signaled. Information on the motion vector resolution may be information indicating the precision of a differential motion vector, which will be described later.

[0065] Meanwhile, the inter prediction unit (124) can perform inter prediction using bi-prediction. In the case of bi-prediction, two reference pictures and two motion vectors indicating the block position most similar to the current block within each reference picture are used. The inter prediction unit (124) selects a first reference picture and a second reference picture from reference picture list 0 (RefPicList0) and reference picture list 1 (RefPicList1), respectively, and searches for a block similar to the current block within each reference picture to generate a first reference block and a second reference block. Then, the first reference block and the second reference block are averaged or weighted averaged to generate a prediction block for the current block. Then, motion information including information on two reference pictures used to predict the current block and information on two motion vectors is transmitted to the entropy encoding unit (155). Here, reference picture list 0 may be composed of pictures that are before the current picture in display order among the restored pictures, and reference picture list 1 may be composed of pictures that are after the current picture in display order among the restored pictures. However, this is not necessarily limited to this, and restored pictures that are after the current picture in display order may be additionally included in reference picture list 0, and conversely, restored pictures that are before the current picture may be additionally included in reference picture list 1.

[0066] Various methods can be used to minimize the number of bits required to encode motion information.

[0067] For example, if the reference picture and motion vector of the current block are identical to those of a neighboring block, the motion information of the current block can be transmitted to the image decoding device by encoding information that can identify the neighboring block. This method is called 'merge mode'.

[0068] In merge mode, the inter prediction unit (124) selects a predetermined number of merge candidate blocks (hereinafter referred to as 'merge candidates') from the surrounding blocks of the current block.

[0069] As the surrounding blocks for deriving merge candidates, all or part of the left block (A0), the lower left block (A1), the upper block (B0), the upper right block (B1), and the upper left block (B2) adjacent to the current block within the current picture may be used, as illustrated in FIG. 4. In addition, a block located within a reference picture (which may or may not be the same as the reference picture used to predict the current block) other than the current picture in which the current block is located may be used as a merge candidate. For example, a block co-located with the current block within the reference picture or blocks adjacent to the block at the co-located block may be additionally used as a merge candidate. If the number of merge candidates selected by the method described above is less than a preset number, a 0 vector is added to the merge candidates.

[0070] The inter prediction unit (124) uses these surrounding blocks to construct a merge list containing a predetermined number of merge candidates. Among the merge candidates included in the merge list, the merge candidate to be used as motion information of the current block is selected and merge index information for identifying the selected candidate is generated. The generated merge index information is encoded by the entropy encoding unit (155) and transmitted to the video decoding device.

[0071] Merge Skip mode is a special case of merge mode. After quantization, when all transform coefficients for entropy encoding are close to zero, only neighboring block selection information is transmitted without transmitting residual signals. By utilizing merge skip mode, relatively high encoding efficiency can be achieved for low-motion images, still images, and screen content images.

[0072] Hereinafter, merge mode and merge skip mode are collectively referred to as merge / skip mode.

[0073] Another method for encoding motion information is Advanced Motion Vector Prediction (AMVP) mode.

[0074] In AMVP mode, the inter prediction unit (124) derives predicted motion vector candidates for the motion vector of the current block using neighboring blocks of the current block. As neighboring blocks used to derive predicted motion vector candidates, all or some of the left block (A0), the lower left block (A1), the upper block (B0), the upper right block (B1), and the upper left block (B2) adjacent to the current block in the current picture as shown in FIG. 4 may be used. In addition, a block located in a reference picture (which may or may not be the same as the reference picture used to predict the current block) other than the current picture in which the current block is located may be used as the neighboring block used to derive predicted motion vector candidates. For example, a block co-located with the current block in the reference picture or blocks adjacent to the block in the co-located block may be used. If the number of motion vector candidates is less than a preset number by the method described above, a 0 vector is added to the motion vector candidates.

[0075] The inter prediction unit (124) derives predicted motion vector candidates using the motion vectors of these surrounding blocks, and determines a predicted motion vector for the motion vector of the current block using the predicted motion vector candidates. Then, the predicted motion vector is subtracted from the motion vector of the current block to produce a differential motion vector.

[0076] The predicted motion vector can be obtained by applying a predefined function (e.g., median, mean, etc.) to the predicted motion vector candidates. In this case, the image decoding device also knows the predefined function. In addition, since the surrounding blocks used to derive the predicted motion vector candidates are blocks that have already been encoded and decoded, the image decoding device also already knows the motion vectors of the surrounding blocks. Therefore, the image encoding device does not need to encode information to identify the predicted motion vector candidates. Therefore, in this case, information about the differential motion vector and information about the reference picture used to predict the current block are encoded.

[0077] Alternatively, the predicted motion vector can be determined by selecting one of the predicted motion vector candidates. In this case, information for identifying the selected predicted motion vector candidate is additionally encoded, along with information about the differential motion vector and the reference picture used to predict the current block.

[0078] The subtractor (130) subtracts the prediction block generated by the intra prediction unit (122) or inter prediction unit (124) from the current block to generate a residual block.

[0079] The transformation unit (140) transforms residual signals within a residual block having pixel values ​​in a spatial domain into transform coefficients in a frequency domain. The transformation unit (140) may transform the residual signals within the residual block using the entire size of the residual block as a transformation unit, or may divide the residual block into a plurality of sub-blocks and use the sub-blocks as transformation units to perform the transformation. Alternatively, the residual signals may be transformed using only the transformation domain sub-block as a transformation unit by dividing the sub-blocks into two sub-blocks, that is, a transformation domain and a non-transform domain. Here, the transformation domain sub-block may be one of two rectangular blocks having a size ratio of 1:1 with respect to the horizontal axis (or vertical axis). In this case, a flag (cu_sbt_flag) indicating that only a sub-block has been converted, directionality (vertical / horizontal) information (cu_sbt_horizontal_flag), and / or position information (cu_sbt_pos_flag) are encoded by the entropy encoding unit (155) and signaled to the image decoding device. In addition, the size of the conversion area sub-block may have a size ratio of 1:3 with respect to the horizontal axis (or vertical axis), and in this case, a flag (cu_sbt_quad_flag) distinguishing the corresponding division is additionally encoded by the entropy encoding unit (155) and signaled to the image decoding device.

[0080] Meanwhile, the transformation unit (140) can individually perform transformations on the residual block in the horizontal and vertical directions. For the transformation, various types of transformation functions or transformation matrices can be used. For example, a pair of transformation functions for horizontal transformation and vertical transformation can be defined as a Multiple Transform Set (MTS). The transformation unit (140) can select one transformation function pair with the best transformation efficiency among the MTS and transform the residual block in the horizontal and vertical directions, respectively. Information (mts_idx) on the transformation function pair selected among the MTS is encoded by the entropy encoding unit (155) and signaled to the image decoding device.

[0081] The quantization unit (145) quantizes the transform coefficients output from the transform unit (140) using quantization parameters and outputs the quantized transform coefficients to the entropy encoding unit (155). The quantization unit (145) may directly quantize a related residual block without transformation for a certain block or frame. The quantization unit (145) may also apply different quantization coefficients (scaling values) according to the positions of the transform coefficients within the transform block. The quantization matrix applied to the quantized transform coefficients arranged in two dimensions may be encoded and signaled to an image decoding device.

[0082] The rearrangement unit (150) can perform rearrangement of coefficient values ​​for quantized residual values.

[0083] The reordering unit (150) can change a two-dimensional coefficient array into a one-dimensional coefficient sequence by using coefficient scanning. For example, the reordering unit (150) can output a one-dimensional coefficient sequence by scanning from the DC coefficient to the coefficients of the high-frequency region by using a zig-zag scan or a diagonal scan. Depending on the size of the transformation unit and the intra prediction mode, a vertical scan that scans the two-dimensional coefficient array in the column direction or a horizontal scan that scans the two-dimensional block-shaped coefficients in the row direction may be used instead of the zig-zag scan. That is, depending on the size of the transformation unit and the intra prediction mode, the scanning method to be used may be determined among the zig-zag scan, the diagonal scan, the vertical scan, and the horizontal scan.

[0084] The entropy encoding unit (155) generates a bitstream by encoding a sequence of one-dimensional quantized transform coefficients output from the rearrangement unit (150) using various encoding methods such as CABAC (Context-based Adaptive Binary Arithmetic Code) and Exponential Golomb.

[0085] In addition, the entropy encoding unit (155) encodes information related to block division, such as CTU size, CU division flag, QT division flag, MTT division type, and MTT division direction, so that the image decoding device can divide the block in the same manner as the image encoding device. In addition, the entropy encoding unit (155) encodes information about a prediction type indicating whether the current block is encoded by intra prediction or inter prediction, and encodes intra prediction information (i.e., information about an intra prediction mode) or inter prediction information (information about an encoding mode of motion information (merge mode or AMVP mode), a merge index in the case of a merge mode, and a reference picture index and a differential motion vector in the case of an AMVP mode) according to the prediction type. In addition, the entropy encoding unit (155) encodes information related to quantization, that is, information about a quantization parameter and information about a quantization matrix.

[0086] The inverse quantization unit (160) inversely quantizes the quantized transform coefficients output from the quantization unit (145) to generate transform coefficients. The inverse transform unit (165) transforms the transform coefficients output from the inverse quantization unit (160) from the frequency domain to the spatial domain to restore the residual block.

[0087] An adder (170) adds the restored residual block and the predicted block generated by the prediction unit (120) to restore the current block. The pixels within the restored current block are used as reference pixels when intra-predicting the next block.

[0088] The loop filter unit (180) performs filtering on restored pixels to reduce blocking artifacts, ringing artifacts, blurring artifacts, etc. that occur due to block-based prediction and transformation / quantization. The loop filter unit (180) may include all or part of a deblocking filter (182), a sample adaptive offset (SAO) filter (184), and an adaptive loop filter (ALF, 186) as an in-loop filter.

[0089] The deblocking filter (182) filters the boundaries between restored blocks to remove blocking artifacts caused by block-based encoding / decoding, and the SAO filter (184) and the ALF (186) perform additional filtering on the deblocking-filtered image. The SAO filter (184) and the ALF (186) are filters used to compensate for the differences between restored pixels and original pixels caused by lossy coding. The SAO filter (184) improves not only subjective image quality but also encoding efficiency by applying an offset in units of CTUs. In contrast, the ALF (186) performs block-based filtering, and compensates for distortion by applying different filters by distinguishing the edges and degrees of variation of the corresponding block. Information on filter coefficients to be used in the ALF can be encoded and signaled to an image decoding device.

[0090] The restored blocks filtered through the deblocking filter (182), SAO filter (184), and ALF (186) are stored in the memory (190). When all blocks within a picture are restored, the restored picture can be used as a reference picture for inter-predicting blocks within a picture to be encoded later.

[0091] The video encoding device can store the bitstream of encoded video data on a non-transitory storage medium or transmit it to the video decoding device using a communication network.

[0092] FIG. 5 is an exemplary block diagram of an image decoding device capable of implementing the techniques of the present disclosure. Hereinafter, the image decoding device and its subcomponents will be described with reference to FIG. 5.

[0093] The video decoding device may be configured to include an entropy decoding unit (510), a rearrangement unit (515), an inverse quantization unit (520), an inverse transformation unit (530), a prediction unit (540), an adder (550), a loop filter unit (560), and a memory (570).

[0094] Similar to the video encoding device of FIG. 1, each component of the video decoding device may be implemented in hardware, software, or a combination of hardware and software. Furthermore, the functions of each component may be implemented in software, with a microprocessor executing the software functions corresponding to each component.

[0095] The entropy decoding unit (510) decodes the bitstream generated by the image encoding device to extract information related to block division, thereby determining the current block to be decoded, and extracts prediction information, information on residual signals, etc. required to restore the current block.

[0096] The entropy decoding unit (510) extracts information about the CTU size from the Sequence Parameter Set (SPS) or the Picture Parameter Set (PPS), determines the size of the CTU, and divides the picture into CTUs of the determined size. Then, the CTU is determined as the top layer of the tree structure, i.e., the root node, and the CTU is divided using the tree structure by extracting division information about the CTU.

[0097] For example, when splitting a CTU using the QTBTTT structure, first, the first flag (QT_split_flag) related to the splitting of QT is extracted, and each node is split into four nodes of the lower layer. Then, for the nodes corresponding to the leaf nodes of QT, the second flag (mtt_split_flag) related to the splitting of MTT and the split direction (vertical / horizontal) and / or split type (binary / ternary) information are extracted, and the corresponding leaf nodes are split into the MTT structure. Accordingly, each node below the leaf nodes of QT are split recursively into the BT or TT structure.

[0098] As another example, when splitting a CTU using the QTBTTT structure, the CU split flag (split_cu_flag) indicating whether the CU is split is first extracted, and if the block is split, the first flag (QT_split_flag) may be extracted. During the splitting process, each node may undergo zero or more repeated QT splits followed by zero or more repeated MTT splits. For example, a CTU may undergo an MTT split right away, or conversely, may undergo only multiple QT splits.

[0099] As another example, when splitting a CTU using the QTBT structure, the first flag (QT_split_flag) related to the splitting of QT is extracted, and each node is split into four nodes of the lower layer. Furthermore, for nodes corresponding to leaf nodes of QT, a split flag (split_flag) indicating whether to further split into BTs and splitting direction information are extracted.

[0100] Meanwhile, when the entropy decoding unit (510) determines the current block to be decoded by using the division of the tree structure, it extracts information on the prediction type indicating whether the current block is intra-predicted or inter-predicted. If the prediction type information indicates intra-prediction, the entropy decoding unit (510) extracts syntax elements for intra-prediction information (intra-prediction mode) of the current block. If the prediction type information indicates inter-prediction, the entropy decoding unit (510) extracts syntax elements for inter-prediction information, i.e., information indicating a motion vector and a reference picture referenced by the motion vector.

[0101] Additionally, the entropy decoding unit (510) extracts information about the quantized transform coefficients of the current block as information related to quantization and information about residual signals.

[0102] The rearrangement unit (515) can change the sequence of one-dimensional quantized transform coefficients entropy-decoded in the entropy decoding unit (510) back into a two-dimensional coefficient array (i.e., block) in the reverse order of the coefficient scanning performed by the image encoding device.

[0103] The inverse quantization unit (520) inversely quantizes the quantized transform coefficients and inversely quantizes the quantized transform coefficients using the quantization parameters. The inverse quantization unit (520) may also apply different quantization coefficients (scaling values) to the quantized transform coefficients arranged in two dimensions. The inverse quantization unit (520) may perform inverse quantization by applying a matrix of quantized coefficients (scaling values) from an image encoding device to a two-dimensional array of quantized transform coefficients.

[0104] The inverse transform unit (530) inversely transforms the inverse quantized transform coefficients from the frequency domain to the spatial domain to restore residual signals, thereby generating a residual block for the current block.

[0105] In addition, when the inverse transform unit (530) inversely transforms only a portion of a transform block (sub-block), it extracts a flag (cu_sbt_flag) indicating that only a sub-block of the transform block has been transformed, directionality (vertical / horizontal) information (cu_sbt_horizontal_flag) of the sub-block, and / or position information (cu_sbt_pos_flag) of the sub-block, and inversely transforms the transform coefficients of the corresponding sub-block from the frequency domain to the spatial domain to restore residual signals, and fills “0” values ​​with residual signals for areas that have not been inversely transformed, thereby generating a final residual block for the current block.

[0106] In addition, when MTS is applied, the inverse transform unit (530) determines a transform function or a transform matrix to be applied in the horizontal and vertical directions using MTS information (mts_idx) signaled from the image encoding device, and performs inverse transform on the transform coefficients within the transform block in the horizontal and vertical directions using the determined transform function.

[0107] The prediction unit (540) may include an intra prediction unit (542) and an inter prediction unit (544). The intra prediction unit (542) is activated when the prediction type of the current block is intra prediction, and the inter prediction unit (544) is activated when the prediction type of the current block is inter prediction.

[0108] The intra prediction unit (542) determines the intra prediction mode of the current block among a plurality of intra prediction modes from the syntax elements for the intra prediction mode extracted from the entropy decoding unit (510), and predicts the current block using reference pixels around the current block according to the intra prediction mode.

[0109] The inter prediction unit (544) uses the syntax elements for the inter prediction mode extracted from the entropy decoding unit (510) to determine the motion vector of the current block and the reference picture referenced by the motion vector, and predicts the current block using the motion vector and the reference picture.

[0110] An adder (550) adds the residual block output from the inverse transform unit (530) and the predicted block output from the inter prediction unit (544) or the intra prediction unit (542) to restore the current block. The pixels within the restored current block are used as reference pixels when intra-predicting a block to be decoded later.

[0111] The loop filter unit (560) may include a deblocking filter (562), an SAO filter (564), and an ALF (566) as in-loop filters. The deblocking filter (562) deblocks the boundaries between restored blocks to remove blocking artifacts caused by block-by-block decoding. The SAO filter (564) and the ALF (566) perform additional filtering on restored blocks after deblocking filtering to compensate for differences between restored pixels and original pixels caused by lossy coding. The filter coefficients of the ALF are determined using information about filter coefficients decoded from the non-stream.

[0112] The restored blocks filtered through the deblocking filter (562), SAO filter (564), and ALF (566) are stored in the memory (570). When all blocks within a picture are restored, the restored picture is used as a reference picture for inter-predicting blocks within a picture to be encoded later.

[0113] The present embodiment relates to encoding and decoding of images (video) as described above. More specifically, when generating a predictor of a current block using a single or multiple reference blocks in inter prediction, a video coding method and device are provided for reconstructing an optimal blending parameter list by modifying elements in a blending parameter list to be used for each reference block to optimal values ​​or arranging the order of the elements based on previously restored information.

[0114] The following embodiments may be performed by an inter prediction unit (124) within a video encoding apparatus. In addition, the following embodiments may be performed by an inter prediction unit (544) within a video decoding apparatus.

[0115] The video encoding device can generate signaling information related to the present embodiment in terms of rate distortion optimization in encoding the current block. The video encoding device can encode the signaling information using the entropy encoding unit (155) and then transmit it to the video decoding device. The video decoding device can decode the signaling information related to the decoding of the current block from the bitstream using the entropy decoding unit (510).

[0116] In the following description, the term "target block" may be used interchangeably with the current block or coding unit (CU). Alternatively, the term "target block" may also refer to a portion of a coding unit.

[0117] Also, a value of a flag being true indicates that the flag is set to 1. Also, a value of a flag being false indicates that the flag is set to 0.

[0118] The decoder side includes all or part of the inverse quantization unit (160), the inverse transform unit (165), the prediction unit (120), the adder (170), the loop filter unit (180), and the memory (190) in the video encoding device illustrated in FIG. 1. Alternatively, the decoder side includes all or part of the inverse quantization unit (520), the inverse transform unit (530), the prediction unit (540), the adder (550), the loop filter unit (560), and the memory (570) in the video decoding device illustrated in FIG. 5. With respect to a series of decoding processes, the decoder side of the video encoding device and the decoder side of the video decoding device perform the same operation. The video encoding device determines information related to the operation of the decoder side and signals the determined information to the video decoding device. The video decoding device can decode the signaled information and operate the decoder side based on the decoded information.

[0119] I. Weighted Prediction (WP) and Bi-prediction with CU-level Weight (BCW)

[0120] Weighted Prediction (WP) technology is a technology that corrects the brightness between pictures on a picture-by-picture basis by approximating the brightness change between different pictures with a linear relationship. WP technology is effective in compressing images with fade-ins or fade-outs and is used in HEVC and VVC. When WP technology is used, a video encoding device signals a flag indicating whether to use WP technology to a video decoding device on a picture-by-picture basis, and transmits weights and offsets for brightness correction using a picture header or slice header to indicate the degree of brightness correction. Therefore, the same weights and offsets are used within a picture or slice. WP can be applied to both uni-prediction and bi-prediction. Uni-prediction is performed by using the predictor P for the current block according to Equation 1. uni can be generated, and the positive prediction is the predictor P for the current block according to Equation 2. bi can be created.

[0121]

[0122]

[0123] Here, P0 and P1 are reference blocks within the L0 reference picture and L1 reference picture, respectively, w0 and w1 represent weights for the corresponding reference blocks, and O0 and O1 represent offsets for the corresponding reference blocks. However, when using the WP technique, the weights and offsets signaled at the picture or slice level may not be optimal for some blocks. Accordingly, VVC applies the BCW (Bi-prediction with CU-level Weight) technique, which can use different weights for reference blocks at the CU level in bi-prediction.

[0124] BCW technology can use different weights on a CU basis when predicting the current block by weighting two reference blocks in both predictions. Predictor P for the current block bi is generated according to mathematical formula 3.

[0125]

[0126] Here, P0 and P1 are reference blocks within the L0 reference picture and the L1 reference picture, respectively, and w represents the weight of P1. To indicate w, the video encoding device can signal bcw_idx (when using AMVP mode) or inherit one of the merge candidate lists constructed using neighboring blocks of the current block (when using merge mode).

[0127] To apply the BCW technique, a weight list containing weight candidates is defined identically in a video encoding device and a video decoding device. The video encoding device can transmit the applied weight to the video decoding device by signaling an index (bcw_idx) indicating one candidate in the weight list. However, even when using the BCW technique, the values ​​of the elements in the weight list used to signal bcw_idx may not always be optimal. For example, as shown in Table 1, the existing BCW technique cannot select a value between the weight w and the weight w+1.

[0128]

[0129] Additionally, the order of weights corresponding to each bcw_idx may not always be optimal. As shown in Table 1, since weights are listed in a fixed order without considering the weights of surrounding blocks, encoding bcw_idx using existing BCW techniques may be inefficient.

[0130] Hereinafter, weights, offsets, etc. required to generate a predictor of a current block based on a single or multiple reference blocks are collectively referred to as blending parameters. According to an embodiment, a blending parameter list may be constructed using at least one of the weights and offsets. In the present disclosure, one element in the blending parameter list may be indicated using an index. In such an embodiment, the above-described index may be referred to as a blending parameter. According to an embodiment, the blending parameter may also refer to at least one of the weights and offsets.

[0131] The following embodiments are described with a focus on a video decoding device, but can be implemented in the same or similar manner in a video encoding device. Alternatively, the following embodiments are described with a focus on the decoder side of a video decoding device, but can also be implemented in the same or similar manner in the decoder side of a video encoding device.

[0132] II. Embodiments according to the present disclosure

[0133] Below, the i-th reference block P among N reference blocks (where N is a natural number) i Weight for w i , and the offset is O i In this case, the predictor P of the current block is expressed as pred can be calculated as in mathematical formula 4.

[0134]

[0135] According to existing inter prediction techniques that generate a predictor for the current block using a reference block and its blending parameters, the blending parameters are signaled on a picture or slice basis, such as WP, or on a block basis, such as BCW, or are inherited from neighboring blocks.

[0136] When signaling blending parameters based on the same blending parameter list in all blocks, the values ​​of the blending parameters in the list may not always be optimal. The order of the blending parameters corresponding to each index may also not always be optimal, resulting in inefficient encoding of the index. The problems of the existing techniques described above can be solved by reconstructing an optimal blending parameter list for the current block from a predefined blending parameter list based on various restored information, such as pixels, transform coefficients, encoding information, and motion vectors, as presented in the present disclosure.

[0137] <Example 1> Modifying elements of the blending parameter list on the decoder side

[0138] FIG. 6 is an exemplary diagram showing modification and reconfiguration of blending parameters according to one embodiment of the present disclosure.

[0139] In this implementation example, as shown in FIG. 6, the image encoding device and the image decoding device reconstruct an optimal blending parameter list by modifying elements within a predefined blending parameter list. Therefore, the modified blending parameter list can be used identically by the image encoding device and the image decoding device.

[0140] The video decoding device can determine whether to modify an element in the blending parameter list by referring to the reconfiguration_flag value. Hereinafter, if the reconfiguration_flag value is 'true (or 1)', the video decoding device is assumed to modify an element in the blending parameter list using the restored information. On the other hand, if the reconfiguration_flag value is 'false (or 0)', the video decoding device is assumed not to modify an element in the blending parameter list. As an example, the video encoding device can explicitly signal the reconfiguration_flag to the video decoding device. As another example, the video decoding device can implicitly generate the reconfiguration_flag according to a predetermined standard. When explicitly signaling whether to modify the blending parameter list, a flag indicating whether to modify the blending parameter list can be transmitted at at least one level among the picture / slice / tile / CTU (Coding Tree Unit) / CU levels.

[0141] The syntax elements required to apply this implementation example to BCW are as follows. At least one or more of the syntax elements may be used.

[0142] pps_blending_param_modification_enabled_flag: This flag indicates whether to modify elements in the blending parameter list on a picture-by-picture basis, and can have values ​​of 0 and 1. If this flag value is 0, all blocks in the current picture do not modify elements in the blending parameter list, and both pps_blending_param_modification_explicit_flag and blending_param_modification_flag are inferred to be 0. On the other hand, if this value is 1, the video decoding device can determine whether to modify elements in the blending parameter list of the current block based on pps_blending_param_modification_explicit_flag and blending_param_modification_flag.

[0143] pps_blending_param_modification_explicit_flag: This flag indicates whether to explicitly signal whether to modify elements in the blending parameter list for the current block when the elements in the blending parameter list in the current picture can be modified (i.e., pps_blending_param_modification_enabled_flag is 1), or whether to implicitly determine whether to modify elements, and can have values ​​of 0 and 1. When the value of this flag is 0, the video decoding device can implicitly determine whether to modify elements in the blending parameter list of each block in the current picture based on a certain criterion (without a separate signal). On the other hand, when the value of this flag is 1, the video decoding device can determine whether to modify the blending parameter list of the current block based on blending_param_modification_flag.

[0144] blending_param_modification_flag: This flag indicates whether to modify elements in the blending parameter list on a block-by-block basis, and can have values ​​of 0 and 1. If the value of this flag is 0, the image decoding device does not modify the blending parameter list of the current block. If the value of this flag is 1, the image decoding device can modify the blending parameter list of the current block using the method according to the present implementation example.

[0145] Syntax transmission according to this implementation example can be realized according to the pseudocode shown in Table 2.

[0146]

[0147] Here, blending_param_modification_flag can correspond to the reconfiguration_flag described above.

[0148] In order to implicitly determine whether to modify an element in a blending parameter list according to a given criterion, the decoder side of a video encoding device and the decoder side of a video decoding device can determine whether to modify an element in a blending parameter list according to the following identical method.

[0149] 1. How to determine the reconfiguration_flag value based on information in the current block

[0150] 2. How to determine the reconfiguration_flag value based on the matching cost

[0151] 3. How to determine the reconfiguration_flag value of the current block by referencing the reconfiguration_flag value of the surrounding blocks.

[0152] Below, the specific operations of the above-described methods are described.

[0153] First, the video decoding device can determine whether to modify elements in the blending parameter list based on information about the current block. Information about the current block includes information such as the current block's channels (e.g., luma and chroma channels), width, height, size, aspect ratio, position, and information about the frame / slice that contains the current block (e.g., I (Intra), B (Bi-predictive), P (Predictive) frame / slice).

[0154] For example, you can determine whether to modify elements in the blending parameter list based on the block size as follows: If the current block size is greater than 256, reconfiguration_flag can be set to '1'. Conversely, if the current block size is less than or equal to 256, reconfiguration_flag can be set to '0'.

[0155] As another example, it is possible to determine whether to modify an element in the blending parameter list based on the position of the current block as follows. If the current block is located on a boundary associated with at least one of a picture / slice / tile / CTU, reconfiguration_flag can be set to '0'. On the other hand, if the current block is not located on a boundary associated with at least one of a picture / slice / tile / CTU, reconfiguration_flag can be set to '1'.

[0156] Second, the video decoding device can determine whether to modify an element in the blending parameter list based on a matching cost. The video decoding device can reconstruct the list according to a method for modifying an element in the blending parameter list (to be described later), calculate a matching cost based on the elements in the reconstructed list, and determine whether to modify an element in the blending parameter list based on the matching cost. Here, the matching cost represents the difference between predicted values ​​and restored values ​​when a preset area is predicted using elements in the blending parameter list, and can be calculated based on a loss function such as SAD (Sum of Absolute Differences) or SATD (Sum of Absolute Transformed Differences).

[0157] As a preset area for calculating the matching cost, an area with the highest spatial similarity to the current block or reference block may be set. For example, a portion of an area adjacent to the current block or reference block may be selected as the aforementioned calculation area. As shown in Fig. 7, the (a) top, (b) left, or (c) top and left areas adjacent to the current block or reference block may be set as the calculation area for the matching cost.

[0158] When calculating the matching cost as described above, the higher the similarity between the predicted value and the reconstructed pixel value, the lower the matching cost. Therefore, elements in the list can be modified to reduce the matching cost. For example, the average, the top or bottom N matching costs, the median, etc. of the elements in the list can be defined as representative values ​​of the matching cost. The image decoding device can determine whether to modify elements in the blending parameter list according to the representative value of the matching cost. For example, if the matching cost after modification is smaller than before modification, the image decoding device can set the reconfiguration_flag value to '1'. On the other hand, if the matching cost after modification is larger than before modification, the image decoding device can set the reconfiguration_flag value to '0'.

[0159] Third, the video decoding device can determine whether to modify an element in the blending parameter list by referring to whether to modify an element in the blending parameter list of the surrounding blocks. Depending on whether to modify an element in the blending parameter list of the current block and adjacent blocks, or blocks indicated by the block vector (BV) / motion vector (MV) of the adjacent blocks, the video decoding device can determine whether to modify an element in the blending parameter list of the current block by using one of the following methods.

[0160] ① With respect to a block containing pixels existing at a predefined location around the current block, whether or not to modify elements in the blending parameter list of the block is set to whether or not to modify elements in the blending parameter list of the current block.

[0161] ② Whether or not an element in the blending parameter list of surrounding blocks of the current block is modified is checked, and whether or not an element in the blending parameter list used in a larger number of surrounding blocks based on the number of blocks is modified is set as whether or not an element in the blending parameter list of the current block is modified.

[0162] ③ Whether or not an element in the blending parameter list of surrounding blocks of the current block is modified is checked, and whether or not an element in the blending parameter list used in surrounding blocks with a larger area based on the block area is modified is set as whether or not an element in the blending parameter list of the current block is modified.

[0163] ④ With respect to the blocks surrounding the current block whose aspect ratio and size are the same (or most similar) to those of the current block, whether or not to modify elements in the blending parameter list of the block is set to whether or not to modify elements in the blending parameter list of the current block.

[0164] Below, the specific operations of the above-described methods are described.

[0165] As an example, with respect to a block including pixels existing at predefined positions around a current block, an image decoding device may determine whether to modify an element in a blending parameter list of the corresponding block based on whether to modify an element in a blending parameter list of the current block. The predefined position may be specified in various ways, as shown in Fig. 8. If a block including the corresponding pixel is valid based on a number in a pixel, the image decoding device may set whether to modify an element in a blending parameter list of the corresponding block based on whether to modify an element in a blending parameter list of the current block. Here, if a block is valid, this may indicate a case where the block was decoded before the current block.

[0166] For example, if the distribution of whether or not to modify elements in the blending parameter list of surrounding blocks is as shown in Fig. 9, since the current block is located at the upper boundary of the picture, the block corresponding to pixel 1 is invalid. In the example of Fig. 9, since the block corresponding to pixel 2 is valid, the image decoding device can determine whether or not to modify elements in the blending parameter list of the corresponding block, as the value of 'true (or 1)'.

[0167] As another example, the video decoding device can check whether elements in the blending parameter list of neighboring blocks of the current block are modified, and set whether elements in the blending parameter list used in more neighboring blocks based on the number of blocks to be modified as elements in the blending parameter list of the current block. For example, assume that whether elements in the blending parameter list of neighboring blocks are modified is distributed as in Fig. 10. The video decoding device can determine whether elements in the blending parameter list of the current block are modified as 'true' used in 4 out of 7 neighboring blocks.

[0168] By checking the frequency of whether elements in the blending parameter lists of surrounding blocks are modified, if the number of 'true' and 'false' is the same, the priority can be designated as follows. For example, whether elements in the blending parameter list of the block closest to the upper left pixel of the current block are modified can be determined by whether elements in the blending parameter list of the current block are modified. Alternatively, after calculating the sum of the areas of blocks having whether elements in the same blending parameter list are modified, the image decoding device can determine whether elements in the blending parameter list related to blocks with a larger area are modified by whether elements in the blending parameter list of the current block are modified.

[0169] As another example, the image decoding device can check whether elements in the blending parameter lists of neighboring blocks of the current block are modified, and determine whether elements in the blending parameter lists used in neighboring blocks with a larger area based on the block area are modified based on whether elements in the blending parameter lists of the current block are modified. For example, if the modifications of elements in the blending parameter lists of neighboring blocks are distributed as in Fig. 10, the image decoding device can determine whether elements in the blending parameter lists of the current block are modified, which is 'false', as whether elements in the blending parameter lists used in more pixels are modified.

[0170] By checking the area related to whether the element in the blending parameter list of the surrounding blocks is modified, if the area of ​​'true' and 'false' is the same, the priority can be designated as follows. For example, whether the element in the blending parameter list of the block closest to the upper left pixel of the current block is modified can be determined by whether the element in the blending parameter list of the current block is modified. Alternatively, after calculating the number of blocks having whether the element in the same blending parameter list is modified, the image decoding device can determine whether the element in the blending parameter list used in a larger number of blocks is modified by whether the element in the blending parameter list of the current block is modified.

[0171] As another example, with respect to a block among the surrounding blocks of the current block that has the same aspect ratio and size as (or is most similar to) those of the current block, the image decoding device can set whether to modify an element in the blending parameter list of the block based on whether to modify an element in the blending parameter list of the current block. For example, if the modification of an element in the blending parameter list of surrounding blocks is distributed as in FIG. 10, the image decoding device can determine 'false', which is whether to modify an element in the blending parameter list of a block that has the same aspect ratio and size as the current block (e.g., a surrounding block located at the top in FIG. 10), based on whether to modify an element in the blending parameter list of the current block.

[0172] The method for determining whether to modify an element in the blending parameter list of the current block is as described above, and below, when the determination of whether to modify an element in the blending parameter list is 'true', a method for modifying an element in the blending parameter list using restored information is described.

[0173] To modify an element in a blending parameter list, an image decoding device may select an area for obtaining restored information using at least one of the following methods.

[0174] 1. Some of the restored areas within a preset distance adjacent to the current block or reference block.

[0175] 2. Part of the area of ​​another channel corresponding to the current block or reference block.

[0176] 3. Part of the area of ​​another channel corresponding to the area adjacent to the current block or reference block.

[0177] 4. Some restored areas exist in preset locations.

[0178] The region of another channel corresponding to the current block or reference block refers to a region that is at the same position in another channel as the channel containing the current block or reference block, or a region that is at a position determined according to a predefined relational expression or condition. In addition, the region of another channel corresponding to a region adjacent to the current block or reference block refers to a region that is at the same position in another channel as the channel containing the adjacent region, or a region that is at a position determined according to a predefined relational expression or condition. In this case, if the sizes of the channels are different, the same position may be a position with the same relative position within the channel.

[0179] <Example 1-1> Removing single or multiple elements in a list

[0180] In the present implementation, the video decoding device can reconstruct the blending parameter list by removing one or more elements from the predefined blending parameter list based on the restored information. For example, the video decoding device can reconstruct the blending parameter list by removing elements that are not used in neighboring blocks of the current block from the blending parameter list. The video decoding device can reconstruct the blending parameter list by removing elements with a high matching cost from the blending parameter list. The matching cost can be calculated according to the method described in the process of determining whether to modify an element in the blending parameter list using the matching cost. The number of elements to be removed from the blending parameter list can be signaled at the picture / slice / tile / CTU / CU level or determined as a preset value.

[0181] For example, in unidirectional prediction and bidirectional prediction, let the elements in the blending parameter list be assumed as weights for each reference block, and let the predefined blending parameter list be {4, 5, 3, 7, 1}. The index represents the index for each weight in {4, 5, 3, 7, 1} (for example, the index of weight 4 is 0, and the index of weight 7 is 3).

[0182] As an example, the video decoding device can modify the blending parameter list by removing elements in the blending parameter list that are not used in the surrounding blocks of the current block, as follows. Assume that 4, 5, and 7 are used as elements in the blending parameter list in the surrounding blocks of the current block, as shown in FIG. 11. The video decoding device can remove elements 3 and 1, which are unused elements in the blending parameter list, as shown in FIG. 12.

[0183] As another example, the image decoding device can reconstruct the blending parameter list by removing elements with high matching costs from the blending parameter list as follows.

[0184] The video decoding device sets template regions for the reference block and the current block. The template region may be set based on at least one of the methods for selecting a region for obtaining restored information, as described in Example 1.

[0185] The video decoding device calculates a matching cost for elements within the blending parameter list. The matching cost, as described in Example 1, represents the difference between the predicted values ​​and the reconstructed values ​​when predicting a preset template region using elements within the blending parameter list. The matching cost can be calculated based on a loss function such as SAD or SATD.

[0186] The video decoding device can selectively remove elements from the blending parameter list by comparing matching costs. As shown in Fig. 13, elements with high matching costs can be removed from the blending parameter list. The number of elements to be removed can be signaled at the picture / slice / tile / CTU / CU level or determined as a preset value.

[0187] <Example 1-2> Adding single or multiple elements to a list

[0188] In the present implementation, the video decoding device can reconstruct the blending parameter list by adding one or more elements to a predefined blending parameter list based on the restored information. For example, the video decoding device can reconstruct the blending parameter list by adding a blending parameter corresponding to an element with a low matching cost as a new element within the restored list (e.g., a blending parameter list used in a neighboring block of the current block or a predefined blending parameter list). The number of elements to be added to the blending parameter list can be signaled at the picture / slice / tile / CTU / CU level or determined as a preset value.

[0189] For example, in unidirectional prediction and bidirectional prediction, let the elements in the blending parameter list be assumed as weights for each reference block, and let the predefined blending parameter list be {4, 5, 3, 7, 1}. The index represents the index for each weight in {4, 5, 3, 7, 1} (for example, the index of weight 4 is 0, and the index of weight 7 is 3).

[0190] As an example, the video decoding device can reconstruct the blending parameter list by adding elements in the blending parameter list used in the surrounding blocks of the current block as follows. Assume that 2, 4, 5, and 6 are used as elements in the blending parameter list in the surrounding blocks of the current block, as in FIG. 14. The video decoding device can add 2 and 6 as new elements to the front or back of the blending parameter list, as in FIG. 15 or FIG. 16.

[0191] As another example, the video decoding device can reconstruct the blending parameter list by adding elements with low matching costs as follows.

[0192] The video decoding device sets template regions for the reference block and the current block. The template region may be set based on at least one of the methods for selecting a region for obtaining restored information, as described in Example 1.

[0193] The video decoding device calculates a matching cost for elements within the blending parameter list. The matching cost, as described in Example 1, represents the difference between the predicted values ​​for a preset template region using elements within the blending parameter list and the reconstructed values. The matching cost can be calculated based on a loss function such as SAD or SATD.

[0194] The video decoding device can compare matching costs to add new elements to the blending parameter list. For example, the average of elements in the blending parameter list with low matching costs can be added. As shown in Fig. 17, 6, which is the average of elements 5 and 7, which have low matching costs, can be added as a new element. The new element can be added to the front or back of the blending parameter list. The number of elements to be added can be signaled at the picture / slice / tile / CTU / CU level or determined by a preset value.

[0195] <Example 1-3> Selectively use Example 1-1 and Example 1-2.

[0196] In this implementation example, the video encoding device signals an index indicating one of Implementation Example 1-1, Implementation Example 1-2, Application of Implementation Example 1-1 and then Implementation Example 1-2, and Application of Implementation Example 1-2 and then Implementation Example 1-1, and the video decoding device selects an element modification method in the blending parameter list according to the signaled index. The syntax elements required to apply this implementation example to BCW are as follows.

[0197] blending_param_modification_idx: This index indicates how to modify an element in the blending parameter list, and can have values ​​of 0, 1, 2, and 3. When the value of this index is 0, the video decoding device can remove a single or multiple elements from the blending parameter list based on Implementation Example 1-1. When the value of this index is 1, the video decoding device can add a single or multiple elements to the blending parameter list based on Implementation Example 1-2. When the value of this index is 2, the video decoding device can sequentially use Implementation Example 1-1 and Implementation Example 1-2 to remove a single or multiple elements from the blending parameter list, and then add a single or multiple elements. When the value of this index is 3, the video decoding device can sequentially use Implementation Example 1-2 and Implementation Example 1-1 to add a single or multiple elements to the blending parameter list, and then remove a single or multiple elements.

[0198] Syntax transmission according to this implementation example can be realized according to the pseudocode shown in Table 3.

[0199]

[0200] <Example 2> Sorting elements of the blending parameter list

[0201] FIG. 18 is an exemplary diagram showing the alignment of blending parameters according to one embodiment of the present disclosure.

[0202] In this implementation example, as shown in FIG. 18, the image encoding device and the image decoding device reconstruct an optimal blending parameter list by reordering the order of elements within a predefined blending parameter list. Accordingly, the sorted blending parameter list can be used identically by the image encoding device and the image decoding device.

[0203] The video decoding device can determine whether to sort elements in the blending parameter list by referring to the reconfiguration_flag value. In the present implementation example, reconfiguration_flag is a flag indicating whether to sort elements in the blending parameter list. Hereinafter, if the reconfiguration_flag value is 'true (or 1)', it is assumed that the video decoding device sorts elements in the blending parameter list using restored information. On the other hand, if the reconfiguration_flag value is 'false (or 0)', it is assumed that the video decoding device does not sort elements in the blending parameter list.

[0204] As an example, a video encoding device can explicitly signal a reconfiguration_flag to a video decoding device. As another example, a video decoding device can implicitly generate a reconfiguration_flag according to a predetermined criterion (without a separate signal). When explicitly signaling a reconfiguration_flag, a flag indicating whether elements in a blending parameter list are ordered at at least one of the picture / slice / tile / CTU (Coding Tree Unit) / CU levels can be transmitted. The signaling of the reconfiguration_flag can be specifically explained by replacing the content related to modification in the description of explicitly signaling whether to modify the blending parameter list in Implementation Example 1 with reordering.

[0205] In this implementation example, the content related to sorting can correspond to the content related to modification among the descriptions related to implementation example 1.

[0206] In order to implicitly determine whether to sort elements in the blending parameter list according to a predetermined criterion, the decoder side of the video encoding device and the decoder side of the video decoding device can determine whether to sort elements in the blending parameter list according to the same method as follows.

[0207] 1. How to determine the reconfiguration_flag value based on information in the current block

[0208] 2. How to determine the reconfiguration_flag value based on the matching cost

[0209] 3. How to determine the reconfiguration_flag value of the current block by referencing the reconfiguration_flag value of the surrounding blocks.

[0210] Each method can be explained, specifically, by replacing the content related to modification in the description of implicitly determining whether to modify the blending parameters in Example 1 with an alignment.

[0211] The method for determining the reconfiguration_flag value of the current block is as described above, and below, a method for sorting elements in the blending parameter list using restored information when the reconfiguration_flag value is 'true' is described.

[0212] In order to sort elements in the blending parameter list, the image decoding device may use at least one of the methods for selecting a region for obtaining restored information described in Implementation Example 1 as a method for selecting a region for obtaining restored information.

[0213] <Example 2-1> Sorting all elements in a list

[0214] In this implementation example, the image decoding device arranges the order of all elements in a predefined blending parameter list.

[0215] As an example, the video decoding device can sort elements in the blending parameter list based on matching costs as follows.

[0216] The video decoding device sets template regions for the reference block and the current block. The template region may be set based on at least one of the methods for selecting a region for obtaining restored information, as described in Example 1.

[0217] The video decoding device calculates a matching cost for elements within the blending parameter list. The matching cost, as described in Example 1, represents the difference between the predicted values ​​and the reconstructed values ​​when predicting a preset template region using elements within the blending parameter list. The matching cost can be calculated based on a loss function such as SAD or SATD.

[0218] The video decoding device can sort elements within the blending parameter list by comparing matching costs. As shown in Fig. 19, elements within the blending parameter list with lower matching costs can be listed with priority.

[0219] <Example 2-2> Sorting all elements in the list by dividing them into multiple groups

[0220] In this implementation example, the video decoding device divides all elements in a predefined blending parameter list into multiple groups, and sorts the divided groups. For example, the video decoding device can divide all elements into a group of elements used in blocks surrounding a current block and a group of remaining elements, and sort the divided groups. The video decoding device can divide all elements into a group of elements included in a preset range (e.g., an average of elements in the blending parameter list ± N) and a group of remaining elements, and sort the divided groups. Here, N can be signaled at the picture / slice / tile / CTU / CU level or determined as a preset value. The sorting of the divided groups can include inter-group sorting and intra-group sorting.

[0221] For example, in unidirectional prediction and bidirectional prediction, let the elements in the blending parameter list be assumed as weights for each reference block, and let the predefined blending parameter list be {4, 5, 3, 7, 1}. The index represents the index for each weight in {4, 5, 3, 7, 1} (for example, the index of weight 4 is 0, and the index of weight 7 is 3).

[0222] As an example, an image decoding device can divide all elements into a group of elements used in blocks surrounding the current block and a group of remaining elements. Assuming that elements 4, 5, and 7 are used as elements in a blending parameter list in blocks surrounding the current block, as shown in FIG. 11, the image decoding device can divide all elements into a group including elements 4, 5, and 7 and a group including elements 3 and 1, as shown in FIG. 20, and sort the order of elements in the group including elements 4, 5, and 7 based on the matching cost.

[0223] As another example, the image decoding device may divide the entire elements into groups of elements included in a preset range (e.g., average ± N of elements in the blending parameter list) and groups of remaining elements, and perform inter-group alignment and intra-group alignment, as follows.

[0224] The video decoding device sets template regions for the reference block and the current block. The template region may be set based on at least one of the methods for selecting a region for obtaining restored information, as described in Example 1.

[0225] The video decoding device calculates a matching cost for elements within the blending parameter list. The matching cost, as described in Example 1, represents the difference between the predicted values ​​and the reconstructed values ​​when predicting a preset template region using elements within the blending parameter list. The matching cost can be calculated based on a loss function such as SAD or SATD.

[0226] The video decoding device divides the elements in the blending parameter list into multiple groups. As shown in Fig. 21, the video decoding device can generate a group including elements 3, 4, and 5 and a group including elements 1 and 7 by dividing all elements into groups of elements that fall within a preset range (e.g., the average ± N of the elements in the blending parameter list) and the remaining groups.

[0227] The video decoding device can sort elements within a blending parameter list by comparing matching costs. Sorting of the partitioned groups may include inter-group sorting and intra-group sorting. Inter-group sorting sorts the order of elements within a group by comparing the representative matching costs of each group. The representative matching costs may include the average SAD of each group, the minimum SAD of each group, etc. Intra-group sorting sorts the order of elements within a group based on the matching costs of the elements within the group.

[0228] <Example 2-3> Selecting and sorting specific elements within a list

[0229] In this implementation example, the video decoding device selects specific elements within a predefined blending parameter list and sorts the selected elements. For example, the video decoding device may select elements within a blending parameter list used in blocks surrounding a current block and sort the elements by comparing matching costs associated with the selected elements. The video decoding device may select specific elements within a predefined blending parameter list and sort the elements by comparing matching costs associated with the selected elements. The matching cost may be calculated according to the method described in the process of determining whether to modify an element within a blending parameter list using the matching cost.

[0230] For example, in unidirectional prediction and bidirectional prediction, let the elements in the blending parameter list be assumed as weights for each reference block, and let the predefined blending parameter list be {4, 5, 3, 7, 1}. The index represents the index for each weight in {4, 5, 3, 7, 1} (for example, the index of weight 4 is 0, and the index of weight 7 is 3).

[0231] As an example, the video decoding device can select elements in the blending parameter list used in the surrounding blocks of the current block and sort the selected elements as follows. Assume that 1, 4, and 7 are used as elements in the blending parameter list in the surrounding blocks of the current block, as shown in FIG. 22. The video decoding device can sort the selected elements by selecting only the predefined blending parameters 1, 4, and 7, as shown in FIG. 23, and comparing the matching costs. Unselected elements can be placed before or after the selected elements.

[0232] As another example, the video decoding device can select specific elements from a predefined blending parameter list and sort the selected elements by comparing matching costs, as follows.

[0233] The video decoding device sets template regions for the reference block and the current block. The template region may be set based on at least one of the methods for selecting a region for obtaining restored information, as described in Example 1.

[0234] The video decoding device selects specific elements to be sorted within the blending parameter list. For example, a predetermined number of blending parameters may be selected in descending order of index.

[0235] The video decoding device calculates the matching cost of the selected elements. The matching cost represents the difference between the predicted values ​​and the reconstructed values ​​when predicting a preset template region using elements in the blending parameter list, as described in Example 1. The matching cost can be calculated based on loss functions such as SAD and SATD.

[0236] The video decoding device can reconstruct the blending parameter list by sorting the selected elements by comparing matching costs associated with the selected elements.

[0237] <Example 2-4> Selectively use Example 2-1, Example 2-2, and Example 2-3.

[0238] In this implementation example, the video encoding device signals an index indicating one of Implementation Example 2-1, Implementation Example 2-2, and Implementation Example 2-3, and the video decoding device selects a method for arranging the order of elements in the blending parameter list according to the signaled index. The syntax elements required to apply this implementation example to BCW are as follows.

[0239] blending_param_reordering_idx: This index indicates the ordering method of elements in the blending parameter list, and can have values ​​of 0, 1, and 2. When the value of this index is 0, the video decoding device can sort all elements in the blending parameter list based on Implementation Example 2-1. When the value of this index is 1, the video decoding device can divide all elements in the blending parameter list into multiple groups based on Implementation Example 2-2, and sort the divided groups. When the value of this index is 2, the video decoding device can select specific elements in the blending parameter list based on Implementation Example 2-3, and sort the selected elements.

[0240] <Example 3> Selectively use Examples 1 and 2.

[0241] FIG. 24 is an exemplary diagram showing the reconfiguration of blending parameters according to one embodiment of the present disclosure.

[0242] In this implementation example, as shown in FIG. 24, the image encoding device and the image decoding device reconstruct the optimal blending parameter list by modifying elements in the blending parameter list and rearranging the order of the elements based on the restored information. Therefore, the reconstructed blending parameter list can be used identically by the image encoding device and the image decoding device.

[0243] The video decoding device can determine whether to reconfigure elements in the blending parameter list by referring to the reconfiguration_flag value. In the present implementation example, reconfiguration_flag is a flag indicating whether to reconfigure elements in the blending parameter list. Hereinafter, if the reconfiguration_flag value is 'true (or 1)', it is assumed that the video decoding device reconfigures elements in the blending parameter list using restored information. On the other hand, if the reconfiguration_flag value is 'false (or 0)', it is assumed that the video decoding device does not reconfigure elements in the blending parameter list. The blending parameter list can be reconfigured based on modification of elements in the blending parameter list and arrangement of the order of elements. The same method as in Implementation Example 1 can be used as a method for modifying elements in the list, and the same method as in Implementation Example 2 can be used as a method for arranging the order of elements in the list.

[0244] As an example, a video encoding device can explicitly signal a reconfiguration_flag to a video decoding device. As another example, the video decoding device can implicitly generate a reconfiguration_flag according to a predetermined criterion (without a separate signal). When explicitly signaling a reconfiguration_flag, a flag indicating whether elements in a blending parameter list are reconfigured at at least one of the picture / slice / tile / CTU (Coding Tree Unit) / CU levels can be transmitted. The signaling of the reconfiguration_flag can be described, specifically, by replacing the content related to modification in the description of explicitly signaling whether to modify the blending parameter list in Implementation Example 1 with reconfiguration.

[0245] In this implementation example, the content related to reconstruction can correspond to the content related to modification among the descriptions related to implementation example 1.

[0246] In order to implicitly determine whether to reconstruct the blending parameter list according to a predetermined criterion, the decoder side of the video encoding device and the decoder side of the video decoding device can determine whether to reconstruct the blending parameter list according to the following identical method.

[0247] 1. How to determine the reconfiguration_flag value based on information in the current block

[0248] 2. How to determine the reconfiguration_flag value based on the matching cost

[0249] 3. How to determine the reconfiguration_flag value of the current block by referencing the reconfiguration_flag value of the surrounding blocks.

[0250] Each method can be explained, specifically, by replacing the content related to modification in the description of implicitly determining whether to modify the blending parameters in Example 1 with reconstruction.

[0251] In this implementation example, a video encoding device signals an index indicating one of implementation example 1, implementation example 2, application of implementation example 1 followed by implementation example 2, and application of implementation example 2 followed by implementation example 1, and the video decoding device selects a method for reconstructing a blending parameter list according to the signaled index. The syntax elements required to apply this implementation example to BCW are as follows.

[0252] blending_param_reconfiguration_idx: This index indicates a method of reconfiguring the blending parameter list, and can have values ​​of 0, 1, 2, and 3. When the value of this index is 0, the video decoding device can modify the elements in the blending parameter list based on implementation example 1. When the value of this index is 1, the video decoding device can sort the order of the elements in the blending parameter list based on implementation example 2. When the value of this index is 1, the video decoding device can sequentially use implementation example 1 and implementation example 2 to modify the elements in the blending parameter list, and then sort the order of the elements. When the value of this index is 1, the video decoding device sequentially uses implementation example 2 and implementation example 1 to sort the elements in the blending parameter list, and then modify the elements.

[0253] Hereinafter, a method for reconstructing blending parameters according to Example 1 using the illustrations of FIGS. 25 and 26 is described. Here, the reconstructing may include modification and / or reordering of the blending parameter list.

[0254] FIG. 25 is a flowchart illustrating a method of encoding a current block performed by an image encoding device according to one embodiment of the present disclosure.

[0255] An image encoding device obtains an index indicating one or more reference blocks, a blending parameter list, and one of the elements in the blending parameter list for a current block (S2500). Here, each element in the blending parameter list is used for weighted combining of one or more reference blocks.

[0256] The video encoding device can obtain a list of blending parameters used in blocks surrounding the current block. The video encoding device can obtain a list of blending parameters defined in advance from a higher level.

[0257] The video encoding device obtains a first flag indicating whether to reconstruct the blending parameter list (S2502).

[0258] As an example, a video encoding device may obtain a first flag from a higher level. Thereafter, the video encoding device may encode the first flag.

[0259] As another example, the video encoding device can implicitly derive the first flag based on information about the current block, a matching cost according to application of a blending parameter list reconstruction method, whether the blending parameter list of the surrounding block is reconstructed, etc.

[0260] The video encoding device can use the location of the current block as information about the current block. For example, if the current block is located at a boundary associated with at least one of a picture / slice / tile / CTU, the video encoding device can set the value of the first flag to false. On the other hand, if the current block is not located at a boundary associated with at least one of a picture / slice / tile / CTU, the video encoding device can set the value of the first flag to true.

[0261] The video encoding device may calculate a first matching cost by applying a blending parameter list to a template of a current block and templates of one or more reference blocks. Here, the first matching cost may represent a difference between reconstructed pixel values ​​and predicted pixel values ​​with respect to the template of the current block. The video encoding device may reconstruct a blending parameter list according to a blending parameter list reconstructing method. The video encoding device may calculate a second matching cost by applying the reconstructed blending parameter list to the template of the current block and templates of one or more reference blocks. The video encoding device may set a value of a first flag based on the first matching cost and the second matching cost. For example, when the second matching cost is less than the first matching cost, the value of the first flag may be set to true. On the other hand, when the second matching cost is greater than or equal to the first matching cost, the value of the first flag may be set to false.

[0262] Representative values ​​can be calculated as the first matching cost and the second matching cost. The representative value corresponding to the first matching cost can be calculated based on the matching cost of each element in the blending parameter list. The representative value corresponding to the second matching cost can be calculated based on the matching cost of each element in the reconstructed blending parameter list.

[0263] The video encoding device can implicitly derive the first flag based on whether to reconstruct the blending parameter list of the surrounding blocks as follows.

[0264] The video encoding device can set the value of the first flag depending on whether to reconstruct the blending parameter list of a block including pixels existing at a predefined location around the current block.

[0265] The video encoding device can check whether a blending parameter list of surrounding blocks of a current block is reconstructed, and set a value of a first flag depending on whether a blending parameter list used in more surrounding blocks is reconstructed based on the number of blocks.

[0266] The video encoding device can check whether to reconstruct the blending parameter list of surrounding blocks of the current block, and set the value of the first flag depending on whether to reconstruct the blending parameter list used in surrounding blocks having a larger area in terms of area.

[0267] The video encoding device can set the value of the first flag based on whether to reconstruct the blending parameter list of a block among the surrounding blocks of the current block, with respect to a block having the same aspect ratio and size as or most similar to the current block.

[0268] The video encoding device checks the first flag (S2504).

[0269] If the first flag is true (Yes in S2504), the video encoding device performs the following steps.

[0270] The video encoding device reconstructs the blending parameter list based on the restoration information of the current block using a blending parameter list reconstruction method (S2506). Here, the restoration information represents information utilized on the decoder side of the video encoding device.

[0271] The video encoding device can reconstruct the blending parameter list by removing elements in the blending parameter list that are not used in blocks surrounding the current block. The video encoding device can reconstruct the blending parameter list by adding elements with low matching costs in the blending parameter list used in blocks surrounding the current block to the blending parameter list.

[0272] The video encoding device can reconstruct the blending parameter list by sorting the order of all elements in the blending parameter list. The video encoding device can reconstruct the blending parameter list by dividing all elements in the blending parameter list into multiple groups and sorting the divided groups. The video encoding device can reconstruct the blending parameter list by selecting specific elements in the blending parameter list and sorting the selected elements.

[0273] The image encoding device extracts weights from the reconstructed blending parameter list based on the index (S2508).

[0274] The video encoding device generates a prediction block of the current block by weighting and combining one or more reference blocks based on weights (S2510).

[0275] The video encoding device can encode an index.

[0276] If the first flag is false (No in S2504), the video encoding device may omit reconstructing the blending parameter list. The video encoding device may generate a prediction block of the current block using the acquired blending parameter list. The video encoding device may encode an index.

[0277] Thereafter, the video encoding device can generate a residual block by subtracting the predicted block from the current block. The video encoding device can apply transformation / quantization to the residual block to generate quantized transform coefficients, and encode the quantized transform coefficients to generate a bitstream.

[0278] FIG. 26 is a flowchart illustrating a method for restoring a current block performed by an image decoding device according to one embodiment of the present disclosure.

[0279] The video decoding device obtains one or more reference blocks and a blending parameter list for the current block (S2600).

[0280] The video decoding device can obtain a list of blending parameters used in blocks surrounding the current block. The video decoding device can obtain a list of blending parameters defined in advance from a higher level.

[0281] The video decoding device decodes an index indicating one of the elements in the blending parameter list (S2602). Here, each element in the blending parameter list is used for weighted combining of one or more reference blocks.

[0282] The video decoding device obtains a first flag indicating whether to reconstruct the blending parameter list (S2604).

[0283] As an example, the video decoding device can explicitly decode the first flag from the bitstream.

[0284] As another example, the image decoding device can implicitly derive the first flag based on information about the current block, a matching cost according to application of a blending parameter estimation method, whether blending parameters of surrounding blocks are estimated, etc.

[0285] The video decoding device may use the location of the current block as information about the current block. For example, if the current block is located on a boundary associated with at least one of a picture / slice / tile / CTU, the video decoding device may set the value of the first flag to false. Conversely, if the current block is not located on a boundary associated with at least one of a picture / slice / tile / CTU, the video decoding device may set the value of the first flag to true.

[0286] An image decoding device may calculate a first matching cost by applying a blending parameter list to a template of a current block and templates of one or more reference blocks. Here, the first matching cost may represent a difference between reconstructed pixel values ​​and predicted pixel values ​​with respect to the template of the current block. The image decoding device may reconstruct a blending parameter list according to a blending parameter list reconstructing method. The image decoding device may calculate a second matching cost by applying the reconstructed blending parameter list to the template of the current block and templates of one or more reference blocks. The image decoding device may set a value of a first flag based on the first matching cost and the second matching cost. For example, when the second matching cost is less than the first matching cost, the value of the first flag may be set to true. On the other hand, when the second matching cost is greater than or equal to the first matching cost, the value of the first flag may be set to false.

[0287] Representative values ​​can be calculated as the first matching cost and the second matching cost. The representative value corresponding to the first matching cost can be calculated based on the matching cost of each element in the blending parameter list. The representative value corresponding to the second matching cost can be calculated based on the matching cost of each element in the reconstructed blending parameter list.

[0288] The video decoding device can implicitly derive the first flag based on whether the blending parameter list of the surrounding blocks is reconstructed as follows.

[0289] The video decoding device can set the value of the first flag depending on whether to reconstruct the blending parameter list of a block including pixels existing at a predefined location around the current block.

[0290] The video decoding device can check whether a blending parameter list of surrounding blocks of the current block is reconstructed, and set a value of a first flag depending on whether a blending parameter list used in more surrounding blocks is reconstructed based on the number of blocks.

[0291] The video decoding device can check whether the blending parameter list of the surrounding blocks of the current block is reconstructed, and set the value of the first flag depending on whether the blending parameter list used in the surrounding blocks having a larger area is reconstructed.

[0292] The video decoding device may set the value of the first flag based on whether to reconstruct the blending parameter list of a block among the surrounding blocks of the current block, the block having the same aspect ratio and size as or most similar to the current block.

[0293] The video decryption device checks the first flag (S2606).

[0294] If the first flag is true (Yes in S2606), the video decoding device performs the following steps.

[0295] The video decoding device reconstructs the blending parameter list based on the restoration information of the current block using a blending parameter list reconstruction method (S2608). Here, the restoration information represents information utilized on the decoder side of the video decoding device.

[0296] The video decoding device can reconstruct the blending parameter list by removing elements in the blending parameter list that are not used in blocks surrounding the current block.

[0297] The video decoding device can calculate the matching cost of each element in the blending parameter list by applying each element to the template of the current block and the template of one or more reference blocks. The video decoding device can reconstruct the blending parameter list by removing elements with high matching costs from the blending parameter list.

[0298] The video decoding device can reconstruct the blending parameter list by adding elements with low matching costs among the blending parameter lists used in the surrounding blocks of the current block to the blending parameter list.

[0299] The video decoding device can calculate the matching cost of each element in the blending parameter list by applying each element to the template of the current block and the template of one or more reference blocks. The video decoding device can reconstruct the blending parameter list by calculating the average of elements with low matching costs in the blending parameter list and adding the average to the blending parameter list.

[0300] The video decoding device can reconstruct the blending parameter list by sorting the order of all elements in the blending parameter list. The video decoding device can calculate the matching cost of each element by applying each element in the blending parameter list to the template of the current block and the template of one or more reference blocks. The video decoding device can sort the elements in the blending parameter list by comparing the matching costs.

[0301] The video decoding device can divide all elements in the blending parameter list into groups of elements used in blocks surrounding the current block and groups of remaining elements, and sort the divided groups. The video decoding device can calculate the matching cost of each element by applying each element in the element group to the template of the current block and the template of one or more reference blocks. The video decoding device can reconstruct the blending parameter list by sorting the elements in the element groups by comparing the matching costs.

[0302] An image decoding device can divide all elements in a blending parameter list into groups of elements falling within a preset range and groups of remaining elements, and sort the divided groups. Each element in the blending parameter list can be applied to a template of a current block and a template of one or more reference blocks to calculate a matching cost of each element. Sorting of the divided groups can include inter-group sorting and intra-group sorting. Inter-group sorting sorts the order between groups by comparing representative matching cost values ​​of each group. Intra-group sorting sorts the order of elements within a group based on the matching costs of the elements within the group. By performing inter-group sorting and intra-group sorting, the image decoding device can reconstruct the blending parameter list.

[0303] The video decoding device can select elements from a blending parameter list used in blocks surrounding the current block, apply the selected elements to the template of the current block and the templates of one or more reference blocks, and calculate the matching cost of each selected element. The video decoding device can reconstruct the blending parameter list by sorting the selected elements based on the comparison of the matching costs.

[0304] The video decoding device can select specific elements within the blending parameter list (e.g., based on an index), apply the selected elements to the template of the current block and the templates of one or more reference blocks, and calculate the matching cost of each selected element. The video decoding device can reconstruct the blending parameter list by sorting the selected elements based on the matching costs.

[0305] The image decoding device extracts weights from the reconstructed blending parameter list based on the index (S2610).

[0306] The video decoding device generates a prediction block of the current block by weighting and combining one or more reference blocks based on weights (S2612).

[0307] If the first flag is false (No in S2606), the image decoding device may omit reconstructing the blending parameter list. The image decoding device may generate a prediction block of the current block using the acquired blending parameter list.

[0308] Thereafter, the video decoding device can decode quantized transform coefficients from the bitstream and apply inverse quantization / inverse transformation to the quantized transform coefficients to generate a residual block. The video decoding device can generate a restored block of the current block by adding the residual block and the prediction block.

[0309] Although the flowchart / timing diagram of this specification describes each process as being executed sequentially, this is merely an illustrative description of the technical idea of ​​one embodiment of the present disclosure. In other words, a person of ordinary skill in the art to which one embodiment of the present disclosure belongs may modify and apply various modifications and variations by changing the order described in the flowchart / timing diagram without departing from the essential characteristics of one embodiment of the present disclosure, or by executing one or more of the processes in parallel. Therefore, the flowchart / timing diagram is not limited to a chronological order.

[0310] It should be understood that the exemplary embodiments described above can be implemented in many different ways. The functions or methods described in one or more examples can be implemented in hardware, software, firmware, or any combination thereof. It should be understood that the functional components described herein are labeled as "units" to further emphasize their implementation independence.

[0311] Meanwhile, the various functions or methods described in this embodiment may also be implemented as instructions stored on a non-transitory storage medium that can be read and executed by one or more processors. Non-transitory storage media include, for example, all types of storage devices that store data in a form readable by a computer system. For example, non-transitory storage media include storage media such as erasable programmable read-only memory (EPROM), flash drives, optical drives, magnetic hard drives, and solid-state drives (SSDs).

[0312] The above description is merely an example of the technical idea of ​​the present embodiment, and those skilled in the art to which the present embodiment pertains may make various modifications and variations without departing from the essential characteristics of the present embodiment. Therefore, the present embodiments are not intended to limit the technical idea of ​​the present embodiment, but to explain it, and the scope of the technical idea of ​​the present embodiment is not limited by these embodiments. The protection scope of the present embodiment should be interpreted by the following claims, and all technical ideas within a scope equivalent thereto should be interpreted as being included in the scope of rights of the present embodiment.

[0313]

[0314]

[0315] CROSS-REFERENCE TO RELATED APPLICATION

[0316] This patent application claims priority to Korean patent application No. 10-2024-0050896, filed on April 16, 2024, the entire contents of which are incorporated herein by reference.

Claims

1. In a method for restoring a current block performed by an image decryption device, A step of obtaining one or more reference blocks and a list of blending parameters for the current block; A step of decoding an index indicating one of the elements in the blending parameter list, wherein each element in the blending parameter list is used for weighted combining of one or more reference blocks; A step of obtaining a first flag indicating whether to reconstruct the above blending parameter list; and Step of checking the above first flag Including, If the above first flag is true, A step of reconstructing the blending parameter list based on the restoration information of the current block using a blending parameter list reconstruction method; A step of extracting weights from a reconstructed blending parameter list based on the above index; and A step of generating a prediction block of the current block by weighting and combining one or more reference blocks based on the weights. A method further comprising:

2. In paragraph 1, The step of acquiring the above first flag is: comprising a step of implicitly deriving the first flag; The step of inducing the above first flag is: A step of calculating a first matching cost by applying the blending parameter list to the template of the current block and the template of the one or more reference blocks, wherein the first matching cost represents a difference between restored pixel values ​​and predicted pixel values ​​with respect to the template of the current block; A step of reconstructing the blending parameter list according to the above blending parameter list reconstruction method; A step of calculating a second matching cost by applying the reconstructed blending parameter list to the template of the current block and the template of one or more reference blocks; and A step of setting the value of the first flag based on the first matching cost and the second matching cost. A method comprising:

3. In paragraph 2, The step of calculating the second matching cost is: A method for calculating a representative value as the second matching cost, wherein the representative value is calculated based on the matching cost of each element in the reconstructed blending parameter list.

4. In paragraph 1, The step of acquiring the above first flag is: comprising a step of implicitly deriving the first flag; The step of inducing the above first flag is: A method for setting the value of the first flag depending on whether the blending parameter list of the block is reconstructed, with respect to a block containing pixels existing at a predefined location around the current block.

5. In paragraph 1, The step of reconstructing the above blending parameter list is: A method for removing elements in a list of unused blending parameters from surrounding blocks of the current block.

6. In paragraph 1, The step of reconstructing the above blending parameter list is: A step of calculating a matching cost of each element by applying each element in the blending parameter list to the template of the current block and the template of one or more reference blocks; and A step of removing elements with high matching costs from the above blending parameter list. A method comprising:

7. In paragraph 1, The step of reconstructing the above blending parameter list is: A method for adding an element having a low matching cost within the blending parameter list used in the surrounding blocks of the current block to the blending parameter list.

8. In paragraph 1, The step of reconstructing the above blending parameter list is: A step of calculating a matching cost of each element by applying each element in the blending parameter list to the template of the current block and the template of one or more reference blocks; and A step of calculating an average of elements with a low matching cost in the above blending parameter list and adding the average to the above blending parameter list. A method comprising:

9. In paragraph 1, The step of reconstructing the above blending parameter list is: A step of calculating a matching cost of each element by applying each element in the blending parameter list to the template of the current block and the template of one or more reference blocks; and A step of sorting elements in the blending parameter list by comparing the matching cost of each element. A method comprising:

10. In paragraph 1, The step of reconstructing the above blending parameter list is: A step of dividing all elements in the blending parameter list into a group of elements used in the surrounding blocks of the current block and a group of remaining elements; A step of calculating a matching cost of each element by applying each element in the above element group to the template of the current block and the template of one or more reference blocks; and A step of sorting elements within the element group by comparing the matching cost of each element. A method comprising:

11. In paragraph 1, The step of reconstructing the above blending parameter list is: A step of dividing all elements in the above blending parameter list into a group of elements included in a preset range and a group of remaining elements; A step of calculating a matching cost of each element by applying each element in the blending parameter list to the template of the current block and the template of one or more reference blocks; and A step for performing inter-group and intra-group sorting by comparing the matching cost of each element. A method comprising:

12. In a method for encoding a current block performed by a video encoding device, A step of obtaining one or more reference blocks for the current block, a blending parameter list, and an index indicating one of the elements in the blending parameter list, wherein each element in the blending parameter list is used for weighted combining of the one or more reference blocks; A step of obtaining a first flag indicating whether to reconstruct the above blending parameter list; and Step of checking the above first flag Including, If the above first flag is true, A step of reconstructing the blending parameter list based on the restoration information of the current block using a blending parameter list reconstruction method; A step of extracting weights from a reconstructed blending parameter list based on the above index; and A step of generating a prediction block of the current block by weighting and combining one or more reference blocks based on the weights. A method further comprising:

13. In paragraph 12, The step of acquiring the above first flag is: comprising a step of implicitly deriving the first flag; The step of inducing the above first flag is: A step of calculating a first matching cost by applying the blending parameter list to the template of the current block and the template of the one or more reference blocks, wherein the first matching cost represents a difference between restored pixel values ​​and predicted pixel values ​​with respect to the template of the current block; A step of reconstructing the blending parameter list according to the above blending parameter list reconstruction method; A step of calculating a second matching cost by applying the reconstructed blending parameter list to the template of the current block and the template of one or more reference blocks; and A step of setting the value of the first flag based on the first matching cost and the second matching cost. A method comprising:

14. In paragraph 13, The step of calculating the second matching cost is: A method for calculating a representative value as the second matching cost, wherein the representative value is calculated based on the matching cost of each element in the reconstructed blending parameter list.

15. In paragraph 12, The step of acquiring the above first flag is: comprising a step of implicitly deriving the first flag; The step of inducing the above first flag is: A method for setting the value of the first flag by referring to whether to reconstruct the blending parameter list of the surrounding blocks of the current block.

16. In a method for providing video data to a video decoding device, A step of encoding the above video data into a bitstream; and A step of transmitting the above bitstream to the image decoding device Including, The step of encoding the above video data is: A step of obtaining one or more reference blocks for a current block, a blending parameter list, and an index indicating one of the elements in the blending parameter list, wherein each element in the blending parameter list is used for weighted combining of the one or more reference blocks; A step of obtaining a first flag indicating whether to reconstruct the above blending parameter list; and Step of checking the above first flag Including, If the above first flag is true, A step of reconstructing the blending parameter list based on the restoration information of the current block using a blending parameter list reconstruction method; A step of extracting weights from a reconstructed blending parameter list based on the above index; and A step of generating a prediction block of the current block by weighting and combining one or more reference blocks based on the weights. A method further comprising:

Images