Regression-based weighted sum prediction method

Abstract

The present embodiment provides a regression-based weighted sum prediction method. In the present embodiment, an image decoding device acquires prediction mode information of a plurality of prediction blocks, and determines prediction modes of the prediction blocks on the basis of the prediction mode information. The prediction blocks are used for prediction of the current block, and the prediction modes include an inter prediction mode and an intra prediction mode. The image decoding device configures a prediction information list of each prediction mode, configures an integrated candidate list by combining the prediction information lists of the prediction modes, and reorders the integrated candidate list. The image decoding device predicts the current block on the basis of the reordered integrated candidate list.

Description

Regression-based weighted sum prediction method

[0001] The present disclosure relates to an image encoding / decoding method, an apparatus, and a recording medium for storing a bitstream. More specifically, it relates to a prediction method that derives a weighted sum coefficient based on a restored region around a current block and a plurality of reference blocks, and generates a final prediction block of a current block by weighting a plurality of prediction blocks based on the derived weighted sum coefficient.

[0002] The following description merely provides background information related to the present invention and does not constitute prior art.

[0003] Because video data contains a large amount of data compared to audio or still image data, storing or transmitting it as is without compression processing requires significant hardware resources, including memory.

[0004] Therefore, typically when storing or transmitting video data, the encoder compresses the video data for storage or transmission, and the decoder receives the compressed video data, decompresses it, and plays it. Such video compression technologies include H.264 / AVC, HEVC (High Efficiency Video Coding), and VVC (Versatile Video Coding), which improves coding efficiency by more than 30% compared to HEVC.

[0005] However, as video size, resolution, and frame rates are gradually increasing, and the amount of data that needs to be encoded is also growing accordingly, a new compression technology is required that offers better encoding efficiency and higher image quality improvement effects than existing compression technologies.

[0006] The present disclosure aims to provide an image encoding / decoding method and apparatus that derives weighted sum coefficients based on a restored region around a current block and a plurality of reference blocks, and generates a final predicted block of a current block by weighting summing a plurality of predicted blocks based on the derived weighted sum coefficients, and a recording medium that stores a bitstream generated by said image encoding method / apparatus.

[0007] According to an embodiment of the present disclosure, a method for image decoding to restore a current block comprises the steps of: acquiring prediction mode information of a plurality of prediction blocks and determining the prediction modes of the prediction blocks based on the prediction mode information, wherein the prediction blocks are used to predict the current block and the prediction modes include an inter-prediction mode and an intra-prediction mode; constructing a prediction information list of each prediction mode and combining the prediction information lists of the prediction modes to construct an integrated candidate list; rearranging the integrated candidate list; and predicting the current block based on the rearranged integrated candidate list.

[0008] According to another embodiment of the present disclosure, the step of predicting the current block comprises: generating the prediction blocks based on the reordered integrated candidate list; determining reference samples in the surrounding restored region of the current block and in the surrounding restored region of a plurality of reference blocks; calculating weighted sum coefficients of the prediction blocks using the reference samples, wherein the weighted sum coefficients are defined by a regression equation based on a pixel position within each prediction block; and predicting the current block by weighting the prediction blocks based on the weighted sum coefficients.

[0009] According to another embodiment of the present disclosure, a video encoding method for encoding a current block comprises the steps of: acquiring prediction mode information of a plurality of prediction blocks and determining the prediction modes of the prediction blocks based on the prediction mode information, wherein the prediction blocks are used to predict the current block and the prediction modes include an inter-prediction mode and an intra-prediction mode; constructing a prediction information list of each prediction mode and combining the prediction information lists of the prediction modes to construct an integrated candidate list; rearranging the integrated candidate list; and predicting the current block based on the rearranged integrated candidate list.

[0010] According to another embodiment of the present disclosure, a method for providing video data to a video decoder comprises: encoding the video data into a bitstream; and transmitting the bitstream to the video decoder, wherein the step of encoding the video data comprises: acquiring prediction mode information of a plurality of prediction blocks and determining the prediction modes of the prediction blocks based on the prediction mode information, wherein the prediction blocks are used for prediction of a current block and the prediction modes include an inter-prediction mode and an intra-prediction mode; constructing a prediction information list of each prediction mode and combining the prediction information lists of the prediction modes to construct an integrated candidate list; rearranging the integrated candidate list; and predicting the current block based on the rearranged integrated candidate list.

[0011] As described above, by providing a video encoding / decoding method, an apparatus, and a recording medium storing a bitstream generated by the video encoding method / apparatus according to the present embodiment, it is possible to improve video encoding efficiency and video quality.

[0012] FIG. 1 is an exemplary block diagram of an image encoding device capable of implementing the technologies of the present disclosure.

[0013] Figure 2 is a diagram illustrating a method for dividing blocks using a QTBTTT (QuadTree plus BinaryTree TernaryTree) structure.

[0014] FIGS. 3a and 3b are diagrams showing a plurality of intra prediction modes including wide-angle intra prediction modes.

[0015] Figure 4 is an example diagram of the surrounding blocks of the current block.

[0016] FIG. 5 is an exemplary block diagram of an image decoding device capable of implementing the technologies of the present disclosure.

[0017] Figure 6 is an example diagram schematically illustrating the regression-based GPM (Geometric Partitioning Mode).

[0018] FIG. 7 is a block diagram showing in detail a part of an image decoding device according to one embodiment of the present disclosure.

[0019] FIG. 8 is an illustrative diagram showing the determination of reference samples according to one embodiment of the present disclosure.

[0020] FIGS. 9 and FIGS. 10 are illustrative diagrams showing the determination of reference samples according to other embodiments of the present disclosure.

[0021] FIG. 11 is an illustrative diagram showing the determination of reference samples according to another embodiment of the present disclosure.

[0022] FIGS. 12 and FIGS. 13 are illustrative diagrams showing the determination of reference samples according to another embodiment of the present disclosure.

[0023] FIGS. 14 and FIGS. 15 are illustrative diagrams showing the determination of reference samples according to another embodiment of the present disclosure.

[0024] FIG. 16 is a flowchart illustrating a method for predicting a current block according to one embodiment of the present disclosure.

[0025] Hereinafter, embodiments of the present invention will be described in detail with reference to the exemplary drawings. It should be noted that in assigning reference numerals to the components of each drawing, the same components are given the same reference numeral whenever possible, even if they are shown in different drawings. Furthermore, in describing these embodiments, if it is determined that a detailed description of related known components or functions could obscure the essence of these embodiments, such detailed description is omitted.

[0026] FIG. 1 is an exemplary block diagram of an image encoding device capable of implementing the technologies of the present disclosure. Hereinafter, the image encoding device and its sub-components will be described with reference to FIG. 1.

[0027] The video encoding device may be configured to include a picture splitting unit (110), a prediction unit (120), a subtractor (130), a conversion unit (140), a quantization unit (145), a reordering unit (150), an entropy encoding unit (155), an inverse quantization unit (160), an inverse conversion unit (165), an adder (170), a loop filter unit (180), and a memory (190).

[0028] Each component of the video encoding device may be implemented in hardware or software, or as a combination of hardware and software. Additionally, the function of each component may be implemented in software, and a microprocessor may be implemented to execute the software function corresponding to each component.

[0029] A single image (video) consists of one or more sequences containing multiple pictures. Each picture is divided into multiple regions, and encoding is performed for each region. For example, a single picture is divided into one or more tiles and / or slices. Here, one or more tiles can be defined as a tile group. Each tile or slice is divided into one or more Coding Tree Units (CTUs). And each CTU is divided into one or more Coding Units (CUs) by a tree structure. Information applicable to each CU is encoded as the syntax of the CU, and information applicable to all CUs included in a single CTU is encoded as the syntax of the CTU. Additionally, information applicable to all blocks within a single slice is encoded as the syntax of the slice header, and information applicable to all blocks constituting one or more pictures is encoded in the Picture Parameter Set (PPS) or the picture header. Furthermore, information commonly referenced by multiple pictures is encoded in a Sequence Parameter Set (SPS). Also, information commonly referenced by one or more SPSs is encoded in a Video Parameter Set (VPS). Additionally, information commonly applicable to a single tile or tile group may be encoded as the syntax of a tile or tile group header. The syntax included in the SPS, PPS, slice header, and tile or tile group header may be referred to as high-level syntax.

[0030] The picture division unit (110) determines the size of the CTU. Information regarding the size of the CTU (CTU size) is encoded as a syntax of SPS or PPS and transmitted to an image decoder.

[0031] The picture division unit (110) divides each picture constituting the image into multiple CTUs having a predetermined size, and then recursively divides the CTUs using a tree structure. The leaf nodes in the tree structure become the CUs, which are the basic units of encoding.

[0032] The tree structure may be a QuadTree (QT) in which an upper node (or parent node) is divided into four lower nodes (or child nodes) of equal size, a BinaryTree (BT) in which an upper node is divided into two lower nodes, a TernaryTree (TT) in which an upper node is divided into three lower nodes in a 1:2:1 ratio, or a structure that combines two or more of these QT, BT, and TT structures. For example, a QTBT (QuadTree plus BinaryTree) structure may be used, or a QTBTTT (QuadTree plus BinaryTree TernaryTree) structure may be used. Here, BTTT combined may be referred to as an MTT (Multiple-Type Tree).

[0033] Figure 2 is a diagram illustrating a method for dividing blocks using a QTBTTT structure.

[0034] As illustrated in FIG. 2, the CTU can first be split into a QT structure. Quadtree splitting can be repeated until the size of the splitting block reaches the minimum block size of the leaf node allowed in QT (MinQTSize). A first flag (QT_split_flag) indicating whether each node of the QT structure is split into four nodes of the lower layer is encoded by the entropy encoder (155) and signaled to the image decoder. If the leaf node of the QT is not larger than the maximum block size of the root node allowed in BT (MaxBTSize), it can be further split into one or more of the BT structure or TT structure. In the BT structure and / or TT structure, multiple splitting directions may exist. For example, there may be two directions in which the block of the corresponding node is split horizontally and vertically. As shown in Figure 2, when MTT splitting begins, a second flag (mtt_split_flag) indicating whether the nodes have been split, and if splitting has occurred, a flag indicating the splitting direction (vertical or horizontal) and / or the splitting type (binary or ternary) are encoded by the entropy encoding unit (155) and signaled to the image decoding device.

[0035] Alternatively, prior to encoding the first flag (QT_split_flag) indicating whether each node is split into four nodes of the lower layer, the CU split flag (split_cu_flag) indicating whether the node is split may be encoded. If the value of the CU split flag (split_cu_flag) indicates that it is not split, the block of the corresponding node becomes a leaf node in the split tree structure and becomes a coding unit (CU), which is the basic unit of encoding. If the value of the CU split flag (split_cu_flag) indicates that it is split, the video encoding device starts encoding from the first flag in the manner described above.

[0036] When QTBT is used as another example of a tree structure, there may be two types: a type that divides the block of the corresponding node horizontally into two blocks of the same size (i.e., symmetric horizontal splitting) and a type that divides it vertically (i.e., symmetric vertical splitting). A splitting flag (split_flag) indicating whether each node of the BT structure is split into a block of a lower layer and splitting type information indicating the type of splitting are encoded by the entropy encoding unit (155) and transmitted to the image decoding device. Meanwhile, there may also be an additional type that divides the block of the corresponding node into two blocks of an asymmetric shape. The asymmetric shape may include a shape that divides the block of the corresponding node into two rectangular blocks with a size ratio of 1:3, or a shape that divides the block of the corresponding node diagonally.

[0037] A CU can have various sizes depending on the QTBT or QTBTTT partitioning from a CTU. Hereinafter, the block corresponding to the CU to be encoded or decoded (i.e., the leaf node of QTBTTT) is referred to as the 'current block'. Depending on the adoption of QTBTTT partitioning, the shape of the current block may be not only square but also rectangular.

[0038] The prediction unit (120) predicts the current block and generates a prediction block. The prediction unit (120) includes an intra prediction unit (122) and an inter prediction unit (124).

[0039] Generally, current blocks within a picture can each be predictively coded. Typically, the prediction of a current block can be performed using an intra-prediction technique (using data from the picture containing the current block) or an inter-prediction technique (using data from a picture coded prior to the picture containing the current block). Inter-prediction includes both unidirectional and bidirectional prediction.

[0040] The intra prediction unit (122) predicts pixels within the current block using pixels (reference pixels) located around the current block within the current picture containing the current block. Multiple intra prediction modes exist depending on the prediction direction. For example, as shown in FIG. 3a, multiple intra prediction modes may include two non-directional modes, including Planar mode and DC mode, and 65 directional modes. The surrounding pixels to be used and the calculation formula are defined differently for each prediction mode.

[0041] For efficient directional prediction for a rectangular current block, directional modes (intra-prediction modes 67 through 80 and -1 through -14) illustrated by dashed arrows in FIG. 3b may be additionally used. These may be referred to as "wide angle intra-prediction modes." In FIG. 3b, the arrows indicate corresponding reference samples used for prediction and do not indicate the prediction direction. The prediction direction is opposite to the direction indicated by the arrows. Wide angle intra-prediction modes are modes that perform prediction in the opposite direction of a specific directional mode without additional bit transmission when the current block is rectangular. Among the wide angle intra-prediction modes, some wide angle intra-prediction modes available for the current block may be determined by the ratio of the width to the height of the rectangular current block. For example, wide-angle intra prediction modes with an angle less than 45 degrees (intra prediction modes 67 to 80) are available when the current block is a rectangular shape with a height less than the width, and wide-angle intra prediction modes with an angle greater than -135 degrees (intra prediction modes -1 to -14) are available when the current block is a rectangular shape with a width less than the height.

[0042] The intra prediction unit (122) can determine the intra prediction mode to use for encoding the current block. In some examples, the intra prediction unit (122) may encode the current block using several intra prediction modes and select an appropriate intra prediction mode to use from the tested modes. For example, the intra prediction unit (122) may calculate the rate-distortion values ​​using a rate-distortion analysis of several tested intra prediction modes and select the intra prediction mode having the best rate-distortion features among the tested modes.

[0043] The intra prediction unit (122) selects one intra prediction mode among a plurality of intra prediction modes and predicts the current block using a calculation formula and surrounding pixels (reference pixels) determined according to the selected intra prediction mode. Information regarding the selected intra prediction mode is encoded by the entropy encoding unit (155) and transmitted to the image decoding device.

[0044] The inter prediction unit (124) generates a prediction block for the current block using a motion compensation process. The inter prediction unit (124) searches for the block most similar to the current block within a reference picture that is encoded and decoded before the current picture, and generates a prediction block for the current block using the searched block. Then, it generates a motion vector (MV) corresponding to the displacement between the current block in the current picture and the prediction block in the reference picture. Generally, motion estimation is performed on the lumina component, and the motion vector calculated based on the lumina component is used for both the lumina component and the chroma component. Motion information including information about the reference picture used to predict the current block and information about the motion vector is encoded by the entropy encoding unit (155) and transmitted to the image decoding device.

[0045] The inter prediction unit (124) may perform interpolation on a reference picture or reference block to increase the accuracy of the prediction. That is, subsamples between two consecutive integer samples are interpolated by applying filter coefficients to a plurality of consecutive integer samples including those two integer samples. When the process of searching for the block most similar to the current block is performed for the interpolated reference picture, the motion vector can be expressed with precision in fractional units rather than precision in integer sample units. The precision or resolution of the motion vector can be set differently for each unit of the target area to be encoded, such as a slice, tile, CTU, CU, etc. When such Adaptive Motion Vector Resolution (AMVR) is applied, information regarding the motion vector resolution to be applied to each target area must be signaled for each target area. For example, if the target area is a CU, information regarding the motion vector resolution applied to each CU is signaled. The information regarding the motion vector resolution may be information indicating the precision of the difference motion vector described later.

[0046] Meanwhile, the inter prediction unit (124) can perform inter prediction using bi-prediction. In the case of bi-prediction, two reference pictures and two motion vectors representing the block location most similar to the current block within each reference picture are used. The inter prediction unit (124) selects a first reference picture and a second reference picture from the reference picture list 0 (RefPicList0) and reference picture list 1 (RefPicList1), respectively, and generates a first reference block and a second reference block by searching for a block similar to the current block within each reference picture. Then, it generates a prediction block for the current block by averaging or weighting the first reference block and the second reference block. Then, it transmits motion information containing information about the two reference pictures used to predict the current block and information about the two motion vectors to the entropy encoding unit (155). Here, reference picture list 0 consists of restored pictures that are prior to the current picture in the display order, and reference picture list 1 may consist of restored pictures that are prior to the current picture in the display order. However, this is not necessarily limited to this, and restored pictures prior to the current picture in the display order may be additionally included in reference picture list 0, and conversely, restored pictures prior to the current picture may be additionally included in reference picture list 1.

[0047] Various methods can be used to minimize the amount of bits required to encode motion information.

[0048] For example, if the reference picture and motion vector of the current block are identical to the reference picture and motion vector of a neighboring block, the motion information of the current block can be transmitted to an image decoder by encoding information that can identify the neighboring block. This method is called 'merge mode'.

[0049] In merge mode, the inter prediction unit (124) selects a predetermined number of merge candidate blocks (hereinafter referred to as 'merge candidates') from the surrounding blocks of the current block.

[0050] As for the surrounding blocks for deriving merge candidates, as shown in FIG. 4, all or part of the left block (A0), bottom-left block (A1), top block (B0), top-right block (B1), and top-left block (B2) adjacent to the current block within the current picture may be used. Additionally, a block located within a reference picture (which may be the same as or different from the reference picture used to predict the current block) other than the current picture where the current block is located may be used as a merge candidate. For example, a block located at the same position as the current block within the reference picture (co-located block) or a block adjacent to that same position may be additionally used as a merge candidate. If the number of merge candidates selected by the method described above is less than a preset number, a 0 vector is added to the merge candidates.

[0051] The inter prediction unit (124) constructs a merge list containing a predetermined number of merge candidates using these surrounding blocks. Among the merge candidates included in the merge list, it selects a merge candidate to be used as movement information for the current block and generates merge index information to identify the selected candidate. The generated merge index information is encoded by the entropy encoding unit (155) and transmitted to the image decoding device.

[0052] Merge skip mode is a special case of merge mode; after quantization, when all transform coefficients for entropy coding are close to zero, only neighbor block selection information is transmitted without transmitting residual signals. By utilizing merge skip mode, relatively high coding efficiency can be achieved in images with minimal motion, still images, and screen content images.

[0053] Hereinafter, merge mode and merge skip mode will be collectively referred to as merge / skip mode.

[0054] Another method for encoding motion information is the AMVP (Advanced Motion Vector Prediction) mode.

[0055] In AMVP mode, the inter-prediction unit (124) derives predicted motion vector candidates for the motion vector of the current block using the surrounding blocks of the current block. As surrounding blocks used to derive predicted motion vector candidates, all or part of the left block (A0), bottom-left block (A1), top block (B0), top-right block (B1), and top-left block (B2) adjacent to the current block within the current picture shown in FIG. 4 may be used. Additionally, blocks located within a reference picture (which may be the same as or different from the reference picture used to predict the current block) other than the current picture where the current block is located may be used as surrounding blocks to derive predicted motion vector candidates. For example, blocks located at the same position as the current block within the reference picture (co-located blocks) or blocks adjacent to the blocks at the same position may be used. If the number of motion vector candidates is less than a preset number by the method described above, a 0 vector is added to the motion vector candidates.

[0056] The inter prediction unit (124) derives predicted motion vector candidates using the motion vectors of the surrounding blocks and determines a predicted motion vector for the current block's motion vector using the predicted motion vector candidates. Then, it calculates a difference motion vector by subtracting the predicted motion vector from the current block's motion vector.

[0057] Predicted motion vectors can be obtained by applying a predefined function (e.g., median, mean operation, etc.) to the predicted motion vector candidates. In this case, the image decoder is also aware of the predefined function. Furthermore, since the surrounding blocks used to derive the predicted motion vector candidates have already been encoded and decoded, the image decoder is also aware of the motion vectors of those surrounding blocks. Therefore, the image decoder does not need to encode information to identify the predicted motion vector candidates. Consequently, in this case, information regarding the difference motion vector and the reference picture used to predict the current block is encoded.

[0058] Meanwhile, the predicted motion vector may be determined by selecting one of the predicted motion vector candidates. In this case, information for identifying the selected predicted motion vector candidate is additionally encoded, along with information about the difference motion vector and information about the reference picture used to predict the current block.

[0059] The subtractor (130) generates a residual block by subtracting the prediction block generated by the intra prediction unit (122) or the inter prediction unit (124) from the current block.

[0060] The conversion unit (140) converts residual signals within a residual block having pixel values ​​in a spatial domain into conversion coefficients in the frequency domain. The conversion unit (140) can convert the residual signals within the residual block using the entire size of the residual block as the conversion unit, or it can divide the residual block into multiple sub-blocks and use the sub-blocks as the conversion unit to perform the conversion. Alternatively, it can divide the residual signals into two sub-blocks, a conversion area and a non-conversion area, and use only the conversion area sub-block as the conversion unit to convert the residual signals. Here, the conversion area sub-block may be one of two rectangular blocks having a size ratio of 1:1 with respect to the horizontal axis (or vertical axis). In this case, a flag (cu_sbt_flag) indicating that only the sub-block has been converted, direction (vertical / horizontal) information (cu_sbt_horizontal_flag) and / or position information (cu_sbt_pos_flag) are encoded by the entropy encoding unit (155) and signaled to the image decoding device. Additionally, the size of the converted area sub-block may have a size ratio of 1:3 with respect to the horizontal axis (or vertical axis), and in this case, a flag (cu_sbt_quad_flag) distinguishing the corresponding division is additionally encoded by the entropy encoding unit (155) and signaled to the image decoding device.

[0061] Meanwhile, the transformation unit (140) can perform transformations on the residual block individually in the horizontal and vertical directions. For the transformation, various types of transformation functions or transformation matrices may be used. For example, a pair of transformation functions for horizontal transformation and vertical transformation can be defined as a Multiple Transform Set (MTS). The transformation unit (140) can select one pair of transformation functions with the best transformation efficiency among the MTS and transform the residual block in the horizontal and vertical directions, respectively. Information (mts_idx) regarding the selected pair of transformation functions among the MTS is encoded by the entropy encoding unit (155) and signaled to the image decoder.

[0062] The quantization unit (145) quantizes the transformation coefficients output from the transformation unit (140) using quantization parameters and outputs the quantized transformation coefficients to the entropy encoding unit (155). The quantization unit (145) may quantize the associated residual block directly without transformation for any block or frame. The quantization unit (145) may apply different quantization coefficients (scaling values) depending on the position of the transformation coefficients within the transformation block. The quantization matrix applied to the quantized transformation coefficients arranged in two dimensions can be encoded and signaled to an image decoder.

[0063] The reordering unit (150) can perform reordering of coefficient values ​​for quantized residual values.

[0064] The reordering unit (150) can convert a two-dimensional coefficient array into a one-dimensional coefficient sequence using coefficient scanning. For example, the reordering unit (150) can output a one-dimensional coefficient sequence by scanning from DC coefficients to coefficients in the high-frequency range using a zig-zag scan or a diagonal scan. Depending on the size of the conversion unit and the intra-prediction mode, a vertical scan that scans the two-dimensional coefficient array in the column direction and a horizontal scan that scans the two-dimensional block-shaped coefficients in the row direction may be used instead of a zig-zag scan. That is, depending on the size of the conversion unit and the intra-prediction mode, the scanning method to be used among a zig-zag scan, a diagonal scan, a vertical scan, and a horizontal scan may be determined.

[0065] The entropy encoding unit (155) generates a bitstream by encoding a sequence of one-dimensional quantized transformation coefficients output from the reordering unit (150) using various encoding methods such as CABAC (Context-based Adaptive Binary Arithmetic Code) and Exponential Golomb.

[0066] Additionally, the entropy encoding unit (155) encodes information related to block division, such as CTU size, CU division flag, QT division flag, MTT division type, and MTT division direction, so that the video decoder can divide the block in the same way as the video encoding unit. Additionally, the entropy encoding unit (155) encodes information regarding a prediction type indicating whether the current block is encoded by intra prediction or by inter prediction, and encodes intra prediction information (i.e., information regarding the intra prediction mode) or inter prediction information (information regarding the encoding mode of motion information (merge mode or AMVP mode), the merge index in the case of merge mode, and the reference picture index and difference motion vector in the case of AMVP mode) according to the prediction type. Additionally, the entropy encoding unit (155) encodes information related to quantization, i.e., information regarding quantization parameters and information regarding the quantization matrix.

[0067] The inverse quantization unit (160) inversely quantizes the quantized transformation coefficients output from the quantization unit (145) to generate transformation coefficients. The inverse transformation unit (165) converts the transformation coefficients output from the inverse quantization unit (160) from the frequency domain to the spatial domain to restore the residual block.

[0068] The adder (170) restores the current block by adding the restored residual block and the prediction block generated by the prediction unit (120). The pixels within the restored current block are used as reference pixels when intra-predicting the next block in sequence.

[0069] The loop filter section (180) performs filtering on the restored pixels to reduce blocking artifacts, ringing artifacts, blurring artifacts, etc. caused by block-based prediction and transformation / quantization. The loop filter section (180) may include all or part of a deblocking filter (182), a SAO (Sample Adaptive Offset) filter (184), and an ALF (Adaptive Loop Filter, 186) as an in-loop filter.

[0070] The deblocking filter (182) filters the boundaries between restored blocks to remove blocking artifacts caused by block-unit encoding / decoding, and the SAO filter (184) and ALF (186) perform additional filtering on the deblocking filtered image. The SAO filter (184) and ALF (186) are filters used to compensate for the difference between restored pixels and original pixels caused by lossy coding. The SAO filter (184) improves not only subjective image quality but also encoding efficiency by applying an offset in CTU units. In contrast, the ALF (186) performs block-unit filtering, and compensates for distortion by applying different filters by distinguishing the degree of edge and change of the corresponding block. Information regarding the filter coefficients to be used in the ALF can be encoded and signaled to an image decoder.

[0071] The restored blocks filtered through the deblocking filter (182), SAO filter (184), and ALF (186) are stored in memory (190). Once all blocks within a picture are restored, the restored picture can be used as a reference picture for inter-predicting blocks within a picture to be encoded later.

[0072] The video encoding device can store the bitstream of encoded video data on a non-transient recording medium or transmit it to a video decoding device using a communication network.

[0073] FIG. 5 is an exemplary block diagram of an image decoding device capable of implementing the technologies of the present disclosure. Hereinafter, the image decoding device and its sub-components will be described with reference to FIG. 5.

[0074] The image decoding device may be configured to include an entropy decoding unit (510), a reordering unit (515), an inverse quantization unit (520), an inverse transformation unit (530), a prediction unit (540), an adder (550), a loop filter unit (560), and a memory (570).

[0075] Similar to the image encoding device of FIG. 1, each component of the image decoding device may be implemented in hardware or software, or in combination of hardware and software. Additionally, the function of each component may be implemented in software, and a microprocessor may be implemented to execute the function of the software corresponding to each component.

[0076] The entropy decoding unit (510) determines the current block to be decoded by decoding the bitstream generated by the video encoding device and extracting information related to block division, and extracts prediction information, information on residual signals, etc., necessary to restore the current block.

[0077] The entropy decoding unit (510) extracts information about the CTU size from the SPS (Sequence Parameter Set) or PPS (Picture Parameter Set) to determine the size of the CTU and divides the picture into CTUs of the determined size. Then, the CTU is determined as the top layer of the tree structure, i.e., the root node, and divides the CTU using the tree structure by extracting division information for the CTU.

[0078] For example, when splitting a CTU using a QTBTTT structure, first, a first flag (QT_split_flag) related to QT splitting is extracted to split each node into four nodes of the lower layer. Then, for the nodes corresponding to the leaf nodes of QT, a second flag (mtt_split_flag) related to MTT splitting and splitting direction (vertical / horizontal) and / or splitting type (binary / ternary) information are extracted to split the corresponding leaf nodes into an MTT structure. Accordingly, each node below the leaf nodes of QT is recursively split into a BT or TT structure.

[0079] As another example, when splitting a CTU using the QTBTTT structure, a CU splitting flag (split_cu_flag) indicating whether to split the CU is first extracted, and if the block is split, a first flag (QT_split_flag) is extracted. During the splitting process, each node may undergo zero or more iterative MTT splittings after zero or more iterative QT splittings. For example, the CTU may undergo MTT splitting immediately, or conversely, only multiple QT splittings may occur.

[0080] As another example, when splitting a CTU using a QTBT structure, a first flag (QT_split_flag) related to the splitting of QT is extracted to split each node into four nodes of the lower layer. Then, for the nodes corresponding to the leaf nodes of QT, a split flag (split_flag) indicating whether to further split into BTs and split direction information are extracted.

[0081] Meanwhile, when the entropy decoding unit (510) determines the current block to be decoded using the division of the tree structure, it extracts information regarding the prediction type indicating whether the current block is intra-predicted or inter-predicted. If the prediction type information indicates intra-predicted, the entropy decoding unit (510) extracts syntax elements for the intra-predicted information (intra-predicted mode) of the current block. If the prediction type information indicates inter-predicted, the entropy decoding unit (510) extracts syntax elements for the inter-predicted information, namely information indicating the motion vector and the reference picture that the motion vector refers to.

[0082] Additionally, the entropy decoding unit (510) extracts information regarding quantization-related information and information regarding residual signals, as well as information regarding the quantized transformation coefficients of the current block.

[0083] The reordering unit (515) can change the sequence of one-dimensional quantized transformation coefficients entropy-decoded in the entropy decoding unit (510) back into a two-dimensional coefficient array (i.e., block) in the reverse order of the coefficient scanning order performed by the image encoding device.

[0084] The inverse quantization unit (520) inversely quantizes the quantized transformation coefficients and inversely quantizes the quantized transformation coefficients using quantization parameters. The inverse quantization unit (520) may apply different quantization coefficients (scaling values) to the quantized transformation coefficients arranged in two dimensions. The inverse quantization unit (520) may perform inverse quantization by applying a matrix of quantization coefficients (scaling values) from an image encoding device to a two-dimensional array of quantized transformation coefficients.

[0085] The inverse transformation unit (530) generates a residual block for the current block by inversely transforming the inversely quantized transformation coefficients from the frequency domain to the spatial domain and restoring the residual signals.

[0086] Additionally, when the inverse transformation unit (530) inversely transforms only a part of the transformation block (sub-block), it extracts a flag (cu_sbt_flag) indicating that only the sub-block of the transformation block has been transformed, information on the directionality (vertical / horizontal) of the sub-block (cu_sbt_horizontal_flag) and / or information on the position of the sub-block (cu_sbt_pos_flag), restores residual signals by inversely transforming the transformation coefficients of the corresponding sub-block from the frequency domain to the spatial domain, and creates a final residual block for the current block by filling the areas that have not been inversely transformed with "0" values ​​of residual signals.

[0087] Additionally, when MTS is applied, the inverse transformation unit (530) determines a transformation function or transformation matrix to be applied in the horizontal and vertical directions, respectively, using MTS information (mts_idx) signaled from the video encoding device, and performs an inverse transformation on the transformation coefficients within the transformation block in the horizontal and vertical directions using the determined transformation function.

[0088] The prediction unit (540) may include an intra prediction unit (542) and an inter prediction unit (544). The intra prediction unit (542) is activated when the prediction type of the current block is an intra prediction, and the inter prediction unit (544) is activated when the prediction type of the current block is an inter prediction.

[0089] The intra prediction unit (542) determines the intra prediction mode of the current block among a plurality of intra prediction modes from the syntax elements for the intra prediction mode extracted from the entropy decoding unit (510), and predicts the current block using reference pixels around the current block according to the intra prediction mode.

[0090] The inter prediction unit (544) determines the motion vector of the current block and the reference picture that the motion vector refers to using the syntax elements for the inter prediction mode extracted from the entropy decoding unit (510), and predicts the current block using the motion vector and the reference picture.

[0091] The adder (550) restores the current block by adding the residual block output from the inverse transformation unit (530) and the prediction block output from the inter prediction unit (544) or the intra prediction unit (542). The pixels within the restored current block are used as reference pixels when intra-predicting the block to be decoded later.

[0092] The loop filter section (560) may include a deblocking filter (562), an SAO filter (564), and an ALF (566) as an in-loop filter. The deblocking filter (562) deblocks the boundaries between restored blocks to remove blocking artifacts caused by block-unit decoding. The SAO filter (564) and the ALF (566) perform additional filtering on the restored blocks after deblocking filtering to compensate for the difference between the restored pixels and the original pixels caused by lossy coding. The filter coefficients of the ALF are determined using information about the filter coefficients decoded from the bitstream.

[0093] The restored blocks filtered through the deblocking filter (562), SAO filter (564), and ALF (566) are stored in memory (570). When all blocks within a picture are restored, the restored picture is used as a reference picture to inter-predict blocks within the picture to be encoded later.

[0094] The present embodiment relates to the encoding and decoding of an image (video) as described above. More specifically, the present invention provides an image encoding / decoding method and apparatus that derives a weighted sum coefficient based on a reconstructed region around a current block and a plurality of reference blocks, and generates a final prediction block of a current block by weighting a plurality of prediction blocks based on the derived weighted sum coefficient, and a recording medium that stores a bitstream generated by the image encoding method / apparatus.

[0095] The following embodiments may be performed by a prediction unit (120) within a video encoding apparatus. Additionally, the following embodiments may be performed by a prediction unit (540) within a video decoding apparatus.

[0096] The video encoding device can generate signaling information related to the present embodiment in terms of rate distortion optimization during the encoding of the current block. The video encoding device can encode the signaling information using the entropy encoding unit (155) and then transmit it to the video decoder. The video decoder can decode the signaling information related to the decoding of the current block from the bitstream using the entropy decoder (510).

[0097] In the following description, the term 'target block' may be used interchangeably with 'current block' or 'Coding Unit (CU).' Alternatively, 'target block' may refer to a specific area of ​​a Coding Unit.

[0098] Also, a value of one flag being true indicates that the flag is set to 1. Also, a value of one flag being false indicates that the flag is set to 0.

[0099] The decoder-side includes all or part of an inverse quantizer (160), an inverse transform (165), a prediction unit (120), an adder (170), a loop filter (180), and a memory (190) in the image encoding device illustrated in FIG. 1. Alternatively, the decoder-side includes all or part of an inverse quantizer (520), an inverse transform (530), a prediction unit (540), an adder (550), a loop filter (560), and a memory (570) in the image decoding device illustrated in FIG. 5. In relation to a series of decoding processes, the decoder-side of the image encoding device and the decoder-side of the image decoding device perform the same operation. The image encoding device determines information related to the operation of the decoder-side and signals the determined information to the image decoding device. The image decoding device can decode the signaled information and operate the decoder-side based on the decoded information.

[0100] I. Regression-based GPM (Geometric Partitioning Mode)

[0101] Figure 6 is an example diagram schematically illustrating regression-based GPM.

[0102] In the next-generation VVC technology, the Enhanced Compression Model (ECM), the regression-based GPM generates two prediction blocks for the current block as shown in Fig. 6, derives a weighted sum matrix based on the restored area around the current block and the restored area around the two reference blocks, and generates the final prediction block of the current block by weighting the two prediction blocks using the derived weighted sum matrix.

[0103] The regression-based GPM technique constructs an inter-prediction candidate list for the current block, combines unidirectional prediction candidates from the constructed candidate list to form a pair candidate list, and determines two prediction blocks by signaling / parsing a single index for the pair candidate list. The pair candidate list can be reordered based on template matching between adjacent regions of the current block and the reference blocks. As the original reconstructed regions for deriving the weighted sum matrix, one top line and one left line adjacent to the current block and the two reference blocks are used. An affine linear function of the form ax+by+c is derived based on the original reconstructed regions around the current block and the two reference blocks, and a weight matrix can be calculated based on the derived affine linear function. In this case, x and y in the affine linear function represent pixel positions within the current block relative to the top-left pixel position (0,0) of the current block. a, b, and c are parameters derived based on an optimization method utilizing the aforementioned adjacent regions.

[0104] Regression-based SGPM (Spatial GPM) is a technology that extends SGPM, an intra-prediction technology, to apply the aforementioned regression-based GPM to two intra-prediction blocks. Regression-based SGPM adds four prediction candidates for regression-based SGPM to the prediction candidate list of the existing SGPM and signals / parses the corresponding list index. The prediction candidates for regression-based SGPM are constructed by combining two of the intra-prediction modes included in the MPM list and the block vectors of the neighboring blocks of the current block.

[0105] The following embodiments are described with reference to an image decoding device, but may be implemented identically or similarly in an image encoding device. Alternatively, the following embodiments are described with reference to the decoder side of the image decoding device, but may be implemented identically or similarly in the decoder side of an image encoding device.

[0106] II. Embodiments according to the present disclosure

[0107] FIG. 7 is a block diagram showing in detail a part of an image decoding device according to one embodiment of the present disclosure.

[0108] Hereinafter, "current block" and "current coding block" have the same meaning and may be used interchangeably. "current prediction block" and "current prediction unit block" have the same meaning and may be used interchangeably.

[0109] In this specification, the term 'and / or' includes a combination of a plurality of described items or any of a plurality of described items.

[0110] In this specification, weighted sum coefficients, weighted sum matrix, and blending matrix are used interchangeably.

[0111] The image decoder according to the present embodiment determines a prediction and conversion unit, and for a current block corresponding to the determined unit, performs prediction and inverse conversion using a determined prediction technique and prediction mode, thereby finally generating a restored block of the current block. As exemplified in FIG. 7, this can be performed by the inverse conversion unit (530), prediction unit (540), and adder (550) of the image decoder. Meanwhile, the same operations as exemplified in FIG. 7 can be performed by the inverse conversion unit (165), picture splitting unit (110), prediction unit (120), and adder (170) of the image encoding device. At this time, the image decoder uses encoding information parsed from a bitstream, but the image encoding device may use encoding information set from a higher level in terms of minimizing rate distortion. Hereinafter, for convenience, the present embodiment is described with the image decoder as the focus.

[0112] As shown in the example of FIG. 5, the prediction unit (540) includes an intra prediction unit (542) and an inter prediction unit (544) according to the prediction technology, but as shown in FIG. 7, the prediction unit (540) may include all or part of a prediction unit determination unit (602), a prediction technology determination unit (604), a prediction mode determination unit (606), and a prediction execution unit (608).

[0113] The prediction unit determination unit (602) determines a prediction unit (PU). The prediction technology determination unit (604) determines a prediction technology (e.g., intra prediction, inter prediction, or IBC (Intra Block Copy) mode, palette mode, etc.) for the prediction unit. The prediction mode determination unit (606) determines a detailed prediction mode for the prediction technology. The prediction execution unit (608) generates a prediction block of the current block according to the determined prediction mode.

[0114] The inverse transformation unit (530) includes all or part of the inverse transformation unit determination unit (610), the inverse transformation kernel determination unit (612), and the inverse transformation execution unit (614). The inverse transformation unit determination unit (610) determines a transformation unit for the inverse quantization signals (i.e., inverse quantization transformation coefficients) of the current block, and for the determined transformation unit, can correct the prediction samples of the transformation unit prior to performing the inverse transformation. The inverse transformation kernel determination unit (612) determines an inverse transformation kernel, and the inverse transformation execution unit (614) generates residual samples by inversely transforming the transformation unit expressed by the inverse quantization transformation coefficients.

[0115] Hereinafter, the transform unit (TU) in terms of encoding can be used compatiblely with the transform block. The inverse transform unit and inverse transform block in terms of decoding correspond to the TU and the transform block, respectively. Therefore, the inverse transform unit and the inverse transform block can be used compatiblely with the TU and the transform block.

[0116] A transformation unit represents a unit that determines whether to perform an inverse transformation in relation to transformation coefficients and transmits information regarding the inverse transformation. An image decoder determines a kernel to be applied as the transformation unit and performs an inverse transformation of the transformation coefficients based on the determined kernel. At least one kernel may be determined for a single transformation unit. According to an embodiment, an N-order inverse transformation may be applied, and the size of the transformation unit and the inverse transformation kernel may not be the same for each order. For example, when an N-order inverse transformation is applied to a current transformation unit block (hereinafter referred to as the transformation block or current transformation block), the size of the kernel used for the inverse transformation of each order may not be the same. That is, the image decoder may apply an inverse transformation to some of the input transformation coefficients.

[0117] The adder (550) adds the prediction block and the residual samples to generate a recovery block. The recovery block is stored in memory and can subsequently be used to predict other blocks.

[0118] The aforementioned prediction and restoration process can be performed independently for each color channel of the input image. If the color format of the input video is a YUV format (YUV420, YUV411, YUV422, YUV444, etc.), the image decoder can perform prediction and restoration of the chroma component after performing prediction and restoration of the luminance component. That is, the luminance component and the chroma component can be restored sequentially by the components exemplified in FIG. 7. Alternatively, if the color format is a YUV format, the image decoder can restore the luminance component and the chroma component separately. Meanwhile, if the color format of the input video is RGB, the image encoder can perform a color format conversion from RGB to YUV and then encode the converted video. Here, in the case of a YUV format, the color format represents the correspondence between the pixels of the luminance component and the pixels of the chroma component.

[0119] In relation to the inverse transformation, the entropy decoder (510) decodes the transformation coefficients. When a second transformation is applied, the entropy decoder (510) decodes the quantized second transformation coefficients. When a second transformation is not applied, the entropy decoder (510) decodes the quantized first transformation coefficients. The entropy decoder (510) parses information such as the quantization method and quantization parameter information.

[0120] The inverse quantization unit (520) generates inverse quantization conversion coefficients by inversely quantizing the decoded quantization conversion coefficients based on information such as the quantization method and quantization parameter information.

[0121] The inverse transform unit (530) inversely transforms the TU expressed in inverse quantization transform coefficients to generate residual samples. As shown in FIG. 7, the inverse transform unit (530) may include an inverse transform unit determination unit (610), an inverse transform kernel determination unit (612), and an inverse transform execution unit (614).

[0122] The inverse transformation unit determination unit (610) can determine one TU or a sub-block formed by dividing one TU into multiple parts as the target of transformation. For example, the TU may be the entire current block that is the target of prediction, or a part of the current block.

[0123] The inverse transformation kernel determination unit (612) can determine a separable vertical and horizontal direction first-order inverse transformation kernel and / or an inseparable second-order inverse transformation kernel, or determine an inseparable first-order inverse transformation kernel.

[0124] The inverse transformation execution unit (614) can inverse transform the inverse quantized transformation coefficients using the inverse transformation kernel determined by the inverse transformation kernel determination unit (612). The inverse transformation execution unit (614) can perform inseparable first-order inverse transformation, inseparable second-order inverse transformation, first-order inverse transformation, etc.

[0125] As an example, whether to perform an inseparable first-order inverse transformation and whether to perform an inseparable second-order inverse transformation can be determined using signaling / parsing, and can be implicitly determined based on the size of the current transformation block, etc. According to an embodiment, the operation of the inverse transformation execution unit (614) may be omitted.

[0126] The prediction unit determination unit (602) determines the size and shape of the block to be predicted, and may use all direct or indirect information transmitted from the image encoding device. The prediction unit determination unit (602) may use direct information related to the size and shape of the current block, and may utilize information that can influence the determination of the size and shape of the current block, such as the number of divisions, depth, shape of division, direction of division, size information related to the minimum divided block, and division information / prediction mode of the encoded surrounding block.

[0127] The prediction unit determined by the prediction unit determination unit (602) may be a current block, one of the sub-blocks into which the current block is divided, a set of pixels, or a single pixel. The prediction unit may include size information and shape information for performing prediction of chroma components and luminance components.

[0128] The prediction unit can be determined dependently or independently for the chroma and luminance components. Dependent determination means that the prediction units of the luminance or chroma components are not determined for each component individually; rather, once the prediction unit of one component is determined, the units of other components are determined in corresponding sizes and shapes. In this case, one component may correspond to one or more of the luminance and chroma components. That is, the luminance component may be determined based on the information of the chroma components. As an example, the prediction unit of the chroma component may be a size corresponding to the prediction unit of the luminance component according to the color format. When determined dependently, information regarding the prediction unit of the component determined dependently—that is, other components corresponding to one component—may be omitted. Independent determination means that the prediction units of the luminance and chroma components are determined separately. When determined independently, information regarding the prediction units for each component may be signaled separately.

[0129] The prediction technology determination unit (604) determines the prediction technology for the prediction unit. As described above, the prediction technology may be one of inter-prediction, intra-prediction, IBC mode, and palette mode. At this time, the prediction technology for the chroma component may be determined in the same way as the prediction technology for the corresponding luma component without signaling and parsing separate information. The aforementioned prediction technology may be determined by signaling / parsing a plurality of flags and / or parameters.

[0130] As an example, the prediction technique may include regression-based weighted sum prediction. Here, the regression-based weighted sum prediction derives weighted sum coefficients based on the restored regions around the current block and multiple reference blocks, and generates the final prediction block of the current block by weighting multiple prediction blocks based on the derived weighted sum coefficients.

[0131] The prediction mode determination unit (606) determines a detailed prediction mode of the current prediction unit block in relation to the prediction technology. The prediction mode may represent a method for determining pixel values ​​of the prediction block, such as the prediction direction of intra prediction, the number of reference pictures in inter prediction, a mode for transmitting motion vectors in inter prediction, an affine mode, GPM, etc.

[0132] For example, when the prediction technique of the current block is a palette mode, the image decoder constructs a color palette table containing multiple color information and a color index map containing indices of the color palette table in relation to the pixels of the current block. The image decoder can generate a prediction block of the current block by mapping the aforementioned indices to the color information of the color palette table. The image decoder can construct a color palette table based on the utilization of color information included in the color palette tables of blocks restored according to the previous palette mode and / or the parsing of color information. Each index of the color index map may be determined by directly parsing the index value, determined by copying the index value of an adjacent pixel, or derived based on the index value of an adjacent pixel.

[0133] As another example, when the prediction technique for the current block is in IBC mode, the image decoder may construct a block vector information list, determine block vector information by parsing an index for the block vector information list, and generate a predicted block of the current block based on the determined block vector information. An index for the block vector information list may be signaled from the image encoding device. A block vector represents the displacement between the current block and a region previously restored prior to the current block in the current picture. As an example, the image decoder may generate a final block vector by adding a Block Vector Predictor (BVP) determined from the block vector information list and a Block Vector Difference (BVD) signaled from the image encoding device. A predicted block of the current block may be generated based on the final block vector.

[0134] If the prediction technique of the current block is intra prediction, the prediction mode of the current block is directional prediction mode. Planar mode (Horizontal Planar, Vertical Planar, or Regular Planar). It may be a mode for generating a prediction block of the current block based on at least one of the following modes: DC mode, EIP (Extrapolation filter-based Intra Prediction) prediction mode, matrix-based prediction mode (e.g., MIP), template matching-based prediction mode (IntraTMP (Intra Template Matching Prediction)), spatial geometric partitioning mode (SGPM), or prediction mode based on inter-component correlation (e.g., CCLM (Cross component linear model), MMLM (Multi model CCLM), CCCM (Convolutional cross component model), LBCCP (Local boosting cross component prediction), BVG-CCCM (Block vector guided CCCM), CCLM with slope adjustment, Multi model CCCM, GLCCCM (Gradient and location based CCCM), CCP-merge, GLM (Gradient linear model), DDCCP (Decoder derived cross component prediction), etc.).

[0135] For example, if the prediction technique of the current block is intra prediction and the prediction mode is directional prediction mode, planar mode, or DC mode, the image decoder can generate a prediction block of the current block using prediction mode information and previously restored reference samples around the current block. Here, the prediction mode information can be determined by parsing angle information of the prediction direction. The angle information can be signaled from the image encoder. Alternatively, the image decoder can determine the prediction mode information by combining the prediction mode information of the current block and neighboring blocks to construct a prediction mode candidate list and parsing the index information of the prediction mode candidate list. The index information can be signaled from the image encoder.

[0136] For example, if the prediction technique of the current block is intra prediction, the prediction mode of the current block may be a DIMD (Decoder-side Intra Mode Derivation) mode. The DIMD mode implicitly derives a prediction mode based on reconstructed samples within a pre-reconstructed region surrounding the current block, generates prediction blocks of the current block based on at least one derived mode, and generates a final prediction block of the current block using a weighted sum of the prediction blocks. For example, the DIMD mode may apply a boundary filter to the reconstructed samples within the pre-reconstructed region to calculate directional occurrence information of surrounding pixels, and derive a prediction mode based on directional information having a high value.

[0137] As another example, if the prediction technique of the current block is intra prediction, the prediction mode of the current block may be OBIC (Occurrence-based Intra Coding) mode. The OBIC mode implicitly derives a prediction mode based on the prediction modes of blocks within a previously restored region surrounding the current block, generates prediction blocks of the current block based on at least one derived mode, and generates a final prediction block of the current block using a weighted sum of the prediction blocks. As an example, the OBIC mode can accumulate prediction mode information of restored samples within a previously restored region to calculate the frequency of occurrence for each prediction mode, and derive a prediction mode based on prediction modes having a high frequency of occurrence.

[0138] As another example, if the prediction technique of the current block is intra prediction, the prediction mode of the current block may be a Template-based Intra Mode Derivation (TIMD) mode. The TIMD mode implicitly derives a prediction mode based on restored samples within a restored region around the current block, generates prediction blocks of the current block based on at least one derived mode, and generates a final prediction block of the current block using a weighted sum of the prediction blocks. As an example, the TIMD mode may generate predicted values ​​of the restored region by predicting the restored region around the current block according to a plurality of intra prediction modes, calculate the error (or cost) between the sample values ​​of the restored region and the predicted values, and derive a prediction mode based on prediction modes having a small error.

[0139] As described above, when prediction mode information is derived and determined, multiple prediction modes may be determined. Additionally, the image decoder may generate prediction blocks according to multiple prediction modes and generate the final prediction block of the current block by weighting the generated prediction blocks.

[0140] As an example, when the current block is predicted according to a directional mode, the image decoder may generate a predicted block by weighting reference pixels existing in both directions based on the prediction direction of the current directional mode. In this case, the weights used for the weighted sum may be determined based on information such as the size of the current block, the prediction mode of the current block, and the location of the predicted pixel.

[0141] As another example, if the current block is predicted in DC mode, the prediction block can be generated by weighting the prediction block created in DC mode with reference pixels located at the top and left. In this case, the weights used for the weighted sum can be determined based on information such as the size of the current block and the location of the predicted pixel.

[0142] For example, if the current block is a chroma block and the prediction technique for the current block is intra prediction, the current block can be predicted according to DM (Direct Mode). DM mode can perform prediction of the current chroma block based on a prediction method applied to the luminance block at a position corresponding to the current chroma block, or a prediction method predefined between the video encoding device and the video decoder.

[0143] For example, if the prediction technique of the current block is intra prediction and the prediction mode is EIP, the current prediction sample of the current block can be generated by applying a filter to the previously recovered reference samples around the current block and the previous prediction samples within the current block. In this case, the filter can be calculated based on the previously recovered region around the current block. Alternatively, a filter candidate list can be constructed using the filters of blocks decoded in a filter-based prediction mode, and the filter can be determined by parsing the index of the filter candidate list. The index of the filter candidate list can be signaled from the video encoding device to the video decoder.

[0144] As an example, if the prediction technique of the current block is intra prediction, the prediction mode of the current block may be a Matrix-based Intra Prediction (MIP) mode. In MIP mode, a prediction block of the current block can be generated based on the multiplication of a matrix with previously restored reference samples around the current block. A matrix is ​​defined according to an agreement between the video encoder and the video decoder, and an index representing the defined matrix can be signaled / parsed. A matrix can be determined by signaling / parsing the matrix values. Alternatively, a matrix can be derived based on the intra prediction information of the current block.

[0145] For example, if the prediction technique of the current block is intra prediction, the prediction mode of the current block may be the Intra Template Matching Prediction (IntraTMP) mode. The IntraTMP mode defines a pre-recovered area surrounding the current block as a template, performs template matching in the pre-recovered surrounding area of ​​the current block to search for the template with the smallest error, and can generate a prediction block based on the searched template. The template may also include areas that are not adjacent to the current block.

[0146] For example, if the prediction technique of the current block is intra prediction and the current block is a chroma block, the prediction mode may be a cross-component prediction mode based on the correlation between components (CCLM (Cross Component Linear Model), CCCM (Convolutional Cross Component Model), etc.). The cross-component prediction mode can derive a linear or non-linear model between the reconstructed chroma samples around the current chroma block and the reconstructed lumina samples around the lumina block corresponding to the current chroma block, and apply the derived model to the corresponding lumina block to generate a prediction block of the current chroma block.

[0147] For example, if the prediction technique of the current block is intra prediction, the prediction mode of the current block may be a Spatial Geometry Partitioning Mode (SGPM). The Spatial Geometry Partitioning Mode can divide the current block into one or more sub-regions according to geometric partitioning, generate a prediction block for each sub-region based on intra prediction or IBC mode, and then generate a prediction block for the current block by weighting the prediction blocks based on a weighted sum matrix. According to an embodiment, the SGPM can construct an integrated candidate list by combining intra prediction modes and block vector candidates for the current block, and determine the prediction mode for each sub-region by signaling / parsing an index for the integrated candidate list. For example, geometric partitioning can be derived based on the prediction mode determined from the integrated candidate list. The weighted sum matrix can be determined according to an agreement between the video encoding device and the video decoder, or by signaling / parsing an index representing weighted sum matrices defined according to the agreement.

[0148] Hereinafter, the intra prediction mode represents one of the detailed prediction modes as described above when the prediction technology of the current block is intra prediction.

[0149] According to an embodiment, the integrated candidate list may include candidates for prediction based on a regression-based weighted sum method. When an index indicating a candidate related to the regression-based weighted sum method is signaled / parsed, a prediction block of the current block may be generated by weighting the prediction blocks for each sub-region according to the regression-based weighted sum method. The regression-based weighted sum method may derive a weighted sum matrix based on the restored region around the current block and the restored region around the reference block of each sub-region, and generate a prediction block of the current block by weighting each prediction block based on the derived weighted sum matrix. For example, whether to apply the regression-based weighted sum method may be determined by signaling / parsing a 1-bit flag.

[0150] For example, if the prediction technique of the current block is inter-prediction, the prediction mode of the current block may be a mode that generates a prediction block of the current block using parsed motion information of the current block or motion compensation based on motion information. Motion compensation refers to a process of deriving a prediction block of the current block using motion information in a previously restored region of the same frame as the frame containing the current block, and / or in a previously restored region of a frame that is temporally earlier or later than the frame containing the current block. If the motion information of the current block includes multiple movements, multiple prediction blocks may be generated using motion compensation based on each movement, and the prediction block of the current block may be generated by weighting the multiple prediction blocks. In this case, one or more of the prediction blocks may include samples of previously restored regions within the same frame as the current block.

[0151] For example, if the prediction technique of the current block is inter-prediction, the image decoder may construct an inter-prediction information list and determine the inter-prediction information by parsing the index of the inter-prediction information list. The index of the inter-prediction information list may be signaled from the image encoding device. Subsequently, the image decoder may generate a prediction block of the current block using motion information based on the inter-prediction information as a predictor. The inter-prediction information list may include one or more of the following: inter-prediction information of an area spatially adjacent to the current block in the current picture; inter-prediction information of an area identical to or adjacent to the location of the current block in the reference picture; inter-prediction information of an area spatially non-adjacent to the current block in the current picture; inter-prediction information of an area previously restored prior to the current block in the current picture; or inter-prediction information constructed by combining the inter-prediction information of the inter-prediction information list. The reference picture may represent a picture that is temporally past or future from the current picture, or a picture generated by temporally interpolating multiple past and future pictures. The inter-prediction information may include information such as reference picture information and motion information.

[0152] As an example, if the prediction technique of the current block is inter-prediction, the prediction mode of the current block can be determined as AMVP (Advanced Motion Vector Prediction) mode or merge mode by signaling / parsing a 1-bit flag.

[0153] The AMVP mode calculates a motion vector by combining the Motion Vector Predictor (MVP) and the Motion Vector Difference (MVD), and generates a prediction block for the current block using the calculated motion vector. Motion information is derived from the aforementioned inter-prediction information list, and the Motion Vector Predictor can be determined by the derived motion information. The Motion Vector Difference is determined by the video encoding device and can be signaled to the video decoder. According to an embodiment, if the current block uses two or more motion information, the Motion Vector Difference for one of the motion information is signaled / parsed, and the parsed Motion Vector Difference can be scaled to derive the Motion Vector Difference related to the remaining motion information.

[0154] The merge mode determines a motion vector based on a motion vector predictor and generates a prediction block for the current block using the determined motion vector. Motion information is derived from the aforementioned inter-prediction information list, and the motion vector predictor can be determined from the derived motion information. As an example, an index for a predefined motion vector difference table is signaled / parsed according to an agreement between a video encoding device and a video decoder, and the motion vector difference indicated by that index is combined with the motion vector predictor to determine the motion vector. The motion vector can be expressed with a precision of fractional pixel units, less than or equal to integer pixel units. Fractional pixel information can be generated by applying an interpolation filter to integer pixels.

[0155] For example, if the prediction technique of the current block is inter-prediction, the prediction mode of the current block may be determined as one of a mixed mode of intra-prediction and inter-prediction, an affine mode, or a Geometric Partitioning Mode (GPM), or a mode using two or more modes.

[0156] Hereinafter, modes based on affine transformations and affine modes are used interchangeably, and predictions based on affine transformations and affine predictions are used interchangeably.

[0157] Meanwhile, if the prediction technology of the current block is a technology that combines intra and inter prediction, the final prediction block of the current block can be generated by weighting the prediction block based on intra prediction and the prediction block based on inter prediction. For example, statistical values ​​can be calculated in a restored area of ​​the current picture, an intra prediction mode can be derived based on the calculated statistical values, and the derived prediction mode can be used. Motion information can be determined based on template matching, which defines the surrounding restored area of ​​the current block and the restored area of ​​the reference picture as templates and utilizes information from some or all pixels of the said template. Alternatively, motion information signaled from a video encoding device may be used.

[0158] For example, if the prediction technique of the current block is inter-prediction and the prediction mode is affine mode, the image decoder can construct a list for the affine mode and determine Control Point Motion Vector (CPMV) information by parsing the index of the list. The index of the list can be signaled from the image encoding device. The image decoder can derive an affine transform model based on the control point motion vector information, calculate motion vectors in sub-block units based on the affine transform model, and generate a prediction block of the current block based on the motion vectors in sub-block units.

[0159] The control point motion vector information may include two or more motion vectors corresponding to the top-left, top-right, and bottom-left vertex positions of the current block. The list for the affine mode may include one or more of the following: control point motion vector information of areas spatially adjacent or non-adjacent to the current block in the current picture; control point motion vector information of areas adjacent or non-adjacent to the position of the current block in the reference picture; control point motion vector information generated by combining motion vectors of areas spatially adjacent or non-adjacent to the current block in the current picture; control point motion vector information generated by combining motion vectors of areas adjacent or non-adjacent to the position of the current block in the reference picture; control point motion vector information generated based on affine transformation parameter information of an area restored prior to the current block in the current picture; or control point motion vector information configured by combining the control point motion vector information of the list for the affine mode.

[0160] For example, if the prediction technique of the current block is inter-prediction, the prediction mode of the current block may be a geometric partitioning mode. In the GPM, the image decoder divides the current block into two or more sub-blocks according to geometric partitioning, generates prediction blocks of the sub-blocks according to one or more motion compensations based on the parsed motion information and prediction mode information of the current block, and generates a final prediction block of the current block by weighting the generated multiple prediction blocks. According to an embodiment, the image decoder may construct an inter-prediction information list for the current block, construct an integration candidate list by combining the inter-prediction information of the list, and determine a prediction mode for each sub-region by parsing an index for the integration candidate list. An index for the integration candidate list may be signaled from the image encoding device. According to an embodiment, when inter-prediction is determined and a geometric partitioning mode is determined, at least one block among the sub-blocks within the current block may be predicted according to inter-prediction, affine mode, etc., or at least one block may be predicted according to the IBC mode. According to an embodiment, at least one of the subblocks can be predicted according to intra prediction.

[0161] According to an embodiment, geometric partitioning can be determined by signaling / parsing information including the distance between the center of the current block and the boundary line, the angle formed between the center of the current block and the boundary line, etc. According to an embodiment, the integrated candidate list may include a geometric partitioning mode in addition to inter-prediction information. The image decoder defines a template reflecting the geometric partitioning boundary in an area adjacent to the current block and the reference block of each candidate, respectively, performs template matching to calculate an error, and can rearrange the integrated candidate list based on the calculated error.

[0162] According to an embodiment, a weighted sum matrix is ​​used to perform a weighted sum of prediction blocks. The weighted sum matrix may be determined according to an agreement between the image encoding device and the image decoder, or the weighted sum matrix may be determined by signaling / parsing an index representing weighted sum matrices defined according to an agreement between the image encoding device and the image decoder.

[0163] Hereinafter, the Inter Prediction Mode represents one of the detailed prediction modes described above when the prediction technique of the current block is Inter Prediction. The IBC Mode and Affine Mode may be processed separately from the Inter Prediction Mode.

[0164] As an example, if the prediction technique of the current block is inter prediction, the prediction mode of the current block can be determined as regression-based weighted sum prediction by signaling / parsing a 1-bit flag. Regression-based weighted sum prediction can generate two or more prediction blocks for the current block using an inter prediction mode, an IBC mode, an intra prediction mode, and / or an affine mode, derive a weighted sum matrix based on the restored area around the current block and the restored area around each reference block, and generate the final prediction block of the current block by weighting the prediction blocks based on the derived weighted sum matrix.

[0165] In this specification, regression-based weighted sum prediction method, regression-based weighted sum prediction, regression-based weighted sum mode, regression-based weighted sum prediction mode, weighted sum prediction method, weighted sum prediction mode, and weighted sum mode are used interchangeably. For example, the regression-based weighted sum mode may be a prediction method that is an extension of the regression-based GPM mode.

[0166] In this specification, regression-based weighted sum prediction calculates a weight matrix defined by a regression equation by weighting multiple prediction blocks of the current block. For example, prediction blocks may be generated according to a mode such as GPM, SGPM, etc. Prediction blocks may be generated according to a mode that combines intra prediction and inter prediction. Alternatively, prediction blocks may be generated according to an affine mode.

[0167] As an example, if the prediction mode of the current block is regression-based weighted sum prediction, the image decoder can determine the prediction mode of each prediction block by parsing multiple flags or information. For example, if regression-based weighted sum prediction is applied to two prediction blocks, the image decoder can determine the prediction mode of each prediction block as follows.

[0168] Hereinafter, two prediction blocks are referred to as the first prediction block and the second prediction block, and the prediction modes of the first prediction block and the second prediction block are referred to as the first prediction mode and the second prediction mode, respectively.

[0169] The video decoder decodes information for determining a first prediction mode. If the decoded information of the first prediction mode indicates an inter prediction, the first prediction mode is determined as an inter prediction mode. If the decoded information of the first prediction mode indicates an intra / IBC prediction, the first prediction mode is determined as an intra prediction / IBC mode. If the decoded information of the first prediction mode does not indicate both an inter prediction and an intra / IBC prediction, the first prediction mode is determined as an affine mode, and the second prediction mode may also be determined as an affine mode.

[0170] The video decoder decodes information for determining a second prediction mode. If the decoded information of the second prediction mode indicates an inter prediction, the second prediction mode is determined as an inter prediction mode. If the decoded information of the second prediction mode does not indicate an inter prediction, the second prediction mode is determined as an affine mode.

[0171] As described above, two prediction modes can be determined while excluding the combination of (Intra Prediction / IBC mode, Intra Prediction / IBC mode).

[0172] As another example, if the prediction mode of the current block is a regression-based weighted sum mode, a table may be defined according to an agreement between the image encoding device and the image decoder. Subsequently, the prediction mode of each prediction block may be determined by signaling / parsing an index for the table. For example, when applying regression-based weighted sum prediction to two prediction blocks, the image decoder may determine the prediction mode for generating each prediction block by parsing an index for the table defined as in Table 1.

[0173]

[0174] Based on Table 1, two prediction modes can be determined while excluding combinations of (Intra Prediction / IBC mode, Intra Prediction / IBC mode). According to an embodiment, the order of the prediction modes defined in the table can be adaptively determined based on the prediction mode information of the area adjacent to the current block.

[0175] As an example, if the prediction technique of the current block is a regression-based weighted sum prediction, the image decoder may generate two or more prediction blocks based on an inter-prediction mode, an IBC mode, an intra-prediction mode, and / or an affine mode, derive a weighted sum matrix based on a restored region around the current block and a restored region around each reference block, and generate a final prediction block of the current block by weighting the prediction blocks based on the derived weighted sum matrix. For example, the final prediction block may be generated according to a mode that combines the IBC mode and the intra-prediction mode. As another example, the prediction mode of each prediction block may be determined based on the prediction mode of the current block and the adjacent block.

[0176] As an example, an image decoder may determine the prediction mode of each prediction block by parsing a number of flags or information. For example, when applying regression-based weighted sum prediction to two prediction blocks, the image decoder may determine the prediction mode of each prediction block as follows.

[0177] The video decoder decodes information for determining a first prediction mode. If the decoded information of the first prediction mode indicates an inter prediction, the first prediction mode is determined as an inter prediction mode. If the decoded information of the first prediction mode indicates an intra / IBC prediction, the first prediction mode is determined as an intra prediction / IBC mode. If the decoded information of the first prediction mode does not indicate both an inter prediction and an intra / IBC prediction, the first prediction mode may be determined as an affine mode.

[0178] The video decoder decodes information for determining a second prediction mode. If the decoded information of the second prediction mode indicates inter prediction, the second prediction mode is determined as an inter prediction mode. If the decoded information of the second prediction mode indicates intra / IBC prediction, the second prediction mode is determined as an intra prediction / IBC mode. If the decoded information of the second prediction mode does not indicate both inter prediction and intra / IBC prediction, the second prediction mode may be determined as an affine mode.

[0179] As another example, when applying regression-based weighted sum prediction to two prediction blocks, the image decoder can determine the prediction mode of each prediction block by parsing information for determining the prediction mode and a flag indicating whether the two prediction blocks are generated according to the same prediction mode as follows.

[0180] The video decoder decodes a flag indicating whether the first prediction mode and the second prediction mode are the same.

[0181] If the aforementioned flag is true, the image decoder decodes information for determining a prediction mode. If the decoded information indicates an inter prediction, the first prediction mode and the second prediction mode are determined as an inter prediction mode. If the decoded information indicates an intra / IBC prediction, the first prediction mode and the second prediction mode are determined as an intra prediction / IBC mode. If the decoded information does not indicate both an inter prediction and an intra / IBC prediction, the first prediction mode and the second prediction mode may be determined as an affine mode.

[0182] On the other hand, if the aforementioned flag is false, the image decoding device decodes information for determining a prediction mode. If the decoded information does not include affine prediction, the first prediction mode and the second prediction mode are determined as the inter prediction mode and the intra prediction / IBC mode, respectively. If the decoded information includes affine prediction but does not include intra / IBC prediction, the first prediction mode and the second prediction mode are determined as the inter prediction mode and the affine mode, respectively. If the decoded information includes affine prediction and intra / IBC prediction, the first prediction mode and the second prediction mode may be determined as the intra prediction / IBC mode and the affine mode, respectively.

[0183] As described above, two prediction modes can be determined, including a combination of (Intra Prediction / IBC mode, Intra Prediction / IBC mode).

[0184] As another example, if the prediction mode of the current block is a regression-based weighted sum mode, a table may be defined according to an agreement between the image encoding device and the image decoder. Subsequently, the prediction mode of each prediction block may be determined by signaling / parsing an index for the table. For example, when applying regression-based weighted sum prediction to two prediction blocks, the image decoder may determine the prediction mode for generating each prediction block by parsing an index for the table defined as in Table 2.

[0185]

[0186] Based on Table 2, two prediction modes can be determined, including combinations of (Intra Prediction / IBC mode, Intra Prediction / IBC mode). According to an embodiment, the order of the prediction modes defined in the table can be adaptively determined based on the prediction mode information of the region adjacent to the current block.

[0187] The prediction execution unit (608) generates the final prediction block of the current block according to the determined prediction technology and prediction mode.

[0188] As an example, the prediction execution unit (608) performs a prediction according to the prediction mode determined as described above to generate a prediction block of the current block, and the adder (550) generates a restoration block by adding the prediction block of the current block and the residual samples (i.e., residual blocks).

[0189] Hereinafter, a process for an image decoder to generate a final prediction block of a current block is described, where the prediction technique and / or prediction mode of the current block is a regression-based weighted sum prediction.

[0190] The video decoder parses prediction mode information of multiple prediction blocks to predict the current block. For example, multiple prediction blocks may be generated based on prediction mode information related to an inter-prediction mode, an IBC mode, an intra-prediction mode, and / or an affine mode. Each prediction block may be generated according to the same prediction mode or according to a different prediction mode. The video decoder may determine each prediction mode by parsing multiple flags and / or information. Multiple flags and / or information may be transmitted from the video encoder. Each prediction mode may be determined by signaling / parsing an index for a table defined according to an agreement between the video encoder and the video decoder. Alternatively, the video decoder may derive each prediction mode based on prediction mode information of a region adjacent to the current block.

[0191] The image decoder may construct a list of prediction information for each determined prediction mode and construct an integrated candidate list by combining the information of the constructed lists. The prediction information list includes candidates for each prediction mode, and the integrated candidate list includes candidates for the prediction modes. For example, when generating N prediction blocks, each candidate in the integrated candidate list may include prediction information for generating N prediction blocks. Here, N is a positive integer satisfying N≥2. The image decoder may define regions adjacent to the current block and regions adjacent to the reference block of each candidate as templates, perform template matching based on the templates to calculate an error, and rearrange the integrated candidate list based on the calculated error. As the error, at least one of loss functions such as SAD (Sum of Absolute Differences), SATD (Sum of Absolute Transformed Differences), MAE (Mean Absolute Error), MSE (Mean Squared Error), etc., may be used.

[0192] As another example, the integrated candidate list may include a geometric partitioning mode in addition to prediction information. The image decoder defines templates that reflect geometric partitioning boundaries in the region adjacent to the current block and the region adjacent to the reference block of each candidate, performs template matching based on the templates to calculate an error, and can rearrange the candidates within the integrated candidate list based on the calculated error.

[0193] For example, if the prediction mode for generating N (N≥2) prediction blocks is determined to be an inter-prediction mode, the image decoder constructs an inter-prediction information list for the current block and constructs an integrated candidate list by combining the unidirectional prediction candidates of the constructed inter-prediction information list. The image decoder can generate prediction blocks by parsing one index of the integrated candidate list. According to an embodiment, the bidirectional prediction candidates of the inter-prediction information list may be used as they are as prediction candidates for the integrated candidate list, or the integrated candidate list may be constructed by combining a portion of the information related to the bidirectional prediction candidates with the unidirectional prediction candidates.

[0194] The inter prediction information list may be constructed based on inter prediction information of areas spatially adjacent to the current block in the current picture, inter prediction information of areas identical to or adjacent to the location of the current block in the reference picture, inter prediction information of areas spatially non-adjacent to the current block in the current picture, inter prediction information of areas previously restored prior to the current block in the current picture, and / or inter prediction information constructed by combining the inter prediction information of the inter prediction information list. The reference picture may be a picture that is temporally past or future than the current picture, or a picture generated by temporally interpolating one or more past and / or future pictures. The inter prediction information may include information such as reference picture information and motion information.

[0195] As another example, if the prediction modes for generating N (N≥2) prediction blocks are all determined to be intra prediction / IBC modes, the image decoder constructs an integrated candidate list by combining intra prediction modes for the current block and block vector information. The image decoder can generate prediction blocks by parsing one index of the integrated candidate list. According to an embodiment, the integrated candidate list may be constructed by combining only intra prediction modes for the current block or by combining only block vector information. The intra prediction modes may include intra prediction modes in the MPM list of the current block and / or intra prediction modes derived based on previously restored regions around the current block. The block vector information may include block vector information of regions spatially adjacent to the current block, block vector information of regions spatially non-adjacent to the current block, and / or block vector information of regions previously restored prior to the current block.

[0196] As another example, if the prediction mode for generating N (N≥2) prediction blocks is determined to be an affine mode, the image decoder constructs an affine mode-based prediction information list in relation to the current block and constructs an integrated candidate list by combining the unidirectional prediction candidates of the constructed prediction information list. The image decoder can generate prediction blocks by parsing a single index for the integrated candidate list. A single index may be signaled from the image encoding device. According to an embodiment, the bidirectional prediction candidates of the affine mode-based prediction information list may be used as they are as prediction candidates for the integrated candidate list, or the integrated candidate list may be constructed by combining a portion of the information related to the bidirectional prediction candidates with the unidirectional prediction candidates.

[0197] The affine mode-based prediction information list may be constructed based on control point motion vector information of areas spatially adjacent or non-adjacent to the current block in the current picture, control point motion vector information of areas adjacent or non-adjacent to the position of the current block in the reference picture, control point motion vector information generated by combining motion vectors of areas spatially adjacent or non-adjacent to the current block in the current picture, control point motion vector information generated by combining motion vectors of areas adjacent or non-adjacent to the position of the current block in the reference picture, control point motion vector information based on affine transformation parameter information of areas restored prior to the current block in the current picture, and / or control point motion vector information constructed by combining control point motion vector information of the affine mode-based prediction information list. The prediction information related to the affine mode may include reference picture information, control point motion vector information including two or more motion vectors corresponding to the top-left, top-right, and bottom-left vertex positions of the current block, etc.

[0198] As another example, when the prediction modes for generating N (N≥2) prediction blocks are determined to be different prediction modes, the image decoder constructs prediction information lists corresponding to the prediction modes and constructs an integrated candidate list by combining the candidates of prediction information within each prediction information list. The image decoder can generate prediction blocks by parsing one index for the integrated candidate list. One index may be signaled from the image encoding device. According to an embodiment, when two prediction blocks are generated in an intra prediction / IBC mode and an affine mode, respectively, the image decoder constructs a prediction information list containing intra prediction mode and block vector information and constructs an affine mode-based prediction information list. The image decoder can construct an integrated candidate list by combining the candidates of prediction information included in each list.

[0199] As another example, when prediction modes for generating N (N≥2) prediction blocks are determined, the image decoder may construct prediction information lists corresponding to the prediction modes, parse N indices for the prediction information lists, and generate multiple prediction blocks based on the parsed N indices. The N indices may be signaled from the image encoding device.

[0200] The image decoding device can determine candidates corresponding to multiple prediction blocks from an integrated candidate list based on an index, and generate multiple prediction blocks for the current block using the prediction mode information of the determined candidates.

[0201] As an example, if the prediction mode of the prediction block is an intra prediction mode, the image decoder can generate the prediction block using previously restored reference samples around the current block based on the intra prediction mode determined from the list.

[0202] As another example, when the prediction mode of the prediction block is an inter-prediction mode, the image decoder can generate the prediction block by performing motion compensation based on motion information determined from a list. In this case, the motion compensation determines a reference block from the signal of the restored area of ​​the current picture and / or the signal of the restored area of ​​the reference picture using the motion information, and generates the prediction block based on the reference block.

[0203] As another example, when the prediction mode of the prediction block is IBC mode, the image decoder can determine a reference block from the signal of the restored area of ​​the current picture based on block vector information determined from a list, and generate a prediction block based on the reference block.

[0204] As another example, when the prediction mode of the prediction block is an affine mode, the image decoder derives an affine transformation model using control point motion vector information determined from a list, and calculates motion vectors in sub-block units based on the affine transformation model. The image decoder can generate reference blocks in sub-block units based on the calculated motion vectors. As the affine transformation model, either a 6-parameter model or a 4-parameter model may be used. When a 6-parameter model is used, the control point motion vector information may include three motion vectors corresponding to the top-left, top-right, and bottom-left vertex positions of the current block, and when a 4-parameter model is used, the control point motion vector information may include two motion vectors corresponding to the top-left and top-right, or top-left and bottom-left vertex positions of the current block. When a 6-parameter model is used, the motion vector of each sub-block is calculated as in Equation 1, and when a 4-parameter model is used, the motion vector of each sub-block can be calculated as in Equation 2.

[0205]

[0206]

[0207] Here, cpmv i represents the motion vector of the i-th control point. W and H represent the width and height of the current block, respectively. (x,y) represents the center coordinates of each sub-block, and mv represents the motion vector of each sub-block.

[0208] The image decoding device can determine reference samples for deriving weighted sum coefficients in the restored region around the current block and a plurality of reference blocks.

[0209] As an example, as shown in FIG. 8, reference samples may include n lines adjacent to the top of the current block in the current picture, m lines adjacent to the left of the current block, and / or an m×n area adjacent to the top-left of the current block. In FIG. 8, w and h represent the width and height of the current block, respectively.

[0210] As another example, when the prediction mode of the prediction block is an inter-prediction mode, as shown in FIG. 9, the reference samples may include n lines adjacent to the top of the reference block in the reference picture, m lines adjacent to the left of the reference block, and / or an m×n area adjacent to the top-left of the reference block.

[0211] As another example, when the prediction mode of the prediction block is IBC mode, as shown in FIG. 10, the reference samples may include n lines adjacent to the top of the reference block in the current picture, m lines adjacent to the left of the reference block, and / or an m×n area adjacent to the top-left of the reference block.

[0212] As another example, when the prediction mode of the prediction block is an intra prediction mode, as shown in FIG. 11, n lines adjacent to the top of the current block in the current picture, m lines adjacent to the left of the current block, and / or an m×n area adjacent to the top-left of the current block may be defined as a reference area. The image decoder generates prediction samples of the reference area by using the intra prediction information of the prediction block to perform intra prediction on the reference area based on the pixel values ​​of the areas adjacent to the reference area. At this time, the reference samples may include prediction samples of the reference area.

[0213] As another example, when the prediction mode of the prediction block is an affine mode, as shown in FIG. 12, the reference samples may include n lines adjacent to the top of each reference block located on the top boundary of the reference blocks in sub-block units in the reference picture, m lines adjacent to the left of each reference block located on the left boundary, and / or an m×n area adjacent to the top left of the reference block located on the top-left boundary.

[0214] As another example, when the prediction mode of the prediction block is an affine mode and the current picture is referenced, as shown in FIG. 13, the reference samples may include n lines adjacent to the top of each reference block located on the top boundary of the reference blocks in sub-block units in the current picture, m lines adjacent to the left of each reference block located on the left boundary of the reference block, and / or an m×n area adjacent to the top left of the reference block located on the top-left boundary of the reference block.

[0215] As another example, when the prediction mode of the prediction block is an affine mode, the reference samples may include an area adjacent to a reference block located on the top boundary among the reference blocks in sub-block units in the reference picture, and / or an area adjacent to a reference block located on the left boundary.

[0216] As an example, as shown in FIG. 14, reference samples may include a w×n area adjacent to the top of a reference block located at the top-left boundary of a reference block in a sub-block unit in a reference picture, an m×h area adjacent to the left of a reference block located at the top-left boundary, and / or an m×n area adjacent to the top-left of a reference block located at the top-left boundary.

[0217] As an example, as shown in FIG. 15, the reference samples may include a w×n area adjacent to the top of the uppermost reference block among the reference blocks in sub-block units in the reference picture, and / or an m×h area adjacent to the left of the leftmost reference block.

[0218] The image decoder can derive weighted sum coefficients using determined reference samples. For example, the image decoder can define weighted sum coefficients according to pixel coordinates as a regression equation having K (where K is a positive integer) parameters, and derive weighted sum coefficients by applying an optimization method based on the determined reference samples. Alternatively, the image decoder can parse K parameters. The K parameters may be signaled from the image encoding device. In this case, the process of determining reference samples in the current block and each reference block may be omitted. According to an embodiment, the image decoder may, among the K parameters, j(j <K) 개의 파라미터를 파싱하여 결정하고, 결정된 참조샘플들을 사용하여 K-j 개의 파라미터를 유도할 수 있다.

[0219] For example, when performing regression-based weighted sum prediction on two prediction blocks, the image decoder can define the weighted sum coefficients according to pixel coordinates as a regression equation with three parameters, as shown in Equation 3. Using reference samples determined from the restored region around the current block and the two prediction blocks, the image decoder can derive the weighted sum coefficients by applying an optimization method to a loss function as shown in Equation 4.

[0220]

[0221]

[0222] Here, x and y represent pixel coordinates, and w0(x,y) and w1(x,y) represent weighted sum coefficients at the pixel coordinates of each prediction block. a, b, and c represent the parameters of the regression equation calculated according to the optimization method. Additionally, r(x,y) represents the pixel value of the reference sample determined in the current block, and r0(x,y) and r1(x,y) represent the pixel values ​​of the reference sample determined in each prediction block. In this case, the coordinates of the top-left pixel in the current block and the two prediction blocks are defined as (0,0), and the relative coordinates determined based on the position of the top-left pixel are represented as pixel coordinates.

[0223] The video decoder can generate the final prediction block of the current block by weighting multiple prediction blocks using derived weighted sum coefficients.

[0224] As an example, when performing regression-based weighted sum prediction on two prediction blocks, the image decoder can generate the final prediction block of the current block by performing a weighted sum on the two prediction blocks as shown in Equations 5 and 6.

[0225]

[0226]

[0227] Here, x and y represent pixel coordinates, and w0(x,y) and w1(x,y) represent weighted sum coefficients at the pixel coordinates of each prediction block. a, b, and c represent the parameters of the regression equation calculated according to the optimization method. Additionally, p(x,y) represents the pixel value of the final prediction block, p0(x,y) and p1(x,y) represent the pixel values ​​of each prediction block, and n represents an integer greater than or equal to 1. In this case, the coordinates of the top-left pixel in the current block and the two prediction blocks are defined as (0,0), and the relative coordinates determined based on the position of the top-left pixel are represented as pixel coordinates.

[0228] Hereinafter, a method for predicting the current block based on regression-based weighted sum prediction using the illustration of FIG. 16 is described. The illustration of FIG. 16 can be performed by an image encoding device and an image decoding device. FIG. 16 is described based on the image encoding device, and if necessary, the operation by the image decoding device is additionally described.

[0229] FIG. 16 is a flowchart illustrating a method for predicting a current block according to one embodiment of the present disclosure.

[0230] The video encoding device acquires a flag indicating whether the current block is predicted according to regression-based weighted sum prediction.

[0231] The video encoding device may obtain the aforementioned flag from a high level. Alternatively, the video encoding device may determine the flag in terms of rate distortion optimization. The video encoding device may encode the flag. The video decoding device may decode a flag from the bitstream indicating whether the current block is predicted according to regression-based weighted sum prediction.

[0232] If the aforementioned flag indicates that the current block is predicted according to regression-based weighted sum prediction, the image encoding device may perform the following steps.

[0233] The video encoding device acquires prediction mode information of a plurality of prediction blocks and determines the prediction modes of the prediction blocks based on the prediction mode information (S1600). Here, the prediction blocks are used for prediction of the current block. The prediction modes may include an inter prediction mode, an intra prediction mode, an affine mode, and / or an IBC mode.

[0234] The video encoding device can obtain the aforementioned prediction mode information from a higher level. Alternatively, the video encoding device can determine the prediction mode information in terms of rate distortion optimization. The video encoding device can encode the prediction mode information. The video decoding device can decode the prediction mode information of a plurality of prediction blocks from the bitstream.

[0235] The video encoding device constructs a list of prediction information for each prediction mode and combines the lists of prediction information for the prediction modes to construct an integrated candidate list (S1602).

[0236] The video encoding device can rearrange the integration candidate list. For example, the video encoding device defines regions adjacent to the current block and regions adjacent to the reference block of each candidate in the integration candidate list as templates. The video encoding device performs template matching based on the templates to calculate an error, and can rearrange the integration candidate list based on the calculated error.

[0237] The video encoding device generates prediction blocks based on the integrated candidate list (S1604).

[0238] The video encoding device can obtain an index for the integration candidate list from a higher level. Alternatively, the video encoding device can determine the index in terms of rate distortion optimization. The video encoding device can encode the index. The video decoder can decode the index for the integration candidate list from the bitstream.

[0239] The video encoding device can determine candidates for prediction blocks from an integrated candidate list based on an index. The video encoding device can generate prediction blocks based on the prediction mode information of the candidates.

[0240] The video encoding device determines reference samples in the surrounding restored area of ​​the current block and in the surrounding restored area of ​​a plurality of reference blocks (S1606).

[0241] Reference samples of the current block can be determined as shown in FIG. 8. For example, the reference samples may include at least one of at least one line adjacent to the top of the current block in the current picture, at least one line adjacent to the left of the current block, or at least one pixel adjacent to the top-left of the current block.

[0242] As an example, when a prediction block is predicted according to an inter-prediction mode, reference samples can be determined as shown in FIG. 9. For instance, the reference samples may include at least one of at least one line adjacent to the top of the reference block corresponding to the prediction block in the reference picture, at least one line adjacent to the left of the reference block, or at least one pixel adjacent to the top-left of the reference block.

[0243] As another example, when a prediction block is predicted according to an intra prediction mode, a reference region is defined as in FIG. 11, and reference samples can be determined based on the reference region. For instance, an image encoding device may define a reference region that includes at least one of at least one line adjacent to the top of the current block, at least one line adjacent to the left of the current block, or at least one pixel adjacent to the top-left corner of the current block in the current picture. The image encoding device may generate prediction samples of the reference region by intra-predicting the reference region based on pixel values ​​of regions adjacent to the reference region using the intra prediction information of the prediction block. At this time, the reference samples may include the prediction samples of the reference region.

[0244] As an example, when a prediction block is predicted according to an affine mode, reference samples can be determined as shown in FIG. 12. For instance, the reference samples may include at least one of the following: at least one line adjacent to the top of each reference block located at the top boundary of the reference blocks in the sub-block unit in the reference picture, at least one line adjacent to the left of each reference block located at the left boundary, or at least one pixel adjacent to the top left of the reference block located at the top left boundary.

[0245] As another example, when a prediction block is predicted according to an affine mode, reference samples can be determined as shown in FIG. 14. For instance, the reference samples may include at least one pixel adjacent to the top of a reference block located at the top-left boundary of a reference block in a reference picture, at least one pixel adjacent to the left of a reference block located at the top-left boundary, or at least one pixel adjacent to the top-left of a reference block located at the top-left boundary.

[0246] The video encoding device calculates weighted sum coefficients of prediction blocks using reference samples (S1608). The weighted sum coefficients are defined by a regression equation based on pixel positions within each prediction block.

[0247] The video encoding device can calculate parameters that constitute the regression equation by utilizing a loss function based on reference samples.

[0248] The video encoding device generates the final prediction block of the current block by weighting the prediction blocks based on the weighted sum coefficients (S1610).

[0249] Subsequently, the video encoding device can generate a residual block by subtracting the final prediction block from the current block. The video encoding device can generate transformation coefficients by applying transformation / quantization to the residual block and encode the generated transformation coefficients.

[0250] The image decoder can decode the quantized transform coefficients of the current block from the bitstream and generate a residual block by applying inverse quantization / inverse transform to the quantized transform coefficients. The image decoder can restore the current block by adding the prediction block and the residual block.

[0251] Although the flowcharts and timing diagrams in this specification describe each process as being executed sequentially, this is merely an illustrative explanation of the technical concept of one embodiment of the present disclosure. In other words, a person skilled in the art to which one embodiment of the present disclosure belongs may modify and adapt the flowcharts and timing diagrams in various ways, such as changing the order described in the flowcharts and timing diagrams or executing one or more of the processes in parallel, without departing from the essential characteristics of one embodiment of the present disclosure; therefore, the flowcharts and timing diagrams are not limited to a chronological order.

[0252] It should be understood that the exemplary embodiments described above may be implemented in many different ways. The functions or methods described in one or more examples may be implemented in hardware, software, firmware, or any combination thereof. It should be understood that the functional components described herein are labeled as "...unit" to particularly emphasize their implementation independence.

[0253] Meanwhile, the various functions or methods described in the present embodiment may be implemented as instructions stored in a non-transient recording medium that can be read and executed by one or more processors. A non-transient recording medium includes, for example, any type of recording device in which data is stored in a form readable by a computer system. For example, a non-transient recording medium includes storage media such as an EPROM (erasable programmable read-only memory), a flash drive, an optical drive, a magnetic hard drive, and a solid-state drive (SSD).

[0254] The above description is merely an illustrative explanation of the technical concept of the present embodiment, and a person skilled in the art to which the present embodiment belongs would be able to make various modifications and variations within the scope of the essential characteristics of the present embodiment. Accordingly, the present embodiments are intended to explain, not limit, the technical concept of the present embodiment, and the scope of the technical concept of the present embodiment is not limited by these embodiments. The scope of protection of the present embodiment shall be interpreted by the claims below, and all technical concepts within an equivalent scope shall be interpreted as being included within the scope of rights of the present embodiment.

[0255]

[0256] CROSS-REFERENCE TO RELATED APPLICATION

[0257] This patent application claims priority to Korean patent application No. 10-2025-0003483 filed on January 9, 2025, the entire contents of which are incorporated into this patent application by reference.

Claims

1. In a video decoding method for restoring the current block, A step of obtaining prediction mode information of a plurality of prediction blocks and determining the prediction modes of the prediction blocks based on the prediction mode information, wherein the prediction blocks are used for the prediction of the current block, and the prediction modes include an inter-prediction mode and an intra-prediction mode; A step of constructing a list of prediction information for each prediction mode, and combining the lists of prediction information for the prediction modes to construct an integrated candidate list; A step of rearranging the above integrated candidate list; and A step of predicting the current block based on a reordered integrated candidate list A method including 2. In Paragraph 1, The method further includes the step of decoding a flag indicating whether the current block is predicted according to a regression-based weighted sum prediction, and A method in which prediction mode information of a plurality of prediction blocks is obtained when the above flag indicates that the current block is predicted according to regression-based weighted sum prediction.

3. In Paragraph 1, A method characterized in that the above prediction modes further include an affine mode and an IBC (Intra Block Copy) mode.

4. In Paragraph 1, The step of rearranging the above integrated candidate list is, A step of defining regions adjacent to the current block and regions adjacent to the reference block of each candidate in the integrated candidate list as templates; A step of calculating an error by performing template matching based on the above templates; and Step of rearranging the above integrated candidate list based on the calculated error A method including 5. In Paragraph 1, The step of predicting the current block above is, A step of generating the prediction blocks based on the above-mentioned rearranged integrated candidate list; A step of determining reference samples in the surrounding restored region of the current block and the surrounding restored region of a plurality of reference blocks; A step of calculating weighted sum coefficients of the prediction blocks using the reference samples above, wherein the weighted sum coefficients are defined by a regression equation based on pixel positions within each prediction block; and A step of predicting the current block by weighting the prediction blocks based on the weighted sum coefficients. A method including 6. In Paragraph 5, The step of generating the above prediction blocks is, A step of decrypting the index for the above integrated candidate list; A step of determining candidates for the prediction blocks from the integrated candidate list based on the above index; and Step of generating the prediction blocks based on the prediction mode information of the above candidates A method including 7. In Paragraph 5, The above reference samples are, A method comprising, in the current picture, at least one of at least one line adjacent to the top of the current block, at least one line adjacent to the left of the current block, or at least one pixel adjacent to the top-left of the current block.

8. In Paragraph 5, When the prediction block is predicted according to the above inter-prediction mode, The above reference samples are, A method comprising, in a reference picture, at least one of at least one line adjacent to the top of the reference block corresponding to the prediction block, at least one line adjacent to the left of the reference block, or at least one pixel adjacent to the top-left of the reference block.

9. In Paragraph 5, When the prediction block is predicted according to the above intra prediction mode, The step of generating the above reference samples is, A step of defining a reference area in the current picture that includes at least one of at least one line adjacent to the top of the current block, at least one line adjacent to the left of the current block, or at least one pixel adjacent to the top-left of the current block; A step of intra-predicting the reference region based on pixel values ​​of a region adjacent to the reference region using intra-prediction information of the prediction block; and Step of determining the predicted samples of the above reference region as the above reference samples A method including 10. In Paragraph 5, If the above prediction modes include an affine mode and the prediction block is predicted according to the above affine mode, The above reference samples are, A method comprising, in a reference picture, at least one of the following: at least one line adjacent to the top of each reference block located at the top boundary of the sub-block unit reference blocks, at least one line adjacent to the left of each reference block located at the left boundary, or at least one pixel adjacent to the top-left of the reference block located at the top-left boundary.

11. In Paragraph 5, The step of calculating the above weighted sum coefficients is, A method for calculating parameters that constitute the regression equation by utilizing a loss function based on the above reference samples.

12. In a video encoding method for encoding a current block, A step of obtaining prediction mode information of a plurality of prediction blocks and determining the prediction modes of the prediction blocks based on the prediction mode information, wherein the prediction blocks are used for the prediction of the current block, and the prediction modes include an inter-prediction mode and an intra-prediction mode; A step of constructing a list of prediction information for each prediction mode, and combining the lists of prediction information for the prediction modes to construct an integrated candidate list; A step of rearranging the above integrated candidate list; and A step of predicting the current block based on a reordered integrated candidate list A method including 13. In Paragraph 12, A step of obtaining a flag indicating whether the current block is predicted according to regression-based weighted sum prediction; and Step of encoding the above flag Includes more, A method in which prediction mode information of a plurality of prediction blocks is obtained when the above flag indicates that the current block is predicted according to a regression-based weighted sum prediction.

14. In Paragraph 12, The step of predicting the current block above is, A step of generating the prediction blocks based on the above-mentioned rearranged integrated candidate list; A step of determining reference samples in the surrounding restored region of the current block and the surrounding restored region of a plurality of reference blocks; A step of calculating weighted sum coefficients of the prediction blocks using the reference samples above, wherein the weighted sum coefficients are defined by a regression equation based on pixel positions within each prediction block; and A step of predicting the current block by weighting the prediction blocks based on the weighted sum coefficients. A method including 15. A method for providing video data to a video decoder, A step of encoding the above video data into a bitstream; and Step of transmitting the above bitstream to the above video decoder Includes, The step of encoding the above video data is, A step of obtaining prediction mode information of a plurality of prediction blocks and determining the prediction modes of the prediction blocks based on the prediction mode information, wherein the prediction blocks are used for the prediction of a current block, and the prediction modes include an inter-prediction mode and an intra-prediction mode; A step of constructing a list of prediction information for each prediction mode, and combining the lists of prediction information for the prediction modes to construct an integrated candidate list; A step of rearranging the above integrated candidate list; and A step of predicting the current block based on a reordered integrated candidate list A method including

Images