Method for geometric partitioning-based prediction using adaptive weighted sum

Abstract

The present embodiment discloses a method for geometric partitioning-based prediction using an adaptive weighted sum. In the present embodiment, an image decoding device decodes an index indicating a combination of information about a geometric partitioning mode of the current block and information about a prediction mode of each sub-region of the current block. Here, each sub-region is generated by partitioning the current block. The image decoding device constructs a list including combinations of geometric partitioning mode candidates and prediction mode candidates of each sub-region, and rearranges the combinations in the list. The image decoding device determines, from the rearranged list, the geometric partitioning mode of the current block and the prediction mode of each sub-region of the current block on the basis of the index. The image decoding device predicts the current block on the basis of the geometric partitioning mode and the prediction mode of each sub-region.

Description

Method for geometric partitioning-based prediction using adaptive weighted sum

[0001] The present disclosure relates to an image encoding / decoding method, an apparatus, and a recording medium for storing a bitstream, and more specifically, to a geometric partitioning-based prediction method using an adaptive weighted sum.

[0002] The following description merely provides background information related to the present invention and does not constitute prior art.

[0003] Because video data contains a large amount of data compared to audio or still image data, storing or transmitting it as is without compression processing requires significant hardware resources, including memory.

[0004] Therefore, typically when storing or transmitting video data, the encoder compresses the video data for storage or transmission, and the decoder receives the compressed video data, decompresses it, and plays it. Such video compression technologies include H.264 / AVC, HEVC (High Efficiency Video Coding), and VVC (Versatile Video Coding), which improves coding efficiency by more than 30% compared to HEVC.

[0005] However, as video size, resolution, and frame rates are gradually increasing, and the amount of data that needs to be encoded is also growing accordingly, a new compression technology is required that offers better encoding efficiency and higher image quality improvement effects than existing compression technologies.

[0006] The present disclosure aims to provide an image encoding / decoding method and apparatus that explicitly and / or implicitly determine the region where a weighted sum is performed and the weighted sum coefficients used in the weighted sum in a prediction based on geometric partitioning of a current block, and a recording medium that stores a bitstream generated by said image encoding method / apparatus.

[0007] According to an embodiment of the present disclosure, a method for image decoding to restore a current block is provided, comprising the steps of: decoding an index indicating a combination of information on a geometric partitioning mode of the current block and information on a prediction mode of each sub-region of the current block, wherein each sub-region is generated by partitioning the current block; forming a list including combinations of a candidate geometric partitioning mode and a candidate prediction mode of each sub-region; rearranging combinations within the list; determining the geometric partitioning mode of the current block and the prediction mode of each sub-region of the current block from the rearranged list based on the index; and predicting the current block based on the geometric partitioning mode and the prediction mode of each sub-region.

[0008] According to another embodiment of the present disclosure, a video encoding method for encoding a current block comprises: obtaining an index indicating a combination of information on a geometric partitioning mode of the current block and information on a prediction mode of each sub-region of the current block, wherein each sub-region is created by partitioning the current block; forming a list including combinations of a candidate geometric partitioning mode and a candidate prediction mode of each sub-region; rearranging combinations within the list; determining the geometric partitioning mode of the current block and the prediction mode of each sub-region of the current block from the rearranged list based on the index; predicting the current block based on the geometric partitioning mode and the prediction mode of each sub-region; and encoding the index indicating the combination.

[0009] According to another embodiment of the present disclosure, a method for providing video data to an image decoder comprises: encoding the video data into a bitstream; and transmitting the bitstream to the image decoder, wherein the step of encoding the video data comprises: obtaining an index indicating a combination of information on a geometric partitioning mode of a current block and information on a prediction mode of each sub-region of the current block, wherein each sub-region is created by partitioning the current block; constructing a list including combinations of a candidate geometric partitioning mode and a candidate prediction mode of each sub-region; rearranging the combinations within the list; determining the geometric partitioning mode of the current block and the prediction mode of each sub-region of the current block from the rearranged list based on the index; predicting the current block based on the geometric partitioning mode and the prediction mode of each sub-region; and encoding the index indicating the combination.

[0010] As described above, by providing a video encoding / decoding method and device according to the present embodiment, and a recording medium storing a bitstream generated by the video encoding method / device, it is possible to improve video encoding efficiency and video quality.

[0011] FIG. 1 is an exemplary block diagram of an image encoding device capable of implementing the technologies of the present disclosure.

[0012] Figure 2 is a diagram illustrating a method for dividing blocks using a QTBTTT (QuadTree plus BinaryTree TernaryTree) structure.

[0013] FIGS. 3a and 3b are diagrams showing a plurality of intra prediction modes including wide-angle intra prediction modes.

[0014] Figure 4 is an example diagram of the surrounding blocks of the current block.

[0015] FIG. 5 is an exemplary block diagram of an image decoding device capable of implementing the technologies of the present disclosure.

[0016] FIG. 6 is a block diagram showing in detail a part of an image decoding device according to one embodiment of the present disclosure.

[0017] FIG. 7 is an exemplary diagram showing sub-regions according to a geometric division mode according to one embodiment of the present disclosure.

[0018] FIGS. 8a and 8b are flowcharts illustrating the execution of a geometric partitioning-based prediction mode according to one embodiment of the present disclosure.

[0019] FIG. 10 is an exemplary diagram showing prediction blocks of sub-regions according to one embodiment of the present disclosure.

[0020] FIG. 11 is an exemplary diagram showing surrounding areas of current blocks and predicted blocks according to one embodiment of the present disclosure.

[0021] FIG. 12 is an exemplary diagram showing a reference line of a surrounding restored area according to one embodiment of the present disclosure.

[0022] FIG. 13 is an exemplary diagram illustrating a weighted sum method and the acquisition of a weighted sum region according to one embodiment of the present disclosure.

[0023] FIG. 14 is an exemplary diagram showing weighted sum regions and weighted sum coefficients according to one embodiment of the present disclosure.

[0024] FIG. 15 is an exemplary diagram showing a weighted sum region according to one embodiment of the present disclosure.

[0025] FIG. 16 is an exemplary diagram showing a surrounding restored area of ​​a current block according to one embodiment of the present disclosure.

[0026] FIGS. 17a to 17c are exemplary diagrams illustrating the cost calculation of weighted sum region candidates according to one embodiment of the present disclosure.

[0027] FIG. 18 is an exemplary diagram showing a weighted sum region according to another embodiment of the present disclosure.

[0028] Figures 19a and 19b are flowcharts illustrating a method for determining an inverse transformation kernel.

[0029] FIG. 20 is a flowchart illustrating a method for predicting a current block according to one embodiment of the present disclosure.

[0030] Hereinafter, embodiments of the present invention will be described in detail with reference to the exemplary drawings. It should be noted that in assigning reference numerals to the components of each drawing, the same components are given the same reference numeral whenever possible, even if they are shown in different drawings. Furthermore, in describing these embodiments, if it is determined that a detailed description of related known components or functions could obscure the essence of these embodiments, such detailed description is omitted.

[0031] FIG. 1 is an exemplary block diagram of an image encoding device capable of implementing the technologies of the present disclosure. Hereinafter, the image encoding device and its sub-components will be described with reference to FIG. 1.

[0032] The video encoding device may be configured to include a picture splitting unit (110), a prediction unit (120), a subtractor (130), a conversion unit (140), a quantization unit (145), a reordering unit (150), an entropy encoding unit (155), an inverse quantization unit (160), an inverse conversion unit (165), an adder (170), a loop filter unit (180), and a memory (190).

[0033] Each component of the video encoding device may be implemented in hardware or software, or as a combination of hardware and software. Additionally, the function of each component may be implemented in software, and a microprocessor may be implemented to execute the software function corresponding to each component.

[0034] A single image (video) consists of one or more sequences containing multiple pictures. Each picture is divided into multiple regions, and encoding is performed for each region. For example, a single picture is divided into one or more tiles and / or slices. Here, one or more tiles can be defined as a tile group. Each tile or slice is divided into one or more Coding Tree Units (CTUs). And each CTU is divided into one or more Coding Units (CUs) by a tree structure. Information applicable to each CU is encoded as the syntax of the CU, and information applicable to all CUs included in a single CTU is encoded as the syntax of the CTU. Additionally, information applicable to all blocks within a single slice is encoded as the syntax of the slice header, and information applicable to all blocks constituting one or more pictures is encoded in the Picture Parameter Set (PPS) or the picture header. Furthermore, information commonly referenced by multiple pictures is encoded in a Sequence Parameter Set (SPS). Also, information commonly referenced by one or more SPSs is encoded in a Video Parameter Set (VPS). Additionally, information commonly applicable to a single tile or tile group may be encoded as the syntax of a tile or tile group header. The syntax included in the SPS, PPS, slice header, and tile or tile group header may be referred to as high-level syntax.

[0035] The picture splitting unit (110) determines the size of the CTU. Information regarding the size of the CTU (CTU size) is encoded as a syntax of SPS or PPS and transmitted to an image decoding device.

[0036] The picture division unit (110) divides each picture constituting the image into multiple CTUs having a predetermined size, and then recursively divides the CTUs using a tree structure. The leaf nodes in the tree structure become the CUs, which are the basic units of encoding.

[0037] The tree structure may be a QuadTree (QT) in which an upper node (or parent node) is divided into four lower nodes (or child nodes) of equal size, a BinaryTree (BT) in which an upper node is divided into two lower nodes, a TernaryTree (TT) in which an upper node is divided into three lower nodes in a 1:2:1 ratio, or a structure that combines two or more of these QT, BT, and TT structures. For example, a QTBT (QuadTree plus BinaryTree) structure may be used, or a QTBTTT (QuadTree plus BinaryTree TernaryTree) structure may be used. Here, BTTT combined may be referred to as an MTT (Multiple-Type Tree).

[0038] Figure 2 is a diagram illustrating a method for dividing blocks using a QTBTTT structure.

[0039] As illustrated in FIG. 2, the CTU can first be split into a QT structure. Quadtree splitting can be repeated until the size of the splitting block reaches the minimum block size of the leaf node allowed in QT (MinQTSize). A first flag (QT_split_flag) indicating whether each node of the QT structure is split into four nodes of the lower layer is encoded by the entropy encoder (155) and signaled to the image decoder. If the leaf node of the QT is not larger than the maximum block size of the root node allowed in BT (MaxBTSize), it can be further split into one or more of the BT structure or TT structure. In the BT structure and / or TT structure, multiple splitting directions may exist. For example, there may be two directions in which the block of the corresponding node is split horizontally and vertically. As shown in Figure 2, when MTT splitting begins, a second flag (mtt_split_flag) indicating whether the nodes have been split, and if splitting has occurred, a flag indicating the splitting direction (vertical or horizontal) and / or the splitting type (binary or ternary) are encoded by the entropy encoding unit (155) and signaled to the image decoding device.

[0040] Alternatively, prior to encoding the first flag (QT_split_flag) indicating whether each node is split into four nodes of the lower layer, the CU split flag (split_cu_flag) indicating whether the node is split may be encoded. If the value of the CU split flag (split_cu_flag) indicates that it is not split, the block of the corresponding node becomes a leaf node in the split tree structure and becomes a coding unit (CU), which is the basic unit of encoding. If the value of the CU split flag (split_cu_flag) indicates that it is split, the video encoding device starts encoding from the first flag in the manner described above.

[0041] When QTBT is used as another example of a tree structure, there may be two types: a type that divides the block of the corresponding node horizontally into two blocks of the same size (i.e., symmetric horizontal splitting) and a type that divides it vertically (i.e., symmetric vertical splitting). A splitting flag (split_flag) indicating whether each node of the BT structure is split into a block of a lower layer and splitting type information indicating the type of splitting are encoded by the entropy encoding unit (155) and transmitted to the image decoding device. Meanwhile, there may also be an additional type that divides the block of the corresponding node into two blocks of an asymmetric shape. The asymmetric shape may include a shape that divides the block of the corresponding node into two rectangular blocks with a size ratio of 1:3, or a shape that divides the block of the corresponding node diagonally.

[0042] A CU can have various sizes depending on the QTBT or QTBTTT partitioning from a CTU. Hereinafter, the block corresponding to the CU to be encoded or decoded (i.e., the leaf node of QTBTTT) is referred to as the 'current block'. Depending on the adoption of QTBTTT partitioning, the shape of the current block may be not only square but also rectangular.

[0043] The prediction unit (120) predicts the current block and generates a prediction block. The prediction unit (120) includes an intra prediction unit (122) and an inter prediction unit (124).

[0044] Generally, current blocks within a picture can each be predictively coded. Typically, the prediction of a current block can be performed using an intra-prediction technique (using data from the picture containing the current block) or an inter-prediction technique (using data from a picture coded prior to the picture containing the current block). Inter-prediction includes both unidirectional and bidirectional prediction.

[0045] The intra prediction unit (122) predicts pixels within the current block using pixels (reference pixels) located around the current block within the current picture containing the current block. Multiple intra prediction modes exist depending on the prediction direction. For example, as shown in FIG. 3a, multiple intra prediction modes may include two non-directional modes, including Planar mode and DC mode, and 65 directional modes. The surrounding pixels to be used and the calculation formula are defined differently for each prediction mode.

[0046] For efficient directional prediction for a rectangular current block, directional modes (intra-prediction modes 67 through 80 and -1 through -14) illustrated by dashed arrows in FIG. 3b may be additionally used. These may be referred to as "wide angle intra-prediction modes." In FIG. 3b, the arrows indicate corresponding reference samples used for prediction and do not indicate the prediction direction. The prediction direction is opposite to the direction indicated by the arrows. Wide angle intra-prediction modes are modes that perform prediction in the opposite direction of a specific directional mode without additional bit transmission when the current block is rectangular. Among the wide angle intra-prediction modes, some wide angle intra-prediction modes available for the current block may be determined by the ratio of the width to the height of the rectangular current block. For example, wide-angle intra-prediction modes with an angle less than 45 degrees (intra-prediction modes 67 to 80) are available when the current block is a rectangular shape with a height less than the width, and wide-angle intra-prediction modes with an angle greater than -135 degrees (intra-prediction modes -1 to -14) are available when the current block is a rectangular shape with a width greater than the height.

[0047] The intra prediction unit (122) can determine the intra prediction mode to use for encoding the current block. In some examples, the intra prediction unit (122) may encode the current block using several intra prediction modes and select an appropriate intra prediction mode to use from the tested modes. For example, the intra prediction unit (122) may calculate the rate-distortion values ​​using a rate-distortion analysis of several tested intra prediction modes and select the intra prediction mode having the best rate-distortion features among the tested modes.

[0048] The intra prediction unit (122) selects one intra prediction mode among a plurality of intra prediction modes and predicts the current block using a calculation formula and surrounding pixels (reference pixels) determined according to the selected intra prediction mode. Information regarding the selected intra prediction mode is encoded by the entropy encoding unit (155) and transmitted to the image decoding device.

[0049] The inter prediction unit (124) generates a prediction block for the current block using a motion compensation process. The inter prediction unit (124) searches for the block most similar to the current block within a reference picture that is encoded and decoded before the current picture, and generates a prediction block for the current block using the searched block. Then, it generates a motion vector (MV) corresponding to the displacement between the current block in the current picture and the prediction block in the reference picture. Generally, motion estimation is performed on the lumina component, and the motion vector calculated based on the lumina component is used for both the lumina component and the chroma component. Motion information including information about the reference picture used to predict the current block and information about the motion vector is encoded by the entropy encoding unit (155) and transmitted to the image decoding device.

[0050] The inter prediction unit (124) may perform interpolation on a reference picture or reference block to increase the accuracy of the prediction. That is, subsamples between two consecutive integer samples are interpolated by applying filter coefficients to a plurality of consecutive integer samples including those two integer samples. When the process of searching for the block most similar to the current block is performed for the interpolated reference picture, the motion vector can be expressed with precision in fractional units rather than precision in integer sample units. The precision or resolution of the motion vector can be set differently for each unit of the target area to be encoded, such as slice, tile, CTU, CU, etc. When such Adaptive Motion Vector Resolution (AMVR) is applied, information regarding the motion vector resolution to be applied to each target area must be signaled for each target area. For example, if the target area is a CU, information regarding the motion vector resolution applied to each CU is signaled. The information regarding the motion vector resolution may be information indicating the precision of the difference motion vector described later.

[0051] Meanwhile, the inter prediction unit (124) can perform inter prediction using bi-prediction. In the case of bi-prediction, two reference pictures and two motion vectors representing the block location most similar to the current block within each reference picture are used. The inter prediction unit (124) selects a first reference picture and a second reference picture from the reference picture list 0 (RefPicList0) and reference picture list 1 (RefPicList1), respectively, and generates a first reference block and a second reference block by searching for a block similar to the current block within each reference picture. Then, it generates a prediction block for the current block by averaging or weighting the first reference block and the second reference block. Then, it transmits motion information containing information about the two reference pictures used to predict the current block and information about the two motion vectors to the entropy encoding unit (155). Here, reference picture list 0 consists of restored pictures that are prior to the current picture in the display order, and reference picture list 1 may consist of restored pictures that are prior to the current picture in the display order. However, this is not necessarily limited to this, and restored pictures prior to the current picture in the display order may be additionally included in reference picture list 0, and conversely, restored pictures prior to the current picture may be additionally included in reference picture list 1.

[0052] Various methods can be used to minimize the amount of bits required to encode motion information.

[0053] For example, if the reference picture and motion vector of the current block are identical to the reference picture and motion vector of a neighboring block, the motion information of the current block can be transmitted to an image decoder by encoding information that can identify the neighboring block. This method is called 'merge mode'.

[0054] In merge mode, the inter prediction unit (124) selects a predetermined number of merge candidate blocks (hereinafter referred to as 'merge candidates') from the surrounding blocks of the current block.

[0055] As for the surrounding blocks for deriving merge candidates, as shown in FIG. 4, all or part of the left block (A0), bottom-left block (A1), top block (B0), top-right block (B1), and top-left block (B2) adjacent to the current block within the current picture may be used. Additionally, a block located within a reference picture (which may be the same as or different from the reference picture used to predict the current block) other than the current picture where the current block is located may be used as a merge candidate. For example, a block located at the same position as the current block within the reference picture (co-located block) or a block adjacent to that same position may be additionally used as a merge candidate. If the number of merge candidates selected by the method described above is less than a preset number, a 0 vector is added to the merge candidates.

[0056] The inter prediction unit (124) constructs a merge list containing a predetermined number of merge candidates using these surrounding blocks. Among the merge candidates included in the merge list, it selects a merge candidate to be used as movement information for the current block and generates merge index information to identify the selected candidate. The generated merge index information is encoded by the entropy encoding unit (155) and transmitted to the image decoding device.

[0057] Merge skip mode is a special case of merge mode; after quantization, when all transform coefficients for entropy coding are close to zero, only neighbor block selection information is transmitted without transmitting residual signals. By utilizing merge skip mode, relatively high coding efficiency can be achieved in images with minimal motion, still images, and screen content images.

[0058] Hereinafter, merge mode and merge skip mode will be collectively referred to as merge / skip mode.

[0059] Another method for encoding motion information is the AMVP (Advanced Motion Vector Prediction) mode.

[0060] In AMVP mode, the inter-prediction unit (124) derives predicted motion vector candidates for the motion vector of the current block using the surrounding blocks of the current block. As surrounding blocks used to derive predicted motion vector candidates, all or part of the left block (A0), bottom-left block (A1), top block (B0), top-right block (B1), and top-left block (B2) adjacent to the current block within the current picture shown in FIG. 4 may be used. Additionally, blocks located within a reference picture (which may be the same as or different from the reference picture used to predict the current block) other than the current picture where the current block is located may be used as surrounding blocks to derive predicted motion vector candidates. For example, blocks located at the same position as the current block within the reference picture (co-located blocks) or blocks adjacent to the blocks at the same position may be used. If the number of motion vector candidates is less than a preset number by the method described above, a 0 vector is added to the motion vector candidates.

[0061] The inter prediction unit (124) derives predicted motion vector candidates using the motion vectors of the surrounding blocks and determines a predicted motion vector for the current block's motion vector using the predicted motion vector candidates. Then, it calculates a difference motion vector by subtracting the predicted motion vector from the current block's motion vector.

[0062] Predicted motion vectors can be obtained by applying a predefined function (e.g., median, mean operation, etc.) to the predicted motion vector candidates. In this case, the image decoder is also aware of the predefined function. Furthermore, since the surrounding blocks used to derive the predicted motion vector candidates have already been encoded and decoded, the image decoder is also aware of the motion vectors of those surrounding blocks. Therefore, the image decoder does not need to encode information to identify the predicted motion vector candidates. Consequently, in this case, information regarding the difference motion vector and the reference picture used to predict the current block is encoded.

[0063] Meanwhile, the predicted motion vector may be determined by selecting one of the predicted motion vector candidates. In this case, information for identifying the selected predicted motion vector candidate is additionally encoded, along with information about the difference motion vector and information about the reference picture used to predict the current block.

[0064] The subtractor (130) generates a residual block by subtracting the prediction block generated by the intra prediction unit (122) or the inter prediction unit (124) from the current block.

[0065] The conversion unit (140) converts residual signals within a residual block having pixel values ​​in a spatial domain into conversion coefficients in the frequency domain. The conversion unit (140) can convert the residual signals within the residual block using the entire size of the residual block as the conversion unit, or it can divide the residual block into multiple sub-blocks and use the sub-blocks as the conversion unit to perform the conversion. Alternatively, it can divide the residual signals into two sub-blocks, a conversion area and a non-conversion area, and use only the conversion area sub-block as the conversion unit to convert the residual signals. Here, the conversion area sub-block may be one of two rectangular blocks having a size ratio of 1:1 with respect to the horizontal axis (or vertical axis). In this case, a flag (cu_sbt_flag) indicating that only the sub-block has been converted, direction (vertical / horizontal) information (cu_sbt_horizontal_flag) and / or position information (cu_sbt_pos_flag) are encoded by the entropy encoding unit (155) and signaled to the image decoding device. Additionally, the size of the converted area sub-block may have a size ratio of 1:3 with respect to the horizontal axis (or vertical axis), and in this case, a flag (cu_sbt_quad_flag) distinguishing the corresponding division is additionally encoded by the entropy encoding unit (155) and signaled to the image decoding device.

[0066] Meanwhile, the transformation unit (140) can perform transformations on the residual block individually in the horizontal and vertical directions. For the transformation, various types of transformation functions or transformation matrices may be used. For example, a pair of transformation functions for horizontal transformation and vertical transformation can be defined as a Multiple Transform Set (MTS). The transformation unit (140) can select one pair of transformation functions with the best transformation efficiency among the MTS and transform the residual block in the horizontal and vertical directions, respectively. Information (mts_idx) regarding the selected pair of transformation functions among the MTS is encoded by the entropy encoding unit (155) and signaled to the image decoder.

[0067] The quantization unit (145) quantizes the transformation coefficients output from the transformation unit (140) using quantization parameters and outputs the quantized transformation coefficients to the entropy encoding unit (155). The quantization unit (145) may quantize the associated residual block directly without transformation for any block or frame. The quantization unit (145) may apply different quantization coefficients (scaling values) depending on the position of the transformation coefficients within the transformation block. The quantization matrix applied to the quantized transformation coefficients arranged in two dimensions can be encoded and signaled to an image decoder.

[0068] The reordering unit (150) can perform reordering of coefficient values ​​for quantized residual values.

[0069] The reordering unit (150) can convert a two-dimensional coefficient array into a one-dimensional coefficient sequence using coefficient scanning. For example, the reordering unit (150) can output a one-dimensional coefficient sequence by scanning from DC coefficients to coefficients in the high-frequency range using a zig-zag scan or a diagonal scan. Depending on the size of the conversion unit and the intra-prediction mode, a vertical scan that scans the two-dimensional coefficient array in the column direction and a horizontal scan that scans the two-dimensional block-shaped coefficients in the row direction may be used instead of a zig-zag scan. That is, depending on the size of the conversion unit and the intra-prediction mode, the scanning method to be used among a zig-zag scan, a diagonal scan, a vertical scan, and a horizontal scan may be determined.

[0070] The entropy encoding unit (155) generates a bitstream by encoding a sequence of one-dimensional quantized transformation coefficients output from the reordering unit (150) using various encoding methods such as CABAC (Context-based Adaptive Binary Arithmetic Code) and Exponential Golomb.

[0071] Additionally, the entropy encoding unit (155) encodes information related to block division, such as CTU size, CU division flag, QT division flag, MTT division type, and MTT division direction, so that the video decoder can divide the block in the same way as the video encoding unit. Additionally, the entropy encoding unit (155) encodes information regarding a prediction type indicating whether the current block is encoded by intra prediction or by inter prediction, and encodes intra prediction information (i.e., information regarding the intra prediction mode) or inter prediction information (information regarding the encoding mode of motion information (merge mode or AMVP mode), the merge index in the case of merge mode, and the reference picture index and difference motion vector in the case of AMVP mode) according to the prediction type. Additionally, the entropy encoding unit (155) encodes information related to quantization, i.e., information regarding quantization parameters and information regarding the quantization matrix.

[0072] The inverse quantization unit (160) inversely quantizes the quantized transformation coefficients output from the quantization unit (145) to generate transformation coefficients. The inverse transformation unit (165) converts the transformation coefficients output from the inverse quantization unit (160) from the frequency domain to the spatial domain to restore the residual block.

[0073] The adder (170) restores the current block by adding the restored residual block and the prediction block generated by the prediction unit (120). The pixels within the restored current block are used as reference pixels when intra-predicting the next block in sequence.

[0074] The loop filter section (180) performs filtering on the restored pixels to reduce blocking artifacts, ringing artifacts, blurring artifacts, etc. caused by block-based prediction and transformation / quantization. The loop filter section (180) may include all or part of a deblocking filter (182), a SAO (Sample Adaptive Offset) filter (184), and an ALF (Adaptive Loop Filter, 186) as an in-loop filter.

[0075] The deblocking filter (182) filters the boundaries between restored blocks to remove blocking artifacts caused by block-unit encoding / decoding, and the SAO filter (184) and ALF (186) perform additional filtering on the deblocking filtered image. The SAO filter (184) and ALF (186) are filters used to compensate for the difference between restored pixels and original pixels caused by lossy coding. The SAO filter (184) improves not only subjective image quality but also encoding efficiency by applying an offset in CTU units. In contrast, the ALF (186) performs block-unit filtering, and compensates for distortion by applying different filters by distinguishing the degree of edge and change of the corresponding block. Information regarding the filter coefficients to be used in the ALF can be encoded and signaled to an image decoder.

[0076] The restored blocks filtered through the deblocking filter (182), SAO filter (184), and ALF (186) are stored in memory (190). Once all blocks within a picture are restored, the restored picture can be used as a reference picture for inter-predicting blocks within a picture to be encoded later.

[0077] The video encoding device can store the bitstream of encoded video data on a non-transient recording medium or transmit it to a video decoding device using a communication network.

[0078] FIG. 5 is an exemplary block diagram of an image decoding device capable of implementing the technologies of the present disclosure. Hereinafter, the image decoding device and its sub-components will be described with reference to FIG. 5.

[0079] The image decoding device may be configured to include an entropy decoding unit (510), a reordering unit (515), an inverse quantization unit (520), an inverse transformation unit (530), a prediction unit (540), an adder (550), a loop filter unit (560), and a memory (570).

[0080] Similar to the image encoding device of FIG. 1, each component of the image decoding device may be implemented in hardware or software, or in combination of hardware and software. Additionally, the function of each component may be implemented in software, and a microprocessor may be implemented to execute the function of the software corresponding to each component.

[0081] The entropy decoding unit (510) determines the current block to be decoded by decoding the bitstream generated by the video encoding device and extracting information related to block division, and extracts prediction information, information on residual signals, etc., necessary to restore the current block.

[0082] The entropy decoding unit (510) extracts information about the CTU size from the SPS (Sequence Parameter Set) or PPS (Picture Parameter Set) to determine the size of the CTU and divides the picture into CTUs of the determined size. Then, the CTU is determined as the top layer of the tree structure, i.e., the root node, and divides the CTU using the tree structure by extracting division information for the CTU.

[0083] For example, when splitting a CTU using a QTBTTT structure, first, a first flag (QT_split_flag) related to QT splitting is extracted to split each node into four nodes of the lower layer. Then, for the nodes corresponding to the leaf nodes of QT, a second flag (mtt_split_flag) related to MTT splitting and splitting direction (vertical / horizontal) and / or splitting type (binary / ternary) information are extracted to split the corresponding leaf nodes into an MTT structure. Accordingly, each node below the leaf nodes of QT is recursively split into a BT or TT structure.

[0084] As another example, when splitting a CTU using the QTBTTT structure, a CU splitting flag (split_cu_flag) indicating whether to split the CU is first extracted, and if the block is split, a first flag (QT_split_flag) is extracted. During the splitting process, each node may undergo zero or more iterative MTT splittings after zero or more iterative QT splittings. For example, the CTU may undergo MTT splitting immediately, or conversely, only multiple QT splittings may occur.

[0085] As another example, when splitting a CTU using a QTBT structure, a first flag (QT_split_flag) related to the splitting of QT is extracted to split each node into four nodes of the lower layer. Then, for the nodes corresponding to the leaf nodes of QT, a split flag (split_flag) indicating whether to further split into BTs and split direction information are extracted.

[0086] Meanwhile, when the entropy decoding unit (510) determines the current block to be decoded using the division of the tree structure, it extracts information regarding the prediction type indicating whether the current block is intra-predicted or inter-predicted. If the prediction type information indicates intra-predicted, the entropy decoding unit (510) extracts syntax elements for the intra-predicted information (intra-predicted mode) of the current block. If the prediction type information indicates inter-predicted, the entropy decoding unit (510) extracts syntax elements for the inter-predicted information, namely information indicating the motion vector and the reference picture that the motion vector refers to.

[0087] Additionally, the entropy decoding unit (510) extracts information regarding quantization-related information and information regarding residual signals, as well as information regarding the quantized transformation coefficients of the current block.

[0088] The reordering unit (515) can change the sequence of one-dimensional quantized transformation coefficients entropy-decoded in the entropy decoding unit (510) back into a two-dimensional coefficient array (i.e., block) in the reverse order of the coefficient scanning order performed by the image encoding device.

[0089] The inverse quantization unit (520) inversely quantizes the quantized transformation coefficients and inversely quantizes the quantized transformation coefficients using quantization parameters. The inverse quantization unit (520) may apply different quantization coefficients (scaling values) to the quantized transformation coefficients arranged in two dimensions. The inverse quantization unit (520) may perform inverse quantization by applying a matrix of quantization coefficients (scaling values) from an image encoding device to a two-dimensional array of quantized transformation coefficients.

[0090] The inverse transformation unit (530) generates a residual block for the current block by inversely transforming the inversely quantized transformation coefficients from the frequency domain to the spatial domain and restoring the residual signals.

[0091] Additionally, when the inverse transformation unit (530) inversely transforms only a part of the transformation block (sub-block), it extracts a flag (cu_sbt_flag) indicating that only the sub-block of the transformation block has been transformed, information on the directionality (vertical / horizontal) of the sub-block (cu_sbt_horizontal_flag) and / or information on the position of the sub-block (cu_sbt_pos_flag), restores residual signals by inversely transforming the transformation coefficients of the corresponding sub-block from the frequency domain to the spatial domain, and creates a final residual block for the current block by filling the areas that have not been inversely transformed with "0" values ​​of residual signals.

[0092] Additionally, when MTS is applied, the inverse transformation unit (530) determines a transformation function or transformation matrix to be applied in the horizontal and vertical directions, respectively, using MTS information (mts_idx) signaled from the video encoding device, and performs an inverse transformation on the transformation coefficients within the transformation block in the horizontal and vertical directions using the determined transformation function.

[0093] The prediction unit (540) may include an intra prediction unit (542) and an inter prediction unit (544). The intra prediction unit (542) is activated when the prediction type of the current block is an intra prediction, and the inter prediction unit (544) is activated when the prediction type of the current block is an inter prediction.

[0094] The intra prediction unit (542) determines the intra prediction mode of the current block among a plurality of intra prediction modes from the syntax elements for the intra prediction mode extracted from the entropy decoding unit (510), and predicts the current block using reference pixels around the current block according to the intra prediction mode.

[0095] The inter prediction unit (544) determines the motion vector of the current block and the reference picture that the motion vector refers to using the syntax elements for the inter prediction mode extracted from the entropy decoding unit (510), and predicts the current block using the motion vector and the reference picture.

[0096] The adder (550) restores the current block by adding the residual block output from the inverse transformation unit (530) and the prediction block output from the inter prediction unit (544) or the intra prediction unit (542). The pixels within the restored current block are used as reference pixels when intra-predicting the block to be decoded later.

[0097] The loop filter section (560) may include a deblocking filter (562), an SAO filter (564), and an ALF (566) as an in-loop filter. The deblocking filter (562) deblocks the boundaries between restored blocks to remove blocking artifacts caused by block-unit decoding. The SAO filter (564) and the ALF (566) perform additional filtering on the restored blocks after deblocking filtering to compensate for the difference between the restored pixels and the original pixels caused by lossy coding. The filter coefficients of the ALF are determined using information about the filter coefficients decoded from the bitstream.

[0098] The restored blocks filtered through the deblocking filter (562), SAO filter (564), and ALF (566) are stored in memory (570). When all blocks within a picture are restored, the restored picture is used as a reference picture to inter-predict blocks within the picture to be encoded later.

[0099] The present embodiment relates to the encoding and decoding of an image (video) as described above. More specifically, in a prediction based on geometric partitioning of a current block, the present invention provides an image encoding / decoding method / device that explicitly and / or implicitly determines the region where a weighted sum is performed and the weighted sum coefficients used in the weighted sum, and a recording medium that stores a bitstream generated by said image encoding method / device.

[0100] The following embodiments may be performed by a prediction unit (120) within a video encoding apparatus. Additionally, the following embodiments may be performed by a prediction unit (540) within a video decoding apparatus.

[0101] The video encoding device can generate signaling information related to the present embodiment in terms of rate distortion optimization during the encoding of the current block. The video encoding device can encode the signaling information using the entropy encoding unit (155) and then transmit it to the video decoder. The video decoder can decode the signaling information related to the decoding of the current block from the bitstream using the entropy decoder (510).

[0102] In the following description, the term 'target block' may be used interchangeably with 'current block' or 'Coding Unit (CU).' Alternatively, 'target block' may refer to a specific area of ​​a Coding Unit.

[0103] Also, a value of one flag being true indicates that the flag is set to 1. Also, a value of one flag being false indicates that the flag is set to 0.

[0104] The decoder-side includes all or part of an inverse quantizer (160), an inverse transform (165), a prediction unit (120), an adder (170), a loop filter (180), and a memory (190) in the image encoding device illustrated in FIG. 1. Alternatively, the decoder-side includes all or part of an inverse quantizer (520), an inverse transform (530), a prediction unit (540), an adder (550), a loop filter (560), and a memory (570) in the image decoding device illustrated in FIG. 5. In relation to a series of decoding processes, the decoder-side of the image encoding device and the decoder-side of the image decoding device perform the same operation. The image encoding device determines information related to the operation of the decoder-side and signals the determined information to the image decoding device. The image decoding device can decode the signaled information and operate the decoder-side based on the decoded information.

[0105] I. GPM (Geometric Partitioning Mode) technology

[0106] The GPM technique of VVC is applicable in the case of inter-prediction. Among the 64 partitioning modes defined according to the agreement between the video encoder and the video decoder, one index is signaled / parsed as the geometric partitioning mode of the current block. Each partitioning mode consists of an angle (angleIdx) and a distance (distanceIdx).

[0107] The current block is divided into two sub-regions according to the geometric partitioning mode. Each of the two sub-regions is inter-predicted, and only unidirectional prediction can be used for each sub-region. After the prediction blocks of each sub-region are generated, the two prediction blocks are weighted summed to generate the final prediction block of the current block. At this time, a weighted sum matrix is ​​used for the weighted sum. Each coefficient in the weighted sum matrix has a value from 0 to 8, and the weighted sum of the two prediction blocks can be performed in the region where coefficients from 1 to 7 exist. At this time, a fixed region within the current block is used as the weighted sum region. Based on the geometric partitioning boundary, the weighted sum regions on both sides have the same size, and the size of one weighted sum region is represented by τ.

[0108] ECM's GPM technology is extended from VVC and can be applied not only to inter-prediction but also to intra-prediction and prediction based on IBC mode (hereinafter, IBC prediction). The GPM technology in intra-prediction is named SGPM (Spatial GPM), and each sub-region is intra-predicted. In SGPM technology, each intra-prediction mode and geometric partitioning mode are not signaled / parsed separately; instead, a single index can be signaled / parsed by combining the information of the two prediction modes and the information of the geometric partitioning mode. The combinations of intra-prediction modes and geometric partitioning modes in each region can be rearranged based on the Sum of Absolute Differences (SAD) using the surrounding restored template regions of the current block (e.g., one sample line adjacent to the left and top of the current block). In the case of SGPM, 9 intra-prediction mode candidates and 26 geometric partitioning modes can be used. After the rearrangement of the combinations, an index indicating one of the top 16 candidates is signaled / parsed.

[0109] If the current block is predicted according to the SGPM mode, a flag indicating whether the SGPM block has a fixed weighted sum area may be defined at the PPS level. If the flag is true, the SGPM block has a fixed weighted sum area τ / 4. If the flag is false, the image decoder implicitly determines the weighted sum area based on the size of the current block without signaling / parsing additional syntax. The weighted sum area may be implicitly determined using the minimum value (M) between the height and the width. The weighted sum area is determined equally on both sides of the geometric partition boundary. If M ≤ 4, the weighted sum area is determined as τ / 2, and if M ≤ 8, the weighted sum area is determined as τ. If M ≤ 16, the weighted sum area is determined as 2τ, if M ≤ 32, the weighted sum area is determined as 4τ, and in the remaining cases, the weighted sum area is determined as 8τ.

[0110] In GPM-based technologies excluding regression-based GPM mode, information regarding the weighted sum region is determined by the signaling / parsing of the index. When the GPM flag is 1, the regression-based GPM mode can be determined by the parsing of an additional flag, the regression-based GPM mode flag. When the regression-based GPM mode flag is 1, two blending matrices used for the weighted sum of the two sub-regions are implicitly derived based on two sub-region prediction mode pair candidates.

[0111] On the other hand, when the regression-based GPM mode flag is 0, the image decoder can parse indices for weighted sum regions. The weighted sum regions represented by each index are identical on both sides with respect to the geometric partitioning boundary. The size of the weighted sum region represented by each index may vary depending on the block size. If the minimum value between the height and width of the current block is less than 32, each index represents the weighted sum regions τ / 4, τ / 2, 2τ, and 4τ. If the minimum value between the height and width of the current block is greater than or equal to 32, each index represents the weighted sum regions τ / 2, 2τ, 4τ, and 8τ.

[0112] The area being weighted in the current block is determined according to the weighted sum area, and each coefficient, i.e., weight, within the weighted sum area has a value of 0 to 32.

[0113] Meanwhile, in the AV1 standard, the difference weighted mode is applicable to bidirectional inter prediction, i.e., the compound mode. When the difference weighted mode is used, type information (0 or 1) is additionally parsed. The difference weighted mode is a mode that performs a weighted sum over the entire block, and the weighted sum coefficients used for the weighted sum are two prediction blocks (P A (i,j), P B It is determined in pixel units as in Equation 1 based on the absolute value of the difference between (i,j).

[0114]

[0115] In Equation 1, (i,j) are two prediction blocks P A , P B It is used to represent the position of each sample within. The Clip3(P,Q,R) function applies a clipping operation to the R value using the minimum value P and the maximum value Q.

[0116] The following embodiments are described with reference to an image decoding device, but may be implemented identically or similarly in an image encoding device. Alternatively, the following embodiments are described with reference to the decoder side of the image decoding device, but may be implemented identically or similarly in the decoder side of an image encoding device.

[0117] II. Embodiments according to the present disclosure

[0118] FIG. 6 is a block diagram showing in detail a part of an image decoding device according to one embodiment of the present disclosure.

[0119] Hereinafter, "current block," "current coding block," "current prediction block," and "current prediction unit block" have the same meaning and can be used interchangeably.

[0120] The image decoder according to the present embodiment determines a prediction and conversion unit, and for a current block corresponding to the determined unit, performs prediction and inverse conversion using a determined prediction technique and prediction mode, thereby finally generating a restored block of the current block. As exemplified in FIG. 6, this can be performed by the inverse conversion unit (530), prediction unit (540), and adder (550) of the image decoder. Meanwhile, the same operations as exemplified in FIG. 6 can be performed by the inverse conversion unit (165), picture splitting unit (110), prediction unit (120), and adder (170) of the image encoding device. At this time, the image decoder uses encoding information parsed from a bitstream, but the image encoding device may use encoding information set from a higher level in terms of minimizing rate distortion. Hereinafter, for convenience, the present embodiment is described with the image decoder as the focus.

[0121] As shown in the example of FIG. 5, the prediction unit (540) includes an intra prediction unit (542) and an inter prediction unit (544) according to the prediction technology, but as shown in FIG. 6, the prediction unit (540) may include all or part of a prediction unit determination unit (602), a prediction technology determination unit (604), a prediction mode determination unit (606), and a prediction execution unit (608).

[0122] The prediction unit determination unit (602) determines a prediction unit (PU). The prediction technology determination unit (604) determines a prediction technology for the prediction unit (e.g., intra prediction, inter prediction, or IBC (Intra Block Copy) mode, palette mode, a technology combining intra and inter prediction, etc.). The prediction mode determination unit (606) determines a detailed prediction mode for the prediction technology. The prediction execution unit (608) generates a prediction block of the current block according to the determined prediction mode.

[0123] The inverse transformation unit (530) includes all or part of the inverse transformation unit determination unit (610), the inverse transformation kernel determination unit (612), and the inverse transformation execution unit (614). The inverse transformation unit determination unit (610) determines a transformation unit for the inverse quantization signals (i.e., inverse quantization transformation coefficients) of the current block, and for the determined transformation unit, can correct the predicted signal of the transformation unit prior to performing the inverse transformation. The inverse transformation kernel determination unit (612) determines an inverse transformation kernel, and the inverse transformation execution unit (614) generates residual samples by inversely transforming the transformation unit expressed by the inverse quantization transformation coefficients.

[0124] Hereinafter, the transform unit (TU) in terms of encoding can be used compatiblely with the transform block. The inverse transform unit and inverse transform block in terms of decoding correspond to the TU and the transform block, respectively. Therefore, the inverse transform unit and the inverse transform block can be used compatiblely with the TU and the transform block.

[0125] A transformation unit represents a unit that determines whether to perform an inverse transformation in relation to transformation coefficients and transmits information regarding the inverse transformation. An image decoder determines a kernel to be applied as the transformation unit and performs an inverse transformation of the transformation coefficients based on the determined kernel. At least one kernel may be determined for a single transformation unit. According to an embodiment, an N-order inverse transformation may be applied, and the size of the transformation unit and the inverse transformation kernel may not be the same for each order. For example, when an N-order inverse transformation is applied to a current transformation unit block (hereinafter referred to as the transformation block or current transformation block), the size of the kernel used for the inverse transformation of each order may not be the same. That is, the image decoder may apply an inverse transformation to some of the input transformation coefficients.

[0126] The adder (550) adds the prediction block and the residual samples to generate a recovery block. The recovery block is stored in memory and can subsequently be used to predict other blocks.

[0127] When the color format of the input video is a YUV format (YUV420, YUV411, YUV422, YUV444, etc.), the video decoder can perform prediction and restoration of the chroma component after performing prediction and restoration of the luminance component. That is, the luminance component and the chroma component can be sequentially restored by the components exemplified in FIG. 6. Meanwhile, when the color format of the input video is RGB, the video encoder can perform a color format conversion from RGB to YUV and then encode the converted video. Here, in the case of a YUV format, the color format represents the correspondence relationship between the pixels of the luminance component and the pixels of the chroma component.

[0128] The prediction unit determination unit (602) determines the size and shape of the block to be predicted, and may use all direct or indirect information transmitted from the image encoding device. The prediction unit determination unit (602) may use direct information related to the size and shape of the current block, and may utilize information that can influence the determination of the size and shape of the current block, such as the number of divisions, depth, shape of division, direction of division, size information related to the minimum divided block, and division information / prediction mode of the encoded surrounding block.

[0129] The prediction unit determined by the prediction unit determination unit (602) may be a current block, one of the sub-blocks into which the current block is divided, a set of pixels, or a single pixel. The prediction unit may include size information and shape information for performing prediction of chroma components and luminance components.

[0130] The prediction unit can be determined dependently or independently for the chroma and luminance components. Dependent determination means that the prediction units of the luminance or chroma components are not determined for each component individually; rather, once the prediction unit of one component is determined, the units of other components are determined in corresponding sizes and shapes. In this case, one component may correspond to one or more of the luminance and chroma components. That is, the luminance component may be determined based on the information of the chroma components. As an example, the prediction unit of the chroma component may be a size corresponding to the prediction unit of the luminance component according to the color format. When determined dependently, information regarding the prediction unit of the component determined dependently—that is, other components corresponding to one component—may be omitted. Independent determination means that the prediction units of the luminance and chroma components are determined separately. When determined independently, information regarding the prediction units for each component may be signaled separately.

[0131] The prediction technology determination unit (604) determines the prediction technology for the prediction unit. As described above, the prediction technology may be one of inter prediction, intra prediction, IBC mode, palette mode, or a technology mixed with intra and inter prediction. At this time, the prediction technology for the chroma component may be determined in the same way as the prediction technology for the corresponding luma component without signaling and parsing of separate information. According to the embodiment, a technology mixed with intra and inter prediction may be included in the inter prediction.

[0132] For example, if the prediction technique of the current block is not intra prediction, the image decoder parses 1-bit flag information. For instance, if the parsed flag indicates a Skip mode, the image decoder determines the prediction mode of the current block as an inter prediction merge mode or an IBC merge mode. In the case of Skip mode, the image decoder can use the prediction signals as restoration signals without the inverse transformation process (i.e., without parsing the residual signals). Here, Skip may be a method of not transmitting motion information (e.g., motion vector, reference picture, reference picture list, etc.) or transmitting motion information using only at least one syntax information, and additionally, a case where the residual signal of the current block is not transmitted.

[0133] On the other hand, if the parsed flag does not indicate a Skip mode for the current block, the prediction technology determination unit (604) can parse a series of 1-bit flags to determine the prediction technology of the current block as one of the technologies such as inter prediction, intra prediction, IBC mode, palette mode, etc.

[0134] For example, if no skip is applied to the current block and the prediction technique is determined to be Inter Prediction or IBC mode, the image decoder parses a 1-bit flag. Depending on the parsed flag, the prediction mode of the current block can be determined to be General Merge mode or AMVP (Advanced Motion Vector Prediction) mode.

[0135] The prediction mode determination unit (606) determines the detailed prediction mode of the current prediction unit block in relation to the prediction technology.

[0136] For example, if the prediction technique of the current block is inter-prediction, the prediction mode of the current block may be a mode that generates a prediction block of the current block using parsed motion information of the current block or motion compensation based on motion information. If the motion information of the current block includes multiple motions, multiple prediction blocks may be generated using motion compensation based on each motion, and the prediction block of the current block may be generated by weighting the multiple prediction blocks. In this case, one or more of the prediction blocks may include samples of coded regions within the same frame as the current block.

[0137] As another example, if the prediction technique for the current block is inter-prediction and intra-predicted samples are used to generate the final prediction block of the current block, the surrounding reconstructed region of the current block can be defined as a template, and the intra-prediction mode can be derived using said template. In this case, the template may also include regions that are not adjacent to the current block. Subsequently, the final prediction block of the current block can be generated using the derived intra-prediction mode. For instance, the final prediction block of the current block can be generated by weighting the intra-prediction block determined by derivation and / or parsing and the inter-prediction block determined by derivation and / or parsing. The weights may vary depending on the location of each sample. The image decoder can adaptively determine the weights using the prediction information of the current block, information on the surrounding reconstructed region, the aspect ratio of the current block, etc.

[0138] As another example, if the prediction technique of the current block is inter-prediction, the prediction mode of the current block may be a geometric partitioning-based prediction mode (hereinafter referred to as the Geometric Partitioning Mode (GPM)). In the GPM, the image decoder divides the current block into two or more sub-blocks according to geometric partitioning, generates prediction blocks of the sub-blocks according to one or more motion compensations based on the motion information and prediction mode information of the parsed current block, and generates final prediction signals of the current block by weighting the generated multiple prediction blocks. According to an embodiment, when inter-prediction is determined and the geometric partitioning mode is determined, at least one block among the sub-blocks within the current block may be predicted according to inter-prediction, affine prediction, etc., or at least one block may be predicted according to the IBC mode. According to an embodiment, at least one block among the sub-blocks may be predicted according to intra-prediction.

[0139] Affine prediction reflects various forms of motion, such as rotation, zoom-in, and zoom-out, in addition to translational motion. Two types of models for affine prediction may exist. One is a model that uses four parameters, namely two Control Point Motion Vectors (CPMVs) for the top-left corner and top-right corner of the target block. The other is a model that uses six parameters, namely three Control Point Motion Vectors for the top-left corner, top-right corner, and bottom-left corner of the target block. Affine prediction can utilize motion vectors computed according to the aforementioned affine models for each sample within the target block. Alternatively, to reduce computational complexity, the target block may be divided into 4×4 subblocks, for example, and affine prediction may be performed on a subblock basis. The video encoding device can encode affine-related syntax elements, including a flag indicating whether affine motion prediction has been applied to a target block, type information indicating the type of the affine model, and motion information indicating the motion vector of each control point, and transmit them to the video decoder. The video decoder can determine the type of the affine model and the control point motion vectors using the signaled syntax, and calculate motion vectors for each 4×4 sub-block within the target block using the affine model.

[0140] When the prediction technique of the current block is intra prediction, the prediction mode of the current block may be a mode for generating the prediction block of the current block based on at least one of the following: a directional prediction mode, a Planar mode (Horizontal Planar or Vertical Planar or Regular Planar), a DC mode, an EIP (Extrapolation filter-based intra prediction) prediction mode, a matrix-based prediction mode (for example, MIP), or a prediction mode based on correlation between components (e.g., CCLM (Cross component linear model), MMLM (Multi model CCLM), CCCM (Convolutional cross component model), LBCCP (Local boosting cross component prediction), BVG-CCCM (Block vector guided CCCM), CCLM with slope adjustment, Multi model CCCM, GLCCCM (Gradient and location based CCCM), CCP-merge, GLM (Gradient linear model), DDCCP (Decoder derived cross component prediction), etc.).

[0141] For example, if the prediction technique of the current block is intra prediction, the prediction mode of the current block may be a matrix-based intra prediction mode (for example, Matrix-based intra prediction (MIP)). The MIP mode may signal / parse and / or derive an index of the matrix based on a matrix defined according to an agreement between the video encoding device and the video decoder, or signal / parse and / or derive the matrix, and generate a prediction block of the current block based on the matrix.

[0142] As another example, if the prediction technique of the current block is intra prediction and directional intra prediction, and a specific directional prediction mode is applied according to the block size, the prediction mode of the current block may be a mode based on matrix multiplication. In this case, the prediction signal of the current block can be generated by using matrix multiplication between the directional matrix and the previously restored reference samples around the current block. The matrix can be predefined according to an agreement between the video encoding device and the video decoder.

[0143] As another example, if the prediction technique of the current block is intra prediction, the prediction mode of the current block may be the intra template matching prediction mode (IntraTMP). The IntraTMP mode defines a pre-recovered area surrounding the current block as a template and can generate a prediction block by performing template matching on the pre-recovered surrounding area of ​​the current block. The template may also include areas that are not adjacent to the current block.

[0144] As another example, if the prediction technique of the current block is intra prediction, the prediction mode of the current block may be a Spatial Geometry Partitioning Mode (SGPM). The Spatial Geometry Partitioning Mode divides the current block into one or more sub-regions according to geometric partitioning, generates prediction blocks for each region using intra prediction modes including different directional prediction modes, planar mode, DC mode, etc., and generates the prediction block of the current block by weighting each prediction block. According to an embodiment, if intra prediction is selected and a Spatial Geometry Partitioning Mode is selected, all sub-blocks within the current block may be predicted according to intra prediction. According to an embodiment, if intra prediction is selected and a Spatial Geometry Partitioning Mode is selected, at least one block among the sub-blocks within the current block may be predicted according to IntraTMP or according to IBC mode.

[0145] As another example, if the prediction technique of the current block is intra prediction, the prediction mode of the current block may be a DIMD (Decoder-side intra mode derivation) mode. The DIMD mode implicitly derives a prediction mode based on reconstructed samples within a pre-reconstructed region surrounding the current block, performs prediction of the current block using at least one derived mode, and generates a final prediction block of the current block using a weighted sum of the prediction blocks. As an example, the directionality of the reconstructed samples within the pre-reconstructed region may be considered during the process of deriving the prediction mode.

[0146] As another example, if the prediction technique of the current block is intra prediction, the prediction mode of the current block may be OBIC (Occurrence-based intra coding) mode. The OBIC mode implicitly derives a prediction mode based on the prediction modes of blocks within a restored region surrounding the current block, performs prediction of the current block according to at least one derived mode, and generates the final prediction block of the current block using the weighted sum of the prediction blocks. As an example, in the process of deriving the prediction mode, both the directionality of the restored samples within the restored region and the occurrence frequency of the prediction modes of the blocks within the restored region may be considered.

[0147] As another example, if the current block is a chroma block and the prediction technique for the current block is intra prediction, the current block can be predicted according to DM (Direct Mode). DM mode can perform prediction of the current chroma block based on a prediction method applied to the luminance block at a position corresponding to the current chroma block, or a predefined prediction method between the video encoding device and the video decoder.

[0148] As another example, the relationship between the surrounding restored chroma samples of the current chroma component and the surrounding restored luminance samples of the luminance region at the location corresponding to the current chroma block is calculated using a linear and / or non-linear model, and a prediction block of the current chroma block can be generated based on one or more of the calculated models.

[0149] As another example, there may be various methods for inducing the intra prediction mode of the current block. Among the multiple methods, one or more methods may be used depending on the signaling / parsing of flags and / or indexes.

[0150] Meanwhile, if the prediction technique for the current block is IBC prediction, a prediction block can be obtained from a pre-recovered region within the frame containing the current block using one or more block vectors, and a final prediction block can be generated based on the prediction block. The video encoding device can signal information of the block vector, and the video decoder can parse information of the block vector. For example, the video decoder can construct a block vector candidate list according to a position and search order defined by an agreement between the video encoding device and the video decoder, and parse information such as an index. The video decoder can obtain final block vector information by correcting the initial block vector information obtained based on the parsed information using a method such as template matching.

[0151] As an example, if the prediction technology of the current block is IBC prediction, the prediction mode of the current block may be an IBC geometric partitioning-based prediction mode (hereinafter referred to as the IBC geometric partitioning mode). The IBC geometric partitioning mode may divide the current block into one or more sub-regions according to geometric partitioning, generate prediction blocks for each region using different block vectors, and generate a final prediction block of the current block by weighting the prediction blocks. According to an embodiment, when the IBC prediction and the geometric partitioning mode are determined, at least one of all sub-blocks within the current block may be predicted according to IBC prediction.

[0152] As another example, if the current block is a chroma block and the prediction technique for the current block is IBC prediction, and a corresponding positional luminance block is recovered during the process of constructing a block vector candidate list or acquiring a block vector, the block vector information of the corresponding positional luminance block may be used. For instance, if the block partitioning structures of the luminance component and the chroma component are identical, information is generated by scaling the block vector information of the corresponding positional luminance block according to a color format, and the scaled information may be used as the block vector of the current chroma block and / or as a block vector candidate. On the other hand, if the block partitioning structures of the luminance component and the chroma component are different, one or more block vectors may be acquired according to a predefined position and order within the corresponding luminance region. Information is generated by scaling the acquired block vector according to a color format, and the scaled information may be used as the block vector of the current chroma block and / or as a block vector candidate.

[0153] Meanwhile, if the prediction technology of the current block is a technology that combines intra and inter prediction, the surrounding reconstructed region of the current block may be defined as a template, and an intra prediction mode may be derived using information of some or all pixels of the said template, or an intra prediction block may be generated based on template matching. A prediction block of the current block may be derived according to the derived intra prediction mode, or a prediction block may be generated based on block vector information signaled from an image encoding device. A final prediction block may be generated by mixing the prediction block / intra prediction block or inter prediction block generated based on template matching and / or block vectors according to a method such as weighted sum.

[0154] The template may include both areas adjacent to the current block and non-adjacent areas. The non-adjacent area may be an area within a certain pixel line distance from the current block. When a non-adjacent area is used as a template, information regarding its usage and / or distance may be transmitted from the image encoding device to the image decoder. For example, information regarding distance may be defined according to an agreement between the image encoding device and the image decoder, and the transmission of such information may be omitted. If defined according to an agreement, the value may be fixed as a specific constant or determined variably based on the horizontal and vertical pixel lengths of the prediction unit, the width of the block, the aspect ratio, etc. Subsequently, the final prediction block of the current block may be generated using the derived intra prediction mode.

[0155] The prediction execution unit (608) generates a final prediction block of the current decoding block (hereinafter used compatiblely with the current block) according to the determined prediction technique and prediction mode.

[0156] As an example, the prediction execution unit (608) performs a prediction according to the prediction mode determined as described above to generate a prediction block of the current block, and the adder (550) generates a restoration block by adding the prediction block of the current block and the residual samples (i.e., residual blocks).

[0157] As an example, after the prediction and restoration of a luminal component block, the prediction and restoration of a corresponding chroma component block may be performed. Here, a block may be a CU (Coding unit), a CTU (Coding tree unit), a set of one or more CUs and / or a set of one or more CTUs.

[0158] In this specification, the aspect ratio represents "width:height".

[0159] In this specification, the term 'and / or' includes a combination of a plurality of described items or any of a plurality of described items.

[0160] For example, when a current block having a size of W×H (W is width, H is height) is predicted according to a geometric partitioning-based prediction mode, the image decoder can divide the current block into one or more sub-regions using geometric boundaries as in FIG. 7 (a), perform a prediction for each sub-region, and generate a final prediction block of the current block by weighting the prediction blocks of the sub-regions.

[0161] Information regarding geometric partition boundaries can be defined using θ representing an angle and ρ representing a distance, as shown in FIG. 7(b). As shown in FIG. 7(b), the angle and distance can be defined relative to the center of the block. Information regarding the angle and distance in the information regarding geometric partition boundaries can be signaled / parsed, respectively. The image decoder can implicitly derive at least one piece of information. Alternatively, an index indicating a combination in a list (or table) composed of combinations of distance and angle can be signaled / parsed. In this case, the list can be predefined by an agreement between the image encoding device and the image decoder.

[0162] As shown in FIG. 7 (c), the weighted sum can be performed in a weighted sum area of ​​a certain range (α and β) that is implicitly and / or explicitly determined based on the geometric division boundary. In this case, α and β may have the same value. If the current block is divided into two sub-regions, each pixel within the weighted sum area may include a prediction sample from at least one sub-region. The applicable ranges of α and β may differ depending on the prediction mode of the sub-region. For example, the range of α and / or the range of β may differ depending on whether the prediction mode of sub-region A or sub-region B is an intra prediction mode or an inter prediction mode. In the case of an intra prediction mode, the range of α and / or the range of β may also differ depending on the angular relationship between the geometric division boundary and the directional prediction mode. For example, the range of α and / or the range of β may differ depending on whether it is a directional mode perpendicular to the geometric division boundary, a directional mode horizontal to it, or any other directional mode.

[0163] Below, the weighted sum region is represented by two factors, α and β, as shown in Fig. 7 (c).

[0164] The following lists of geometric partition modes, candidate lists, and lists may be used interchangeably. Geometric partition modes, geometric partition mode candidates, and candidates may be used interchangeably.

[0165] FIGS. 8a and 8b are flowcharts illustrating the execution of a geometric partitioning-based prediction mode according to one embodiment of the present disclosure.

[0166] The flowcharts shown in FIGS. 8a and 8b can be performed by an image decoding device and an image encoding device.

[0167] As described above, the geometric partitioning-based prediction mode can be applied to inter-prediction techniques, affine prediction techniques, intra-prediction techniques, and / or IBC prediction techniques. For example, as shown in the examples of FIGS. 8a and 8b, whether the geometric partitioning-based prediction mode is performed can be signaled / parsed in the form of a flag.

[0168] As an example, when a geometric partitioning-based prediction mode is performed, as shown in FIG. 8a, the image decoder constructs a list of geometric partitioning modes, parses geometric partitioning information and weighted sum information, constructs a list of prediction modes for each sub-region, and parses the prediction mode information for each sub-region. The image decoder determines the prediction mode of each sub-region using the prediction mode information of each sub-region. The image decoder can rearrange the order of geometric partitioning modes applicable to the current block using the prediction mode of each sub-region and determine the geometric partitioning mode of the current block using the parsed geometric partitioning information. The image decoder obtains information on geometric partitioning boundaries using the geometric partitioning mode and can partition the current block into multiple sub-regions using the obtained information. The image decoder can generate a prediction block for each sub-region by predicting each sub-region according to the prediction mode of each sub-region. The image decoder can generate a final prediction block of the current block by performing a weighted sum of the prediction blocks of the sub-regions based on the weighted sum information.

[0169] As another example, when a geometric partition-based prediction mode is performed, as shown in FIG. 8b, the image decoder constructs a list of geometric partition modes and a list of prediction modes for each sub-region. The image decoder obtains geometric partition information and prediction mode information for each sub-region, and parses weighted sum information. For example, the image decoder can obtain a combination of geometric partition information and information on the prediction modes of each sub-region. The image decoder rearranges the geometric partition modes and the prediction modes of each sub-region, and can determine the geometric partition mode of the current block and the prediction mode of each sub-region using the parsed geometric partition information and the prediction mode information of each sub-region. The image decoder obtains information on geometric partition boundaries using the geometric partition modes and can partition the current block into multiple sub-regions using the obtained information. The image decoder can generate a prediction block for each sub-region by predicting each sub-region according to the prediction mode of each sub-region. The image decoder can generate a final prediction block for the current block by performing a weighted sum of the prediction blocks of the sub-regions based on the weighted sum information.

[0170] As an example, the list of geometric partition modes may consist of a list related to all geometric partition boundaries applicable to the current block. In this case, the list may be determined differently depending on the size of the current block and / or the aspect ratio of the current block.

[0171] The image decoder can generate a candidate list for each sub-region for each geometric partitioning mode. In the process of generating the candidate list, the image decoder may utilize information (recovered pixel values, prediction modes, etc.) of the surrounding restored regions of the current block. Depending on the geometric partitioning mode, the range of the surrounding regions used in the process of generating the candidate list for each sub-region may differ. For example, as shown in the example of FIG. 9, for each geometric partitioning mode, the top region and the left region of the current block may be used in the process of generating the candidate list for each sub-region. In the example of FIG. 9, m and n are integers greater than or equal to 0 or 1, and when m and n are integers greater than or equal to 1, they may be identical. Alternatively, the surrounding restored regions used for each sub-region according to each geometric partitioning mode may be predefined according to an agreement between the image encoding device and the image decoder.

[0172] The image decoding device can rearrange geometric partition modes within the candidate list using the surrounding restored regions of the current block.

[0173] If both prediction blocks are not intra-predicted, the image decoder can rearrange the geometric partitioning modes as follows. If the sub-region is inter-predicted, the prediction block of the sub-region is generated within the reference picture of the current block according to the motion information.

[0174] As shown in the examples of FIGS. 10 and 11, the difference between the surrounding restored area of ​​the current block and the surrounding restored area of ​​the prediction block of each corresponding sub-region, i.e., the cost, can be calculated based on a loss function. In this case, as shown in FIGS. 10 (b) and (c), the partitioning boundary according to the geometric partitioning mode is extended from each sub-region to perform a weighted sum of the surrounding restored samples of the two sub-regions, and the cost between the weighted sum of the restored samples (Fig. 11 (b)) and the surrounding restored samples of the current block (Fig. 11 (a)) can be calculated. As a loss function, at least one of functions such as SAD (Sum of absolute differences), SATD (Sum of absolute transformed differences), MAE (Mean absolute error), MSE (Mean squared error), MRSAD (Mean removal SAD), MRSATD (Mean removal SATD), etc., may be used. Additionally, a weighted sum may not be performed during the process of calculating the cost. That is, based on the extended geometric partition boundary, surrounding reconstruction samples of sub-region A may exist on one side, and surrounding reconstruction samples of sub-region B may exist on the other side.

[0175] If at least one of the two prediction blocks is intra-predicted, the image decoder may rearrange the geometric partitioning modes as follows. If a sub-region is intra-predicted, the sub-region and the prediction block of the sub-region exist at the same location.

[0176] As surrounding restoration samples of the sub-region where intra prediction is performed, a reference line of the restored region can be used, as shown in FIG. 12. The image decoder can predict the restored region according to the intra prediction mode using the reference line of the restored region, and compare the predicted samples of the restored region with the surrounding restoration samples of the sub-region. Subsequently, the cost estimation process can be performed in a manner similar to the method described above.

[0177] As described above, the image decoder can rearrange geometric partition modes by calculating the cost for each geometric partition mode and sorting the costs in ascending or descending order. The image decoder can parse geometric partition information and derive the geometric partition mode of the current block based on the geometric partition information from the rearranged geometric partition modes.

[0178] As shown in FIG. 8b, when geometric partitioning information and information on the prediction mode of each sub-region are determined according to a single piece of information, the image decoder can calculate the cost according to each geometric partitioning mode by also considering the combination of the prediction modes of each sub-region during the aforementioned geometric mode rearrangement process.

[0179] After the geometric partitioning mode of the current block and the prediction mode of each sub-region are determined, the image decoder can determine weighted sum information for generating the final prediction block, as shown in FIG. 8a or FIG. 8b.

[0180] When geometric partitioning-based prediction is applied to the current block, the final prediction sample P of the current block GPM To generate (i,j), prediction samples from two sub-regions (prediction block P of sub-region A predicted according to the prediction mode of sub-region A) A Prediction block P of sub-region B predicted according to (i,j) and the prediction mode of sub-region B B The weighted sum of (i,j)) can be performed as in Equation 2.

[0181]

[0182] In Equation 2, w(i,j) may be a weighted sum matrix composed of the weighted sum coefficients (i.e., weights) of the current block. Each weighted sum coefficient is between 0 and 2, determined by the weighted sum method and the weighted sum region. n It is an integer less than or equal to the following. In this case, n can be determined according to the weighted sum method.

[0183] FIG. 13 is an exemplary diagram illustrating a weighted sum method and the acquisition of a weighted sum region according to one embodiment of the present disclosure.

[0184] The image decoding device can obtain information on the weighted sum method of the current block and the weighted sum area, as shown in the example of FIG. 13.

[0185] In FIG. 13 (a), the image decoder parses a flag indicating whether to perform a linear weighted-sum and, depending on the parsed flag, can determine a linear weighted-sum or a difference-based weighted-sum as the weighted-sum method. The image decoder can determine the weighted-sum region by rearranging the weighted-sum region candidates and parsing information indicating the weighted-sum region of the current block among the rearranged candidates.

[0186] In FIG. 13(b), the image decoder parses a flag indicating whether to perform a linear weighted sum, and can determine a linear weighted sum or a difference-based weighted sum as the weighted sum method according to the parsed flag. Subsequently, the image decoder can derive a weighted sum region.

[0187] In (c) of FIG. 13, the image decoder may determine whether to perform a linear weighted sum or a difference-based weighted sum. The image decoder may determine the weighted sum region by rearranging the weighted sum region candidates and parsing information indicating the weighted sum region of the current block among the rearranged candidates.

[0188] In (d) of FIG. 13, the image decoder may determine whether to perform a linear weighted sum or a difference-based weighted sum. Afterward, the image decoder may determine a weighted sum region.

[0189] As shown in (a) and (b) of FIG. 13 above, the image decoding device can parse a flag indicating whether to perform a linear weighted sum. If the parsed flag is true, a linear weighted sum may be performed. If the parsed flag is false, a difference-based weighted sum may be performed.

[0190] FIG. 14 is an exemplary diagram showing weighted sum regions and weighted sum coefficients according to one embodiment of the present disclosure.

[0191] In linear weighted sums, the weighted sum coefficients are linearly arranged within the weighted sum region according to the geometric partitioning boundaries. For example, when an 8×8 current block is predicted according to geometric partitioning-based prediction, the weighted sum matrix for the weighted sum of the prediction blocks of each sub-region can be constructed as in Fig. 14 (a), where n=3 in Equation 2 and both weighted sum regions (α and β in Fig. 7 (c)) are each 3. When n=3 in Equation 2 and the weighted sum region is each 1, the weighted sum matrix can be constructed as in Fig. 14 (b). As another example, when n=3 in Equation 2 and the weighted sum regions are 3 and 1, respectively, the weighted sum matrix can be constructed as in Fig. 14 (c). In the illustration of Fig. 14, the region where the weighted sum coefficient is not 0 or 8 represents the weighted sum region.

[0192] FIG. 15 is an exemplary diagram showing a weighted sum region according to one embodiment of the present disclosure.

[0193] In difference-based weighted sums, each weighted sum coefficient within the weighted sum region of the weighted sum matrix is ​​prediction block P A (i,j) and P BIt can be determined based on the difference between pixel values ​​of (i,j). For example, when geometric partitioning-based prediction is applied to a current block of size 8×8 and difference-based weighted sum is performed, and when the weighted sum area (α, β in Fig. 7(c)) is 3 each, the weighted sum area within the weighted sum matrix can be determined as in Fig. 15(a). Also, when the weighted sum area is 1 each, the weighted sum area within the weighted sum matrix is ​​determined as in Fig. 15(b), and when the weighted sum area is 3 and 1 each, the weighted sum area within the weighted sum matrix can be determined as in Fig. 15(c).

[0194] For example, the weighted sum coefficients within the weighted sum region in (a) to (c) of FIG. 15 can be determined according to at least one of Equation 3 or Equation 4.

[0195]

[0196]

[0197] In Equations 3 and 4, (i,j) represent the position within the weighted sum region of the weighted sum matrix, respectively. The Clip3(P,Q,R) function applies a clipping operation to the R value using the minimum value P and the maximum value Q.

[0198] The weighted sum matrix w(i,j) of the current block is w A (i,j) and w B It can be determined by at least one of (i,j). For example, w A (i,j) is determined as the weighted sum matrix w(i,j) of Equation 2, or w B (i,j) can be determined as the weighted sum matrix w(i,j) of Equation 2. Or, w A (i,j) represents the weighted sum coefficients applied to the weighted sum region within sub-region A from the weighted sum boundary in FIG. 15, and w B(i,j) may represent weights applied to the weighted sum region within sub-region B from the weighted sum boundary in FIG. 15. In Equations 3 and 4, t and r may be values ​​defined according to an agreement between the image encoding device and the image decoder. Alternatively, they may be values ​​determined by the size of the current block, the aspect ratio, or the bit depth of the input image. For example, in Equations 3 and 4, n, t, and r may be 6, 38, and 16, respectively.

[0199] As another example, after geometric partitioning-based prediction is applied to the current block and the decision to perform a linear weighted sum is determined, the image decoder can determine the weighted sum region. As shown in FIG. 13 (a) and (c), the weighted sum region can be determined by rearrangement and parsing. Depending on the embodiment, the rearrangement process may be omitted. Also, as shown in FIG. 13 (b) and (d), the weighted sum region may be implicitly derived. As another example, whether to perform rearrangement of weighted sum region candidates may be determined according to the weighted sum method. For example, rearrangement of weighted sum region candidates may be performed only when a linear weighted sum is performed, or rearrangement of weighted sum region candidates may be performed only when a difference-based weighted sum is performed.

[0200] The rearrangement process of candidate weighted sum regions is described below. Depending on the size, aspect ratio, and / or weighted sum method of the current block, each weighted sum region (α, β in Fig. 7(c)) can be determined, and candidates for combinations of two weighted sum regions can be defined. When the rearrangement process is performed, the surrounding restored region of the current block and the surrounding restored region of the prediction block of each corresponding sub-region can be used, as exemplified in Figs. 10 and 11. At this time, the surrounding restored region of the current block and the surrounding restored region of the prediction block of each corresponding sub-region can be determined in one of the forms shown in Fig. 16(a) to (d) according to the geometric partitioning boundary of the current block. In Fig. 16, a and b may be integers greater than or equal to 0 or 1. m and n are integers greater than or equal to 0 or 1, and may be identical if both m and n are greater than or equal to 1. As another example, a may be W and b may be H.

[0201] For each weighted sum region candidate, the cost between the surrounding restored region of the current block (Fig. 11 (a)) and the surrounding restored region of the prediction block of each sub-region where the weighted sum was performed (Fig. 11 (b)) is calculated, and the weighted sum region candidates can be rearranged according to the ascending or descending order of the costs. At this time, as in the example of Fig. 10, the image decoder can extend the geometric partition boundary to the surrounding restored region of the prediction block of each sub-region, and the weighted sum matrix for each weighted sum region can also be extended to the surrounding restored region of the prediction block of each sub-region.

[0202] FIGS. 17a to 17c are exemplary diagrams illustrating the cost calculation of weighted sum region candidates according to one embodiment of the present disclosure.

[0203] As an example, among the weighted sum region candidates associated with a current block having a size of 16×8, for a candidate having a weighted sum region such as that shown in FIG. 17a, the image decoder may extend the geometric partition boundary and extend the weighted sum matrix to the surrounding reconstructed region. Using the surrounding reconstructed samples of the prediction block of each sub-region and the extended weighted sum matrix, weighted sum reconstructed samples may be generated as shown in FIG. 17b. When the sub-region is inter-predicted, the prediction block of the sub-region is generated within the reference picture of the current block according to motion information. When the sub-region is intra-predicted, the sub-region and the prediction block of the sub-region exist at the same location. In this case, the weighted sum coefficients for generating the weighted sum region of the surrounding reconstructed region of sub-region A and the surrounding reconstructed region of sub-region B may be configured as the weighted sum matrix in FIG. 17a. For example, when a linear weighted sum is performed, the weighted sum coefficients may be configured as shown in the example of FIG. 14. When a difference-based weighted sum is performed, the weighted sum coefficients can be constructed based on Equations 3 and 4 according to FIG. 15. The cost between the weighted sum reconstructed samples as in FIG. 17b and the samples within the surrounding reconstructed region of the current block as in FIG. 17c can be calculated using a loss function. The cost of each weighted sum region candidate of the current block can be calculated according to the aforementioned method. At least one of functions such as SAD, SATD, MAE, MSE, etc., can be used as the loss function. Meanwhile, regarding the intra-predicted sub-region among the sub-regions of the current block, surrounding reconstructed samples of the intra-predicted sub-region can be generated based on the reference line exemplified in FIG. 12.

[0204] As an example, after the rearrangement process of weighted sum region candidates is performed, the weighted sum region information to be used for the current block among the arranged candidates can be determined by signaling / parsing. At this time, m candidates are rearranged, and the weighted sum region information related to the top n candidates can be signaled / parsed. For example, after 10 candidates are rearranged, a flag can be signaled / parsed as information indicating the weighted sum region of the current block among the top 2 candidates. Or, after 10 candidates are rearranged, an index can be signaled / parsed as information indicating the weighted sum region of the current block among the top 4 candidates.

[0205] The rearrangement process of weighted sum region candidates may be omitted. In this case, the weighted sum region information of the current block can be determined by signaling / parsing the weighted sum region information. For example, each weighted sum region (α, β in Fig. 7(c)) may be signaled / parsed respectively. Alternatively, candidates having the same or different values ​​for each weighted sum region may be configured, and an index may be signaled / parsed as information indicating the weighted sum region of the current block among the candidates. Alternatively, candidates having the same shape for each weighted sum region may be configured, and an index may be signaled / parsed as information indicating the weighted sum region of the current block among the candidates. In this case, the weighted sum region represented by each index may be determined differently based on the size and / or aspect ratio of the current block. That is, the weighted sum region candidates may be configured differently based on at least one of the size or aspect ratio of the current block.

[0206] As another example, a weighted sum region may be implicitly derived in the image decoder. According to the examples of FIGS. 17a through 17c, the image decoder performs a rearrangement of weighted sum candidates, and among the rearranged candidates, the weighted sum region of the candidate with the smallest or largest cost may be used as the weighted sum region of the current block. When a difference-based weighted sum is used, prediction blocks (P of two sub-regions) are used to determine the weighted sum coefficients applied to the weighted sum region. A (i,j) and P B (i,j)) can be used.

[0207] As shown in (c) and (d) of FIG. 13 above, whether to perform a linear weighted sum in the image decoding device can be induced.

[0208] As an example, as shown in the examples of FIGS. 17a to 17c, whether to perform a linear weighted sum can be derived using the surrounding restored area of ​​the current block and the surrounding restored area of ​​the prediction block of each sub-area. In this case, during the process of deriving whether to perform a linear weighted sum, the weighted sum area can be used as a fixed value. For example, α and β representing the weighted sum area can each be fixed to 3. The fixed weighted sum area can be determined based on the size, aspect ratio, etc. of the current block.

[0209] For a fixed weighted sum region, weighted sum samples (Fig. 17b) related to the surrounding restored regions of the prediction blocks of the two sub-regions can be calculated by expanding the weighted sum matrix calculated according to a linear weighted sum as in Fig. 14. Subsequently, a first cost is calculated between the samples of the surrounding restored region of the current block (Fig. 17c) and the weighted sum samples. Additionally, weighted sum samples related to the surrounding restored regions of the prediction blocks of the two sub-regions can be calculated by expanding the weighted sum matrix calculated according to a difference-based weighted sum using Equations 3 and 4 according to Fig. 15. Subsequently, a second cost is calculated between the samples of the surrounding restored region of the current block and the weighted sum samples. By comparing the first cost and the second cost, the weighted sum method that generates a smaller difference for the surrounding restored region of the current block can be determined as the weighted sum method of the current block. At this time, an extended weighted sum matrix consisting only of weight coefficients calculated according to FIG. 15 and Equation 3, and an extended weighted sum matrix consisting only of weight coefficients calculated according to FIG. 15 and Equation 4 may be included as candidates for determining whether to perform a linear weighted sum.

[0210] For example, when a difference-based weighted sum is performed in FIG. 13 (a) and (b), the weighted sum coefficients within the weighted sum region can be determined based on FIG. 15, Equation 3, and Equation 4. For example, the weighted sum coefficients within sub-region A based on the weighted sum boundary can be calculated according to Equation 3, and the weighted sum coefficients within sub-region B based on the weighted sum boundary can be calculated according to Equation 4. Alternatively, all weighted sum coefficients within the weighted sum region may consist only of coefficients calculated according to Equation 3, or all weighted sum coefficients within the weighted sum region may consist only of coefficients calculated according to Equation 4. Alternatively, at some locations within the weighted sum region, the weighted sum coefficients may be calculated according to Equation 3, and at the remaining locations, the weighted sum coefficients may be calculated according to Equation 4. The above-described process may be implicitly determined in the image decoding device according to the embodiment.

[0211] As another example, to construct a weighted sum matrix using coefficients calculated according to Equation 3 at some locations within the weighted sum region and coefficients calculated according to Equation 4 at the remaining locations, the coefficients (weights) at each location are the prediction blocks (P) of the two sub-regions A (i,j) and P B It can be determined as follows using (i,j).

[0212] For the weighted sum region of the current block, P in the weighted sum region within sub-region A A (i,j) and P B (i,j), P in the weighted sum region within sub-region B A (i,j) and P B Using (i,j), P based on the geometric partition boundary A Change in (i,j) and P BThe change in (i,j) can be calculated. In this case, the comparison unit may be the entire block or each sub-block divided into M parts. For example, the change may be the difference between the statistical values ​​(mean or median, etc.) of the predicted samples of the weighted sum regions divided based on geometric partitioning boundaries. P A The change in (i,j) is P within the weighted sum region of sub-region A. A The average value of (i,j) and the weighted sum of sub-region B, P within the region A It could be the difference between the average values ​​of (i,j). P B The change in (i,j) is P within the weighted sum region of sub-region A. B The average value of (i,j) and the weighted sum of sub-region B, P within the region B It can be the difference between the average values ​​of (i,j).

[0213] Meanwhile, in a difference-based weighted sum where the weighting coefficients (weights) are adaptively determined based on location, the weighted sum region must first be determined to calculate the amount of change.

[0214] The weighted sum region can be determined as described above. However, if a rearrangement process of the weighted sum region candidates as described above is performed to determine the weighted sum region, the cost for the weighted sum region candidates used in the rearrangement process cannot be calculated because the weights have not yet been determined. In this case, the weights can be determined using either Equation 3 or Equation 4 in a fixed weighted sum region (e.g., α and β are each 3 in Fig. 7 (c)). Alternatively, the cost can be calculated using at least two of the following methods: a weight determination method using Equation 3, a weight determination method using Equation 4, or a method of determining weights by applying Equation 3 and Equation 4, respectively, to the weighted sum region of sub-regions A and B, and the method with the smallest cost can be selected. The weights can be determined according to the selected method.

[0215] Subsequently, the cost for each weighted sum region candidate is calculated using the determined weights, and the weighted sum region candidates can be rearranged based on the costs. The final weighted sum region can be set as the weighted sum region candidate with the smallest cost. Alternatively, among all or part of the rearranged weighted sum region candidates, the candidate indicated by the index decoded (or parsed) from the bitstream may be set as the final weighted sum region.

[0216] After the weighted sum region for applying the difference-based weighted sum is determined, P A Change in (i,j) and P B The change in (i,j) is calculated, and based on the calculated change, the weight at each position within the weighted sum region can be adaptively determined.

[0217] As an example, as in FIG. 18(a), two prediction blocks (P A , P B To generate prediction samples that approximate the optimal prediction samples using ), geometric partitioning is performed on the current block as shown in FIG. 18(b). When the corresponding weighted sum region is determined, two prediction blocks (P along the geometric partitioning boundary) are formed as shown in FIG. 18(b). A , P B The boundary between ) can be determined. In this case, compared to a weight determination method that uses Equation 3 or Equation 4 for the entire weighted sum area within sub-region A and Equation 3 or Equation 4 for the entire weighted sum area within sub-region B, a weight determination method that uses Equation 3 for some areas within the 'weighted sum area of ​​sub-region A' and Equation 4 for the remaining areas, and a weight determination method that uses Equation 3 for some areas within the 'weighted sum area of ​​sub-region B' and Equation 4 for the remaining areas, can generate prediction samples that are more similar to the optimal prediction samples.

[0218] In addition, if a weight determination method using Equation 3 is applied to some areas within the 'weighted sum area of ​​sub-region A' and Equation 4 is applied to the remaining areas, a weight determination method using only Equation 3 or Equation 4 may be considered for the 'weighted sum area of ​​sub-region B'. Alternatively, if a weight determination method using Equation 3 is applied to some areas within the 'weighted sum area of ​​sub-region B' and Equation 4 is applied to the remaining areas, a weight determination method using only Equation 3 or Equation 4 may be considered for the 'weighted sum area of ​​sub-region A'. In conclusion, without a separate signaling / parsing process, weights can be implicitly and adaptively determined in the image decoder based on the characteristics (e.g., change amount) of the predicted samples.

[0219] As an example, P based on the geometric partition boundary A P is greater than the change in (i,j). B The change in (i,j) is large, and P B If the change in (i,j) is greater than the threshold, P in the weighted sum region within sub-region A A (i,j) and P B A comparison between (i,j) can be performed. In the weighted sum region within sub-region A, P A (i,j) and P B If the difference value between (i,j) is greater than the threshold value, the weighted sum coefficient for that location can be calculated according to Equation 4. In addition, in the weighted sum region within sub-region A, P A (i,j) and P BWhen the difference value of (i,j) is smaller than the threshold value, the weighted sum matrix can be constructed by calculating the weighted sum coefficient of the corresponding position according to Equation 3. All coefficients of the weighted sum region within sub-region B can be calculated according to Equation 3. Alternatively, all coefficients of the weighted sum region within sub-region B can be calculated according to Equation 4. In this case, the threshold values ​​used in the comparison process can be determined in advance according to an agreement between the video encoding device and the video decoder. Additionally, the threshold values ​​can be determined based on the prediction mode of each prediction block, the size of the block, etc.

[0220] The image decoder can predict the current block based on information obtained by deriving and / or parsing geometric division information of the current block, weighted sum method and weighted sum region information, and prediction information of each sub-region. For example, the image decoder can generate a prediction block of each sub-region and generate a final prediction block of the current block by weighting the prediction blocks according to Equation 2.

[0221] The operations related to inverse transformation and restoration are described below.

[0222] In relation to the inverse transformation, the entropy decoder (510) decodes the transformation coefficients. When a second transformation is applied, the entropy decoder (510) decodes the quantized second transformation coefficients. When a second transformation is not applied, the entropy decoder (510) decodes the quantized first transformation coefficients. The entropy decoder (510) parses information such as the quantization method and quantization parameter information.

[0223] The inverse quantization unit (520) generates inverse quantization conversion coefficients by inversely quantizing the decoded quantization conversion coefficients based on information such as the quantization method and quantization parameter information.

[0224] The inverse transform unit (530) inversely transforms the TU expressed in inverse quantization transform coefficients to generate residual samples. As shown in FIG. 6, the inverse transform unit (530) may include an inverse transform unit determination unit (610), an inverse transform kernel determination unit (612), and an inverse transform execution unit (614).

[0225] The inverse transformation unit determination unit (610) can determine one TU or a sub-block formed by dividing one TU into multiple parts as the target of transformation. For example, the TU may be the entire current block that is the target of prediction, or a part of the current block.

[0226] The inverse transformation kernel determination unit (612) can determine a separable vertical and horizontal direction first-order inverse transformation kernel and / or an inseparable second-order inverse transformation kernel, or determine an inseparable first-order inverse transformation kernel.

[0227] The inverse transformation execution unit (614) can inverse transform the inverse quantized transformation coefficients using the inverse transformation kernel determined by the inverse transformation kernel determination unit (612). The inverse transformation execution unit (614) can perform inseparable first-order inverse transformation, inseparable second-order inverse transformation, first-order inverse transformation, etc.

[0228] As an example, whether to perform an inseparable first-order inverse transformation and whether to perform an inseparable second-order inverse transformation can be determined using signaling / parsing, and can be implicitly determined based on the size of the current transformation block, etc. Hereinafter, "current block," "current transformation block," and "current transformation unit block" have the same meaning and can be used interchangeably.

[0229] Figures 19a and 19b are flowcharts illustrating a method for determining an inverse transformation kernel.

[0230] When nst_idx is 0, the image decoder can determine an inverse transform kernel as shown in FIG. 19b. Here, nst_idx represents a non-separable transform index and may correspond to lfnst_idx, i.e., a low-frequency non-separable transform index, in the next-generation VVC, the Enhanced Compression Model (ECM).

[0231] In the case where an ISP (Intra sub-partition) is applied during the prediction process of the current conversion block, or where a MIP (Matrix-based intra prediction) is not used during the prediction process of the current conversion block and the conversion kernel is determined implicitly (ImplicitMTS), the image decoder determines the vertical and horizontal first-order conversion kernels as one of {DCT2 / DCT2, DCT2 / DST7, DST7 / DCT2, DST7 / DST7} according to the size of the current TU.

[0232] In the case where the above information is not fully satisfied and the transformation is performed only on a sub-region within the transformation block (SBT, Subblock Transform), the image decoder determines the vertical and horizontal first transformation kernels as one of {DCT2 / DCT2, DCT8 / DST7, DST7 / DCT8, DST7 / DST7} based on the size of the current TU and the location of the area where the transformation is performed within the current TU.

[0233] If all of the above information is not satisfied, it may be checked whether the transformation kernel is explicitly determined (ExplicitMTS). If the transformation kernel is not explicitly determined, the image decoder determines both the vertical and horizontal first transformation kernels as DCT2. On the other hand, if the transformation kernel is explicitly determined, mts_idx is parsed, and the first transformation kernel can be determined according to the parsed index value. At this time, based on a comparison between the sum of the absolute values ​​of the transformation coefficients within the transformation block and a preset threshold, the range of the index, that is, the size of the kernel set, can be determined as one of 1, 4, or 6.

[0234] When the transformation kernel is explicitly determined, the transformation kernel may be determined differently depending on the prediction technique of the current transformation block as follows. When an intra prediction technique is used, the image decoder determines a kernel set based on the size of the current TU and the prediction mode index, and parses the index indicating one of the transformation kernels (i.e., a transformation kernel pair) within the kernel set. The image decoder obtains information on the vertical and horizontal first-order transformation kernels indicated by the corresponding index from a Look-up Table (LUT) defined according to an agreement between the image encoder and the image decoder, and uses the obtained information as the first-order transformation kernel of the current TU. In this case, if both the height and width of the current TU are less than or equal to 16 and mts_idx is 3, the vertical and horizontal first-order transformation kernels are determined as the Identity Transform (IDTR) according to specific conditions. Meanwhile, the prediction mode index indicates the directional prediction mode. If the intra prediction mode is a non-directional mode, the image decoder may derive the directionality of the prediction signals and use a directional prediction mode mapped to the derived directionality. Additionally, if predicted by MIP, the image decoder may map the prediction mode to a planar mode.

[0235] When an inter-prediction technique is used, the image decoder determines the vertical and horizontal first-order transformation kernels among DCT8 and DST7 based on mts_idx. At this time, if the height and width of the current TU are both less than 16, the vertical and horizontal transformation kernels may be changed to the Kahunen-Loeve Transform (KLT) defined according to the agreement between the image encoder and the image decoder.

[0236] If nst_idx is not 0, the image decoder can determine an inverse transform kernel as shown in FIG. 19a. Based on the size of the current TU, it is implicitly determined whether to perform an inseparable first-order inverse transform, or to perform an inseparable second-order inverse transform and a first-order inverse transform. For example, for blocks smaller than 16×16 (4×4, 4×8, 8×4, 8×8, 8×16, 16×8), an inseparable first-order inverse transform may be performed.

[0237] When the prediction technique of the current TU is directional prediction, that is, intra prediction technique and is in directional prediction mode, the image decoder determines a non-separable first-order transform kernel set or a non-separable second-order transform kernel set according to the nst_idx value. The image decoder determines a non-separable first-order transform kernel or a non-separable second-order transform kernel according to the directionality of the mode used for prediction of the current transform block.

[0238] If the prediction technique of the current transform block is not directional prediction, the image decoder determines a non-separable first-order transform kernel set or a non-separable second-order transform kernel set according to the nst_idx value. The image decoder derives the directionality of the prediction samples of the current transform block and determines a non-separable first-order transform kernel or a non-separable second-order transform kernel according to the derived directionality.

[0239] When a non-separable second-order inverse transform is performed, the image decoder determines a non-separable second-order transform kernel and performs the non-separable second-order inverse transform. Afterwards, a first-order inverse transform is performed. At this time, as the vertical and horizontal first-order transform kernels, a fixed kernel, such as the DCT2 kernel, may be implicitly used.

[0240] Hereinafter, a method for predicting the current block using an adaptive weighted sum in geometric partitioning-based prediction is described using the illustration of FIG. 20. The illustration of FIG. 20 can be performed by an image encoding device and an image decoding device. FIG. 20 is described based on the image encoding device, and if necessary, the operation by the image decoding device is additionally described.

[0241] FIG. 20 is a flowchart illustrating a method for predicting a current block according to one embodiment of the present disclosure.

[0242] The video encoding device obtains an index indicating a combination of information on the geometric division mode of the current block and information on the prediction mode of each sub-region of the current block (S2000). Here, each sub-region can be created by dividing the current block.

[0243] The video encoding device can obtain an index indicating a combination from a higher level. Alternatively, the video encoding device can determine the index in terms of rate distortion optimization. The video encoding device can encode the index. The video decoder can decode the index indicating a combination from the bitstream.

[0244] The video encoding device forms a list containing combinations of geometric partitioning mode candidates and prediction mode candidates for each sub-region (S2002).

[0245] Candidate prediction modes for each sub-region may include inter-prediction, affine mode, IBC prediction, IntraTMP mode, and / or intra-prediction.

[0246] The video encoding device rearranges the combinations in the list (S2004).

[0247] The image encoding device can calculate the cost of each combination based on geometric partitioning mode candidates, prediction mode candidates for each sub-region, and surrounding reconstructed regions of the current block. The image encoding device can rearrange the combinations based on the costs of the combinations.

[0248] As an example, if the prediction mode candidates for each sub-region are inter-prediction, affine mode, IBC prediction, or IntraTMP mode, the image encoding device can calculate the cost for each combination as follows. The image encoding device can generate prediction blocks for each sub-region based on the geometric partitioning mode candidates for each combination and the prediction mode candidates for each sub-region of each combination. The image encoding device can calculate the cost between the surrounding region of the prediction block of each sub-region and the surrounding restored region of the current block using a loss function.

[0249] As another example, if the prediction mode candidate for each sub-region is an intra prediction mode, the image encoding device can calculate the cost for each combination as follows. The image encoding device can generate predicted samples of the surrounding reconstructed region by predicting the surrounding reconstructed region using the geometric partitioning mode candidate of each combination, the prediction mode candidate for each sub-region of each combination, and the reference line of the surrounding reconstructed region of the current block. The image encoding device can calculate the cost between the predicted samples and the samples within the surrounding reconstructed region using a loss function.

[0250] The video encoding device determines the geometric partitioning mode of the current block and the prediction mode of each sub-region of the current block from the rearranged list based on the index (S2006).

[0251] The video encoding device divides the current block into sub-regions based on a geometric division mode (S2008).

[0252] The video encoding device obtains a weighted sum method and a weighted sum region of the current block (S2010),

[0253] As an example, information indicating a weighted sum method may be signaled / parsed. An image encoding device may obtain a flag indicating a weighted sum method from a higher level. Alternatively, the image encoding device may determine the flag in terms of rate distortion optimization. The image encoding device may encode the flag. An image decoder may decode the flag indicating a weighted sum method from a bitstream. Depending on the flag indicating a weighted sum method, the image encoding device may determine a weighted sum method applied to the weighted sum region as a linear weighted sum or a difference-based weighted sum.

[0254] As another example, a weighted sum method can be derived as follows. An image encoding device sets the weighted sum region as a fixed region. The image encoding device calculates a first cost using weighted sum coefficients based on a linear weighted sum based on the fixed region, and calculates a second cost using weighted sum coefficients based on a difference-based weighted sum based on the fixed region. Based on the first cost and the second cost, the image encoding device can determine the weighted sum method as a linear weighted sum or a difference-based weighted sum.

[0255] As an example, information indicating a weighted sum region can be signaled / parsed. The video encoder constructs weighted sum region candidates for the current block using combinations of weighted sum region candidates for each sub-region. The video encoder calculates the cost of each weighted sum region candidate and rearranges the weighted sum region candidates based on the costs of the weighted sum region candidates. The video encoder obtains an index indicating the weighted sum region of the current block from a higher level. Alternatively, the video encoder may determine the index in terms of rate distortion optimization. The video encoder may encode the index. The video decoder may decode the index indicating the weighted sum region from the bitstream. Based on the index indicating the weighted sum region, the video encoder may determine the weighted sum region of the current block from the rearranged weighted sum region candidates.

[0256] As another example, a weighted sum region can be derived as follows. The video encoding device constructs weighted sum region candidates for the current block using combinations of weighted sum region candidates for each sub-region. The video encoding device calculates the cost of each weighted sum region candidate and determines the weighted sum region candidate with the smallest cost as the weighted sum region of the current block.

[0257] In embodiments related to the determination of weighted sum regions, the cost of each weighted sum region candidate can be calculated as follows. The image encoding device generates weighted sum reconstructed samples by applying weighted sum coefficients corresponding to each weighted sum region candidate to the surrounding reconstructed samples of the prediction block of each sub-region. The image encoding device can calculate the cost between the weighted sum reconstructed samples and the samples within the surrounding pre-reconstructed region of the current block using a loss function.

[0258] The video encoding device determines the weighted sum coefficients of the sub-regions based on the weighted sum method and the weighted sum region (S2012).

[0259] When the weighted sum method is a linear weighted sum, the image encoding device can determine the weighted sum coefficients of the sub-regions linearly according to the weighted sum region, as in the example of FIG. 14.

[0260] When the weighted sum method is a difference-based weighted sum, the image encoding device can determine the weighted sum coefficients of the sub-regions according to the weighted sum region by using Equations 3 and 4 according to FIG. 15.

[0261] For example, as shown in FIG. 18 (b), the current block is divided into sub-region A and sub-region B according to geometric division boundaries, and the weighted sum area of ​​the current block includes the weighted sum area within sub-region A and the weighted sum area within sub-region B, and a difference-based weighted sum is applied to the weighted sum area within sub-region A and the weighted sum area within sub-region B.

[0262] The image encoding device, based on the geometric division boundary, has a first change amount (P) of the predicted samples of sub-region A. A Change in (i,j), and second change in predicted samples of sub-region B (P B The amount of change (i,j) is calculated. The image encoding device can adaptively determine the weighted sum coefficient at each position within the weighted sum region of the current block based on the first change amount and the second change amount.

[0263] For example, if the second change amount is greater than the first change amount and the second change amount is greater than a preset threshold, the video encoding device calculates the difference between the predicted sample of sub-region A and the predicted sample of sub-region B at each location within the weighted sum area of ​​sub-region A. Based on the comparison result between the calculated difference and the preset threshold, the video encoding device can adaptively determine the weighted sum coefficients at each location within the weighted sum area of ​​sub-region A using Equation 3 or Equation 4. At this time, the weighted sum coefficients within the weighted sum area of ​​sub-region B may be determined using only one of Equation 3 or Equation 4.

[0264] The video encoding device generates prediction blocks of sub-regions based on the prediction mode of each sub-region, and generates a final prediction block of the current block by weighting the prediction blocks based on weighted sum coefficients (S2014).

[0265] Subsequently, the video encoding device can generate a residual block by subtracting the final prediction block from the current block. The video encoding device can generate transformation coefficients by applying transformation / quantization to the residual block and encode the generated transformation coefficients.

[0266] The image decoder can decode the quantized transform coefficients of the current block from the bitstream and generate a residual block by applying inverse quantization / inverse transform to the quantized transform coefficients. The image decoder can restore the current block by adding the prediction block and the residual block.

[0267] Although the flowcharts and timing diagrams in this specification describe each process as being executed sequentially, this is merely an illustrative explanation of the technical concept of one embodiment of the present disclosure. In other words, a person skilled in the art to which one embodiment of the present disclosure belongs may modify and adapt the flowcharts and timing diagrams in various ways, such as changing the order described in the flowcharts and timing diagrams or executing one or more of the processes in parallel, without departing from the essential characteristics of one embodiment of the present disclosure; therefore, the flowcharts and timing diagrams are not limited to a chronological order.

[0268] It should be understood that the exemplary embodiments described above may be implemented in many different ways. The functions or methods described in one or more examples may be implemented in hardware, software, firmware, or any combination thereof. It should be understood that the functional components described herein are labeled as "...unit" to particularly emphasize their implementation independence.

[0269] Meanwhile, the various functions or methods described in the present embodiment may be implemented as instructions stored in a non-transient recording medium that can be read and executed by one or more processors. A non-transient recording medium includes, for example, any type of recording device in which data is stored in a form readable by a computer system. For example, a non-transient recording medium includes storage media such as an EPROM (erasable programmable read-only memory), a flash drive, an optical drive, a magnetic hard drive, and a solid-state drive (SSD).

[0270] The above description is merely an illustrative explanation of the technical concept of the present embodiment, and a person skilled in the art to which the present embodiment belongs would be able to make various modifications and variations within the scope of the essential characteristics of the present embodiment. Accordingly, the present embodiments are intended to explain, not limit, the technical concept of the present embodiment, and the scope of the technical concept of the present embodiment is not limited by these embodiments. The scope of protection of the present embodiment shall be interpreted by the claims below, and all technical concepts within an equivalent scope shall be interpreted as being included within the scope of rights of the present embodiment.

[0271]

[0272] CROSS-REFERENCE TO RELATED APPLICATION

[0273] This patent application claims priority to Korean patent application No. 10-2024-0147698 filed on October 25, 2024, the entire contents of which are incorporated into this patent application by reference.

Claims

1. In a video decoding method for restoring the current block, A step of decoding an index indicating a combination of information on the geometric partitioning mode of the current block and information on the prediction mode of each sub-region of the current block, wherein each sub-region is generated by partitioning the current block; A step of constructing a list including combinations of geometric partitioning mode candidates and prediction mode candidates for each sub-region; A step of rearranging combinations within the above list; Based on the above index, a step of determining the geometric partitioning mode of the current block and the prediction mode of each sub-region of the current block from the rearranged list; and Predicting the current block based on the geometric division mode and the prediction mode of each sub-region A method including 2. In Paragraph 1, The step of predicting the current block above is, A step of dividing the current block into sub-regions based on the geometric division mode above; A weighted sum method and a step of obtaining the weighted sum region of the current block; A step of determining weighted sum coefficients of the sub-regions based on the above weighted sum method and the above weighted sum region; and A step of generating prediction blocks of the sub-regions based on the prediction mode of each sub-region and weighting the prediction blocks based on the weighted sum coefficients A method including 3. In Paragraph 1, A method in which the prediction mode candidates for each of the above sub-regions include inter-prediction and intra-prediction.

4. In Paragraph 1, The step of rearranging the above combinations is, A step of calculating the cost of each combination based on geometric partitioning mode candidates of each combination, prediction mode candidates of each sub-region of each combination, and surrounding restored regions of the current block; and A step of rearranging the combinations based on the costs of the combinations. A method including 5. In Paragraph 4, The step of calculating the cost of each of the above combinations is, If the prediction mode candidate for each of the above sub-regions is an inter-prediction, A step of generating a prediction block for each sub-region based on a geometric partitioning mode candidate of each combination and a prediction mode candidate of each sub-region of each combination; and Step of calculating the cost between the surrounding area of ​​the above prediction block and the surrounding restored area. A method including 6. In Paragraph 4, The step of calculating the cost of each of the above combinations is, If the prediction mode candidate for each of the above sub-regions is an intra prediction mode, A step of generating prediction samples of the surrounding restored region using a geometric partitioning mode candidate of each combination, a prediction mode candidate of each sub-region of each combination, and a reference line of the surrounding restored region; and A step of calculating the cost between the above-mentioned predicted samples and the samples within the surrounding restored area. A method including 7. In Paragraph 2, The step of obtaining the above-mentioned weighted sum method is, A step of decoding information indicating the weighted sum method from a bitstream; and A step of determining a weighted sum method applied to the weighted sum region as a linear weighted sum or a difference-based weighted sum according to information indicating the above weighted sum method. A method including 8. In Paragraph 7, The above difference-based weighted sum is, A method using weighted sum coefficients calculated based on the difference between the predicted samples of the sub-regions within the weighted sum region.

9. In Paragraph 2, The step of obtaining the above-mentioned weighted sum method is, A step of setting the above weighted sum region as a fixed region; A step of calculating a first cost using weighted sum coefficients according to a linear weighted sum based on the above fixed area; A step of calculating a second cost using weighted sum coefficients based on a difference-based weighted sum based on the above fixed area; and Step of determining the weighting method as the linear weighting or the difference-based weighting based on the first cost and the second cost. A method including 10. In Paragraph 2, The step of obtaining the above-mentioned weighted sum region is, A step of configuring weighted sum region candidates of the current block using a combination of weighted sum region candidates of each of the above sub-regions; A step of calculating the cost of each weighted sum region candidate; A step of rearranging the weighted sum region candidates based on the costs of the weighted sum region candidates; A step of decoding information indicating a weighted sum region of the current block; and A step of determining the weighted sum region of the current block from rearranged weighted sum region candidates based on information indicating the weighted sum region. A method including 11. In Paragraph 10, The step of calculating the cost of each of the above-mentioned weighted sum region candidates is, A step of generating weighted summed samples by applying weighted sum coefficients corresponding to each of the above-mentioned weighted sum region candidates to the surrounding restoration samples of the prediction block of each of the above-mentioned sub-regions; and A step of calculating the cost between the weighted sum of the restored samples and the samples within the surrounding restored area of ​​the current block. A method including 12. In Paragraph 2, The step of obtaining the above-mentioned weighted sum region is, A step of configuring weighted sum region candidates of the current block using a combination of weighted sum region candidates of each of the above sub-regions; A step of calculating the cost of each weighted sum region candidate; and Step of determining the candidate weighted sum region with the smallest cost as the weighted sum region of the current block A method including 13. In Paragraph 2, The step of determining the above weighted sum coefficients is, The above current block is divided into a first sub-region and a second sub-region according to a geometric division boundary, and the weighted sum area of ​​the above current block includes the weighted sum area of ​​the first sub-region and the weighted sum area of ​​the second sub-region, and when a difference-based weighted sum is applied to the weighted sum area of ​​the first sub-region and the weighted sum area of ​​the second sub-region, A step of calculating a first change amount of the prediction samples of the first sub-region and a second change amount of the prediction samples of the second sub-region based on the geometric division boundary; and A step of adaptively determining weighted sum coefficients at each location within the weighted sum region of the current block based on the first change amount and the second change amount. A method including 14. In Paragraph 13, The step of adaptively determining the above weighted sum coefficients is, If the above first change amount is greater than the above second change amount and the above second change amount satisfies a preset condition, A step of calculating the difference between the prediction sample of the first sub-region and the prediction sample of the second sub-region at each position within the weighted sum area of ​​the first sub-region; A step of adaptively determining a weighted sum coefficient at each position within the weighted sum region of the first sub-region using a preset first formula or a preset second formula according to the result of comparison between the above difference and a preset threshold; and A step of determining weighted sum coefficients within the weighted sum region of the second sub-region using only one of the above-preset first formula or the above-preset second formula. A method including 15. In Paragraph 13, The above first change amount is, A method for indicating the difference between the average value of the predicted samples of the first sub-region within the weighted sum region of the first sub-region and the average value of the predicted samples of the first sub-region within the weighted sum region of the second sub-region, based on the geometric division boundary.

16. In a video encoding method for encoding a current block, A step of obtaining an index indicating a combination of information on the geometric division mode of the current block and information on the prediction mode of each sub-region of the current block, wherein each sub-region is created by dividing the current block; A step of constructing a list including combinations of geometric partitioning mode candidates and prediction mode candidates for each sub-region; A step of rearranging combinations within the above list; A step of determining the geometric partitioning mode of the current block and the prediction mode of each sub-region of the current block from the rearranged list based on the above index; A step of predicting the current block based on the geometric division mode and the prediction mode of each sub-region; and Step of encoding an index indicating the above combination A method including 17. In Paragraph 13, The step of predicting the current block above is, A step of dividing the current block into sub-regions based on the geometric division mode above; A weighted sum method and a step of obtaining the weighted sum region of the current block; A step of determining weighted sum coefficients of the sub-regions based on the above weighted sum method and the above weighted sum region; and A step of generating prediction blocks of the sub-regions based on the prediction mode of each sub-region and weighting the prediction blocks based on the weighted sum coefficients A method including 18. A method for providing video data to a video decoder, A step of encoding the above video data into a bitstream; and Step of transmitting the above bitstream to the above video decoder Includes, The step of encoding the above video data is, A step of obtaining an index indicating a combination of information on the geometric partitioning mode of the current block and information on the prediction mode of each sub-region of the current block, wherein each sub-region is created by partitioning the current block; A step of constructing a list including combinations of geometric partitioning mode candidates and prediction mode candidates for each sub-region; A step of rearranging combinations within the above list; A step of determining the geometric partitioning mode of the current block and the prediction mode of each sub-region of the current block from the rearranged list based on the above index; A step of predicting the current block based on the geometric division mode and the prediction mode of each sub-region; and Step of encoding an index indicating the above combination A method including

Images