Video coding method and device, and recording medium storing bitstream

Abstract

An image signal encoding / decoding method and device according to the present disclosure may comprise the steps of: partitioning the current block into a plurality of partitions on the basis of geometric partitioning, wherein the plurality of partitions include a first partition and a second partition; generating a first prediction block for the first partition by performing any one of intra prediction or inter prediction; generating a second prediction block for the second partition by performing any one of intra prediction or inter prediction; and generating a final prediction block for the current block through a weighted sum operation for the first prediction block and the second prediction block, wherein weights for the weighted sum operation may be determined on the basis of a lookup table.

Description

Video coding method and device, and a recording medium storing a bitstream

[0001] The present invention relates to a video signal processing method and apparatus. More specifically, it relates to a method and apparatus for performing geometric segmentation prediction using a LookUp Table (LUT).

[0002] VVC (Versatile Video Coding) improved coding efficiency by additionally supporting a geometric partitioning-based prediction mode, rather than just supporting grid-shaped block partitioning like the existing HEVC.

[0003] VVC supports 64 geometric partitioning modes. The present disclosure aims to support extended geometric partitioning modes by adding geometric partitioning modes.

[0004] Depending on the geometric partitioning mode, each region geometrically partitioned in the CU can be coded as intra prediction or inter prediction. For the two regions divided by geometric partitioning, intra prediction or inter prediction can be performed, and different prediction methods can be performed for the two regions, such as performing screen prediction on the top block and inter prediction on the bottom block.

[0005] The present disclosure aims to provide a geometric partitioning prediction method and apparatus based on a lookup table.

[0006] The present disclosure aims to provide a method and apparatus for the sequential restoration of sub-blocks in geometric partition prediction.

[0007] The present disclosure aims to provide a method and apparatus for determining a geometric partitioning mode for a sub-block in geometric partitioning prediction.

[0008] The present disclosure aims to provide a method and apparatus for determining a prediction mode for a sub-block in geometric partitioning prediction.

[0009] The technical problems to be solved in this disclosure are not limited to those mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art to which this disclosure belongs from the description below.

[0010] A video signal decoding method according to the present disclosure comprises the steps of: dividing a current block into a plurality of partitions based on geometric partitioning, wherein the plurality of partitions include a first partition and a second partition; generating a first prediction block for the first partition by performing either an intra prediction or an inter prediction; generating a second prediction block for the second partition by performing either an intra prediction or an inter prediction; and generating a final prediction block for the current block through a weighted sum operation on the first prediction block and the second prediction block, wherein the weights for the weighted sum operation may be determined based on a lookup table.

[0011] In the image signal decoding method according to the present disclosure, the weight is determined based on a value stored in the lookup table, and the value may be a constant quantized to the power of 2.

[0012] In the image signal decoding method according to the present disclosure, the weight is determined based on a value obtained by scaling a value stored in the lookup table, wherein the scaling can be performed by applying a predefined scaling factor to the value stored in the lookup table.

[0013] In the image signal decoding method according to the present disclosure, the weights are determined based on values ​​that adaptively adjust values ​​stored in the lookup table, wherein the adjustment may be performed based on at least one of the angle for performing the weighted sum operation or the aspect ratio of the current block.

[0014] In the image signal decoding method according to the present disclosure, the weight is determined based on a predetermined non-linear function, wherein the predetermined non-linear function may include at least one of a sigmoid or a trigonometric function.

[0015] In the image signal decoding method according to the present disclosure, the lookup table is composed of one or more lists, and an index for specifying the one or more lists may be signaled.

[0016] In the image signal decoding method according to the present disclosure, the size of the region where the weighted sum operation is performed may be determined to be one of 1 / 4, 1 / 2, 1, 2, or 4 times a predetermined parameter (τ).

[0017] In the image signal decoding method according to the present disclosure, the current block may be any one of a plurality of sub-blocks divided based on an intra-sub-partition mode.

[0018] A video signal encoding method according to the present disclosure comprises the steps of: dividing a current block into a plurality of partitions based on geometric partitioning, wherein the plurality of partitions include a first partition and a second partition; generating a first prediction block for the first partition by performing either an intra prediction or an inter prediction; generating a second prediction block for the second partition by performing either an intra prediction or an inter prediction; and generating a final prediction block for the current block through a weighted sum operation on the first prediction block and the second prediction block, wherein the weights for the weighted sum operation may be determined based on a lookup table.

[0019] A digital storage medium for storing a video bitstream according to the present disclosure, wherein the bitstream is encoded by a video signal encoding method comprising the steps of: dividing a current block into a plurality of partitions based on geometric partitioning, wherein the plurality of partitions include a first partition and a second partition; generating a first prediction block for the first partition by performing either an intra prediction or an inter prediction; generating a second prediction block for the second partition by performing either an intra prediction or an inter prediction; and generating a final prediction block for the current block through a weighted sum operation on the first prediction block and the second prediction block, wherein the weights for the weighted sum operation may be determined based on a lookup table.

[0020] According to the present disclosure, by providing a geometric partitioning prediction method and apparatus based on a lookup table, computational efficiency can be increased and rapid processing speed can be secured.

[0021] According to the present disclosure, by providing a method and apparatus for the sequential restoration of sub-blocks in geometric partitioning prediction, the prediction accuracy of sub-blocks of the rank after restoration can be improved by utilizing previously restored blocks.

[0022] According to the present disclosure, by providing a method and apparatus for determining a geometric partitioning mode for a sub-block in geometric partitioning prediction, signaling bits can be reduced and coding efficiency improved by utilizing a previously restored sub-block.

[0023] According to the present disclosure, by providing a method and apparatus for determining a prediction mode for a sub-block in geometric partitioning prediction, prediction accuracy can be improved by performing prediction on a sub-block basis.

[0024] The effects obtainable from the present disclosure are not limited to those mentioned above, and other unmentioned effects will be clearly understood by those skilled in the art to which the present disclosure belongs from the description below.

[0025] FIG. 1 is a block diagram showing an image encoding device according to the present disclosure.

[0026] FIG. 2 is a block diagram showing an image decoding device according to the present disclosure.

[0027] FIG. 3 is a flowchart of an image signal encoding method according to one embodiment of the present disclosure.

[0028] FIG. 4 is a drawing illustrating a geometric division according to the present disclosure.

[0029] FIG. 5 is a drawing illustrating a geometric division according to the present disclosure.

[0030] FIG. 6 is a diagram showing a non-linear function for deriving weights according to one embodiment of the present disclosure.

[0031] FIG. 7 is a diagram showing the size and weight of the region where a weighted sum operation is performed, according to one embodiment of the present disclosure.

[0032] FIG. 8 is a drawing illustrating a sub-block division according to the present disclosure.

[0033] FIG. 9 is a drawing illustrating a sub-block division according to the present disclosure.

[0034] FIG. 10 is a drawing illustrating a geometric division direction according to the present disclosure.

[0035] FIG. 11 is a drawing showing a case in which the endpoint of a geometric division for a lower block of a restoration line rank according to one embodiment of the present disclosure does not meet a lower block of a post-restoration rank.

[0036] FIG. 12 is a drawing illustrating a geometric division mode for a lower block of rank after restoration according to the present disclosure.

[0037] FIG. 13 is a drawing illustrating a geometric division mode for a lower block of rank after restoration according to the present disclosure.

[0038] FIG. 14 is a drawing illustrating restoration samples referenced by a sub-block according to the present disclosure.

[0039] FIG. 15 is a diagram showing the sub-block prediction order according to the availability of a reference sample, according to one embodiment of the present disclosure.

[0040] FIG. 16 is a diagram illustrating a case in which prediction using template matching is performed on a sub-block basis when predicting a sub-block based on geometric partitioning according to one embodiment of the present disclosure.

[0041] FIG. 17 is a diagram illustrating a case in which prediction using template matching is performed on a sub-block basis when predicting a sub-block based on geometric partitioning according to one embodiment of the present disclosure.

[0042] FIG. 18 is a drawing illustrating block division for prediction and transformation according to the present disclosure.

[0043] FIG. 19 is a flowchart of an image signal decoding method according to one embodiment of the present disclosure.

[0044] A video signal decoding method according to the present disclosure comprises the steps of: dividing a current block into a plurality of partitions based on geometric partitioning, wherein the plurality of partitions include a first partition and a second partition; generating a first prediction block for the first partition by performing either an intra prediction or an inter prediction; generating a second prediction block for the second partition by performing either an intra prediction or an inter prediction; and generating a final prediction block for the current block through a weighted sum operation on the first prediction block and the second prediction block, wherein the weights for the weighted sum operation may be determined based on a lookup table.

[0045] In the image signal decoding method according to the present disclosure, the weight is determined based on a value stored in the lookup table, and the value may be a constant quantized to the power of 2.

[0046] In the image signal decoding method according to the present disclosure, the weight is determined based on a value obtained by scaling a value stored in the lookup table, wherein the scaling can be performed by applying a predefined scaling factor to the value stored in the lookup table.

[0047] In the image signal decoding method according to the present disclosure, the weights are determined based on values ​​that adaptively adjust values ​​stored in the lookup table, wherein the adjustment may be performed based on at least one of the angle for performing the weighted sum operation or the aspect ratio of the current block.

[0048] In the image signal decoding method according to the present disclosure, the weight is determined based on a predetermined non-linear function, wherein the predetermined non-linear function may include at least one of a sigmoid or a trigonometric function.

[0049] In the image signal decoding method according to the present disclosure, the lookup table is composed of one or more lists, and an index for specifying the one or more lists may be signaled.

[0050] In the image signal decoding method according to the present disclosure, the size of the region where the weighted sum operation is performed may be determined to be one of 1 / 4, 1 / 2, 1, 2, or 4 times a predetermined parameter (τ).

[0051] In the image signal decoding method according to the present disclosure, the current block may be any one of a plurality of sub-blocks divided based on an intra-sub-partition mode.

[0052] A video signal encoding method according to the present disclosure comprises the steps of: dividing a current block into a plurality of partitions based on geometric partitioning, wherein the plurality of partitions include a first partition and a second partition; generating a first prediction block for the first partition by performing either an intra prediction or an inter prediction; generating a second prediction block for the second partition by performing either an intra prediction or an inter prediction; and generating a final prediction block for the current block through a weighted sum operation on the first prediction block and the second prediction block, wherein the weights for the weighted sum operation may be determined based on a lookup table.

[0053] A digital storage medium for storing a video bitstream according to the present disclosure, wherein the bitstream is encoded by a video signal encoding method comprising the steps of: dividing a current block into a plurality of partitions based on geometric partitioning, wherein the plurality of partitions include a first partition and a second partition; generating a first prediction block for the first partition by performing either an intra prediction or an inter prediction; generating a second prediction block for the second partition by performing either an intra prediction or an inter prediction; and generating a final prediction block for the current block through a weighted sum operation on the first prediction block and the second prediction block, wherein the weights for the weighted sum operation may be determined based on a lookup table.

[0054] Embodiments of the present invention are described in detail with reference to the drawings attached to this specification so that those skilled in the art can easily implement the invention. However, the present invention may be embodied in various different forms and is not limited to the embodiments described herein. Furthermore, in order to clearly explain the invention in the drawings, parts unrelated to the explanation have been omitted, and similar parts throughout the specification are denoted by similar reference numerals.

[0055] Throughout this specification, when a part is described as being 'connected' to another part, this includes not only cases where they are directly connected, but also cases where they are electrically connected with other elements in between.

[0056] Furthermore, throughout this specification, when a part is described as 'comprising' a certain component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components.

[0057] Additionally, terms such as "first," "second," etc., may be used to describe various components, but said components should not be limited by said terms. These terms are used solely for the purpose of distinguishing one component from another.

[0058] Additionally, in the embodiments relating to the device and method described herein, some components of the device or some steps of the method may be omitted. Also, the order of some components of the device or some steps of the method may be changed. Additionally, other components or other steps may be inserted into some components of the device or some steps of the method.

[0059] In addition, some components or steps of the first embodiment of the present invention may be added to or replace some components or steps of the second embodiment of the present invention.

[0060] Furthermore, the components shown in the embodiments of the present invention are depicted independently to represent different characteristic functions and do not imply that each component consists of separate hardware or a single software unit. That is, for convenience of explanation, each component is described by listing it as a separate component, and at least two of the components may be combined to form a single component, or a single component may be divided into multiple components to perform a function. Such integrated and separated embodiments of each component are also included within the scope of the present invention as long as they do not deviate from the essence of the invention.

[0061] FIG. 1 is a block diagram showing an image encoding device according to the present disclosure.

[0062] Referring to FIG. 1, the image encoding device (100) comprises a picture splitting unit (110), a prediction unit (120, 125), a conversion unit (130), a quantization unit (135), a reordering unit (160), and an entropy encoding unit (165).

[0063] It may include an inverse quantization unit (140), an inverse conversion unit (145), a filter unit (150), and a memory (155).

[0064] The picture splitting unit (110) can split the input picture into at least one processing unit. At this time, the processing unit may be a Prediction Unit (PU), a Transform Unit (TU), or a Coding Unit (CU). In the following embodiments of the present invention, the term "coding unit" may be used to mean a unit that performs coding, or a unit that performs decoding.

[0065] A prediction unit may be divided into shapes such as at least one square or rectangle of the same size within a single encoding unit, or it may be divided such that one of the prediction units within a single encoding unit has a different shape and / or size from another prediction unit. When generating a prediction unit that performs intra prediction based on an encoding unit, if it is not the minimum encoding unit, intra prediction can be performed without dividing into multiple prediction units NxN.

[0066] The prediction unit (120, 125) may include an inter prediction unit (120) that performs inter prediction or inter-frame prediction, and an intra prediction unit (125) that performs intra prediction or intra-frame prediction. It may determine whether to use inter prediction or perform intra prediction for a prediction unit, and determine specific information according to each prediction mode (e.g., intra prediction mode, motion vector, reference picture, etc.). The residual value (residual block) between the generated prediction block and the original block may be input to the conversion unit (130). In addition, the prediction mode information, motion vector information, etc. used for prediction may be encoded together with the residual value in the entropy encoding unit (165) and transmitted to the decoder.

[0067] The inter prediction unit (120) may predict a prediction unit based on information of at least one picture among the previous picture or the subsequent picture of the current picture, and in some cases, may predict a prediction unit based on information of a partially encoded area within the current picture. The inter prediction unit (120) may include a reference picture interpolation unit, a motion prediction unit, and a motion compensation unit.

[0068] In the reference picture interpolation unit, reference picture information is received from memory (155), and pixel information of integer pixels or less can be generated from the reference picture. In the case of luminance pixels, a DCT-based 8-tap interpolation filter with different filter coefficients can be used to generate pixel information of integer pixels or less in 1 / 4 pixel units. In the case of chrominance signals, a DCT-based 4-tap interpolation filter with different filter coefficients can be used to generate pixel information of integer pixels or less in 1 / 8 pixel units.

[0069] The motion prediction unit can perform motion prediction based on a reference picture interpolated by the reference picture interpolation unit. Various methods such as FBMA (Full search-based Block Matching Algorithm), TSS (Three Step Search), and NTS (New Three-Step Search Algorithm) can be used to calculate motion vectors. Based on the interpolated pixels, the motion vector can have motion vector values ​​in units of 1 / 2 or 1 / 4 pixels. The motion prediction unit can predict the current prediction unit by using different motion prediction methods. Various motion prediction methods such as the Skip method, Merge method, AMVP (Advance Motion Vector Prediction) method, and Intra Block Copy method can be used.

[0070] The intra prediction unit (125) can generate a prediction unit based on reference pixel information around the current block, which is pixel information within the current picture. If the surrounding block of the current prediction unit is a block that has performed inter prediction, and the reference pixel is a pixel that has performed inter prediction, the reference pixel included in the block that has performed inter prediction can be replaced with the reference pixel information of the surrounding intra prediction block. That is, if the reference pixel is not available, the unavailable reference pixel information can be replaced with at least one of the available reference pixels.

[0071] Additionally, a residual block can be generated that includes residual value information, which is the difference between the prediction unit that performed the prediction based on the prediction unit generated in the prediction unit (120, 125) and the original block of the prediction unit. The generated residual block can be input to the conversion unit (130).

[0072] In the transformation unit (130), the residual block containing residual value information of the prediction unit generated through the original block and the prediction unit (120, 125) can be transformed using a transformation method such as DCT (Discrete Cosine Transform), DST (Discrete Sine Transform), or KLT. Whether to apply DCT, DST, or KLT to transform the residual block can be determined based on the intra-prediction mode information of the prediction unit used to generate the residual block.

[0073] The quantization unit (135) can quantize the values ​​converted into the frequency domain in the conversion unit (130). The quantization coefficient may vary depending on the block or the importance of the image. The values ​​produced by the quantization unit (135) may be provided to the inverse quantization unit (140) and the reordering unit (160).

[0074] The reordering unit (160) can perform reordering of coefficient values ​​for quantized residual values.

[0075] The reordering unit (160) can convert two-dimensional block-shaped coefficients into one-dimensional vector forms through a coefficient scanning method. For example, the reordering unit (160) can convert the coefficients from DC to high-frequency range coefficients into one-dimensional vector forms by scanning using a Zig-Zag Scan method. Depending on the size of the conversion unit and the intra-prediction mode, a vertical scan that scans two-dimensional block-shaped coefficients in the column direction or a horizontal scan that scans two-dimensional block-shaped coefficients in the row direction may be used instead of the Zig-Zag Scan. That is, depending on the size of the conversion unit and the intra-prediction mode, it can be determined whether to use a Zig-Zag Scan, a vertical scan, or a horizontal scan.

[0076] The entropy encoding unit (165) can perform entropy encoding based on the values ​​calculated by the reordering unit (160). Entropy encoding can use various encoding methods, such as, for example, Exponential Golomb, CAVLC (Context-Adaptive Variable Length Coding), and CABAC (Context-Adaptive Binary Arithmetic Coding). In this regard, the entropy encoding unit (165) can encode residual value coefficient information of the encoding unit from the reordering unit (160) and the prediction unit (120, 125).

[0077] In the inverse quantization unit (140) and inverse transformation unit (145), the values ​​quantized in the quantization unit (135) are inverse quantized, and the values ​​transformed in the transformation unit (130) are inverse transformed. The residual value generated in the inverse quantization unit (140) and inverse transformation unit (145) can be combined with the prediction unit predicted through the motion estimation unit, motion compensation unit, and intra prediction unit included in the prediction unit (120, 125) to generate a reconstructed block.

[0078] The filter unit (150) may include at least one of a deblocking filter, an offset correction unit, and an Adaptive Loop Filter (ALF). The deblocking filter can remove block distortion caused by boundaries between blocks in the restored picture. The offset correction unit can correct the offset from the original image on a pixel-by-pixel basis for the image that has undergone deblocking. To perform offset correction for a specific picture, a method may be used in which pixels included in the image are divided into a certain number of regions, the region to be offset is determined, and the offset is applied to that region, or a method may be used in which the offset is applied by considering the edge information of each pixel. Adaptive Loop Filtering (ALF) may be performed based on a value obtained by comparing the filtered restored image with the original image. After dividing the pixels included in the image into a predetermined group, a filter to be applied to that group is determined, and filtering may be performed differentially for each group.

[0079] The memory (155) can store a restored block or picture produced through the filter unit (150), and the stored restored block or picture can be provided to the prediction unit (120, 125) when performing inter-prediction.

[0080] FIG. 2 is a block diagram showing an image decoding device according to the present disclosure.

[0081] Referring to FIG. 2, the image decoder (200) may include an entropy decoder (210), a reordering unit (215), an inverse quantization unit (220), an inverse transformation unit (225), a prediction unit (230, 235), a filter unit (240), and a memory (245).

[0082] When a video bitstream is input to a video encoder, the input bitstream can be decoded using the reverse procedure of the video encoder.

[0083] The entropy decoding unit (210) can perform entropy decoding in the opposite procedure to that which the entropy encoding unit of the image encoder performed. For example, various methods such as Exponential Golomb, CAVLC (Context-Adaptive Variable Length Coding), and CABAC (Context-Adaptive Binary Arithmetic Coding) can be applied in correspondence with the method performed in the image encoder.

[0084] The entropy decoding unit (210) can decode information related to intra-prediction and inter-prediction performed in the encoder.

[0085] The reordering unit (215) can perform reordering based on the method of reordering the entropy-decoded bitstream in the encoding unit in the entropy decoding unit (210). The coefficients expressed in the form of a one-dimensional vector can be reordered by restoring them to the form of two-dimensional block coefficients.

[0086] The inverse quantization unit (220) can perform inverse quantization based on the coefficient values ​​of the rearranged block and the quantization parameters provided by the encoder.

[0087] The inverse transform unit (225) can perform inverse transforms, i.e., inverse DCT, inverse DST, and inverse KLT, on the transforms, i.e., DCT, DST, and KLT, performed by the transform unit on the quantization result performed by the image encoder. The inverse transform can be performed based on the transmission unit determined by the image encoder. In the inverse transform unit (225) of the image decoder, a transformation technique (e.g., DCT, DST, KLT) can be selectively performed according to multiple pieces of information such as a prediction method, the size of the current block, and the prediction direction.

[0088] The prediction unit (230, 235) can generate a prediction block based on the prediction block generation information provided by the entropy decoding unit (210) and the previously decoded block or picture information provided by the memory (245).

[0089] As described above, when performing intra prediction or intra prediction identical to the operation in the video encoder, if the size of the prediction unit and the size of the transform unit are the same, intra prediction for the prediction unit is performed based on the pixels to the left of the prediction unit, the pixels to the top left, and the pixels to the top; however, if the size of the prediction unit and the size of the transform unit are different when performing intra prediction, intra prediction can be performed using reference pixels based on the transform unit. Additionally, intra prediction using NxN partitioning only for the minimum encoding unit may also be used.

[0090] The prediction unit (230, 235) may include a prediction unit determination unit, an inter prediction unit, and an intra prediction unit. The prediction unit determination unit receives various information, such as prediction unit information input from the entropy decoding unit (210), prediction mode information of the intra prediction method, and motion prediction related information of the inter prediction method, distinguishes the prediction unit in the current encoding unit, and can determine whether the prediction unit performs inter prediction or intra prediction.

[0091] The inter prediction unit (230) can perform inter prediction for the current prediction unit based on information included in at least one picture, either a previous picture or a subsequent picture, of the current picture containing the current prediction unit, using information required for inter prediction of the current prediction unit provided by the video encoder. To perform inter prediction, based on the encoding unit, it can determine whether the motion prediction method of the prediction unit included in the corresponding encoding unit is a Skip Mode, Merge Mode, AMVP Mode, or Intra Block Copy Mode.

[0092] The intra prediction unit (235) can generate a prediction block based on pixel information within the current picture. If the prediction unit is a prediction unit that has performed intra prediction, it can perform intra prediction based on the intra prediction mode information of the prediction unit provided by the image encoder. The intra prediction unit (235) may include an Adaptive Intra Smoothing (AIS) filter, a reference pixel interpolation unit, and a DC filter. The AIS filter is a part that performs filtering on the reference pixel of the current block, and can determine whether to apply the filter based on the prediction mode of the current prediction unit. AIS filtering can be performed on the reference pixel of the current block using the prediction mode of the prediction unit and the AIS filter information provided by the image encoder. If the prediction mode of the current block is a mode that does not perform AIS filtering, the AIS filter may not be applied.

[0093] The reference pixel interpolation unit can generate a reference pixel of an integer value or less by interpolating the reference pixel when the prediction mode of the prediction unit is a prediction unit that performs intra-prediction based on the pixel value interpolated from the reference pixel. If the prediction mode of the current prediction unit is a prediction mode that generates a prediction block without interpolating the reference pixel, the reference pixel may not be interpolated. The DC filter can generate a prediction block through filtering when the prediction mode of the current block is DC mode.

[0094] The restored block or picture may be provided to a filter unit (240). The filter unit (240) may include a deblocking filter, an offset correction unit, and an ALF.

[0095] Information regarding whether a deblocking filter has been applied to the corresponding block or picture can be received from the video encoder, and if a deblocking filter has been applied, information regarding whether a strong filter or a weak filter has been applied. The deblocking filter of the video decoder receives information related to the deblocking filter provided by the video encoder and can perform deblocking filtering on the corresponding block.

[0096] The offset correction unit can perform offset correction on the restored image based on the type of offset correction and offset value information applied to the image during encoding. ALF can be applied to the encoding unit based on information on whether ALF is applied and ALF coefficient information provided by the encoder. This ALF information can be provided included in a specific parameter set.

[0097] The memory (245) can store the restored picture or block so that it can be used as a reference picture or reference block, and can also provide the restored picture to the output unit.

[0098] Below, we will examine in detail the image signal encoding / decoding method proposed in the present disclosure.

[0099] First, the current block described in this disclosure may refer to any one of the multiple sub-blocks obtained by performing a predetermined sub-block division on a coding block. The multiple sub-blocks include a first sub-block and a second sub-block, and may optionally include one or more additional sub-blocks. Accordingly, the current block may be any one of the first sub-block, the second sub-block, or the optionally divided additional sub-blocks.

[0100] FIG. 3 is a flowchart of an image signal encoding method according to one embodiment of the present disclosure.

[0101] Referring to FIG. 3, the current block can be divided into multiple partitions based on geometric division (S310).

[0102] According to the present disclosure, geometric partitioning prediction or geometric partitioning-based prediction may mean a prediction method that divides a block into a plurality of partitions through a line having a geometric partitioning angle and an offset. A geometric partitioning mode may be determined according to a combination of the geometric partitioning angle and the offset.

[0103] Geometric partitioning prediction can be performed on square (NxN) or rectangular (NxM) shaped blocks having a predetermined size. The size of the block may be, for example, 8x8, 8x16, 16x8, 16x16, 16x32, 32x16, 32x32, 32x64, 64x32, 64x64, 8x32, 32x8, 16x64, 64x16, 8x64, or 64x8.

[0104] However, the figures disclosed above are merely examples and may have different values.

[0105] Geometric division of the current block can be performed based on a line having a geometric division angle and an offset. The geometric division angle may indicate the direction of the line and may be referred to as the geometric division direction. The offset may indicate a distance from the center of the block. Here, the block may be a coding block or a sub-block. That is, the offset may be a distance from the center of the coding block or a distance from the center of the sub-block.

[0106] Meanwhile, in the present disclosure, when sub-block partitioning is performed on a coding block for which geometric partitioning prediction is performed, prediction and transformation may be performed based on an intra prediction mode and / or an inter prediction mode.

[0107] That is, intra prediction or inter prediction can be performed on multiple partitions obtained by geometric partitioning. Alternatively, prediction and transformation can be performed based on different prediction modes, such as performing intra prediction (or inter prediction) on at least one of the multiple partitions obtained by geometric partitioning and performing inter prediction (or intra prediction) on the remaining partitions.

[0108] When performing predictions based on intra prediction mode and / or inter prediction mode for multiple partitions, a lookup table can be predefined that takes combinations of geometric partition angles and / or offsets of lines for geometric partitioning as candidates. An index of the determined geometric partitioning mode can be signaled from the image encoder to the image decoder. Since geometric partitioning of the coding block is performed by lines defined according to the signaled index and lookup table, merge indices for the partitions can be additionally signaled.

[0109] FIG. 4 is a drawing illustrating a geometric division according to the present disclosure.

[0110] Figure 4 shows the angles and offsets of various lines for geometrically dividing the current block.

[0111] According to one embodiment of the present disclosure, a lookup table can be defined that takes a combination of the angle and offset of a line for geometric division as candidates.

[0112] Table 1 below is an example of a lookup table used to calculate geometric division angles.

[0113] Angle idx I0123456789Angle Deg.01427456390117135153166COSLUT[i]32312923140-14-23-29-31SINLUT[i]0-8-14-23-29-32-29-23-14-8

[0114] Referring to Table 1, the lookup table can store the values ​​of geometric division angles and indices for identifying them. The values ​​in the lookup table can be quantized into powers of 2 values ​​for shift operations and stored.

[0115] FIG. 5 is a drawing illustrating a geometric division according to the present disclosure.

[0116] Geometric division angles and offsets can be expressed through the Hessian Normal form, and may also be expressed by a specific mathematical formula to be defined and used as a lookup table.

[0117] Meanwhile, FIG. 5 illustrates that 20 angles and 7 offsets for geometric division are supported, but this is merely an example and is not limited thereto.

[0118] In this case, when the partitioning direction is adaptively determined based on the size and / or aspect ratio of the prediction block, when storing the combination of angle and offset in the form of a lookup table, the larger the size of the prediction block, the more the combination can be expanded.

[0119] For example, all applicable combinations can be defined and used.

[0120] For example, a limited number of candidate combinations may be defined, but the angle and / or offset represented by each index of the lookup table may vary depending on the aspect ratio of the prediction block. For example, if the prediction block has a horizontally elongated shape, a geometric division angle closer to the horizontal direction may be replaced with a geometric division angle closer to the diagonal direction or a geometric division angle closer to the vertical direction. Or, if the prediction block has a vertically elongated shape, a geometric division angle closer to the vertical direction may be replaced with a geometric division angle closer to the diagonal direction or a geometric division angle closer to the horizontal direction.

[0121] Meanwhile, according to one embodiment of the present disclosure, when intra prediction and inter prediction are performed together for a plurality of partitions, information regarding which region among the plurality of partitions corresponds to the region for performing intra prediction can be signaled from the image encoder to the image decoder. Alternatively, a prediction mode that can be applied may be defined in the image decoder. For example, it may be defined to perform intra prediction on the left partition or the top partition, which have more available recovery reference samples.

[0122] According to one embodiment of the present disclosure, when performing intra prediction for a plurality of partitions, any one of a plurality of intra prediction modes may be used. The plurality of intra prediction modes may include at least one of an intra prediction mode in a direction parallel to the geometric partition angle, a prediction mode that derives an intra prediction mode by utilizing a previously restored region, a prediction mode that derives an intra prediction mode by utilizing a previously restored region of an MPM list, a prediction mode that uses an intra prediction mode of a surrounding block, an intra prediction mode in a direction perpendicular to the geometric partition angle, and a Planar prediction mode. Some of the plurality of intra prediction modes may be used as intra prediction candidates for a specific partition (e.g., a first partition or a second partition).

[0123] According to one embodiment of the present disclosure, when performing inter prediction for a plurality of partitions, any one of a plurality of inter prediction modes may be used. In this case, a merge mode may be used for each coding block, and a prediction mode that signals the Motion Vector Difference (MVD), a mode that uses template matching, etc., may or may not be used. In this case, the reference sample used as the template area may be a left sample and / or a top sample of the coding block to be predicted, and the area to be used may be determined according to a geometric partitioning method. Motion information may be derived by minimizing the difference between the reconstructed reference sample around the coding block used as the template area and the template area of ​​the reference picture. When performing inter prediction for a plurality of partitions, inter prediction may be performed based on a prediction mode using affine transformation, a prediction mode using affine transformation and template matching, or a prediction mode based on affine transformation and TIP reference block.

[0124] According to one embodiment of the present disclosure, the indices of geometric partition modes may be reordered using template matching. The indices of geometric partition modes may be selected from K available candidates, and the bits that have undergone context coding may be signaled. Here, K may be a natural number less than or equal to L.

[0125] At this time, the prediction mode may be determined at the partition level (hereinafter referred to as Case 1) or at the sub-block level (hereinafter referred to as Case 2). That is, whether prediction is performed in intra prediction mode and / or inter prediction mode may be determined per partition (Case 1), or the prediction mode may be determined at the sub-block level (Case 2). Here, the current block may correspond to any one of a plurality of sub-blocks.

[0126] For example, when a prediction mode is determined per partition, at least one of the following information can be signaled to an image decoder: geometric partitioning information for a reference sub-block, information on the difference in geometric partitioning angles between sub-blocks, information on the prediction mode of a sub-block, and information on the weighted sum operation method. Through the geometric partitioning information for the reference sub-block and the information on the difference in geometric partitioning angles between sub-blocks, the image decoder can determine the geometric partitioning information of all sub-blocks. In this case, the geometric partitioning information for the reference sub-block may be the geometric partitioning information for a first sub-block, or it may not be the geometric partitioning information for a first sub-block.

[0127] For example, when the prediction mode is determined at the sub-block level, geometric partitioning information for a reference sub-block or a combination of geometric partitioning information per sub-block and the prediction mode of the sub-block can be configured as candidates. For example, when intra-prediction and inter-prediction are performed together for multiple partitions, a combination of the intra-prediction mode and the inter-prediction mode can be configured as candidates.

[0128] When intra-prediction and inter-prediction are performed together for multiple partitions, or when inter-prediction is performed for multiple partitions, other methods may be used to reduce signaling information.

[0129] When intra-prediction and inter-prediction are performed together for multiple partitions, some of the selectable prediction modes of the intra-prediction mode may be configured into one or more candidate lists to reduce signaling information. The candidate lists may have a limited size.

[0130] At this time, an index for identifying a candidate can be signaled to the image decoder. By signaling the index for identifying a candidate, the transmission bits can be reduced. A candidate may consist of one or more lists, and if it consists of at least two lists, an index for identifying a list can be signaled to the image decoder.

[0131] Additionally, when intra-prediction and inter-prediction are performed together for multiple partitions, some of the selectable prediction modes of the inter-prediction mode may be configured into one or more candidate lists. The candidate lists may have a limited size.

[0132] At this time, an index for identifying a candidate can be signaled to the image decoder. By signaling the index for identifying a candidate, the transmission bits can be reduced. A candidate may consist of one or more lists, and if it consists of at least two lists, an index for identifying a list can be signaled to the image decoder.

[0133] For example, when inter-prediction is performed on multiple partitions, at least two partitions may share a common MVD value.

[0134] Referring to FIG. 3, a first prediction block for the first partition can be generated (S320).

[0135] Intra-prediction or inter-prediction can be performed on multiple partitions obtained by geometric partitioning. Alternatively, prediction can be performed based on different prediction modes, such as performing intra-prediction (or inter-prediction) on at least one of the multiple partitions obtained by geometric partitioning and performing inter-prediction (or intra-prediction) on the remaining partitions.

[0136] According to one embodiment of the present disclosure, intra-prediction or inter-prediction may be performed for a first partition according to a determined prediction mode. Here, the prediction mode may be determined per partition as described above, or may be determined per sub-block unit.

[0137] For example, when intra prediction is performed on a first partition, any one of a plurality of intra prediction modes may be used. The plurality of intra prediction modes may include at least one of an intra prediction mode in a direction parallel to the geometric partition angle, a prediction mode that derives an intra prediction mode by utilizing a previously restored region, a prediction mode that derives an intra prediction mode by utilizing a previously restored region of the MPM list, a prediction mode that uses an intra prediction mode of a surrounding block, an intra prediction mode in a direction perpendicular to the geometric partition angle, and a Planar prediction mode. Some of the plurality of intra prediction modes may be used as intra prediction candidates for the first partition.

[0138] According to one embodiment of the present disclosure, inter-prediction may be performed for a second partition according to a determined prediction mode. Here, the prediction mode may be determined per partition as described above, or may be determined per sub-block unit.

[0139] For example, when inter prediction is performed on a first partition, any one of multiple inter prediction modes may be used. In this case, a merge mode may be used for each coding block, but a prediction mode that signals the Motion Vector Difference (MVD), a mode that uses template matching, etc., may or may not be used. In this case, the reference samples used as the template area may be the left sample and / or the top sample of the coding block to be predicted, and the area to be used may be determined according to the geometric partitioning method. Motion information can be derived by minimizing the difference between the reconstructed reference samples around the coding block used as the template area and the template area of ​​the reference picture. When inter prediction is performed on multiple partitions, inter prediction may be performed based on a prediction mode using affine transformation, a prediction mode using affine transformation and template matching, or a prediction mode based on affine transformation and TIP reference blocks.

[0140] Referring to FIG. 3, a second prediction block for the second partition can be generated (S330).

[0141] Intra-prediction or inter-prediction can be performed on multiple partitions obtained by geometric partitioning. Alternatively, prediction can be performed based on different prediction modes, such as performing intra-prediction (or inter-prediction) on at least one of the multiple partitions obtained by geometric partitioning and performing inter-prediction (or intra-prediction) on the remaining partitions.

[0142] According to one embodiment of the present disclosure, intra-prediction or inter-prediction may be performed on a second partition according to a determined prediction mode. Here, the prediction mode may be determined per partition as described above, or may be determined per sub-block unit.

[0143] For example, when intra prediction is performed on a second partition, any one of a plurality of intra prediction modes may be used. The plurality of intra prediction modes may include at least one of an intra prediction mode in a direction parallel to the geometric partition angle, a prediction mode that derives an intra prediction mode by utilizing a previously restored region, a prediction mode that derives an intra prediction mode by utilizing a previously restored region of the MPM list, a prediction mode that uses an intra prediction mode of a surrounding block, an intra prediction mode in a direction perpendicular to the geometric partition angle, and a Planar prediction mode. Some of the plurality of intra prediction modes may be used as intra prediction candidates for the first partition.

[0144] According to one embodiment of the present disclosure, inter-prediction may be performed for a second partition according to a determined prediction mode. Here, the prediction mode may be determined per partition as described above, or may be determined per sub-block unit.

[0145] For example, when inter prediction is performed on a second partition, any one of multiple inter prediction modes may be used. In this case, a merge mode may be used for each coding block, but a prediction mode that signals the Motion Vector Difference (MVD), a mode that uses template matching, etc., may or may not be used. In this case, the reference samples used as the template region may be the left sample and / or the top sample of the coding block to be predicted, and the region to be used may be determined according to the geometric partitioning method. Motion information can be derived by minimizing the difference between the reconstructed reference samples around the coding block used as the template region and the template region of the reference picture. When performing inter prediction on multiple partitions, inter prediction may be performed based on a prediction mode using affine transformation, a prediction mode using affine transformation and template matching, or a prediction mode based on affine transformation and TIP reference blocks.

[0146] Referring to FIG. 3, a final prediction block for the current block can be generated through a weighted sum operation for the first prediction block and the second prediction block (S340).

[0147] According to one embodiment of the present disclosure, a final prediction block for the current block can be generated through a weighted sum operation for a first prediction block and a second prediction block, and the process may be understood as a blending of the boundaries or boundary samples of the first prediction block and the second prediction block.

[0148] For example, the weighted sum operation for the first prediction block and the second prediction block can be performed by applying predefined weights.

[0149] For example, the weighted sum operation for the first prediction block and the second prediction block can be performed by applying weights adjusted according to the difference in predicted pixel values.

[0150] For example, the weighted sum operation for the first prediction block and the second prediction block can be performed by applying weights adjusted using reference restoration samples of previously restored surrounding blocks.

[0151] For example, the weighted sum operation for the first prediction block and the second prediction block can be performed by applying weights adjusted according to the difference in POC (Picture Of Count).

[0152] For example, it may be performed by applying all four of the aforementioned methods.

[0153] According to one embodiment of the present disclosure, a predetermined mask matrix may be applied to a first prediction block and a second prediction block to perform a weighted sum operation. The predetermined mask matrix may include weight information for the weighted sum operation.

[0154] FIG. 6 is a diagram showing a non-linear function for deriving weights according to one embodiment of the present disclosure.

[0155] The weights may be determined through a linear function or through a non-linear function. The non-linear function may include at least one of a sigmoid or a trigonometric function.

[0156] For example, weights can be derived according to the following mathematical formula 1.

[0157] [Mathematical Formula 1]

[0158]

[0159] FIG. 7 is a diagram showing the size and weight of the region where a weighted sum operation is performed, according to one embodiment of the present disclosure.

[0160] As described above, a predetermined mask matrix may be applied to the first prediction block and the second prediction block to perform a weighted sum operation.

[0161] Referring to FIG. 7, the size of the region to which a predetermined mask matrix is ​​applied, that is, the region where a weighted sum operation is performed, can be determined as a multiple of τ. For example, the size of the region where a weighted sum operation is performed can be determined as any one of τ / 4, τ / 2, τ, 2τ, or 4τ. Here, τ can be an integer greater than 0. For example, τ can be 2.

[0162] According to one embodiment of the present disclosure, a weighted sum operation for a first prediction block and a second prediction block may be performed based on a regression model.

[0163] According to the regression-based weighted sum operation method, a mathematical model can be derived by modeling the prediction block of the current block with the area around the boundary of the geometric partition as the target region. The mathematical model can be derived in the image decoder using the previously restored samples from the left and top of the current block. The derived mathematical model can take the form of a linear function, for example, ax + by + c = 0.

[0164] When performing a regression-based weighted sum operation, the mask matrix of the coding block can be derived using previously restored reference samples adjacent to the coding block. Alternatively, the mask matrix of the sub-block can be derived using previously restored reference samples at the sub-block level of the coding block.

[0165] According to one embodiment of the present disclosure, weights may be determined based on a lookup table.

[0166] For example, the values ​​of the lookup table for calculating weights may be quantized and fixed as described above. Alternatively, they may be applied after being finely adjusted according to the angle and / or aspect ratio of the block to which the mask matrix is ​​applied.

[0167] For example, weights can be calculated using the values ​​stored in the lookup table as they are, quantized. Alternatively, weights can be calculated using values ​​scaled by applying a predefined scaling factor to the stored values.

[0168] For example, one or more lists for the lookup table may be configured. This is to diversify the areas where weighted sum operations can be performed. In this case, the list and / or an index to specify the area where the weighted sum operation is performed can be signaled to the image decoder.

[0169] Meanwhile, according to one embodiment of the present disclosure, a flag related to a weighted sum operation can be signaled for each level.

[0170] For example, for screen content, the size of the area where the weighted sum operation is performed can optionally be set to τ / 2N via High Level Syntax (HLS). Here, N can be an integer greater than 0.

[0171] For example, at the Picture Parameter Set (PPS) level, it is determined whether to perform a weighted sum operation on boundary samples and signal a 1-bit flag to the image decoder. If no weighted sum operation is performed, the weights can be set to a maximum or minimum value. For example, the maximum value can be 32 and the minimum value can be 0.

[0172] For example, the weighted sum operation can be performed in a linear or non-linear manner. Whether the shape of the region where the weighted sum operation is performed is symmetric or asymmetric is defined in advance, and this can be signaled to the decoder at the CU level.

[0173] For example, to support various weighted sum operation methods, the distance at which the weighted sum operation is performed (also called the blending distance) may be selected and used from among multiple candidates. Alternatively, the candidate list may be composed of different combinations based on at least one of the size of the coding block or sub-block, the aspect ratio of the coding block or sub-block, the geometric partitioning mode, or the sub-block partitioning type, and the index of the candidate list may be signaled. In this case, one or more candidate lists may be used.

[0174] For example, when performing intra prediction and inter prediction together on multiple partitions, regarding the area where inter prediction was performed, the area where weighted sum operations are performed may be set narrowly or not performed at the boundary of the inter prediction area, or conversely, the area where weighted sum operations are performed may be set broadly. In determining the area where weighted sum operations are performed, the image encoder may indicate part of the necessary information based on the distance from the geometric partitioning boundary, or the image decoder may determine part of the necessary information based on motion vectors, QP strengths, POC distances, etc. The area where weighted sum operations are performed may be obtained by calculating the blending distance difference based on the blending distance applied to the area where intra prediction was performed, or it may be obtained by reconstructing or rearranging blending distance candidates.

[0175] For example, when performing inter-prediction on multiple partitions, the motion vectors used for each partition may differ. In this case, the blending distance can be set differently based on the difference value, POC value, QP value, etc. of the motion vectors. Alternatively, when performing inter-prediction on multiple partitions using merge mode, the blending masking coefficient can be set differently depending on whether the referenced motion vector candidate locations are adjacent or far apart.

[0176] For example, when performing inter-prediction on multiple partitions, when using motion vectors of merge candidates, weights during the weighted sum operation may be defined based on the block size of the spatial / temporal adjacent / non-adjacent positions of the motion vectors.

[0177] For example, when intra prediction and inter prediction are performed together for multiple partitions, or when inter prediction is performed for multiple partitions, if it is determined that the line for geometric division has continuity in the reference sample region based on the geometric division angle of the sub-block and the position of the sub-block, a masking factor may be derived from the reference sample region. In this case, at least one of the following methods may be used to determine continuity: the difference in values ​​of the previously restored samples, the structural difference of the previously restored samples, the second derivative, or a convolution operation using a two-dimensional matrix (e.g., a 3x3 matrix).

[0178] However, the above-disclosed embodiments are merely examples, and the method for determining continuity may differ.

[0179] For example, different weighted sum operation methods can be applied to each sub-block, and a mask matrix can be calculated at the coding block level based on the difference pattern of geometric partitioning for each sub-block and then applied at the coding block level.

[0180] For example, when a regression-based weighted sum operation is performed, a predetermined mask matrix can be derived on a sub-block basis using reference samples of the previously restored sub-blocks.

[0181] The present disclosure assumes a case where a sub-block partitioning method is performed on a coding block for which geometric partitioning prediction is performed. Below, the case where a sub-block partitioning method is performed on a coding block for which geometric partitioning prediction is performed will be described in detail.

[0182] According to one embodiment of the present disclosure, sequential prediction can be performed according to a predetermined order when predicting a plurality of sub-blocks. In this case, the sub-block(s) predicted prior to the sub-block currently to be predicted may be understood as the sub-block of the reconstruction priority, and the sub-block currently to be predicted may be understood as the sub-block of the reconstruction priority. In this case, in intra-prediction, the sub-block of the reconstruction priority is predicted using samples of the sub-block of the reconstruction priority, thereby improving the prediction efficiency of the sub-block in the next reconstruction order. For example, directional prediction can be performed using reference samples of the reconstructed sub-block in intra-prediction. For example, when constructing merge candidates in inter-prediction, the reference location of the motion vector can be specified based on the sub-block currently to be predicted. Accordingly, the location of the temporal motion vector candidate may differ for each sub-block. Alternatively, when constructing merge candidates in inter-prediction, the reference location of the motion vector may be specified based on geometrically divided partitions.

[0183] Flag signaling related to sub-block splitting

[0184] According to one embodiment of the present disclosure, a flag associated with a sub-block partitioning method can be signaled to an image decoder.

[0185] For example, a flag indicating whether to perform a sub-block partitioning method in geometric partitioning prediction in High Level Syntax (HLS) and Sequence Parameter Set (SPS) can be signaled to the image decoder.

[0186] For example, in the case of screen content, the variety of sub-block partitioning in HLS can be optionally limited.

[0187] For example, in PPS, it can be determined whether to perform the sub-block partitioning method in geometric partitioning prediction. If it is determined whether to perform, a 1-bit flag can be signaled to the image decoder.

[0188] For example, if a geometric partitioning mode is selected at the coding block level, a flag indicating whether to perform additional sub-block partitioning can be signaled to the image decoder.

[0189] For example, when a sub-block partitioning method is performed, whether the prediction is performed based on either the intra prediction mode and / or the inter prediction mode can be determined per partition as in Case 1 described above. Alternatively, whether the prediction is performed based on either the intra prediction mode or the inter prediction mode can be determined per sub-block as in Case 2 described above.

[0190] For example, for at least one block among spatially adjacent / non-adjacent neighboring blocks, a sub-block partitioning method may be performed only when prediction is performed based on at least one prediction mode among a sub-block-based merge mode, a prediction mode using an affine transformation, or a geometric partitioning-based sub-block prediction mode. Accordingly, if there is no block predicted based on sub-blocks among spatially adjacent / non-adjacent neighboring blocks, a flag indicating whether to perform the sub-block partitioning method may not be signaled, and conversely, if there is at least one block predicted based on sub-blocks, a flag indicating whether to perform the sub-block partitioning method may be signaled. This method has the utility of reducing unnecessary encoding time and unnecessary signaling bits.

[0191] Determining sub-block partitioning type

[0192] According to one embodiment of the present disclosure, a sub-block partitioning type for a sub-block can be determined.

[0193] For example, the direction of sub-block partitioning can be explicitly signaled from the video encoder to the video decoder, or implicitly derived from the video decoder based on the size and / or aspect ratio of the coding block.

[0194] For example, the partition type of a subblock to be predicted may be implicitly determined by referring to the size and / or geometric partition angle of the subblock of the restoration line rank. In this case, the partition type may be determined differently depending on the geometric partition mode and / or intra / inter prediction mode of one or more subblocks of the restoration line rank.

[0195] For example, the sub-block partitioning type may be restricted by referring to the size and / or geometric partitioning angle of the sub-block of the restoration line priority. In this case, an index to specify any one of the restricted sub-block partitioning types may be determined by signaling it to the image decoder.

[0196] For example, the type of division can be determined in the vertical or horizontal direction depending on the size of the coding block. For example, one coding block can be divided into two or four sub-blocks.

[0197] A coding block may be divided in a vertical or horizontal direction and may be divided into multiple sub-blocks of equal size or multiple sub-blocks of non-uniform size.

[0198] FIG. 8 is a drawing illustrating a sub-block division according to the present disclosure.

[0199] More specifically, it relates to a case where a coding block is divided in a vertical or horizontal direction and divided into multiple sub-blocks of equal size.

[0200] The sub-block partitioning method can be performed on coding blocks of size N x 2N or 2 N x N. Alternatively, the sub-block partitioning method can be performed on coding blocks larger than N x 2N or 2 N x N. The minimum size of a block on which the sub-block partitioning method can be performed can be 4 x 8 or 8 x 4.

[0201] As illustrated in FIG. 8, when the size of the coding block is N×2N or 2N×N, the coding block can be divided into two sub-blocks. For example, if the width of the coding block is W and the height is H, it can be divided into two sub-blocks with a size of Wx(H / 2) or two sub-blocks with a size of (W / 2)xH.

[0202] As shown in FIG. 8, when the size of the coding block is greater than N×2N or 2N×N, the coding block can be divided into four. For example, if the width of the coding block is W and the height is H, it can be divided into four sub-blocks with a size of Wx(H / 4) or four sub-blocks with a size of (W / 4)xH.

[0203] Alternatively, the maximum size of the block on which the sub-block partitioning method can be performed may be limited to 64x64, 128x128, or 256x256, etc. This is to account for the unit of data input / output at the pipeline stage of the image decoder hardware. Depending on the maximum size of the selectable coding block, the depth of the block partitioning structure, or variety, the number of horizontal or vertical pixels constituting the sub-block may be limited. For example, the number of horizontal or vertical pixels constituting the sub-block may be 4 or fewer, or 64 or more.

[0204] FIG. 9 is a drawing illustrating a sub-block division according to the present disclosure.

[0205] More specifically, this relates to a case where a coding block is divided in a vertical or horizontal direction and divided into multiple sub-blocks having non-uniform sizes.

[0206] The sub-block partitioning method can be performed on coding blocks of size N x 2N or 2 N x N. Alternatively, the sub-block partitioning method can be performed on coding blocks larger than N x 2N or 2 N x N. The minimum size of a block on which the sub-block partitioning method can be performed can be 4 x 8 or 8 x 4.

[0207] As illustrated in FIG. 9, when the size of the coding block is N×2N or 2N×N, the coding block can be divided into two or three sub-blocks. For example, if the width of the coding block is W and the height is H, it can be divided into two sub-blocks with a size of Wx(H / 4) and one sub-block with a size of Wx(H / 2). Alternatively, it can be divided into one sub-block with a size of WxM and two sub-blocks with a size of WxN. Here, M and N may be different integers greater than or equal to 1.

[0208] As illustrated in FIG. 9, when the size of the coding block is greater than N×2N or 2N×N, the coding block may be divided into two or three sub-blocks. For example, if the width of the coding block is W and the height is H, it may be divided into two sub-blocks with a size of (W / 4)xH and one sub-block with a size of (W / 2)xH. Alternatively, it may be divided into one sub-block with a size of MxH and two sub-blocks with a size of NxH. Here, M and N may be different integers greater than or equal to 1.

[0209] In the present disclosure, whether to perform the sub-block partitioning method or the sub-block type, etc., described above may be configured as candidates in combination with a geometric partitioning mode and / or an intra / inter prediction mode, and an index may be signaled. In this case, the configured combination may vary depending on the size, aspect ratio, etc. of the coding block.

[0210] Meanwhile, according to one embodiment of the present disclosure, a geometric division mode can be determined for each sub-block for the sub-block.

[0211] Determination of geometric partitioning mode for sub-blocks of restoration line rank

[0212] The geometric division mode of the sub-blocks of the restoration line rank can be determined by a combination of geometric division angles and offsets.

[0213] At this time, the combination can be stored in the form of a lookup table, the combination can be stored as a quantized value, and an index for specifying the combination can be signaled to an image decoder.

[0214] Meanwhile, in the present disclosure, geometric division in a specific direction may be additionally possible depending on the sub-block division type and the aspect ratio of the coding block. If the sub-block has a horizontally elongated shape or if the sub-block has a vertically elongated shape, division directions may be added or removed for the wider or narrower sides. Accordingly, the constraints on the number of generally supported division directions may be expanded.

[0215] The following FIG. 10 is a drawing illustrating a geometric division direction according to the present disclosure.

[0216] Generally, there may be N possible splitting directions. Here, N can be 20. In this case, if the sub-block has a horizontally elongated shape or if the sub-block has a vertically elongated shape, an extended splitting direction may be supported.

[0217] For example, as shown in FIG. 10(a), more than 32 different geometric division angles can be supported.

[0218] Alternatively, as shown in FIG. 10(b), a wider variety of geometric division angles may be supported.

[0219] At this time, all extended geometric division angles can be selected and used, but the number of lookup table entries that can be used may be reduced depending on the size and / or aspect ratio of the sub-block. That is, the number of geometric division angles stored in the lookup table may be reduced.

[0220] However, the values ​​disclosed above are merely one embodiment and may have different values.

[0221] Meanwhile, in the present disclosure, if the endpoint of the geometric division for a sub-block of the restoration line priority does not meet a sub-block of the post-restoration priority, the sub-block division method may not be supported. In this case, the lookup table may be modified so that certain combinations are not selected depending on the size and / or aspect ratio of the sub-block.

[0222] FIG. 11 is a drawing showing a case in which the endpoint of a geometric division for a lower block of a restoration line rank according to one embodiment of the present disclosure does not meet a lower block of a post-restoration rank.

[0223] FIG. 11 illustrates an embodiment that can be represented by a single geometric division without being divided into sub-blocks.

[0224] Referring to Fig. 11, if the width of the coding block is W and the height is H, the upper sub-block among two sub-blocks having a size of Wx(H / 2) may correspond to the sub-block of the restoration line priority. In this case, if the endpoint of the geometric division of the sub-block of the restoration line priority does not meet the sub-block of the restoration post-priority at the bottom, the sub-block division method may not be supported.

[0225] Referring to Fig. 11, if the width of the coding block is W and the height is H, the uppermost sub-block among the four sub-blocks having a size of Wx(H / 4) may correspond to the sub-block of the restoration line priority. In this case, if the endpoint of the geometric division of the sub-block of the restoration line priority does not meet the sub-block of the restoration post-priority at the bottom, the sub-block division method may not be supported.

[0226] Referring to Fig. 11, if the width of the coding block is W and the height is H, the leftmost of the two sub-blocks with a size of (W / 2)xH may correspond to the sub-block of the restoration line priority. In this case, if the endpoint of the geometric division of the sub-block of the restoration line priority does not meet the sub-block of the post-restoration priority at the bottom, the sub-block division method may not be supported.

[0227] Referring to Fig. 11, if the width of the coding block is W and the height is H, the leftmost sub-block among the four sub-blocks with a size of (W / 4)xH may correspond to the sub-block of the restoration line priority. In this case, if the endpoint of the geometric division of the sub-block of the restoration line priority does not meet the sub-block of the restoration post-priority at the bottom, the sub-block division method may not be supported.

[0228] In addition, according to one embodiment of the present disclosure, when a geometric partitioning mode for a sub-block of a restoration line rank is determined, a geometric partitioning mode for a sub-block of a post-restoration rank can be determined.

[0229] Determination of geometric partitioning mode for sub-blocks of the rank after restoration

[0230] FIGS. 12 and FIGS. 13 are drawings illustrating a geometric division mode for a sub-block of a rank after restoration according to the present disclosure.

[0231] The geometric partitioning mode can be determined differently for each sub-block. For example, as shown in FIG. 12, the geometric partitioning modes of sub-block PU[0] and sub-block PU[1] can be determined differently, and to signal the geometric partitioning mode of sub-block PU[1], the difference from the geometric partitioning mode of sub-block PU[0] can be formed into a candidate list, and an index to specify the candidate can be signaled. Therefore, when performing geometric partitioning, fine-tuning of the prediction method can be made at the sub-block level by signaling the difference in the geometric partitioning mode of the sub-block of the post-restoration rank with a small amount of bits, based on the geometric partitioning mode of the sub-block of the restoration line rank.

[0232] Referring to FIGS. 12 and 13, when signaling a geometric partitioning mode for a sub-block of the post-restoration rank, the difference in geometric partitioning angles based on the geometric partitioning time of the sub-block of the post-restoration rank ( The difference between angle) and / or offset ( The number of transmitted bits can be minimized by signaling only the offset or by using a lookup table to signal only the index representing the combination of the two parameters.

[0233] Referring to FIGS. 12 and 13, as a constraint on the sub-block, the variety of geometric division angles and / or offsets may be gradually increased or decreased depending on the restoration order. Specifically, referring to FIG. 12, the variety of geometric division angles and / or offsets may be gradually increased or gradually decreased. Referring to FIG. 12, the variety of geometric division angles and / or offsets may be gradually increased and then decreased, or gradually increased and then fixed. That is, By setting the angle to a constant value, the geometric division angle can be induced to gradually increase or decrease by directly signaling it or by signaling an index that indicates the value, and The value of the angle can also be gradually increased or gradually decreased. Alternatively, the sign of the geometric division angle can be signaled.

[0234] At this time, the lower block of the ranking after restoration angle and / or A list of offsets can be generated.

[0235] For example, depending on the difference in aspect ratio with the lower block of the restoration line priority or the difference in geometric partitioning mode with the lower block of the restoration line priority, etc. angle and / or You can determine the offset value and construct a list. In this case, one or more lists may be used, taking into account the size of the sub-block, the aspect ratio of the sub-block, the geometric partitioning mode, etc. Depending on the type of list, angle and / or The value of the offset may change. In this case, angle and / or Depending on the value of offset, the list may be composed of asymmetric combinations, combinations considering mathematical modeling such as circles or parabolas centered on one vertex of a sub-block or coding block, point-symmetric combinations, or combinations that change horizontally and vertically.

[0236] For example, the list can be constructed directly by considering the local variance of the reference samples of the restored sub-block or surrounding blocks.

[0237] For example, since blocks are divided by lines defined according to index and lookup tables, a single index representing a combination of geometric division angles and offset differences for all sub-blocks can be signaled at the CU level.

[0238] For example, the indices constituting each list may be reordered using referenceable restoration samples.

[0239] Meanwhile, according to the present disclosure, intra prediction for a sub-block can be performed according to a determined intra prediction mode.

[0240] Determination of intra-prediction mode for sub-blocks

[0241] For example, when intra prediction is performed on a sub-block, any one of a plurality of intra prediction modes may be used. The plurality of intra prediction modes may include DC prediction mode, Planar prediction mode, horizontal direction prediction mode, vertical direction prediction mode, etc., and various other intra prediction modes may be used. Alternatively, a block vector may be used, or the index of the corresponding mode may be signaled using a geometric partitioning angle and a horizontal or vertical direction mode.

[0242] For example, if intra prediction is performed on a sub-block of the post-restoration rank, the sub-block of the post-restoration rank can perform prediction using samples from the previously restored sub-block.

[0243] At this time, depending on the geometric partitioning mode of the previously restored sub-block, it may be divided into predetermined regions and each may refer to different restoration samples.

[0244] FIG. 14 is a drawing illustrating restoration samples referenced by a sub-block according to the present disclosure.

[0245] As shown in FIG. 14, depending on the geometric partitioning mode of the previously restored subblock, the subblock currently to be predicted may be divided into two regions to refer to different restoration samples.

[0246] Alternatively, as illustrated in FIG. 14, when a sub-block is divided into two regions and each references different restoration samples, boundary samples from before or after the weighted sum operation is performed can be copied and used for the reference location where the reference sample exists. The boundary samples may correspond to samples located at points where multiple partitions are distinguished according to geometric partitioning.

[0247] For example, the order of directions indicated by the index in intra-prediction mode can be redefined based on at least one of the sub-block partition type, the size of the coding block, or the aspect ratio of the coding block. In this case, the redefined index can be signaled by context-coded bits.

[0248] Alternatively, if all geometrically partitioned regions (e.g., first partition, second partition, etc.) are encoded in an intra-prediction mode, the order of the indices for the intra-prediction directionality may be redefined based on at least one of the geometric partitioning angle, the size of the sub-block, or the aspect ratio of the sub-block. In this case, the top reconstructed samples and / or the left reconstructed samples may be used as reference samples. Referenceable samples located at the top may be understood as top reference samples, and referenceable samples located at the bottom may be understood as bottom reference samples. In this case, which region's reference samples are used may be determined according to the geometric partitioning mode.

[0249] For example, if DC prediction mode or Planar prediction mode is selected among the intra prediction modes, only the longer region between the top reference sample or the left reference sample may be used, or both the top reference sample and the left reference sample may be used. Alternatively, to facilitate shift operations, 2N You can also select and use several samples.

[0250] For example, intra prediction can be performed using the previously restored regions and MPM list.

[0251] For example, as reference samples for the intra prediction mode, not only the restored samples of surrounding blocks but also samples of previously restored sub-blocks may be used.

[0252] For example, the MPM list or candidate list may be modified by referring to the coding information of previously restored sub-blocks. This is intended to improve the prediction efficiency of lower-priority sub-blocks for restoration.

[0253] For example, if the sub-block partition type is horizontal and the directional index of the intra prediction mode is smaller than the horizontal mode index (18), the prediction order can be applied differently depending on whether the bottom-left reference samples of the coding block are available.

[0254] FIG. 15 is a diagram showing the sub-block prediction order according to the availability of a reference sample, according to one embodiment of the present disclosure.

[0255] Intra prediction can be performed using the restored region and the MPM list or candidate list. The restored region used for intra prediction may include the region of the restored sub-block as illustrated in FIG. 15.

[0256] At this time, as illustrated in FIG. 15, if the sub-block partition type is horizontal and the directional index of the intra prediction mode is smaller than the horizontal direction mode index (18), and there is no lower-left reference sample of the coding block, prediction can be made sequentially from the sub-block located at the top to the sub-block located at the bottom.

[0257] Alternatively, as illustrated in FIG. 15, if the sub-block partition type is horizontal and the directional index of the intra prediction mode is smaller than the horizontal mode index (18), if there is a reference sample of the bottom left of the coding block, prediction can be made in reverse order from the sub-block located at the bottom to the sub-block located at the top.

[0258] This can be understood to apply in the same way even when the sub-block partition type is vertical and the directional index of the intra prediction mode is greater than the vertical direction mode index (50).

[0259] Meanwhile, intra-prediction can be performed using block vectors.

[0260] For example, the region of a previously restored sub-block can be used as the region of reference samples required for the reordering of template-based geometric partitioning predictions. The reference samples used as templates may include samples to the left and / or top of the coding block. The region of reference samples used can be determined according to the geometric partitioning mode. Information on the block vector can be obtained by minimizing the template difference between the restored reference samples surrounding the coding block used as a template and the reference picture, and the block vector can be defined at the sub-block level. The index of the geometric partitioning mode can be reordered using template matching of the block vectors of surrounding blocks or sub-blocks. The reordered index may be less than the maximum number of geometric partitioning modes and can be signaled by context-coded bits.

[0261] For example, the block vector difference for a coding block or sub-block can be signaled to an image decoder.

[0262] The aforementioned intra prediction modes may consist of one or more lists and may be included as intra prediction candidates or may not be included as intra prediction candidates.

[0263] In addition, the aforementioned intra prediction modes may be signaled at the coding block level or signaled differently for each sub-block. If the intra prediction modes differ for each sub-block, the index may be signaled by configuring only the difference in prediction modes between the sub-blocks of the restoration priority and the prediction mode as candidates. However, among the intra prediction modes, for prediction modes dependent on geometric partitioning modes, such as direction prediction parallel / perpendicular to the geometric partitioning angle, even if only one prediction mode is used uniformly for the coding block, the utility of having different prediction modes for each sub-block can be obtained.

[0264] Meanwhile, according to the present disclosure, inter prediction for a sub-block can be performed according to a determined inter prediction mode.

[0265] Determining the inter-prediction mode for the sub-block

[0266] For example, when using merge mode with inter-prediction mode, adjacent / non-adjacent spatial candidate and adjacent / non-adjacent temporal candidate locations of the restored reference sample can be used when constructing merge candidates. When using merge mode with inter-prediction mode, the location order of candidates to reference motion vectors can be set differently for multiple partitions.

[0267] For example, to represent motion vector information in inter prediction, reference picture lists, reference picture indices, motion vectors, etc., may be directly signaled, or merge mode may be used while signaling MVD values. In this case, when inter prediction is performed on all multiple partitions, the MVD values ​​for the lower blocks of the partitions may be set to the same value to reduce signaling bits. The motion vectors for the lower blocks of the restoration priority may have different values ​​for each partition, while the difference values ​​for the motion vectors for the lower blocks of the post-restoration priority may be the same.

[0268] For example, the MVD may consist of one or more candidate lists. For example, the first candidate list may be configured to include at least one of the candidates 1 / 4-pel, 1 / 2-pel, 1-pel, 2-pel, 3-pel, 4-pel, 8-pel, or 16-pel, and the second candidate list may be configured to include at least one of the candidates 1 / 4-pel, 1 / 2-pel, 1-pel, 2-pel, 4-pel, 8-pel, 16-pel, or 32-pel. An index indicating the MVD value within the candidate list may be signaled from the video encoder to the video decoder. If composed of two or more candidate lists, an index specifying the candidate list may be signaled. To reduce signaling bits, motion vectors smaller than integer values ​​can be excluded from CTC for high-resolution video sequences such as 4K through picture header flags, and a specific constant value can be used by multiplying the MVD values ​​in the candidate list.

[0269] For example, when using merge mode but using prediction mode that signals the MVD value, the availability of template matching at the SPS level can be signaled as a flag. The MVD can be represented by direction and distance, and an index specifying combinations of direction and distance can be signaled from the video encoder to the video decoder. In this case, to reduce the signaling bits, the combination candidates can be reordered through template matching, and accordingly, a smaller list size can be set for the combination candidates.

[0270] For example, a prediction mode using an affine transformation may be used for at least one of the multiple partitions. When performing prediction using an affine transformation, a motion vector offset for transmitting the MVD in merge mode may be added to the three-vertex or two-vertex motion vectors used in the affine transformation. In the case of an affine transformation, if the sub-block currently to be predicted is contained within a motion space that can be represented by an affine transformation, there is a high probability that the sub-prediction block will also be coded in a prediction mode using an affine transformation; conversely, if the sub-prediction block is contained within a motion space represented only by parallel transformation without linear transformation, there is a high probability that the sub-prediction block will also be coded in a prediction mode using parallel transformation. Therefore, depending on the prediction mode of the surrounding blocks of the sub-prediction block, an affine transformation may be possible or impossible. If an affine transformation is impossible, a transmission flag for which an affine transformation can be selected or a prediction mode candidate indicating an affine transformation may be removed.

[0271] For example, a prediction mode candidate using an affine transformation may be refined through template matching. In inter-prediction, when a prediction using template matching is performed, the prediction may be performed in units of coding blocks, and motion vectors may be derived through template matching in units of sub-blocks, as shown in FIGS. 16 and 17 below.

[0272] FIGS. 16 and FIGS. 17 are drawings illustrating a case in which prediction using template matching is performed on a sub-block basis when predicting a sub-block based on geometric partitioning according to one embodiment of the present disclosure.

[0273] Referring to FIG. 16, when intra prediction and inter prediction are performed together for multiple partitions, when making a prediction using template matching, the template region may include at least one of the upper restoration samples of the lower block and / or the left restoration samples of the lower block.

[0274] Referring to FIG. 17, when performing inter-prediction on multiple partitions, when prediction using template matching, the template region may include at least one of the upper restoration samples of the lower block and / or the left restoration samples of the lower block.

[0275] Meanwhile, for example, a reference picture can be generated by scaling at least one motion vector passing through the current picture relative to the current picture, and the generated reference picture can be used to predict the current picture.

[0276] The aforementioned inter prediction modes may consist of one or more lists and may be included as inter prediction candidates or may not be included as inter prediction candidates.

[0277] In addition, the aforementioned inter prediction modes may be signaled at the coding block level or may be signaled differently for each sub-block. If the inter prediction modes differ for each sub-block, the index may be signaled by configuring only the difference in prediction modes between the sub-block of the restoration priority as candidates.

[0278] If the prediction mode differs for each sub-block, the restoration process can be performed sequentially using the aforementioned inter-prediction mode on a sub-block basis; if the prediction mode is the same for all sub-blocks, prediction samples can be derived in a parallel manner on a sub-block basis or in batches on a prediction coding block basis.

[0279] Meanwhile, when performing predictions on lower-ranked blocks after restoration, the MPM list or candidate list may be modified using the coding information of the previously restored lower-ranked blocks.

[0280] FIG. 18 is a drawing illustrating block division for prediction and transformation according to the present disclosure.

[0281] In the present disclosure, the units of the sub-blocks for prediction and transformation may be the same or different.

[0282] A single coding block can be divided into multiple sub-blocks for prediction (also called sub-prediction blocks) based on detailed differences in prediction modes.

[0283] At this time, any one of the multiple sub-prediction blocks may be composed of one or more sub-blocks for transformation (also called sub-transformation blocks).

[0284] Referring to FIG. 18, one coding block can be divided into two sub-prediction blocks (PU[0], PU[1]). One sub-prediction block consists of two sub-transformation blocks, and one coding block can be composed of four sub-transformation blocks (TU[0], TU[1], TU[2], TU[3]).

[0285] FIG. 19 is a flowchart of an image signal decoding method according to one embodiment of the present disclosure.

[0286] Referring to FIG. 19, the current block can be divided into multiple partitions based on geometric division (S1910).

[0287] According to one embodiment of the present disclosure, a lookup table having a combination of line angles and offsets for geometric division as candidates may be defined in advance.

[0288] When the partitioning direction is adaptively determined based on the size and / or aspect ratio of the prediction block, when storing combinations of angles and offsets in the form of a lookup table, the larger the size of the prediction block, the more the combinations can be expanded.

[0289] Meanwhile, according to one embodiment of the present disclosure, when intra prediction and inter prediction are performed together for a plurality of partitions, information regarding which region among the plurality of partitions corresponds to the region for performing intra prediction can be signaled from the image encoder to the image decoder. Alternatively, a prediction mode that can be applied may be defined in the image decoder. For example, it may be defined to perform intra prediction on the left partition or the top partition, which have more available recovery reference samples.

[0290] According to one embodiment of the present disclosure, when performing intra prediction for a plurality of partitions, any one of a plurality of intra prediction modes may be used. The plurality of intra prediction modes may include at least one of an intra prediction mode in a direction parallel to the geometric partition angle, a prediction mode that derives an intra prediction mode by utilizing a previously restored region, a prediction mode that derives an intra prediction mode by utilizing a previously restored region of an MPM list, a prediction mode that uses an intra prediction mode of a surrounding block, an intra prediction mode in a direction perpendicular to the geometric partition angle, and a Planar prediction mode. Some of the plurality of intra prediction modes may be used as intra prediction candidates for a specific partition (e.g., a first partition or a second partition).

[0291] According to one embodiment of the present disclosure, when performing inter prediction for a plurality of partitions, any one of a plurality of inter prediction modes may be used. In this case, a merge mode may be used for each coding block, and a prediction mode that signals the Motion Vector Difference (MVD), a mode that uses template matching, etc., may or may not be used. In this case, the reference sample used as the template area may be a left sample and / or a top sample of the coding block to be predicted, and the area to be used may be determined according to a geometric partitioning method. Motion information may be derived by minimizing the difference between the reconstructed reference sample around the coding block used as the template area and the template area of ​​the reference picture. When performing inter prediction for a plurality of partitions, inter prediction may be performed based on a prediction mode using affine transformation, a prediction mode using affine transformation and template matching, or a prediction mode based on affine transformation and TIP reference block.

[0292] According to one embodiment of the present disclosure, the indices of geometric partition modes may be reordered using template matching. The indices of geometric partition modes may be selected from K available candidates, and the bits that have undergone context coding may be signaled. Here, K may be a natural number less than or equal to L.

[0293] In the present disclosure, prediction and transformation may be performed using an intra prediction mode and / or an inter prediction mode, and whether prediction is performed in an intra prediction mode and / or an inter prediction mode may be determined per partition or at the sub-block level. This is as described with reference to Case 1 and Case 2.

[0294] Regarding the geometric division of the current block, it can be understood as seen in S310 of Fig. 3, so to avoid duplication, a detailed explanation is omitted here.

[0295] Referring to FIG. 19, a first prediction block for the first partition can be generated (S1920).

[0296] Intra-prediction or inter-prediction can be performed on multiple partitions obtained by geometric partitioning. Alternatively, prediction can be performed based on different prediction modes, such as performing intra-prediction (or inter-prediction) on at least one of the multiple partitions obtained by geometric partitioning and performing inter-prediction (or intra-prediction) on the remaining partitions.

[0297] According to one embodiment of the present disclosure, intra-prediction may be performed for a first partition according to a determined prediction mode. Here, the prediction mode may be determined per partition as described above, or may be determined per sub-block unit.

[0298] For example, when intra prediction is performed on a first partition, any one of a plurality of intra prediction modes may be used. The plurality of intra prediction modes may include at least one of an intra prediction mode in a direction parallel to the geometric partition angle, a prediction mode that derives an intra prediction mode by utilizing a previously restored region, a prediction mode that derives an intra prediction mode by utilizing a previously restored region of the MPM list, a prediction mode that uses an intra prediction mode of a surrounding block, an intra prediction mode in a direction perpendicular to the geometric partition angle, and a Planar prediction mode. Some of the plurality of intra prediction modes may be used as intra prediction candidates for the first partition.

[0299] According to one embodiment of the present disclosure, inter-prediction may be performed for a first partition according to a determined prediction mode. Here, the prediction mode may be determined per partition as described above, or may be determined per sub-block unit.

[0300] For example, when inter prediction is performed on a first partition, any one of multiple inter prediction modes may be used. In this case, a merge mode may be used for each coding block, but a prediction mode that signals the Motion Vector Difference (MVD), a mode that uses template matching, etc., may or may not be used. In this case, the reference samples used as the template area may be the left sample and / or the top sample of the coding block to be predicted, and the area to be used may be determined according to the geometric partitioning method. Motion information can be derived by minimizing the difference between the reconstructed reference samples around the coding block used as the template area and the template area of ​​the reference picture. When inter prediction is performed on multiple partitions, inter prediction may be performed based on a prediction mode using affine transformation, a prediction mode using affine transformation and template matching, or a prediction mode based on affine transformation and TIP reference blocks.

[0301] Referring to FIG. 19, a second prediction block for the second partition can be generated (S1930).

[0302] Intra-prediction or inter-prediction can be performed on multiple partitions obtained by geometric partitioning. Alternatively, prediction can be performed based on different prediction modes, such as performing intra-prediction (or inter-prediction) on at least one of the multiple partitions obtained by geometric partitioning and performing inter-prediction (or intra-prediction) on the remaining partitions.

[0303] According to one embodiment of the present disclosure, intra-prediction may be performed on a second partition according to a determined prediction mode. Here, the prediction mode may be determined per partition as described above, or may be determined per sub-block.

[0304] For example, when intra prediction is performed on a second partition, any one of a plurality of intra prediction modes may be used. The plurality of intra prediction modes may include at least one of an intra prediction mode in a direction parallel to the geometric partition angle, a prediction mode that derives an intra prediction mode by utilizing a previously restored region, a prediction mode that derives an intra prediction mode by utilizing a previously restored region of the MPM list, a prediction mode that uses an intra prediction mode of a surrounding block, an intra prediction mode in a direction perpendicular to the geometric partition angle, and a Planar prediction mode. Some of the plurality of intra prediction modes may be used as intra prediction candidates for the second partition.

[0305] According to one embodiment of the present disclosure, inter-prediction may be performed for a second partition according to a determined prediction mode. Here, the prediction mode may be determined per partition as described above, or may be determined per sub-block unit.

[0306] For example, when inter prediction is performed on a second partition, any one of multiple inter prediction modes may be used. In this case, a merge mode may be used for each coding block, but a prediction mode that signals the Motion Vector Difference (MVD), a mode that uses template matching, etc., may or may not be used. In this case, the reference samples used as the template region may be the left sample and / or the top sample of the coding block to be predicted, and the region to be used may be determined according to the geometric partitioning method. Motion information can be derived by minimizing the difference between the reconstructed reference samples around the coding block used as the template region and the template region of the reference picture. When performing inter prediction on multiple partitions, inter prediction may be performed based on a prediction mode using affine transformation, a prediction mode using affine transformation and template matching, or a prediction mode based on affine transformation and TIP reference blocks.

[0307] Regarding the performance of intra prediction and / or inter prediction for sub-blocks or multiple partitions, it can be understood as seen in S320 and S330 of FIG. 3, and as it has been explained in more detail through FIG. 14 to 17, a detailed explanation will be omitted here.

[0308] Referring to FIG. 19, a final prediction block for the current block can be generated through a weighted sum operation for the first prediction block and the second prediction block (S1940).

[0309] According to one embodiment of the present disclosure, a final prediction block for the current block can be generated through a weighted sum operation for a first prediction block and a second prediction block, and the process may be understood as a blending of the boundaries or boundary samples of the first prediction block and the second prediction block.

[0310] For example, the weighted sum operation for the first prediction block and the second prediction block can be performed by applying predefined weights.

[0311] For example, the weighted sum operation for the first prediction block and the second prediction block can be performed by applying weights adjusted according to the difference in predicted pixel values.

[0312] For example, the weighted sum operation for the first prediction block and the second prediction block can be performed by applying weights adjusted using reference restoration samples of previously restored surrounding blocks.

[0313] For example, the weighted sum operation for the first prediction block and the second prediction block can be performed by applying weights adjusted according to the difference in POC (Picture Of Count).

[0314] For example, it may be performed by applying all four of the aforementioned methods.

[0315] According to one embodiment of the present disclosure, a predetermined mask matrix may be applied to a first prediction block and a second prediction block to perform a weighted sum operation. The predetermined mask matrix may include weight information for the weighted sum operation.

[0316] The weights may be determined through a linear function or through a non-linear function. The non-linear function may include at least one of a sigmoid or a trigonometric function. For example, the weights may be derived according to the aforementioned mathematical formula 1.

[0317] According to one embodiment of the present disclosure, a weighted sum operation for a first prediction block and a second prediction block may be performed based on a regression model.

[0318] When performing a regression-based weighted sum operation, the mask matrix of the coding block can be derived using previously restored reference samples adjacent to the coding block. Alternatively, the mask matrix of the sub-block can be derived using previously restored reference samples at the sub-block level of the coding block.

[0319] According to one embodiment of the present disclosure, weights may be determined based on a lookup table.

[0320] For example, the values ​​of the lookup table for calculating weights may be quantized and fixed as described above. Alternatively, they may be applied after being finely adjusted according to the angle and / or aspect ratio of the block to which the mask matrix is ​​applied.

[0321] For example, weights can be calculated using the values ​​stored in the lookup table as they are, quantized. Alternatively, weights can be calculated using values ​​scaled by applying a predefined scaling factor to the stored values.

[0322] For example, one or more lists for the lookup table may be constructed. In this case, an index to specify the list and / or the area where the weighted sum operation is performed may be signaled to the image decoder.

[0323] Meanwhile, according to one embodiment of the present disclosure, a flag related to a weighted sum operation may be signaled for each level.

[0324] Regarding the weighted sum operation for multiple partitions, it can be understood as seen in S340 of FIG. 3, so a detailed explanation is omitted here.

[0325] Meanwhile, the present disclosure assumes a case where a sub-block partitioning method is performed on a coding block for which geometric partitioning prediction is performed.

[0326] According to one embodiment of the present disclosure, sequential prediction (reconstruction) can be performed according to a predetermined order when predicting a plurality of sub-blocks. In this case, sub-block(s) predicted prior to the sub-block currently to be predicted may be understood as sub-blocks with a reconstruction priority, and the sub-block currently to be predicted may be understood as sub-blocks with a reconstruction priority. In this case, in intra-prediction, the sub-blocks with a reconstruction priority are predicted using samples of sub-blocks with a reconstruction priority, thereby improving the prediction efficiency of the sub-blocks in the next reconstruction order. For example, directional prediction can be performed using reference samples of the reconstructed sub-blocks in intra-prediction. For example, when constructing merge candidates in inter-prediction, the reference location of the motion vector can be specified based on the sub-block currently to be predicted. Accordingly, the location of the temporal motion vector candidate may differ for each sub-block. Alternatively, when constructing merge candidates in inter-prediction, the reference location of the motion vector may be specified based on geometrically divided partitions.

[0327] According to one embodiment of the present disclosure, a flag associated with a sub-block partitioning method may be signaled to an image decoder. Since the flag associated with the sub-block partitioning method can be understood as identical to the image encoding method, a detailed description thereof is omitted here to avoid redundancy.

[0328] According to one embodiment of the present disclosure, a sub-block partitioning type for a sub-block can be determined and parsed. Since the sub-block partitioning type can be understood in the same way as in an image encoding method, a detailed description is omitted here to avoid redundancy.

[0329] Meanwhile, according to one embodiment of the present disclosure, a geometric partitioning mode can be determined and parsed for each sub-block. Regarding the determination of the geometric partitioning mode of a sub-block of the reconstruction priority, since it can be understood in the same way as in an image encoding method, a detailed explanation is omitted here to avoid redundancy.

[0330] According to one embodiment of the present disclosure, when a geometric partitioning mode for a sub-block of the restoration priority is determined, a geometric partitioning mode for a sub-block of the post-restoration priority can be determined. Since the determination of the geometric partitioning mode and parsing of the sub-block of the post-restoration priority can be understood in the same way as in image encoding methods, a detailed description is omitted here to avoid redundancy.

[0331] Additionally, according to the present disclosure, intra prediction for a sub-block may be performed according to a determined intra prediction mode. Since the determination and parsing of the intra prediction mode can be understood in the same way as in image encoding methods, a detailed description is omitted here to avoid redundancy.

[0332] According to the present disclosure, inter prediction for a sub-block can be performed according to a determined inter prediction mode. Since the determination and parsing of the inter prediction mode can be understood in the same way as in image encoding methods, a detailed description is omitted here to avoid redundancy.

[0333] The various embodiments of the present disclosure are not intended to list all possible combinations but to describe representative aspects of the present disclosure, and the matters described in the various embodiments may be applied independently or in combination of two or more.

[0334] In addition, various embodiments of the present disclosure may be implemented by hardware, firmware, software, or a combination thereof. In the case of implementation by hardware, it may be implemented by one or more ASICs (Application Specific Integrated Circuits), DSPs (Digital Signal Processors), DSPDs (Digital Signal Processing Devices), PLDs (Programmable Logic Devices), FPGAs (Field Programmable Gate Arrays), general processors, controllers, microcontrollers, microprocessors, etc.

[0335] The scope of the present disclosure includes software or machine-executable instructions (e.g., operating system, application, firmware, program, etc.) that enable an operation according to a method of various embodiments to be executed on a device or computer, and a non-transitory computer-readable medium on which such software or instructions, etc. are stored and executable on a device or computer.

[0336] The present invention can be used to encode / decode images.

Claims

1. A step of dividing the current block into a plurality of partitions based on geometric partitioning, wherein the plurality of partitions include a first partition and a second partition; A step of generating a first prediction block for the first partition by performing either an intra prediction or an inter prediction; A step of generating a second prediction block for the second partition by performing either an intra prediction or an inter prediction; and The method includes the step of generating a final prediction block for the current block through a weighted sum operation for the first prediction block and the second prediction block, wherein A video signal decoding method in which the weights for the above weighted sum operation are determined based on a lookup table.

2. In Paragraph 1, The above weight is determined based on the value stored in the above lookup table, but, A method for decoding an image signal, wherein the above value is a constant quantized to the power of 2.

3. In Paragraph 1, The above weight is determined based on a scaled value of the value stored in the above lookup table, wherein A video signal decoding method in which the above scaling is performed by applying a predefined scaling factor to the value stored in the above lookup table.

4. In Paragraph 1, The above weight is determined based on a value that adaptively adjusts the value stored in the lookup table, wherein A method for decoding an image signal, wherein the above adjustment is performed based on at least one of the angle for performing the weighted sum operation or the aspect ratio of the current block.

5. In Paragraph 1, The above weights are determined based on a predetermined non-linear function, but, A method for decoding an image signal, wherein the above-mentioned predetermined non-linear function comprises at least one of a sigmoid or a trigonometric function.

6. In Paragraph 1, The above lookup table consists of one or more lists, and A video signal decoding method in which an index for specifying one or more of the above lists is signaled.

7. In Paragraph 1, A method for decoding an image signal, wherein the size of the region where the above weighted sum operation is performed is determined to be one of 1 / 4, 1 / 2, 1, 2, or 4 times a predetermined parameter (τ).

8. In Paragraph 1, A video signal decoding method in which the above-mentioned current block is one of a plurality of sub-blocks divided based on an Intra Sub-Partition mode.

9. A step of dividing the current block into a plurality of partitions based on geometric partitioning, wherein the plurality of partitions include a first partition and a second partition; A step of generating a first prediction block for the first partition by performing either an intra prediction or an inter prediction; A step of generating a second prediction block for the second partition by performing either an intra prediction or an inter prediction; and The method includes the step of generating a final prediction block for the current block through a weighted sum operation for the first prediction block and the second prediction block, wherein A video signal encoding method in which weights for the above weighted sum operation are determined based on a lookup table.

10. In a digital storage medium for storing a video bitstream, A digital storage medium that is encoded by a video signal encoding method comprising the steps of: dividing a current block into a plurality of partitions based on geometric partitioning, wherein the plurality of partitions include a first partition and a second partition; generating a first prediction block for the first partition by performing either an intra prediction or an inter prediction; generating a second prediction block for the second partition by performing either an intra prediction or an inter prediction; and generating a final prediction block for the current block through a weighted sum operation on the first prediction block and the second prediction block, wherein the weights for the weighted sum operation are determined based on a lookup table.

Images