Method and apparatus for video coding for efficient transmission of geometric partitioning information

The method efficiently transmits geometric segmentation information by using high-level information and difference calculations to improve inter prediction, addressing the limitations of traditional block-based prediction and enhancing video coding efficiency and image quality.

WO2024237446A1PCT designated stage expired Publication Date: 2024-11-21HYUNDAI MOTOR CO LTD +2
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
PCT/KR2024/003079
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2024-03-08
Filing Date
2024-03-11
Publication Date
2024-11-21

AI Technical Summary

Technical Problem

Current video coding technologies face challenges in efficiently transmitting geometric segmentation information, particularly in predicting and encoding non-square or rectangular blocks, which limits the ability to accurately predict objects of arbitrary shapes in images, leading to suboptimal coding efficiency and picture quality.

Method used

A method and device for efficiently transmitting geometric segmentation information by acquiring high-level information related to geometric segmentation mode, checking for contact points between neighboring blocks, generating division boundary predictors, and using difference information to create split blocks, thereby improving inter prediction accuracy and coding efficiency.

Benefits of technology

This approach enhances video coding efficiency and image quality by enabling more precise geometric division and prediction, overcoming limitations of traditional block-based prediction methods.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure KR2024003079_21112024_PF_FP_ABST
    Figure KR2024003079_21112024_PF_FP_ABST
Patent Text Reader

Abstract

The present embodiment discloses a video coding method and apparatus for efficient transmission of geometric partitioning information. In the present embodiment, an image decoding apparatus acquires high-level information related to application of a geometric partitioning mode and checks whether or not there is a contact point according to a partitioning boundary of neighboring blocks at a block boundary between a neighboring block and the current block. If there are one or more contact points, the image decoding apparatus generates a partitioning boundary predictor of the current block by using partitioning boundary information regarding the neighboring block. The image decoding apparatus acquires differential information regarding the geometric partitioning mode and generates partitioning boundary information regarding the current block by adding the partitioning boundary predictor and the differential information. The image decoding apparatus generates partitioning blocks of the current block on the basis of the high-level information and the partitioning boundary information.
Need to check novelty before this filing date? Find Prior Art

Description

Method and device for video coding for efficient transmission of geometric segmentation information

[0001] The present disclosure relates to a video coding method and device for efficient transmission of geometric segmentation information.

[0002] The content described below merely provides background information related to the present invention and does not constitute prior art.

[0003] Since video data has a large amount of data compared to voice data or still image data, it requires a lot of hardware resources, including memory, to store or transmit it without processing for compression.

[0004] Therefore, when storing or transmitting video data, an encoder is used to compress the video data and store or transmit it, and a decoder receives the compressed video data, decompresses it, and plays it back. Examples of such video compression technologies include H.264 / AVC, HEVC (High Efficiency Video Coding), and VVC (Versatile Video Coding), which improves encoding efficiency by about 30% compared to HEVC.

[0005] However, as the size, resolution, and frame rate of images are gradually increasing, and the amount of data that needs to be encoded is also increasing, a new compression technology that has better encoding efficiency and better image quality improvement than existing compression technologies is required.

[0006] In a video coding method and device, an inter prediction technique generates a prediction block using pixels in temporally adjacent pictures to predict the current block. To improve the performance of inter prediction, techniques are attempted that perform inter prediction in units of blocks, which are the basic units of encoding and decoding, or in units of one or more sub-blocks for a single block. The above-described block-based prediction techniques are limited to prediction using square or rectangular block divisions, and therefore have difficulty in performing predictions for objects of arbitrary shapes within an image.

[0007] To overcome the limitations described above, in addition to square or rectangular block partitioning, methods utilizing arbitrary block partitioning have been attempted. A representative example of arbitrary block partitioning is partitioning rectangular blocks using straight lines with arbitrary angles. During the VVC standardization process, a new technique was proposed: partitioning a block using arbitrary line segments and blending the resulting blocks to produce a single final inter predictor. This technique, named Geometric Partitioning Mode (GPM), was ultimately incorporated into the VVC standard. The aforementioned GPM is used only in merge mode among inter prediction modes.

[0008] In GPM technology, the encoder transmits information about geometric segmentation to the decoder. Therefore, to improve video encoding efficiency and enhance video quality, a method for efficiently transmitting information about geometric segmentation during prediction according to geometric segmentation mode is needed.

[0009] The present disclosure aims to provide a video coding method and device that efficiently transmits information about geometric segmentation and additional information in prediction according to a geometric segmentation mode.

[0010] According to an embodiment of the present disclosure, a method for restoring a current block, performed by an image decoding device, is provided, characterized in that the method further comprises: obtaining high-level information related to application of a geometric segmentation mode; confirming whether there is a neighboring block restored according to the geometric segmentation mode; and confirming whether there is a contact point according to a segmentation boundary of the neighboring blocks at a block boundary between the neighboring block and the current block, and, if there is one or more contact points, generating a segmentation boundary predictor of the current block using segmentation boundary information of the neighboring blocks; obtaining differential information of the geometric segmentation mode; generating segmentation boundary information of the current block by adding the segmentation boundary predictor and the differential information; and generating segment blocks of the current block based on the high-level information and the segmentation boundary information.

[0011] According to another embodiment of the present disclosure, a method for encoding a current block, performed by an image encoding device, is provided, comprising: determining high-level information related to application of a geometric segmentation mode; generating segmentation boundary information of the current block based on the high-level information; generating segmentation blocks of the current block based on the segmentation boundary information; checking whether there exists a neighboring block reconstructed according to the geometric segmentation mode; and checking whether there exists a contact point according to the segmentation boundaries of the neighboring blocks at a block boundary between the neighboring block and the current block, and, if there exists one or more contact points, generating a segmentation boundary predictor of the current block using the segmentation boundary information of the neighboring blocks; and generating difference information by subtracting the segmentation boundary predictor from the segmentation boundary information of the current block.

[0012] According to another embodiment of the present disclosure, a computer-readable recording medium storing a bitstream generated by an image encoding method is provided, characterized in that the image encoding method comprises: determining high-level information related to application of a geometric segmentation mode; generating segmentation boundary information of a current block based on the high-level information; generating segmentation blocks of the current block based on the segmentation boundary information; checking whether there exists a neighboring block restored according to the geometric segmentation mode; and checking whether there exists a contact point according to the segmentation boundaries of the neighboring blocks at a block boundary between the neighboring block and the current block, and if there exists one or more contact points, generating a segmentation boundary predictor of the current block using the segmentation boundary information of the neighboring blocks; and generating difference information by subtracting the segmentation boundary predictor from the segmentation boundary information of the current block.

[0013] As described above, according to the present embodiment, in prediction according to a geometric segmentation mode, a video coding method and device for efficiently transmitting information and additional information about geometric segmentation are provided, thereby improving video encoding efficiency and improving video quality.

[0014] FIG. 1 is an exemplary block diagram of an image encoding device capable of implementing the techniques of the present disclosure.

[0015] Figure 2 is a drawing for explaining a method of dividing a block using the QTBTTT (QuadTree plus BinaryTree TernaryTree) structure.

[0016] FIGS. 3A and 3B are diagrams illustrating multiple intra prediction modes, including wide-angle intra prediction modes.

[0017] Figure 4 is an example diagram of the surrounding blocks of the current block.

[0018] FIG. 5 is an exemplary block diagram of an image decoding device capable of implementing the techniques of the present disclosure.

[0019] Figure 6 is an example diagram showing block division according to geometric division.

[0020] Figures 7a and 7b are examples showing straight lines that divide a block into two.

[0021] Figure 8 is an example diagram conceptually illustrating inter prediction using geometric segmentation.

[0022] Figure 9 is an example diagram showing a GPM (Geometric Partition Mode) merge list used for geometric motion prediction.

[0023] FIGS. 10A to 10D are exemplary diagrams showing prediction of geometric segmentation information according to one embodiment of the present disclosure.

[0024] FIGS. 11A to 11C are exemplary diagrams showing a composite split mode according to one embodiment of the present disclosure.

[0025] FIG. 12 is a flowchart illustrating a method for an image encoding device to encode a current block according to an embodiment of the present disclosure.

[0026] FIG. 13 is a flowchart illustrating a method for an image decoding device to restore a current block according to one embodiment of the present disclosure.

[0027] Hereinafter, embodiments of the present invention will be described in detail with reference to exemplary drawings. When designating components in each drawing, it should be noted that, where possible, identical components are given the same reference numerals, even if they appear in different drawings. Furthermore, in describing the present embodiments, detailed descriptions of related known structures or functions will be omitted if they are deemed to obscure the gist of the present embodiments.

[0028] FIG. 1 is an exemplary block diagram of an image encoding device capable of implementing the techniques of the present disclosure. Hereinafter, the image encoding device and its subcomponents will be described with reference to the illustration in FIG. 1.

[0029] The video encoding device may be configured to include a picture segmentation unit (110), a prediction unit (120), a subtractor (130), a transformation unit (140), a quantization unit (145), a reordering unit (150), an entropy encoding unit (155), an inverse quantization unit (160), an inverse transformation unit (165), an adder (170), a loop filter unit (180), and a memory (190).

[0030] Each component of the video encoding device may be implemented in hardware, software, or a combination of hardware and software. Furthermore, the functions of each component may be implemented in software, with a microprocessor executing the software functions corresponding to each component.

[0031] A single image (video) is composed of one or more sequences containing multiple pictures. Each picture is divided into multiple regions, and encoding is performed for each region. For example, a single picture is divided into one or more tiles and / or slices. Here, one or more tiles can be defined as a tile group. Each tile or slice is divided into one or more Coding Tree Units (CTUs). Each CTU is then divided into one or more Coding Units (CUs) by a tree structure. Information applied to each CU is encoded as the syntax of the CU, and information commonly applied to CUs included in a CTU is encoded as the syntax of the CTU. In addition, information commonly applied to all blocks within a single slice is encoded as the syntax of the slice header, and information applied to all blocks constituting one or more pictures is encoded in the Picture Parameter Set (PPS) or the picture header. Furthermore, information commonly referenced by multiple pictures is encoded in a Sequence Parameter Set (SPS). And, information commonly referenced by one or more SPS is encoded in a Video Parameter Set (VPS). In addition, information commonly applied to one tile or tile group may be encoded as syntax of a tile or tile group header. Syntaxes included in an SPS, PPS, slice header, tile or tile group header may be referred to as high level syntax.

[0032] The picture segmentation unit (110) determines the size of the CTU. Information about the size of the CTU (CTU size) is encoded as the syntax of SPS or PPS and transmitted to the image decoding device.

[0033] The picture segmentation unit (110) divides each picture constituting an image into a plurality of CTUs having a predetermined size, and then recursively divides the CTUs using a tree structure. A leaf node in the tree structure becomes a CU, which is a basic unit of encoding.

[0034] The tree structure may be a QuadTree (QT) in which an upper node (or parent node) is divided into four lower nodes (or child nodes) of the same size, a BinaryTree (BT) in which an upper node is divided into two lower nodes, or a TernaryTree (TT) in which an upper node is divided into three lower nodes in a 1:2:1 ratio, or a structure that mixes two or more of the QT structures, BT structures, and TT structures. For example, a QTBT (QuadTree plus BinaryTree) structure may be used, or a QTBTTT (QuadTree plus BinaryTree TernaryTree) structure may be used. Here, BTTT may be combined and referred to as a MTT (Multiple-Type Tree).

[0035] Figure 2 is a drawing for explaining a method of dividing a block using the QTBTTT structure.

[0036] As illustrated in FIG. 2, a CTU may first be split into a QT structure. The quadtree splitting may be repeated until the size of the splitting block reaches the minimum block size (MinQTSize) of the leaf node allowed in the QT. A first flag (QT_split_flag) indicating whether each node of the QT structure is split into four nodes of the lower layer is encoded by the entropy encoding unit (155) and signaled to the image decoding device. If the leaf node of the QT is not larger than the maximum block size (MaxBTSize) of the root node allowed in the BT, it may be further split into one or more of the BT structure or the TT structure. There may be multiple splitting directions in the BT structure and / or the TT structure. For example, there may be two directions in which the block of the corresponding node is split horizontally and two directions in which the block is split vertically. As illustrated in FIG. 2, when MTT splitting begins, a second flag (mtt_split_flag) indicating whether nodes have been split, and if splitting has occurred, a flag indicating the splitting direction (vertical or horizontal) and / or a flag indicating the splitting type (Binary or Ternary) are encoded by the entropy encoding unit (155) and signaled to the image decoding device.

[0037] Alternatively, before encoding the first flag (QT_split_flag) indicating whether each node is split into four nodes of a lower layer, a CU split flag (split_cu_flag) indicating whether the node is split may be encoded. If the CU split flag (split_cu_flag) value indicates that the node is not split, the block of the corresponding node becomes a leaf node in the split tree structure and becomes a CU (coding unit), which is a basic unit of encoding. If the CU split flag (split_cu_flag) value indicates that the node is split, the video encoding device starts encoding from the first flag in the above-described manner.

[0038] As another example of a tree structure, when QTBT is used, there may be two types: a type that horizontally splits the block of the corresponding node into two blocks of the same size (i.e., symmetric horizontal splitting) and a type that vertically splits it (i.e., symmetric vertical splitting). A split flag (split_flag) indicating whether each node of the BT structure is split into blocks of a lower layer and split type information indicating the type of split are encoded by the entropy encoding unit (155) and transmitted to the image decoding device. Meanwhile, there may additionally be a type that splits the block of the corresponding node into two blocks of an asymmetrical shape. The asymmetric shape may include a shape that splits the block of the corresponding node into two rectangular blocks with a size ratio of 1:3, or a shape that splits the block of the corresponding node in a diagonal direction.

[0039] A CU can have various sizes depending on the QTBT or QTBTTT partitioning from the CTU. Hereinafter, the block corresponding to the CU to be encoded or decoded (i.e., the leaf node of the QTBTTT) is referred to as the "current block." Depending on the QTBTTT partitioning employed, the current block may be rectangular as well as square.

[0040] The prediction unit (120) predicts the current block and generates a prediction block. The prediction unit (120) includes an intra prediction unit (122) and an inter prediction unit (124).

[0041] In general, each current block within a picture can be predictively coded. Prediction of the current block can typically be performed using either intra-prediction (using data from the picture containing the current block) or inter-prediction (using data from a picture coded before the picture containing the current block). Inter-prediction encompasses both unidirectional and bidirectional prediction.

[0042] The intra prediction unit (122) predicts pixels within the current block using pixels (reference pixels) located around the current block within the current picture including the current block. There are multiple intra prediction modes depending on the prediction direction. For example, as shown in Fig. 3a, the multiple intra prediction modes may include two non-directional modes including the Planar mode and the DC mode, and 65 directional modes. The surrounding pixels to be used and the calculation formula are defined differently depending on each prediction mode.

[0043] For efficient directional prediction for a rectangular current block, directional modes (intra prediction modes 67 to 80 and -1 to -14) indicated by dotted arrows in Fig. 3b may be additionally used. These may be referred to as "wide-angle intra-prediction modes." In Fig. 3b, the arrows point to corresponding reference samples used for prediction, and do not indicate the prediction direction. The prediction direction is opposite to the direction indicated by the arrows. Wide-angle intra-prediction modes are modes that perform prediction in the opposite direction of a specific directional mode without additional bit transmission when the current block is rectangular. At this time, among the wide-angle intra-prediction modes, some wide-angle intra-prediction modes available for the current block may be determined based on the ratio of the width and height of the rectangular current block. For example, wide-angle intra prediction modes (intra prediction modes 67 to 80) having an angle less than 45 degrees are available when the current block is a rectangular shape whose height is smaller than its width, and wide-angle intra prediction modes (intra prediction modes -1 to -14) having an angle greater than -135 degrees are available when the current block is a rectangular shape whose width is larger than its height.

[0044] The intra prediction unit (122) can determine an intra prediction mode to be used to encode the current block. In some examples, the intra prediction unit (122) can encode the current block using multiple intra prediction modes and select an appropriate intra prediction mode to be used from the tested modes. For example, the intra prediction unit (122) can calculate bit-rate distortion values ​​using rate-distortion analysis for multiple tested intra prediction modes and select an intra prediction mode with the best bit-rate distortion characteristics among the tested modes.

[0045] The intra prediction unit (122) selects one intra prediction mode from among multiple intra prediction modes and predicts the current block using surrounding pixels (reference pixels) and an operation formula determined according to the selected intra prediction mode. Information about the selected intra prediction mode is encoded by the entropy encoding unit (155) and transmitted to the image decoding device.

[0046] The inter prediction unit (124) generates a prediction block for the current block using a motion compensation process. The inter prediction unit (124) searches for a block most similar to the current block within reference pictures that were encoded and decoded before the current picture, and generates a prediction block for the current block using the searched block. Then, a motion vector (MV) corresponding to the displacement between the current block within the current picture and the prediction block within the reference picture is generated. Generally, motion estimation is performed on the luma component, and the motion vector calculated based on the luma component is used for both the luma component and the chroma component. The motion information including information on the reference picture used to predict the current block and information on the motion vector is encoded by the entropy encoding unit (155) and transmitted to the image decoding device.

[0047] The inter prediction unit (124) may perform interpolation on a reference picture or a reference block to improve prediction accuracy. That is, subsamples between two consecutive integer samples are interpolated by applying filter coefficients to a plurality of consecutive integer samples including the two integer samples. When a process of searching for a block most similar to the current block is performed on the interpolated reference picture, the motion vector can be expressed up to a precision in decimal units rather than a precision in integer sample units. The precision or resolution of the motion vector can be set differently for each target region to be encoded, such as a slice, tile, CTU, CU, etc. When such adaptive motion vector resolution (AMVR) is applied, information on the motion vector resolution to be applied to each target region must be signaled for each target region. For example, when the target region is a CU, information on the motion vector resolution applied to each CU is signaled. Information on the motion vector resolution may be information indicating the precision of a differential motion vector, which will be described later.

[0048] Meanwhile, the inter prediction unit (124) can perform inter prediction using bi-prediction. In the case of bi-prediction, two reference pictures and two motion vectors indicating the block position most similar to the current block within each reference picture are used. The inter prediction unit (124) selects a first reference picture and a second reference picture from reference picture list 0 (RefPicList0) and reference picture list 1 (RefPicList1), respectively, and searches for a block similar to the current block within each reference picture to generate a first reference block and a second reference block. Then, the first reference block and the second reference block are averaged or weighted averaged to generate a prediction block for the current block. Then, motion information including information on two reference pictures used to predict the current block and information on two motion vectors is transmitted to the entropy encoding unit (155). Here, reference picture list 0 may be composed of pictures that are before the current picture in display order among the restored pictures, and reference picture list 1 may be composed of pictures that are after the current picture in display order among the restored pictures. However, this is not necessarily limited to this, and restored pictures that are after the current picture in display order may be additionally included in reference picture list 0, and conversely, restored pictures that are before the current picture may be additionally included in reference picture list 1.

[0049] Various methods can be used to minimize the number of bits required to encode motion information.

[0050] For example, if the reference picture and motion vector of the current block are identical to those of a neighboring block, the motion information of the current block can be transmitted to the image decoding device by encoding information that can identify the neighboring block. This method is called 'merge mode'.

[0051] In merge mode, the inter prediction unit (124) selects a predetermined number of merge candidate blocks (hereinafter referred to as 'merge candidates') from the surrounding blocks of the current block.

[0052] As the surrounding blocks for deriving merge candidates, all or part of the left block (A0), the lower left block (A1), the upper block (B0), the upper right block (B1), and the upper left block (B2) adjacent to the current block within the current picture may be used, as illustrated in FIG. 4. In addition, a block located within a reference picture (which may or may not be the same as the reference picture used to predict the current block) other than the current picture in which the current block is located may be used as a merge candidate. For example, a block co-located with the current block within the reference picture or blocks adjacent to the block at the co-located block may be additionally used as a merge candidate. If the number of merge candidates selected by the method described above is less than a preset number, a 0 vector is added to the merge candidates.

[0053] The inter prediction unit (124) uses these surrounding blocks to construct a merge list containing a predetermined number of merge candidates. Among the merge candidates included in the merge list, the merge candidate to be used as motion information of the current block is selected and merge index information for identifying the selected candidate is generated. The generated merge index information is encoded by the entropy encoding unit (155) and transmitted to the video decoding device.

[0054] Merge Skip mode is a special case of merge mode. After quantization, when all transform coefficients for entropy encoding are close to zero, only neighboring block selection information is transmitted without transmitting residual signals. By utilizing merge skip mode, relatively high encoding efficiency can be achieved for low-motion images, still images, and screen content images.

[0055] Hereinafter, merge mode and merge skip mode are collectively referred to as merge / skip mode.

[0056] Another method for encoding motion information is Advanced Motion Vector Prediction (AMVP) mode.

[0057] In AMVP mode, the inter prediction unit (124) derives predicted motion vector candidates for the motion vector of the current block using neighboring blocks of the current block. As neighboring blocks used to derive predicted motion vector candidates, all or some of the left block (A0), the lower left block (A1), the upper block (B0), the upper right block (B1), and the upper left block (B2) adjacent to the current block in the current picture as shown in FIG. 4 may be used. In addition, a block located in a reference picture (which may or may not be the same as the reference picture used to predict the current block) other than the current picture in which the current block is located may be used as the neighboring block used to derive predicted motion vector candidates. For example, a block co-located with the current block in the reference picture or blocks adjacent to the block in the co-located block may be used. If the number of motion vector candidates is less than a preset number by the method described above, a 0 vector is added to the motion vector candidates.

[0058] The inter prediction unit (124) derives predicted motion vector candidates using the motion vectors of these surrounding blocks, and determines a predicted motion vector for the motion vector of the current block using the predicted motion vector candidates. Then, the predicted motion vector is subtracted from the motion vector of the current block to produce a differential motion vector.

[0059] The predicted motion vector can be obtained by applying a predefined function (e.g., median, mean, etc.) to the predicted motion vector candidates. In this case, the image decoding device also knows the predefined function. In addition, since the surrounding blocks used to derive the predicted motion vector candidates are blocks that have already been encoded and decoded, the image decoding device also already knows the motion vectors of the surrounding blocks. Therefore, the image encoding device does not need to encode information to identify the predicted motion vector candidates. Therefore, in this case, information about the differential motion vector and information about the reference picture used to predict the current block are encoded.

[0060] Alternatively, the predicted motion vector can be determined by selecting one of the predicted motion vector candidates. In this case, information for identifying the selected predicted motion vector candidate is additionally encoded, along with information about the differential motion vector and the reference picture used to predict the current block.

[0061] The subtractor (130) subtracts the prediction block generated by the intra prediction unit (122) or inter prediction unit (124) from the current block to generate a residual block.

[0062] The transformation unit (140) transforms residual signals within a residual block having pixel values ​​in a spatial domain into transform coefficients in a frequency domain. The transformation unit (140) may transform the residual signals within the residual block using the entire size of the residual block as a transformation unit, or may divide the residual block into a plurality of sub-blocks and use the sub-blocks as transformation units to perform the transformation. Alternatively, the residual signals may be transformed using only the transformation domain sub-block as a transformation unit by dividing the sub-blocks into two sub-blocks, that is, a transformation domain and a non-transform domain. Here, the transformation domain sub-block may be one of two rectangular blocks having a size ratio of 1:1 with respect to the horizontal axis (or vertical axis). In this case, a flag (cu_sbt_flag) indicating that only a sub-block has been converted, directionality (vertical / horizontal) information (cu_sbt_horizontal_flag), and / or position information (cu_sbt_pos_flag) are encoded by the entropy encoding unit (155) and signaled to the image decoding device. In addition, the size of the conversion area sub-block may have a size ratio of 1:3 with respect to the horizontal axis (or vertical axis), and in this case, a flag (cu_sbt_quad_flag) distinguishing the corresponding division is additionally encoded by the entropy encoding unit (155) and signaled to the image decoding device.

[0063] Meanwhile, the transformation unit (140) can individually perform transformations on the residual block in the horizontal and vertical directions. For the transformation, various types of transformation functions or transformation matrices can be used. For example, a pair of transformation functions for horizontal transformation and vertical transformation can be defined as a Multiple Transform Set (MTS). The transformation unit (140) can select one transformation function pair with the best transformation efficiency among the MTS and transform the residual block in the horizontal and vertical directions, respectively. Information (mts_idx) on the transformation function pair selected among the MTS is encoded by the entropy encoding unit (155) and signaled to the image decoding device.

[0064] The quantization unit (145) quantizes the transform coefficients output from the transform unit (140) using quantization parameters and outputs the quantized transform coefficients to the entropy encoding unit (155). The quantization unit (145) may directly quantize a related residual block without transformation for a certain block or frame. The quantization unit (145) may also apply different quantization coefficients (scaling values) according to the positions of the transform coefficients within the transform block. The quantization matrix applied to the quantized transform coefficients arranged in two dimensions may be encoded and signaled to an image decoding device.

[0065] The rearrangement unit (150) can perform rearrangement of coefficient values ​​for quantized residual values.

[0066] The reordering unit (150) can change a two-dimensional coefficient array into a one-dimensional coefficient sequence by using coefficient scanning. For example, the reordering unit (150) can output a one-dimensional coefficient sequence by scanning from the DC coefficient to the coefficients of the high-frequency region by using a zig-zag scan or a diagonal scan. Depending on the size of the transformation unit and the intra prediction mode, a vertical scan that scans the two-dimensional coefficient array in the column direction or a horizontal scan that scans the two-dimensional block-shaped coefficients in the row direction may be used instead of the zig-zag scan. That is, depending on the size of the transformation unit and the intra prediction mode, the scanning method to be used may be determined among the zig-zag scan, the diagonal scan, the vertical scan, and the horizontal scan.

[0067] The entropy encoding unit (155) generates a bitstream by encoding a sequence of one-dimensional quantized transform coefficients output from the rearrangement unit (150) using various encoding methods such as CABAC (Context-based Adaptive Binary Arithmetic Code) and Exponential Golomb.

[0068] In addition, the entropy encoding unit (155) encodes information related to block division, such as CTU size, CU division flag, QT division flag, MTT division type, and MTT division direction, so that the image decoding device can divide the block in the same manner as the image encoding device. In addition, the entropy encoding unit (155) encodes information about a prediction type indicating whether the current block is encoded by intra prediction or inter prediction, and encodes intra prediction information (i.e., information about an intra prediction mode) or inter prediction information (information about an encoding mode of motion information (merge mode or AMVP mode), a merge index in the case of a merge mode, and a reference picture index and a differential motion vector in the case of an AMVP mode) according to the prediction type. In addition, the entropy encoding unit (155) encodes information related to quantization, that is, information about a quantization parameter and information about a quantization matrix.

[0069] The inverse quantization unit (160) inversely quantizes the quantized transform coefficients output from the quantization unit (145) to generate transform coefficients. The inverse transform unit (165) transforms the transform coefficients output from the inverse quantization unit (160) from the frequency domain to the spatial domain to restore the residual block.

[0070] An adder (170) adds the restored residual block and the predicted block generated by the prediction unit (120) to restore the current block. The pixels within the restored current block are used as reference pixels when intra-predicting the next block.

[0071] The loop filter unit (180) performs filtering on restored pixels to reduce blocking artifacts, ringing artifacts, blurring artifacts, etc. that occur due to block-based prediction and transformation / quantization. The loop filter unit (180) may include all or part of a deblocking filter (182), a sample adaptive offset (SAO) filter (184), and an adaptive loop filter (ALF, 186) as an in-loop filter.

[0072] The deblocking filter (182) filters the boundaries between restored blocks to remove blocking artifacts caused by block-based encoding / decoding, and the SAO filter (184) and the ALF (186) perform additional filtering on the deblocking-filtered image. The SAO filter (184) and the ALF (186) are filters used to compensate for the differences between restored pixels and original pixels caused by lossy coding. The SAO filter (184) improves not only subjective image quality but also encoding efficiency by applying an offset in units of CTUs. In contrast, the ALF (186) performs block-based filtering, and compensates for distortion by applying different filters by distinguishing the edges and degrees of variation of the corresponding block. Information on filter coefficients to be used in the ALF can be encoded and signaled to an image decoding device.

[0073] The restored blocks filtered through the deblocking filter (182), SAO filter (184), and ALF (186) are stored in the memory (190). When all blocks within a picture are restored, the restored picture can be used as a reference picture for inter-predicting blocks within a picture to be encoded later.

[0074] The video encoding device can store the bitstream of encoded video data on a non-transitory storage medium or transmit it to the video decoding device using a communication network.

[0075] FIG. 5 is an exemplary block diagram of an image decoding device capable of implementing the techniques of the present disclosure. Hereinafter, the image decoding device and its subcomponents will be described with reference to FIG. 5.

[0076] The video decoding device may be configured to include an entropy decoding unit (510), a rearrangement unit (515), an inverse quantization unit (520), an inverse transformation unit (530), a prediction unit (540), an adder (550), a loop filter unit (560), and a memory (570).

[0077] Similar to the video encoding device of FIG. 1, each component of the video decoding device may be implemented in hardware, software, or a combination of hardware and software. Furthermore, the functions of each component may be implemented in software, with a microprocessor executing the software functions corresponding to each component.

[0078] The entropy decoding unit (510) decodes the bitstream generated by the image encoding device to extract information related to block division, thereby determining the current block to be decoded, and extracts prediction information, information on residual signals, etc. required to restore the current block.

[0079] The entropy decoding unit (510) extracts information about the CTU size from the Sequence Parameter Set (SPS) or the Picture Parameter Set (PPS), determines the size of the CTU, and divides the picture into CTUs of the determined size. Then, the CTU is determined as the top layer of the tree structure, i.e., the root node, and the CTU is divided using the tree structure by extracting division information about the CTU.

[0080] For example, when splitting a CTU using the QTBTTT structure, first, the first flag (QT_split_flag) related to the splitting of QT is extracted, and each node is split into four nodes of the lower layer. Then, for the nodes corresponding to the leaf nodes of QT, the second flag (mtt_split_flag) related to the splitting of MTT and the split direction (vertical / horizontal) and / or split type (binary / ternary) information are extracted, and the corresponding leaf nodes are split into the MTT structure. Accordingly, each node below the leaf nodes of QT are split recursively into the BT or TT structure.

[0081] As another example, when splitting a CTU using the QTBTTT structure, the CU split flag (split_cu_flag) indicating whether the CU is split is first extracted, and if the block is split, the first flag (QT_split_flag) may be extracted. During the splitting process, each node may undergo zero or more repeated QT splits followed by zero or more repeated MTT splits. For example, a CTU may undergo an MTT split right away, or conversely, may undergo only multiple QT splits.

[0082] As another example, when splitting a CTU using the QTBT structure, the first flag (QT_split_flag) related to the splitting of QT is extracted, and each node is split into four nodes of the lower layer. Furthermore, for nodes corresponding to leaf nodes of QT, a split flag (split_flag) indicating whether to further split into BTs and splitting direction information are extracted.

[0083] Meanwhile, when the entropy decoding unit (510) determines the current block to be decoded by using the division of the tree structure, it extracts information on the prediction type indicating whether the current block is intra-predicted or inter-predicted. If the prediction type information indicates intra-prediction, the entropy decoding unit (510) extracts syntax elements for intra-prediction information (intra-prediction mode) of the current block. If the prediction type information indicates inter-prediction, the entropy decoding unit (510) extracts syntax elements for inter-prediction information, i.e., information indicating a motion vector and a reference picture referenced by the motion vector.

[0084] Additionally, the entropy decoding unit (510) extracts information about the quantized transform coefficients of the current block as information related to quantization and information about residual signals.

[0085] The rearrangement unit (515) can change the sequence of one-dimensional quantized transform coefficients entropy-decoded in the entropy decoding unit (510) back into a two-dimensional coefficient array (i.e., block) in the reverse order of the coefficient scanning performed by the image encoding device.

[0086] The inverse quantization unit (520) inversely quantizes the quantized transform coefficients and inversely quantizes the quantized transform coefficients using the quantization parameters. The inverse quantization unit (520) may also apply different quantization coefficients (scaling values) to the quantized transform coefficients arranged in two dimensions. The inverse quantization unit (520) may perform inverse quantization by applying a matrix of quantized coefficients (scaling values) from an image encoding device to a two-dimensional array of quantized transform coefficients.

[0087] The inverse transform unit (530) inversely transforms the inverse quantized transform coefficients from the frequency domain to the spatial domain to restore residual signals, thereby generating a residual block for the current block.

[0088] In addition, when the inverse transform unit (530) inversely transforms only a portion of a transform block (sub-block), it extracts a flag (cu_sbt_flag) indicating that only a sub-block of the transform block has been transformed, directionality (vertical / horizontal) information (cu_sbt_horizontal_flag) of the sub-block, and / or position information (cu_sbt_pos_flag) of the sub-block, and inversely transforms the transform coefficients of the corresponding sub-block from the frequency domain to the spatial domain to restore residual signals, and fills “0” values ​​with residual signals for areas that have not been inversely transformed, thereby generating a final residual block for the current block.

[0089] In addition, when MTS is applied, the inverse transform unit (530) determines a transform function or a transform matrix to be applied in the horizontal and vertical directions using MTS information (mts_idx) signaled from the image encoding device, and performs inverse transform on the transform coefficients within the transform block in the horizontal and vertical directions using the determined transform function.

[0090] The prediction unit (540) may include an intra prediction unit (542) and an inter prediction unit (544). The intra prediction unit (542) is activated when the prediction type of the current block is intra prediction, and the inter prediction unit (544) is activated when the prediction type of the current block is inter prediction.

[0091] The intra prediction unit (542) determines the intra prediction mode of the current block among a plurality of intra prediction modes from the syntax elements for the intra prediction mode extracted from the entropy decoding unit (510), and predicts the current block using reference pixels around the current block according to the intra prediction mode.

[0092] The inter prediction unit (544) uses the syntax elements for the inter prediction mode extracted from the entropy decoding unit (510) to determine the motion vector of the current block and the reference picture referenced by the motion vector, and predicts the current block using the motion vector and the reference picture.

[0093] An adder (550) adds the residual block output from the inverse transform unit (530) and the predicted block output from the inter prediction unit (544) or the intra prediction unit (542) to restore the current block. The pixels within the restored current block are used as reference pixels when intra-predicting a block to be decoded later.

[0094] The loop filter unit (560) may include a deblocking filter (562), an SAO filter (564), and an ALF (566) as in-loop filters. The deblocking filter (562) deblocks the boundaries between restored blocks to remove blocking artifacts caused by block-by-block decoding. The SAO filter (564) and the ALF (566) perform additional filtering on restored blocks after deblocking filtering to compensate for differences between restored pixels and original pixels caused by lossy coding. The filter coefficients of the ALF are determined using information about filter coefficients decoded from the non-stream.

[0095] The restored blocks filtered through the deblocking filter (562), SAO filter (564), and ALF (566) are stored in the memory (570). When all blocks within a picture are restored, the restored picture is used as a reference picture for inter-predicting blocks within a picture to be encoded later.

[0096]

[0097] The present embodiment relates to encoding and decoding of images (videos) as described above. More specifically, in prediction according to a geometric segmentation mode, a video coding method and device for efficiently transmitting information about geometric segmentation and additional information are provided.

[0098] The following embodiments may be performed by an inter prediction unit (124) within a video encoding apparatus. Additionally, they may be performed by an inter prediction unit (544) within a video decoding apparatus.

[0099] The video encoding device can generate signaling information related to the present embodiment in terms of rate distortion optimization in encoding the current block. The video encoding device can encode the signaling information using the entropy encoding unit (155) and then transmit it to the video decoding device. The video decoding device can decode the signaling information related to the decoding of the current block from the bitstream using the entropy decoding unit (510).

[0100] In the following description, the term "target block" may be used interchangeably with the current block or coding unit (CU). Alternatively, the term "target block" may also refer to a portion of a coding unit.

[0101] Also, a value of a flag being true indicates that the flag is set to 1. Also, a value of a flag being false indicates that the flag is set to 0.

[0102] I. Merge / Skip Mode of Inter-Prediction

[0103] Hereinafter, a method for constructing a merge candidate list of motion information in the merge / skip mode of inter prediction is described. To support the merge / skip mode, a video encoding device can construct a merge candidate list by selecting a preset number of merge candidates (e.g., 6).

[0104] The video encoding device searches for spatial merge candidates. The video encoding device searches for spatial merge candidates from surrounding blocks, as illustrated in FIG. 4. Up to four spatial merge candidates can be selected.

[0105] A video encoding device searches for temporal merge candidates. The video encoding device may add a co-located block as a temporal merge candidate, which is a block located in the same location as the current block within a reference picture (which may or may not be the same as the reference picture used to predict the current block) other than the current picture in which the target block is located. Only one temporal merge candidate may be selected.

[0106] A video encoding device searches for HMVP (History-based Motion Vector Predictor) candidates. The video encoding device can store the motion vectors of the previous h CUs (where h is a natural number) in a table and use them as merge candidates. The table size is 6, and the motion vectors of the previous CUs are stored in a First-in First Out (FIFO) manner. This indicates that up to 6 HMVP candidates are stored in the table. The video encoding device can set the most recent motion vectors among the HMVP candidates stored in the table as merge candidates.

[0107] The video encoding device searches for PAMVP (Pairwise Average MVP) candidates. The video encoding device can set the motion vector average of the first and second candidates in the merge candidate list as the merge candidate.

[0108] If the merge candidate list cannot be filled even after performing all of the above-described search processes (i.e., if the preset number cannot be filled), the video encoding device adds a zero motion vector as a merge candidate.

[0109] In terms of optimizing encoding efficiency, a video encoding device can determine a merge index that indicates one candidate within a merge candidate list. The video encoding device can then derive a motion vector predictor (MVP) from the merge candidate list using the merge index, and then determine the MVP as the motion vector of the current block. Furthermore, the video encoding device can signal the merge index to a video decoding device.

[0110] The video encoding device uses the same motion vector transmission method as the merge mode in the skip mode, but does not transmit the residual block corresponding to the difference between the current block and the predicted block.

[0111] The method for constructing the aforementioned merge candidate list can be performed in the same manner by a video decoding device. The video decoding device can decode the merge index. The video decoding device can then derive an MVP from the merge candidate list using the merge index, and then determine the MVP as the motion vector of the current block.

[0112] The following embodiments are described with a focus on an image decoding device, but can be implemented identically or similarly in an image encoding device.

[0113] II. Geometric Partitioning Mode (GPM)

[0114] The following embodiments are described with a focus on an image decoding device, but can be implemented identically or similarly in an image encoding device.

[0115] In VVC (Versatile Video Coding), GPM (extrapolation base Intra Prediction, EIP) uses prediction units of various shapes other than rectangular ones.

[0116] Figure 6 is an example diagram showing block division according to geometric division.

[0117] The video decoding device divides the current block into two parts based on a straight line perpendicular to a line segment having a constant angle (θ) and a constant distance (ρ) from the center of the block. Hereinafter, the two divided blocks are referred to as the first block partition and the second block partition. The term "block partition" is used interchangeably with the term "partitioned block." The straight line that divides the current block into two parts is referred to as the partitioning boundary.

[0118] Here, the center of the block represents a virtual position where the position at half the height of the block and the position at half the width of the block intersect with respect to the current block before division. The angle (θ) represents the angle rotated counterclockwise from the virtual horizontal axis passing through the center of the block to the line segment perpendicular to the division boundary. The distance (ρ) represents the distance between the center of the block and the division boundary.

[0119] As described above, the straight line that divides the block, i.e., the division boundary, divides the current block into two different block divisions. The image decoding device uses the geometric division described above to divide the current block into two blocks, a first block division and a second block division, based on the division boundary, for performing separate predictions.

[0120] As an example, in a conventional GPM, information on segmentation boundaries, such as the example in FIG. 6, is signaled from an image encoding device to an image decoding device. The image decoding device can decode geometric block segments of the current block using the information on the parsed segmentation boundaries. Here, the information on the segmentation boundaries can include an angle (θ) and a distance (ρ) based on the center of the block. In addition, the image encoding device can additionally signal information indicating whether or not to apply the GPM to the image decoding device.

[0121] Hereinafter, information on the segmentation boundary according to the geometric segmentation mode is used interchangeably with geometric segmentation mode information, geometric segmentation information, or segmentation information.

[0122] Figures 7a and 7b are examples showing straight lines that divide a block into two.

[0123] As another example, a lookup table including combinations of angles and distances for dividing a current block into a first block division and a second block division, as shown in FIGS. 7A and 7B and Table 1, may be constructed. Thereafter, an index indicating a combination of angles and distances within the lookup table may be signaled from a video encoding device to a video decoding device. The combination of angles and distances corresponding to each index may be defined based on a fixed lookup table according to an agreement between the video encoding device and the video decoding device. As another example, the lookup table may be adaptively reconfigured according to a pre-agreed rule.

[0124] As described above, the geometric segmentation shape is based on the segmentation boundary, which is a straight line representing the division of the block. Information about this straight line may include an index distanceIdx representing the distance (ρ) from the center of the block to the segmentation boundary, and an index angleIdx representing the angle (θ) of the line segment perpendicular to the segmentation boundary. The index representing the angle of the line segment perpendicular to the segmentation boundary may be set as illustrated in Fig. 7a. In addition, 64 geometric segmentation shapes according to these angles and distances may be set as illustrated in Fig. 7b.

[0125] 64 geometric partition shapes can be signaled using the merge_gpm_partition_idx syntax, which is an index indicating a geometric partition shape, as shown in Table 1. That is, the shapes that divide the current block into the first block partition and the second block partition according to various angles and various distances can be efficiently signaled using a single index.

[0126]

[0127] The index distanceIdx derived from the example in Fig. 7b is a value that excludes the size of the current block. Therefore, the actual distance between a pixel in the current block and a straight line can be calculated using the size information of the current block, the index angleIdx indicating the angle, and the index distanceIdx indicating the distance. Here, the actual distance is a value expressed in pixel units.

[0128] Meanwhile, weights for each pixel within the current block can be calculated using the actual distance. For example, for a pixel within the first block partition, as the actual distance between the pixel and the straight line increases, the weight of the predictor of the first block partition, as described above, may increase, and the weight of the predictor of the second block partition may decrease. For pixels located on the partition boundary, the two predictors may use weights with the same value. In this case, the sum of the weights of the two predictors for a single pixel remains 1.

[0129] Figure 8 is an example diagram conceptually illustrating inter prediction using geometric segmentation.

[0130] As in the example of Fig. 8, for two different block divisions of the current block existing in the current picture, the video decoding device performs prediction using each motion vector (mv0 or mv1). The video decoding device applies a weighted sum-based blending process to the prediction block for the first block division and the prediction block for the second block division to generate a final prediction block of the current block. In the example of Fig. 8, PART_0 represents the first block division, and PART_1 represents the second block division. In addition, the reference block of PART_0 represents the prediction block for the first block division, and the reference block of PART_1 represents the prediction block for the second block division.

[0131] In order to perform inter prediction of the current block as illustrated in FIG. 8, the image decoding device obtains a first prediction block using a motion vector for the first block division, and obtains a second prediction block using a motion vector for the second block division. At this time, the image decoding device, in the process of obtaining the prediction block for each block division, may obtain the prediction block in a form in which different weights are multiplied according to pixel positions as described above. In addition, the image decoding device may use a shift operation and a clipping operation in the process of generating a final prediction block from the prediction blocks in a form in which weights are multiplied.

[0132] Figure 9 is an example diagram showing a GPM (Geometric Partition Mode) merge list used for geometric motion prediction.

[0133] In performing geometric motion prediction as illustrated in FIG. 8, the image decoding device can select motion information for motion prediction from a merge list, as illustrated in FIG. 9, and then use the selected motion information.

[0134] However, unlike the existing block-based motion prediction technology, for geometric motion prediction, the image decoding device performs unidirectional prediction for one block division by limiting the prediction directionality, as illustrated in FIG. 9. This is because, compared to block-based motion prediction, when motion prediction is performed according to bidirectional prediction for each block division in geometric motion prediction, the memory bandwidth used for prediction doubles. Therefore, to efficiently solve the aforementioned memory bandwidth increase problem, a technology for limiting the prediction directionality for each block division can be applied.

[0135] In the case where the direction of prediction is limited for each block partition in geometric motion prediction, a GPM merge list for geometric motion prediction can be generated using an existing merge list rule. To generate a GPM merge list, the image decoding device first configures the merge list as described above. Thereafter, the image decoding device can generate a GPM merge list for geometric motion prediction from the merge list according to the prediction direction and the order in the list. At this time, the image decoding device adds motion information in the L0 direction to the GPM merge list to generate a merge candidate for geometric motion prediction of the first block partition. In addition, the image decoding device can add motion information in the L1 direction to the GPM merge list to generate a merge candidate for geometric motion prediction of the second block partition. That is, the image decoding device can derive unidirectional motion information in one direction from the existing bidirectional motion information, and then add the derived motion information to the GPM merge list.

[0136] As in the example of Fig. 9, in the existing geometric motion prediction method, the image decoding device can directly use the merge candidates in the GPM merge list as motion information for motion prediction of the first block division and the second block division.

[0137] Meanwhile, the video decoding device can derive motion information related to GPM in the AMVP mode of inter prediction similarly to what was described above.

[0138] Below, a method for efficiently transmitting additional information in addition to the aforementioned geometric segmentation information (i.e., information related to segmentation boundaries) is described.

[0139] The following embodiments are described with a focus on an image decoding device, but can be implemented identically or similarly in an image encoding device.

[0140] III. Embodiments according to the present disclosure

[0141] In this embodiment, for efficient transmission of additional information related to geometric segmentation, geometric segmentation mode information is predicted and a difference is transmitted to supplement the prediction. Here, the additional information includes high-level information related to the geometric segmentation mode and difference information of the geometric segmentation mode.

[0142] An image decoding device decodes high-level information from a bitstream to determine whether geometric segmentation is applied to an image to be decoded. In addition, the high-level information may include one or more of information indicating the number of blocks segmented according to a geometric segmentation mode, a minimum size of a block segmented according to the geometric segmentation mode, a minimum horizontal size (i.e., minimum width), a minimum vertical size (i.e., minimum height), and mask information applicable for blending processing in the geometric segmentation mode. Here, the size may be the number of pixels.

[0143] The video decoding device decodes block-by-block geometric partitioning mode information for block-by-block decoding. The video decoding device determines whether the geometric partitioning mode is applied to each block. At this time, a flag indicating whether the geometric partitioning mode is applied to each block may be used. If the current block is partitioned according to the geometric partitioning mode, the video decoding device derives the partition boundary information of the current block based on the number of partitions or the number of partition blocks presented in the higher-level information.

[0144] The image decoding device checks whether there are surrounding blocks restored in geometric partitioning mode. The image decoding device checks whether there are contact points between the surrounding blocks restored in geometric partitioning mode and the current block according to the partition boundaries of the surrounding blocks. If the aforementioned contact points exist, the image decoding device predicts the partition boundary information of the current block using the contact points and the partition boundary information of the surrounding blocks. Hereinafter, the predicted partition boundary information of the current block is referred to as a partition boundary predictor. For example, the partition boundary predictor may include distances and angles as exemplified in Table 1.

[0145] The image decoding device derives segmentation boundary information of the current block according to the geometric segmentation mode by using the differential information of the geometric segmentation mode parsed in relation to the segmentation boundary predictor. For example, the image decoding device can generate segmentation boundary information of the current block by adding the segmentation boundary predictor and the parsed differential information.

[0146] The image decoding device generates prediction signals of the current block by performing block-by-block prediction according to a geometric segmentation mode, for example, using merge mode. Furthermore, the image decoding device can generate final prediction signals of the current block by applying a mask filter for blending processing to the prediction signals around the segmentation boundary.

[0147] As described above, the higher-level information may include whether block segmentation is applied at the higher level, the number of segmented blocks, the minimum width of the segmented blocks, the minimum number of horizontal pixels, the minimum number of vertical pixels, mask filter information, etc. As an example, the higher-level information may be signaled based on various combinations of the above-described information. The higher-level information may be transmitted at various levels, such as frames, slices, frame groups, slice groups, and video groups.

[0148] As an example, when transmitting high-level information related to geometric segmentation mode, the syntax in Table 2 may be applied.

[0149]

[0150] In Table 2, the image decoding device parses gpm_flag, which is a flag indicating whether the geometric segmentation mode is applied at a high level. If the parsed flag is true and the geometric segmentation mode is applied, the image decoding device parses the number of segment blocks and the minimum size (width) of the segment blocks. The image decoding device obtains the number of segment blocks using num_gpm1. For example, the number of segment blocks is calculated by adding 2 to num_gpm1. For example, when num_gpm1 is 0, the number of segment blocks is 2. At this time, segmentation is performed once according to the geometric segmentation mode. In addition, when num_gpm1 is 1, segmentation is performed twice, so that the number of segment blocks according to the geometric segmentation mode is 3. In the present embodiment, the minimum width of the segment block cannot be smaller than the minimum size of the prediction block defined in the encoding and decoding method to which the present embodiment is applied. As an example, an image decoding device can calculate the size of a split block according to a geometric splitting mode by adding the minimum size of the prediction block and gpm_size. At this time, the prediction block also includes a sub-block split from the current block.

[0151] In Table 2, ae(v) represents syntax elements that are context-adaptive arithmetic encoded.

[0152] As an example, when transmitting high-level information related to geometric segmentation mode, the syntax in Table 3 may be applied.

[0153]

[0154] In Table 3, the image decoding device parses num_gpm2 to check whether the geometric segmentation mode is applied and the number of segmented blocks. num_gpm2 indicates the number of segments according to the application of the geometric segmentation mode. For example, if num_gpm2 is 0, the current block is not segmented according to the geometric segmentation mode. In other words, the geometric segmentation mode is not applied to the current block. If num_gpm2 is 1, the current block is segmented once, and the number of segmented blocks according to the geometric segmentation mode is 2. In addition, if num_gpm2 is 2, the current block is segmented twice, and the number of segmented blocks according to the geometric segmentation mode is 3. Therefore, the image decoding device parses the minimum size (width) of the segmented block if num_gpm2 is not 0.

[0155] FIGS. 10A to 10D are exemplary diagrams showing prediction of geometric segmentation information according to one embodiment of the present disclosure.

[0156] In this embodiment, a segmentation boundary predictor for the current block can be generated using segmentation information from blocks restored prior to the current block. By generating such a segmentation boundary predictor, the image encoding device can effectively transmit geometric segmentation information. In FIGS. 10A to 10D , the hashed block represents the current block, and the unhashed blocks represent previously decrypted blocks.

[0157] The image decoding device according to the present embodiment predicts segmentation information of the current block using geometric segmentation mode information of the surrounding blocks when a geometric segmentation mode is applied to the surrounding decoding blocks of the current block. As in the example of Fig. 10a, a case is described where the geometric segmentation mode is applied to one or more of the surrounding decoding blocks, and a point of contact exists between the segmentation boundary according to the applied segmentation mode and the block boundary of the current block. The image decoding device can predict the segmentation boundary of the current block based on the aforementioned point of contact. The image decoding device generates a segmentation boundary predictor of the current block by predicting a straight line extending the segmentation boundary of the decoding block as the segmentation boundary of the current block. The image decoding device generates final segmentation boundary information of the current block by adding differential information to the segmentation boundary predictor. Here, the differential information is included in a lookup table defined according to an agreement between the image encoding device and the image decoding device, as shown in Table 4. The difference information includes a combination of delta_distance (hereinafter referred to as distance difference) and delta_angle (hereinafter referred to as angle difference) that are added to the segmentation boundary predictor.

[0158]

[0159] The video encoding device generates distance differences and angle differences by subtracting a segmentation boundary predictor from the segmentation boundary information of the current block. The video encoding device derives an index gpm_partition_idx indicating a combination of distance differences and angle differences using the predefined lookup table shown in Table 4, and then signals the derived index to the video decoding device. The video decoding device decodes the aforementioned index and then derives difference information, i.e., a combination of distance differences and angle differences, using the index and Table 4. The video decoding device generates a segmentation boundary predictor using geometric segmentation mode information of the surrounding blocks, and then adds the derived difference information and the segmentation boundary predictor to generate final segmentation boundary information of the current block. The video decoding device determines the segmentation boundary of the current block according to the final segmentation boundary information.

[0160] In the lookup table shown in Table 4, the number of angular differences mapped to delta_angleIdx is 31, but this is not necessarily limited to 31. That is, the number of angular differences can adaptively change depending on the angle of the predicted segmentation boundary and the size / shape of the block to which GPM is applied. In Table 4, the angular differences depend on the index angleIdx, which represents the angles exemplified in Table 1.

[0161] As an example, the angular spacing between delta_angleIdx values ​​(i.e., the spacing between angular differences) may be a non-zero interval of k degrees. For example, k may be an angle that evenly divides 180 degrees. In this case, the angular spacing may be fixed according to an agreement between an image encoding device and an image decoding device. Alternatively, k or the number of 180 degrees divided into k degrees may be transmitted using a higher-level syntax. As another example, k may not be a fixed value. In this case, the values ​​mapped to delta_angleIdx may be transmitted as a lookup table. Alternatively, the values ​​mapped to delta_angleIdx may be fixed according to an agreement between an image encoding device and an image decoding device.

[0162] In the lookup table shown in Table 4, the distance difference mapped to delta_distanceIdx is not necessarily 0. That is, the distance difference may vary depending on the location of the predicted segmentation boundary and the size of the block to which the geometric segmentation mode is applied. In Table 4, the distance difference depends on the index distanceIdx, which represents the distance illustrated in Table 1.

[0163] Meanwhile, the number N of indices gpm_partition_idx, which indicate the combination of distance difference and angle difference, may be limited in terms of optimizing encoding efficiency. The number N of indices may vary depending on the width, height, area, shape, etc. of the block to which the geometric partitioning mode is applied. In this case, the maximum value of N may be fixed according to an agreement between the video encoding device and the video decoding device. Alternatively, the maximum value of N may be transmitted using a higher-level syntax.

[0164] Based on the number of indices N, when adaptively constructing a lookup table, the following combinations of distance differences and angle differences can be excluded. That is, if the corrected partition boundary based on the difference information indicated by gpm_partition_idx is identical to the block boundary of the current block, the corresponding combination of delta_angleIdx and delta_distanceIdx is excluded from constructing the lookup table.

[0165] In the lookup table shown in Table 4, when gpm_partition_Idx is 0, the predicted partition boundary represents the same straight line as the partition boundary of the current block. When gpm_partition_Idx is 1, the current block is partitioned at the same position as the contact point of the boundary, and the angle of the partition boundary is an angle rotated by 1 angleIdx in the clockwise direction with respect to the contact point. In addition, when gpm_partition_Idx is 2, the current block is partitioned at the same position as the contact point of the boundary, and the angle of the partition boundary is an angle rotated by 1 angleIdx in the counterclockwise direction with respect to the contact point.

[0166] In this embodiment, the size of the divided block must satisfy the minimum size of the divided block. Therefore, the lookup table is configured so that the corrected divided block satisfies the minimum size according to the combination of the distance difference and the angle difference. For example, after generating a divided boundary by adding the difference information indicated by gpm_partition_idx 0 to the divided boundary predictor, if any of the divided blocks according to the generated divided boundary does not satisfy the minimum size, the corresponding index may be excluded from the lookup table according to the agreement between the image encoding device and the image decoding device. At this time, the angle and distance combination of index 1 may be changed to index 0. As described above, the index indicating the combination of the angle difference and the distance difference within the lookup table may be adaptively determined.

[0167] Meanwhile, if there are no blocks encoded in the geometric partitioning mode in the surroundings, or if there is no point of contact between the partitioning boundary of the block encoded in the geometric partitioning mode and the block boundary of the current block, the generation of the partitioning boundary predictor is omitted. In this case, the image decoding device can generate geometric partitioning information of the current block using only the information about the geometric partitioning mode transmitted from the image encoding device.

[0168] As another example for effective transmission of geometric segmentation information, the continuity between the segmentation boundary of a block to which GPM is applied among surrounding coding blocks and the segmentation boundary of the current block can be utilized. For example, a lookup table can be adaptively constructed using a combination of angleIdx and distanceIdx that create a point of contact that is identical or similar at the boundary between the surrounding blocks and the current block. An image encoding device can derive an index indicating the segmentation boundary of the current block from the adaptively constructed lookup table and then transmit the derived index to an image decoding device. A lookup table including cases for the angles and distances that constitute the existing merge_gpm_partition_Idx can be newly constructed while excluding merge_gpm_partition_Idx that has a condition of not being split into an identical or similar position at the boundary of the point of contact. Alternatively, a lookup table can be newly constructed by reordering the indices according to the order of proximity to the point of contact. At this time, as the value of merge_gpm_partition_Idx increases, the lookup table can be adaptively configured so that the continuity related to the contact point at the boundary of the current block decreases. This is because, due to the characteristics of the image, the probability that the partition boundary exists discontinuously, as in the example of Fig. 10b, is lower than the probability that it exists continuously, as in the example of Fig. 10a.

[0169] As another example, when there are two or more extended partition boundaries in the current block (i.e., when there are two or more contact points), as in the example of FIG. 10c or FIG. 10d, the image decoding device may generate differential information by parsing multiple gpm_partition_idx. Meanwhile, the number of partition blocks of the current block cannot be greater than the number of geometric partition blocks transmitted from the upper level. When the number of partition blocks is greater than the number of geometric partition blocks transmitted from the upper level, the image decoding device decodes differential information corresponding to a higher priority in the decoding order, as in the example of Table 5.

[0170]

[0171] FIGS. 11A to 11C are exemplary diagrams showing a composite split mode according to one embodiment of the present disclosure.

[0172] When geometric segmentation information is transmitted as in FIG. 11b for a geometric segmentation such as the example of FIG. 11a, quadtree segmentation information may be transmitted for the upper block and geometric segmentation information for each segmented block may be transmitted. In the composite segmentation mode according to the present embodiment, for a geometric segmentation such as the example of FIG. 11a, the upper block is segmented in a composite of two modes, that is, the existing QT-based segmentation mode and the geometric segmentation mode, as in the example of FIG. 11c. At this time, two segmentation modes may be transmitted as segmentation information at the upper block level. That is, instead of segmenting the upper block according to the quadtree example and then transmitting geometric segmentation mode information for each of the segmented blocks A, B, C, and D, by applying the quadtree segmentation and the geometric segmentation at the same level, the transmission efficiency of the geometric segmentation information may be improved.

[0173] Hereinafter, using the cities of FIGS. 12 and 13, a method for generating segmented blocks of a current block according to application of a geometric segmentation mode is described.

[0174] FIG. 12 is a flowchart illustrating a method for an image encoding device to encode a current block according to an embodiment of the present disclosure.

[0175] The image encoding device determines high-level information related to applying a geometric segmentation mode (S1200).

[0176] Here, the high-level information includes one or more of whether a geometric segmentation mode is applied at the high level, the number of segmented blocks according to the geometric segmentation mode, the minimum size of the segmented blocks according to the geometric segmentation mode, or mask information applicable for blending processing. In terms of rate distortion optimization, the image encoding device can determine the high-level information.

[0177] The video encoding device determines a flag indicating the application of a geometric segmentation mode to the current block (S1202). In terms of rate distortion optimization, the video encoding device may determine a flag indicating the application of a geometric segmentation mode.

[0178] The video encoding device checks the flag (S1204).

[0179] If the flag indicating application of the geometric segmentation mode is false (No in S1204), the image encoding device encodes the current block using a method other than the geometric segmentation mode.

[0180] On the other hand, if the flag is true (Yes in S1204), the video encoding device performs the following steps.

[0181] The video encoding device generates segmentation boundary information of the current block based on higher-level information (S1206). For example, the segmentation boundary information may include distances and angles as exemplified in Table 1.

[0182] The image encoding device generates segment blocks of the current block based on segmentation boundary information (S1208).

[0183] The video encoding device checks whether there is a surrounding block restored according to the GPM (S1210).

[0184] The video encoding device checks whether there is a point of contact between the current block and the surrounding blocks according to the division boundary of the surrounding blocks (S1212).

[0185] If there is one or more contacts (Yes in S1210 and S1212), the video encoding device performs the following steps.

[0186] The image encoding device generates a segmentation boundary predictor of the current block using segmentation boundary information of surrounding blocks (S1214).

[0187] The video encoding device generates differential information by subtracting a segmentation boundary predictor from the segmentation boundary information of the current block (S1216). For example, the differential information includes a combination of distance differences and angle differences, as shown in Table 4.

[0188] The image encoding device derives an index corresponding to the differential information from the lookup table (S1218).

[0189] As an example, if there is more than one dot, the image encoding device derives an index corresponding to the difference information from a predefined lookup table.

[0190] As another example, when there is one or more points, the video encoding device adaptively generates a lookup table based on the points and higher-level information. For example, the video encoding device configures the lookup table so that it does not include an index indicating the aforementioned difference information when the corrected segmentation boundary based on the difference information is identical to the block boundary of the current block. The video encoding device configures the lookup table so that the corrected segmentation block satisfies the minimum size according to the application of the geometric segmentation mode according to the combination of the distance difference and the angle difference. Alternatively, the video encoding device configures the lookup table using combinations of the distance difference and the angle difference that create the points that match or are similar to each other.

[0191] The image encoding device derives an index corresponding to the differential information from the generated lookup table.

[0192] The video encoding device encodes an index (S1220).

[0193] Meanwhile, if there are no surrounding blocks restored in the geometric segmentation mode (No in S1210) or no dots exist (No in S1212), the image encoding device omits the generation of the segmentation boundary predictor and the generation of differential information.

[0194] FIG. 13 is a flowchart illustrating a method for an image decoding device to restore a current block according to one embodiment of the present disclosure.

[0195] The image decoding device obtains high-level information related to applying a geometric segmentation mode (S1300).

[0196] Here, the high-level information includes one or more of whether a geometric segmentation mode is applied at the high level, the number of segmentation blocks according to the geometric segmentation mode, the minimum size of the segmentation block according to the geometric segmentation mode, or mask information applicable for blending processing.

[0197] The video decoding device decodes a flag indicating application of a geometric segmentation mode to the current block from the bitstream (S1302).

[0198] The video decoding device checks the flag (S1304).

[0199] If the flag indicating application of the geometric segmentation mode is false (No in S1304), the image decoding device restores the current block using a method other than the geometric segmentation mode.

[0200] On the other hand, if the aforementioned flag is true (Yes in S1304), the image decoding device performs the following steps.

[0201] The image decoding device checks whether there are any surrounding blocks restored according to the geometric segmentation mode (S1306).

[0202] The video decoding device checks whether there is a contact point at the block boundary between the surrounding blocks and the current block according to the division boundary of the surrounding blocks (S1308).

[0203] If there is one or more contact points (Yes in S1306 and S1308), the image decoding device performs the following steps.

[0204] The image decoding device generates a segmentation boundary predictor of the current block using segmentation boundary information of surrounding blocks (S1310).

[0205] The image decoding device obtains differential information in geometric segmentation mode (S1312).

[0206] As an example, a video decoding device decodes an index from a bitstream and, using the decoded index, derives differential information from a predefined lookup table.

[0207] As another example, the image decoding device adaptively generates a lookup table based on the contact points and higher-level information. For example, the image decoding device configures the lookup table so that, if the corrected segmentation boundary based on the difference information is identical to the block boundary of the current block, the lookup table does not include an index indicating the aforementioned difference information. The image decoding device configures the lookup table so that the corrected segmentation block satisfies the minimum size according to the geometric segmentation mode application according to the combination of the distance difference and the angle difference. Alternatively, the image decoding device configures the lookup table using combinations of the distance difference and the angle difference that generate the contact points identically or similarly.

[0208] The video decoding device uses the decrypted index to derive differential information from the generated lookup table.

[0209] The image decoding device generates segmentation boundary information of the current block by adding the segmentation boundary predictor and differential information (S1314).

[0210] The image decoding device generates segment blocks of the current block based on the upper level information and segmentation boundary information (S1316).

[0211] Meanwhile, if there are no surrounding blocks restored in the geometric segmentation mode (No in S1306) or none of the aforementioned points exist (No in S1308), the image decoding device omits generation of the segmentation boundary predictor and acquisition of differential information.

[0212] Although the flowchart / timing diagram of this specification describes each process as being executed sequentially, this is merely an illustrative description of the technical idea of ​​one embodiment of the present disclosure. In other words, a person of ordinary skill in the art to which one embodiment of the present disclosure belongs may modify and apply various modifications and variations by changing the order described in the flowchart / timing diagram without departing from the essential characteristics of one embodiment of the present disclosure, or by executing one or more of the processes in parallel. Therefore, the flowchart / timing diagram is not limited to a chronological order.

[0213] It should be understood that the exemplary embodiments described above can be implemented in many different ways. The functions or methods described in one or more examples can be implemented in hardware, software, firmware, or any combination thereof. It should be understood that the functional components described herein are labeled as "units" to further emphasize their implementation independence.

[0214] Meanwhile, the various functions or methods described in this embodiment may also be implemented as instructions stored on a non-transitory storage medium that can be read and executed by one or more processors. Non-transitory storage media include, for example, all types of storage devices that store data in a form readable by a computer system. For example, non-transitory storage media include storage media such as erasable programmable read-only memory (EPROM), flash drives, optical drives, magnetic hard drives, and solid-state drives (SSDs).

[0215] The above description is merely an example of the technical idea of ​​the present embodiment, and those skilled in the art will appreciate that various modifications and variations can be made without departing from the essential characteristics of the present embodiment. Therefore, the present embodiments are not intended to limit the technical idea of ​​the present embodiment, but rather to explain it, and the scope of the technical idea of ​​the present embodiment is not limited by these embodiments. The scope of protection of the present embodiment should be interpreted by the claims below, and all technical ideas within a scope equivalent thereto should be interpreted as being included in the scope of rights of the present embodiment.

[0216] (Explanation of symbols)

[0217] 124: Inter Prediction Department

[0218] 155: Entropy encoding unit

[0219] 510: Entropy decryption unit

[0220] 544: Inter Prediction Department

[0221]

[0222] CROSS-REFERENCE TO RELATED APPLICATION

[0223] This patent application claims priority to Korean patent application No. 10-2023-0062811, filed in Korea on May 16, 2023, and Korean patent application No. 10-2024-0033416, filed in Korea on March 8, 2024, the entire contents of which are incorporated herein by reference.

Claims

1. A method for restoring a current block performed by a video decoding device, A step of obtaining high-level information related to applying a geometric segmentation mode; A step of checking whether there is a surrounding block restored according to the above geometric division mode; and Including a step of checking whether there is a contact point according to the division boundary of the surrounding blocks at the block boundary between the surrounding block and the current block, If one or more of the above contacts exist, A step of generating a segmentation boundary predictor of the current block using segmentation boundary information of the surrounding blocks; A step of obtaining differential information of the above geometric segmentation mode; A step of generating segmentation boundary information of the current block by adding the segmentation boundary predictor and the differential information; and A step of generating split blocks of the current block based on the above-mentioned high-level information and the above-mentioned split boundary information. A method, characterized in that it further comprises:

2. In paragraph 1, A step of decoding a flag indicating application of the geometric segmentation mode to the current block from the bitstream; and Steps to check the above flags Including more, A method characterized in that, if the above flag is true, a step of checking whether there is a surrounding block restored according to the geometric segmentation mode is performed.

3. In paragraph 1, A method further comprising a step of decoding the higher-level information from a bitstream, wherein the higher-level information includes at least one of whether the geometric segmentation mode is applied at the higher level, the number of segment blocks according to the geometric segmentation mode, the minimum size of the segment blocks according to the geometric segmentation mode, or mask information applicable for blending processing.

4. In paragraph 3, A method characterized in that the geometric segmentation mode is not applied when the number of the above segmentation blocks is 0.

5. In paragraph 1, The above segmentation boundary information is, A method characterized by including a distance and an angle for application of the above geometric segmentation mode.

6. In paragraph 1, The above differential information is, A method characterized by including a distance difference and an angular difference for adding to the above segmentation boundary predictor.

7. In paragraph 1, The step of obtaining the above differential information is: A step of decoding an index from a bitstream; and A step of deriving the difference information from a predefined lookup table using the above index. A method, characterized by including:

8. In paragraph 7, A method, characterized in that the interval between angular differences in the above-described lookup table according to the increase of the above-described index is set to a preset angle.

9. In paragraph 1, The step of obtaining the above differential information is: A step of decrypting an index from a bitstream; A step of generating a lookup table based on the above contact point and the above upper level information; and A step of deriving the difference information from the lookup table using the above index. A method, characterized by including:

10. In paragraph 9, The steps for creating the above lookup table are: A method characterized in that the lookup table is configured not to include an index indicating the differential information when the corrected segmentation boundary based on the differential information is identical to the block boundary of the current block.

11. In paragraph 9, The steps for creating the above lookup table are: A method characterized in that the lookup table is configured such that the corrected segmentation block satisfies the minimum size according to the application of the geometric segmentation mode according to the combination of the distance difference and the angle difference.

12. In paragraph 9, The steps for creating the above lookup table are: A method characterized by using combinations of distance differences and angle differences that match or similarly generate the above contact points.

13. A method for encoding a current block performed by a video encoding device, A step of determining high-level information related to applying a geometric segmentation mode; A step of generating segmentation boundary information of the current block based on the above high-level information; A step of generating split blocks of the current block based on the split boundary information; A step of checking whether there is a surrounding block restored according to the above geometric division mode; and It includes a step of checking whether there is a contact point according to the division boundary of the surrounding blocks at the block boundary between the surrounding block and the current block, If one or more of the above contacts exist, A step of generating a segmentation boundary predictor of the current block using segmentation boundary information of the surrounding blocks; and A step of generating difference information by subtracting the segmentation boundary predictor from the segmentation boundary information of the current block. A method, characterized in that it further comprises:

14. In Article 14, a step of determining a flag indicating application of the geometric division mode to the current block; and Steps to check the above flags Including more, A method characterized in that, if the above flag is true, a step of generating segmentation boundary information of the current block is performed.

15. In paragraph 14, The above differential information is, A method characterized by including a distance difference and an angular difference for adding to the above segmentation boundary predictor.

16. In paragraph 14, If one or more of the above contacts exist, A step of deriving an index corresponding to the differential information from a predefined lookup table; and Step of encoding the above index A method, characterized in that it further comprises:

17. In paragraph 14, If one or more of the above contacts exist, A step of generating a lookup table based on the above contact point and the above upper level information; A step of deriving an index corresponding to the difference information from the lookup table; and Step of encoding the above index A method, characterized in that it further comprises:

18. A computer-readable recording medium storing a bitstream generated by an image encoding method, wherein the image encoding method comprises: A step of determining high-level information related to applying a geometric segmentation mode; A step of generating segmentation boundary information of a current block based on the above high-level information; A step of generating split blocks of the current block based on the split boundary information; A step of checking whether there is a surrounding block restored according to the above geometric division mode; and It includes a step of checking whether there is a contact point according to the division boundary of the surrounding blocks at the block boundary between the surrounding block and the current block, If one or more of the above contacts exist, A step of generating a segmentation boundary predictor of the current block using segmentation boundary information of the surrounding blocks; and A step of generating difference information by subtracting the segmentation boundary predictor from the segmentation boundary information of the current block. A recording medium, characterized by including:

Citation Information

Patent Citations

  • Wireless power transmission apparatus and method for registering at least one cooking device

    KR1020230160676A

  • Citron Fermenting Apparatus And Method of Making Cold-Fermented Citron Tea

    KR1020240149757A

  • A method of measuring bone mineral density of based on machine learning using radiographic image of the hip joint taken by X-ray

    KR102578943B1

  • Video encoding / decoding method and device for applying, to combined intra / inter-prediction mode or geometric partitioning mode, merge mode using motion vector difference

    WO2023055193A1

  • KR20220101638A