Method and apparatus for video coding with geometric motion prediction

Abstract

Methods and apparatus for video coding with geometric motion prediction are disclosed. In this embodiment, an image decoding device decodes motion information on a first block partition of a current block, and then generates a first prediction block of the current block by using the motion information of the first block partition. The image decoding device decodes a merge candidate index for a second block partition of the current block, and then generates a second prediction block of the current block by using the merge candidate index. The image decoding device derives a geometric block partition type by using the first prediction block and then generates a final prediction block of the current block by mixing the first prediction block and the second prediction block by using the geometric block partition type.

Description

Technical Field

[0001] The invention relates to a video coding and decoding method and device using geometric motion prediction. Background Art

[0002] The statements in this section merely provide background information related to the present invention and may not necessarily constitute prior art.

[0003] Since video data has a large amount of data compared to audio data or still image data, video data requires a large amount of hardware resources (including memory) to store or transmit the video data without being compressed.

[0004] Accordingly, the encoder is generally used to compress and store or transmit video data. The decoder receives the compressed video data, decompresses the received compressed video data, and plays the decompressed video data. Video compression technologies include H.264 / Advanced Video Codec (AVC), High Efficiency Video Coding (HEVC), and Versatile Video Codec (VVC), which improves the coding efficiency of HEVC by about 30% or more.

[0005] However, as image size, resolution and frame rate gradually increase, the amount of data to be encoded will also increase. Accordingly, new compression technologies are needed that provide higher encoding and decoding efficiency and improved image enhancement effects than existing compression technologies.

[0006] In the video coding method and apparatus, the inter-frame prediction technology generates a prediction block by using pixels in a temporally adjacent picture to predict the current block. In order to improve the performance of inter-frame prediction, such a technology has been tried, which performs inter-frame prediction in units of blocks as a basic unit of encoding and decoding, or performs inter-frame prediction on a block in units of one or more sub-blocks. However, the block-based prediction technology described above is limited to prediction using square or rectangular block partitions, so it is difficult to perform prediction on objects with arbitrary shapes in an image.

[0007] In order to overcome the above limitations, in addition to square or rectangular block partitioning, methods using arbitrary block partitioning have been tried. As a representative arbitrary block partitioning, there is a method of partitioning square blocks using straight lines with arbitrary angles. This technology was proposed and discussed during the HEVC standardization process, but was not adopted as a standard technology. Later, during the VVC standardization process, in the inter-frame prediction technology, a technology for partitioning a block using two diagonal segments and mixing the partitioned blocks to generate a final inter-frame prediction factor has been proposed. This technology is named triangle partition mode (TPM) and is adopted in the VVC standard. In addition, during the VVC standardization process, the form of partitioning a block has been expanded to be more diverse. This technology is named geometric partition mode (GPM) and was eventually incorporated into the VVC standard. The above GPM is only used in the merge mode in the inter-frame prediction mode. Therefore, in order to improve video encoding and decoding efficiency and improve video quality, it is necessary to consider improving GPM. Summary of the invention

[0008] Technical issues

[0009] The present invention is dedicated to providing a video coding method and apparatus using geometric motion prediction in inter-frame prediction of a current block. Here, the geometric motion prediction divides the current block into an arbitrary shape, performs different motion predictions on each partitioned block to generate a prediction factor, and mixes the prediction factors based on different weights of predefined pixel units to generate a final inter-frame prediction factor.

[0010] Technical Solution

[0011] At least one aspect of the present invention provides a method for decoding a current block performed by a video decoding device. The method includes decoding motion information for a first block partition of the current block from a bitstream. The method also includes generating a first prediction block of the current block by using the motion information of the first block partition. The method also includes decoding a merge candidate index for a second block partition of the current block from the bitstream. The method also includes obtaining the motion information of the second block partition from a geometric partition mode (GPM) merge list based on the merge candidate index, and then generating a second prediction block of the current block by using the motion information of the second block partition. The method also includes deriving a geometric block partition shape that partitions the current block into a first block partition and a second block partition by using the first prediction block. The method also includes mixing the first prediction block and the second prediction block by using the geometric block partition shape to generate a final prediction block of the current block.

[0012] Another aspect of the present invention provides a method for predicting a current block performed by a video encoding device. The method includes determining motion information for a first block partition of the current block. The method also includes generating a first prediction block of the current block by using the motion information of the first block partition. The method also includes determining a merge candidate index for a second block partition of the current block. The method also includes obtaining the motion information of the second block partition from a geometric partition mode (GPM) merge list based on the merge candidate index, and then generating a second prediction block of the current block by using the motion information of the second block partition. The method also includes deriving a geometric block partition shape that divides the current block into a first block partition and a second block partition by using the first prediction block. The method also includes mixing the first prediction block and the second prediction block by using the geometric block partition shape to generate a final prediction block of the current block.

[0013] Another aspect of the present invention provides a computer-readable recording medium storing a bitstream generated by a video encoding method. The video encoding method includes determining motion information for a first block partition of a current block. The video encoding method also includes generating a first prediction block of the current block by using the motion information of the first block partition. The video encoding method also includes determining a merge candidate index for a second block partition of the current block. The video encoding method also includes obtaining the motion information of the second block partition from a geometric partition mode (GPM) merge list based on the merge candidate index, and then generating a second prediction block of the current block by using the motion information of the second block partition. The video encoding method also includes deriving a geometric block partition shape that divides the current block into a first block partition and a second block partition by using the first prediction block. The video encoding method also includes mixing the first prediction block and the second prediction block by using the geometric block partition shape to generate a final prediction block of the current block.

[0014] Beneficial Effects

[0015] As described above, the present invention provides a video coding method and apparatus that uses geometric motion prediction in inter-frame prediction of a current block. Therefore, the video coding method and apparatus improve video coding efficiency and enhance video quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 is a block diagram of a video encoding device that can implement the technology of the present invention.

[0017] Figure 2 A method of partitioning blocks using a quadtree plus binary tree ternary tree (QTBTTT) structure is shown.

[0018] Figure 3a and Figure 3b Multiple intra prediction modes are shown including a wide-angle intra prediction mode.

[0019] Figure 4Shows the neighboring blocks of the current block.

[0020] Figure 5 is a block diagram of a video decoding device that can implement the technology of the present invention.

[0021] Figure 6 is a schematic diagram illustrating block partitioning for geometric motion prediction according to an embodiment of the present invention.

[0022] Figure 7a and Figure 7b is a schematic diagram showing a straight line bisecting a block according to an embodiment of the present invention.

[0023] Figure 8 is a schematic diagram conceptually illustrating inter-frame prediction using geometric partitioning.

[0024] Fig. 9 is a schematic diagram showing a geometry partition mode (GPM) merge list for geometric motion prediction.

[0025] 10 is a schematic diagram illustrating a GPM advanced motion vector prediction (AMVP) list and a GPM merge list for geometric motion prediction according to an embodiment of the present invention.

[0026] Fig.11 is a schematic diagram illustrating geometric motion prediction using a hybrid AMVP mode and a merge mode according to an embodiment of the present invention.

[0027] Fig.12 is a flowchart illustrating a method for a video decoding apparatus to decode motion information according to an embodiment of the present invention.

[0028] Fig.13 is a flowchart illustrating a method for a video encoding apparatus to predict a current block according to an embodiment of the present invention.

[0029] Fig.14 is a flowchart illustrating a method for a video decoding apparatus to predict a current block according to an embodiment of the present invention. DETAILED DESCRIPTION

[0030] Hereinafter, some embodiments of the present invention will be described in detail with reference to the attached illustrative drawings. In the following description, the same reference numerals represent the same elements, although the elements are shown in different drawings. In addition, in the following description of some embodiments, when it is considered that the detailed description of related known components and functions obscures the subject matter of the present invention, the detailed description of related known components and functions may be omitted for the sake of clarity and brevity.

[0031] Figure 1 is a block diagram of a video encoding device that can implement the technology of the present invention. Figure 1 A video encoding device and components of the device are described in detail.

[0032] The encoding device may include: an image divider 110, a predictor 120, a subtractor 130, a transformer 140, a quantizer 145, a rearrangement unit 150, an entropy encoder 155, an inverse quantizer 160, an inverse transformer 165, an adder 170, a loop filtering unit 180 and a memory 190.

[0033] Each component of the encoding device can be implemented as hardware or software, or as a combination of hardware and software. In addition, the function of each component can be implemented as software, and the microprocessor can also be implemented to execute the function of the software corresponding to each component.

[0034] A video consists of one or more sequences including multiple images. Each image is divided into multiple regions, and encoding is performed on each region. For example, an image is divided into one or more tiles or / and slices. Here, one or more tiles can be defined as a tile group. Each tile or / and slice is divided into one or more coding tree units (coding tree unit, CTU). In addition, each CTU is divided into one or more coding units (coding unit, CU) through a tree structure. The information applied to each coding unit (CU) is encoded as the syntax of the CU, and the information commonly applied to the CU included in a CTU is encoded as the syntax of the CTU. In addition, the information commonly applied to all blocks in a slice is encoded as the syntax of the slice header, and the information applied to all blocks constituting one or more images is encoded as a picture parameter set (Picture Parameter Set, PPS) or a picture header. In addition, the information commonly referenced by multiple images is encoded as a sequence parameter set (Sequence Parameter Set, SPS). In addition, the information commonly referenced by one or more SPSs is encoded as a video parameter set (Video Parameter Set, VPS). In addition, information commonly applied to one tile or tile group may also be encoded as syntax of the tile or tile group header.The syntax included in the SPS, PPS, slice header, tile or tile group header may be referred to as a high-level syntax.

[0035] The image divider 110 determines the size of a coding tree unit (CTU). Information on the size of the CTU (CTU size) is encoded as a syntax of an SPS or a PPS and transmitted to a video decoding device.

[0036] The image divider 110 divides each image constituting a video into a plurality of coding tree units (CTUs) having a predetermined size, and then recursively divides the CTUs by using a tree structure. A leaf node in the tree structure becomes a coding unit (CU), which is a basic unit of coding.

[0037] The tree structure can be a quadtree (quadtree, QT), in which a higher node (or parent node) is split into four lower nodes (or child nodes) of the same size. The tree structure can also be a binary tree (binarytree, BT), in which a higher node is split into two lower nodes. The tree structure can also be a ternary tree (ternarytree, TT), in which a higher node is split into three lower nodes at a ratio of 1:2:1. The tree structure can also be a structure in which two or more structures in the QT structure, BT structure and TT structure are mixed. For example, a quadtree plus binary tree (quadtree plus binarytree, QTBT) structure can be used, or a quadtree plus binary tree ternary tree (quadtree plus binarytreeternarytree, QTBTTT) structure can be used. Here, a binary tree ternary tree (BTTT) is added to the tree structure to be called a multiple-type tree (multiple-type tree, MTT).

[0038] Figure 2 is a schematic diagram for describing a method of partitioning a block by using a QTBTTT structure.

[0039] like Figure 2 As shown, the CTU can be first split into a QT structure. The quadtree splitting can be recursive until the size of the split block reaches the minimum block size (MinQTSize) of the leaf node allowed in QT. The first flag (QT_split_flag) indicating whether each node of the QT structure is split into four nodes of the lower layer is encoded by the entropy encoder 155 and notified to the video decoding device with a signal. When the leaf node of QT is not larger than the maximum block size (MaxBTSize) of the root node allowed in BT, the leaf node can be further split into at least one of a BT structure or a TT structure. There may be multiple splitting directions in the BT structure and / or the TT structure. For example, there may be two directions, namely, a direction for horizontally splitting blocks of the corresponding node and a direction for vertically splitting blocks of the corresponding node. As Figure 2 As shown, when MTT splitting starts, the entropy encoder 155 encodes a second flag (mtt_split_flag) indicating whether the node is split, and a flag indicating the splitting direction (vertical or horizontal) and / or a flag indicating the splitting type (binary or trifurcated) when the node is split, and notifies the video decoding device of the same by signal.

[0040] Alternatively, before encoding the first flag (QT_split_flag) indicating whether each node is split into four nodes of the lower layer, the CU split flag (split_cu_flag) indicating whether the node is split may also be encoded. When the value of the CU split flag (split_cu_flag) indicates that each node is not split, the block of the corresponding node becomes a leaf node in the split tree structure and becomes a CU, which is the basic unit of encoding. When the value of the CU split flag (split_cu_flag) indicates that each node is split, the video encoding device first starts encoding the first flag in the above scheme.

[0041] When QTBT is used as another example of a tree structure, there may be two types, namely, a type in which the block of the corresponding node is horizontally split into two blocks of the same size (i.e., symmetrical horizontal splitting) and a type in which the block of the corresponding node is vertically split into two blocks of the same size (i.e., symmetrical vertical splitting). The entropy encoder 155 encodes a split flag (split_flag) indicating whether each node of the BT structure is split into blocks of the lower layer and split type information indicating the split type, and transmits them to the video decoding device. On the other hand, there may be another type in which the block of the corresponding node is split into two blocks that are asymmetric to each other. The asymmetric form may include a form in which the block of the corresponding node is split into two rectangular blocks with a size ratio of 1:3, or may also include a form in which the block of the corresponding node is split in a diagonal direction.

[0042] The CU may have various sizes according to the QTBT or QTBTTT partitioned from the CTU. Hereinafter, the block corresponding to the CU to be encoded or decoded (i.e., the leaf node of the QTBTTT) is referred to as the "current block". When QTBTTT partitioning is adopted, the shape of the current block may also be a rectangular shape in addition to a square shape.

[0043] The predictor 120 predicts the current block to generate a predicted block. The predictor 120 includes an intra predictor 122 and an inter predictor 124.

[0044] Typically, each of the current blocks in an image may be predictively encoded. Typically, the prediction of the current block may be performed by utilizing an intra-prediction technique (which utilizes data from an image including the current block) or an inter-prediction technique (which utilizes data from an image encoded prior to the image including the current block). Inter-prediction includes both unidirectional prediction and bidirectional prediction.

[0045] The intra-frame predictor 122 predicts pixels in the current block by using pixels (reference pixels) located adjacent to the current block in the current image including the current block. There are multiple intra-frame prediction modes according to the prediction direction. For example, Figure 3a As shown, the plurality of intra prediction modes may include two non-directional modes including a planar mode and a DC mode, and may include 65 directional modes. Neighboring pixels and algorithm equations to be used are defined differently according to each prediction mode.

[0046] In order to perform efficient directional prediction for a current block with a rectangular shape, it is possible to additionally use Figure 3b The directional modes are shown in the dashed arrows (#67 to #80, intra-frame prediction modes #-1 to #-14). The directional modes can be called "wide angle intra-frame prediction modes". Figure 3b In the figure, the arrows indicate the corresponding reference samples used for prediction, rather than representing the prediction direction. The prediction direction is opposite to the direction indicated by the arrow. When the current block has a rectangular shape, the wide-angle intra prediction mode is a mode in which prediction is performed in a direction opposite to the specific direction mode without additional bit transmission. In this case, in the wide-angle intra prediction mode, some wide-angle intra prediction modes available for the current block can be determined by the ratio of the width to the height of the current block having a rectangular shape. For example, when the current block has a rectangular shape with a height less than the width, a wide-angle intra prediction mode (intra prediction modes #67 to #80) with an angle less than 45 degrees is available. When the current block has a rectangular shape with a width greater than the height, a wide-angle intra prediction mode with an angle greater than -135 degrees is available.

[0047] The intra-frame predictor 122 may determine an intra-frame prediction to be used for encoding the current block. In some examples, the intra-frame predictor 122 may encode the current block by utilizing multiple intra-frame prediction modes, and may also select an appropriate intra-frame prediction mode to be used from a test mode. For example, the intra-frame predictor 122 may calculate a rate-distortion value by utilizing a rate-distortion analysis of multiple tested intra-frame prediction modes, and may also select an intra-frame prediction mode having the best rate-distortion characteristic in the test mode.

[0048] The intra-frame predictor 122 selects an intra-frame prediction mode among a plurality of intra-frame prediction modes and predicts the current block by using adjacent pixels (reference pixels) and an algorithm equation determined according to the selected intra-frame prediction mode. Information about the selected intra-frame prediction mode is encoded by the entropy encoder 155 and transmitted to the video decoding device.

[0049] The inter-frame predictor 124 generates a prediction block for the current block by using a motion compensation process. The inter-frame predictor 124 searches for a block that is most similar to the current block in a reference image that has been encoded and decoded earlier than the current image, and generates a prediction block for the current block by using the searched block. In addition, a motion vector (MV) is generated, which corresponds to the displacement between the current block in the current image and the prediction block in the reference image. Typically, motion estimation is performed on the luminance (luma) component, and the motion vector calculated based on the luminance component is used for both the luminance component and the chrominance component. The motion information including information of the reference image and information about the motion vector used to predict the current block is encoded by the entropy encoder 155 and transmitted to the video decoding device.

[0050] The inter-frame predictor 124 may also perform interpolation of a reference image or a reference block to increase the accuracy of the prediction. In other words, a subsample is interpolated between two consecutive integer samples by applying a filter coefficient to a plurality of consecutive integer samples including two integer samples. When performing a process of searching for a block most similar to the current block on the interpolated reference image, a decimal unit precision may be represented for the motion vector instead of an integer sample unit precision. For each target region to be encoded, for example, a unit such as a slice, a tile, a CTU, a CU, etc., the precision or resolution of the motion vector may be set differently. When such an adaptive motion vector resolution (AMVR) is applied, information about the motion vector resolution to be applied to each target region should be signaled for each target region. For example, when the target region is a CU, information about the motion vector resolution applied to each CU is signaled. The information about the motion vector resolution may be information representing the precision of the motion vector difference to be described below.

[0051] On the other hand, the inter-frame predictor 124 may perform inter-frame prediction by utilizing bidirectional prediction. In the case of bidirectional prediction, two reference images and two motion vectors representing the block positions most similar to the current block in each reference image are used. The inter-frame predictor 124 selects a first reference image and a second reference image from a reference image list 0 (RefPicList0) and a reference image list 1 (RefPicList1), respectively. The inter-frame predictor 124 also searches for a block most similar to the current block in the corresponding reference image to generate a first reference block and a second reference block. In addition, a prediction block of the current block is generated by averaging or weighted averaging the first reference block and the second reference block. In addition, motion information including information about two reference images used to predict the current block and including information about two motion vectors is transmitted to the entropy encoder 155. Here, the reference image list 0 may be composed of images that are before the current image in the display order in the pre-reconstructed image, and the reference image list 1 may be composed of images that are after the current image in the display order in the pre-reconstructed image. However, although not particularly limited thereto, the pre-reconstructed image that is after the current image in the display order may be additionally included in the reference image list 0. Conversely, a pre-reconstructed image preceding the current image may also be additionally included in the reference image list 1 .

[0052] In order to minimize the amount of bits consumed for encoding motion information, various methods may be used.

[0053] For example, when the reference image and motion vector of the current block are the same as those of the neighboring block, information capable of identifying the neighboring block is encoded to transmit the motion information of the current block to the video decoding device. This method is called merge mode.

[0054] In the merge mode, the inter predictor 124 selects a predetermined number of merge candidate blocks (hereinafter, referred to as 'merge candidates') from neighboring blocks of the current block.

[0055] As the neighboring blocks for deriving the merge candidate, all or some of the left block A0, the lower left block A1, the upper block B0, the upper right block B1 and the upper left block B2 adjacent to the current block in the current image may be used, such as Figure 4 As shown. In addition, in addition to the current image where the current block is located, a block located in a reference image (which may be the same as or different from the reference image used to predict the current block) may also be used as a merge candidate. For example, a co-located block of the current block in the reference image or a block adjacent to the co-located block may be additionally used as a merge candidate. If the number of merge candidates selected by the above method is less than the preset number, a zero vector is added to the merge candidate.

[0056] The inter-frame predictor 124 configures a merge list including a predetermined number of merge candidates by using adjacent blocks. A merge candidate to be used as motion information of the current block is selected from among the merge candidates included in the merge list, and merge index information for identifying the selected candidate is generated. The generated merge index information is encoded by the entropy encoder 155 and transmitted to the video decoding device.

[0057] Merge skip mode is a special case of merge mode. After quantization, when all transform coefficients for entropy coding are close to zero, only the neighbor block selection information is transmitted without the residual signal. By using merge skip mode, relatively high coding efficiency can be achieved for images with slight motion, still images, screen content images, etc.

[0058] Hereinafter, merge mode and merge skip mode are collectively referred to as merge / skip mode.

[0059] Another method for encoding motion information is the advanced motion vector prediction (AMVP) mode.

[0060] In the AMVP mode, the inter-frame predictor 124 derives a motion vector prediction candidate for a motion vector of a current block by using neighboring blocks of the current block. As neighboring blocks for deriving motion vector prediction candidates, the neighboring blocks may be used. Figure 4 All or some of the left block A0, lower left block A1, upper block B0, upper right block B1 and upper left block B2 adjacent to the current block in the current image shown. In addition, in addition to the current image where the current block is located, blocks located in a reference image (which may be the same as or different from the reference image used to predict the current block) may also be used as adjacent blocks for deriving motion vector prediction candidates. For example, a co-located block of the current block in the reference image or a block adjacent to the co-located block may be used. If the number of motion vector candidates selected by the above method is less than the preset number, a zero vector is added to the motion vector candidates.

[0061] The inter-frame predictor 124 derives a motion vector prediction candidate by using the motion vectors of the neighboring blocks, and determines a motion vector prediction of the motion vector of the current block by using the motion vector prediction candidate. In addition, a motion vector difference is calculated by subtracting the motion vector prediction from the motion vector of the current block.

[0062] The motion vector prediction can be obtained by applying a predefined function (e.g., median and mean calculations, etc.) to the motion vector prediction candidate. In this case, the video decoding device also knows the predefined function. In addition, since the neighboring blocks used to derive the motion vector prediction candidate are blocks that have completed encoding and decoding, the video decoding device may also already know the motion vectors of the neighboring blocks. Therefore, the video encoding device does not need to encode the information for identifying the motion vector prediction candidate. Accordingly, in this case, information about the motion vector difference and information about the reference image used to predict the current block are encoded.

[0063] On the other hand, the motion vector prediction can also be determined by selecting any one of the motion vector prediction candidates. In this case, information for identifying the selected motion vector prediction candidate is additionally encoded together with information about the motion vector difference used to predict the current block and information about the reference image.

[0064] The subtractor 130 generates a residual block by subtracting a prediction block generated by the intra predictor 122 or the inter predictor 124 from the current block.

[0065] The transformer 140 transforms the residual signal in the residual block having the pixel value of the spatial domain into the transform coefficient of the frequency domain. The transformer 140 may transform the residual signal in the residual block by using the entire size of the residual block as the transform unit, or may also divide the residual block into a plurality of sub-blocks, and may perform the transform by using the sub-block as the transform unit. Alternatively, the residual block is divided into two sub-blocks, namely, a transform region and a non-transform region, so as to transform the residual signal by using only the transform region sub-block as the transform unit. Here, the transform region sub-block may be one of two rectangular blocks having a size ratio of 1:1 based on the horizontal axis (or the vertical axis). In this case, the entropy encoder 155 encodes a flag (cu_sbt_flag) indicating that only the sub-block is transformed, and direction (vertical / horizontal) information (cu_sbt_horizontal_flag) and / or position information (cu_sbt_pos_flag), and notifies the video decoding device with a signal. In addition, the size of the transform region subblock may have a size ratio of 1:3 based on the horizontal axis (or vertical axis). In this case, a flag (cu_sbt_quad_flag) for dividing the corresponding partition is additionally encoded by the entropy encoder 155 and signaled to the video decoding apparatus.

[0066] On the other hand, the transformer 140 may perform transformation of the residual block separately in the horizontal direction and the vertical direction. For the transformation, various types of transformation functions or transformation matrices may be used. For example, paired transformation functions for horizontal transformation and vertical transformation may be defined as a multiple transform set (MTS). The transformer 140 may select a transformation function pair with the highest transformation efficiency in the MTS, and may transform the residual block in each of the horizontal and vertical directions. The information (mts_idx) about the transformation function pair in the MTS is encoded by the entropy encoder 155 and notified to the video decoding device with a signal.

[0067] The quantizer 145 quantizes the transform coefficients output from the transformer 140 using the quantization parameters and outputs the quantized transform coefficients to the entropy encoder 155. The quantizer 145 may also quantize the relevant residual block immediately without transforming any block or frame. The quantizer 145 may also apply different quantization coefficients (scaling values) according to the position of the transform coefficients in the transform block. The quantization matrix applied to the quantized transform coefficients arranged in two dimensions may be encoded and signaled to the video decoding device.

[0068] The rearrangement unit 150 may perform rearrangement of coefficient values ​​on the quantized residual value.

[0069] The rearrangement unit 150 can change the 2D coefficient array into a 1D coefficient sequence by using coefficient scanning. For example, the rearrangement unit 150 can use a zig-zag scan or a diagonal scan to scan the DC coefficient to the coefficient of the high frequency region to output a 1D coefficient sequence. Depending on the size of the transform unit and the intra-frame prediction mode, the zig-zag scan can also be replaced by a vertical scan that scans the 2D coefficient array in the column direction and a horizontal scan that scans the 2D block type coefficient in the row direction. In other words, depending on the size of the transform unit and the intra-frame prediction mode, the scanning method to be used can be determined among zig-zag scanning, diagonal scanning, vertical scanning, and horizontal scanning.

[0070] The entropy encoder 155 encodes the sequence of 1D quantized transform coefficients output from the rearrangement unit 150 by using various encoding schemes including Context-based Adaptive Binary Arithmetic Code (CABAC), Exponential Golomb, etc. to generate a bitstream.

[0071] In addition, the entropy encoder 155 encodes information related to block segmentation (e.g., CTU size, CTU segmentation flag, QT segmentation flag, MTT segmentation type, and MTT segmentation direction, etc.) so that the video decoding device can segment the block equivalent to the video encoding device. In addition, the entropy encoder 155 encodes information about the prediction type indicating whether the current block is encoded by intra-frame prediction or inter-frame prediction. The entropy encoder 155 encodes intra-frame prediction information (i.e., information about intra-frame prediction mode) or inter-frame prediction information (merge index in the case of merge mode, and information about reference image index and motion vector difference in the case of AMVP mode) according to the prediction type. In addition, the entropy encoder 155 encodes information related to quantization (i.e., information about quantization parameters and information about quantization matrices).

[0072] The inverse quantizer 160 inversely quantizes the quantized transform coefficient output from the quantizer 145 to generate a transform coefficient. The inverse transformer 165 transforms the transform coefficient output from the inverse quantizer 160 from the frequency domain to the spatial domain to reconstruct a residual block.

[0073] The adder 170 adds the reconstructed residual block and the prediction block generated by the predictor 120 to reconstruct the current block. When intra-prediction is performed on the next block, the pixels in the reconstructed current block are used as reference pixels.

[0074] The loop filtering unit 180 performs filtering on the reconstructed pixels to reduce blocking artifacts, ringing artifacts, blurring artifacts, etc. that occur due to block-based prediction and transform / quantization. The loop filtering unit 180 as an in-loop filter may include all or some of a deblocking filter 182, a sample adaptive offset (SAO) filter 184, and an adaptive loop filter (ALF) 186.

[0075] The deblocking filter 182 filters the boundaries between the reconstructed blocks to remove blocking artifacts that occur due to block unit encoding / decoding, and the SAO filter 184 and the ALF 186 perform additional filtering on the deblocking filtered video. The SAO filter 184 and the ALF 186 are filters for compensating for the difference between the reconstructed pixels and the original pixels that occur due to lossy coding. The SAO filter 184 applies an offset in units of CTU to enhance subjective image quality and encoding efficiency. On the other hand, the ALF 186 performs block unit filtering and applies different filters to compensate for distortion by dividing the boundaries of the corresponding blocks and the degree of the amount of change. Information about the filter coefficients to be used for the ALF can be encoded and notified to the video decoding device with a signal.

[0076] The reconstructed blocks filtered by the deblocking filter 182, the SAO filter 184, and the ALF 186 are stored in the memory 190. When all blocks in one image are reconstructed, the reconstructed image can be used as a reference image for inter-frame prediction of blocks within an image to be subsequently encoded.

[0077] The video encoding device may store a bit stream of encoded video data in a nonvolatile storage medium or transmit the bit stream to a video decoding device through a communication network.

[0078] Figure 5 is a functional block diagram of a video decoding device that can implement the technology of the present invention. Figure 5 , describes a video decoding device and components of the device.

[0079] The video decoding apparatus may include an entropy decoder 510 , a rearrangement unit 515 , an inverse quantizer 520 , an inverse transformer 530 , a predictor 540 , an adder 550 , a loop filtering unit 560 , and a memory 570 .

[0080] Similar to Figure 1 Each component of the video encoding device and the video decoding device can be implemented as hardware or software, or as a combination of hardware and software. In addition, the function of each component can be implemented as software, and the microprocessor can also be implemented to execute the function of the software corresponding to each component.

[0081] The entropy decoder 510 extracts information related to block partitioning by decoding a bitstream generated by a video encoding apparatus to determine a current block to be decoded, and extracts prediction information required to reconstruct the current block and information about a residual signal.

[0082] The entropy decoder 510 determines the size of the CTU by extracting information about the CTU size from a sequence parameter set (SPS) or a picture parameter set (PPS), and divides the image into CTUs of the determined size. In addition, the CTU is determined as the highest level (i.e., the root node) of the tree structure, and the partition information of the CTU can be extracted to partition the CTU by using the tree structure.

[0083] For example, when CTU is segmented by using the QTBTTT structure, the first flag (QT_split_flag) related to the segmentation of QT is first extracted to segment each node into four nodes of the lower layer. In addition, the second flag (mtt_split_flag) related to the segmentation of MTT, the segmentation direction (vertical / horizontal) and / or the segmentation type (binary / tripartite) are extracted with respect to the node corresponding to the leaf node of QT to segment the corresponding leaf node into the MTT structure. As a result, each node below the leaf node of QT is recursively segmented into a BT or TT structure.

[0084] As another example, when a CTU is split by using a QTBTTT structure, a CU split flag (split_cu_flag) indicating whether the CU is split is extracted. When the corresponding block is split, a first flag (QT_split_flag) may also be extracted. During the splitting process, for each node, 0 or more recursive MTT splits may occur after 0 or more recursive QT splits. For example, for a CTU, MTT splits may occur immediately, or, conversely, multiple QT splits may occur only.

[0085] As another example, when CTU is segmented by using the QTBT structure, the first flag (QT_split_flag) related to the segmentation of the QT is extracted to segment each node into four nodes of the lower layer. In addition, the segmentation flag (split_flag) indicating whether the node corresponding to the leaf node of the QT is further segmented into BT and the segmentation direction information are extracted.

[0086] On the other hand, when the entropy decoder 510 determines the current block to be decoded by using the partitioning of the tree structure, the entropy decoder 510 extracts information about the prediction type indicating whether the current block is intra-predicted or inter-predicted. When the prediction type information indicates intra-prediction, the entropy decoder 510 extracts syntax elements of intra-prediction information (intra-prediction mode) for the current block. When the prediction type information indicates inter-prediction, the entropy decoder 510 extracts information representing syntax elements of inter-prediction information, that is, a motion vector and a reference image to which the motion vector refers.

[0087] In addition, the entropy decoder 510 extracts quantization-related information and extracts information on a quantized transform coefficient of the current block as information on a residual signal.

[0088] The rearrangement unit 515 may change the sequence of 1D quantized transform coefficients entropy-decoded by the entropy decoder 510 into a 2D coefficient array (ie, block) again in the reverse order of the coefficient scanning order performed by the video encoding device.

[0089] The inverse quantizer 520 inversely quantizes the quantized transform coefficients, and inversely quantizes the quantized transform coefficients by using quantization parameters. The inverse quantizer 520 may also apply different quantization coefficients (scaling values) to the quantized transform coefficients arranged in 2D. The inverse quantizer 520 may perform inverse quantization by applying a matrix of quantization coefficients (scaling values) from a video encoding device to a 2D array of quantized transform coefficients.

[0090] The inverse transformer 530 reconstructs a residual signal by inversely transforming the inversely quantized transform coefficient from the frequency domain to the spatial domain to generate a residual block of the current block.

[0091] In addition, when the inverse transformer 530 inversely transforms a partial region (sub-block) of the transform block, the inverse transformer 530 extracts a flag (cu_sbt_flag) that only the sub-block of the transform block is transformed, direction (vertical / horizontal) information (cu_sbt_horizontal_flag) of the sub-block, and / or position information (cu_sbt_pos_flag) of the sub-block. The inverse transformer 530 also inversely transforms the transform coefficient of the corresponding sub-block from the frequency domain to the spatial domain to reconstruct the residual signal, and fills the region that is not inversely transformed with a value of "0" as the residual signal to generate a final residual block of the current block.

[0092] In addition, when MTS is applied, the inverse transformer 530 determines the transform index or transform matrix to be applied in each of the horizontal and vertical directions by using the MTS information (mts_idx) signaled from the video encoding device. The inverse transformer 530 also performs inverse transform on the transform coefficients in the transform block in the horizontal and vertical directions by using the determined transform function.

[0093] The predictor 540 may include an intra predictor 542 and an inter predictor 544. The intra predictor 542 is activated when the prediction type of the current block is intra prediction, and the inter predictor 544 is activated when the prediction type of the current block is inter prediction.

[0094] The intra predictor 542 determines an intra prediction mode of a current block among a plurality of intra prediction modes according to syntax elements of the intra prediction mode extracted from the entropy decoder 510. The intra predictor 542 also predicts the current block by using neighboring reference pixels of the current block according to the intra prediction mode.

[0095] The inter predictor 544 determines a motion vector of a current block and a reference image to which the motion vector refers by using a syntax element of the inter prediction mode extracted from the entropy decoder 510 .

[0096] The adder 550 reconstructs the current block by adding the residual block output from the inverse transformer 530 and the prediction block output from the inter predictor 544 or the intra predictor 542. When intra-predicting a subsequent block to be decoded, pixels within the reconstructed current block are used as reference pixels.

[0097] The loop filtering unit 560 as an in-loop filter may include a deblocking filter 562, an SAO filter 564, and an ALF 566. The deblocking filter 562 performs deblocking filtering on the boundary between the reconstructed blocks in order to remove block artifacts that occur due to block unit decoding. The SAO filter 564 and the ALF 566 perform additional filtering on the reconstructed blocks after deblocking filtering in order to compensate for the difference between the reconstructed pixels and the original pixels that occurs due to lossy encoding. The filter coefficients of the ALF are determined by using information about the filter coefficients decoded from the bitstream.

[0098] The reconstructed blocks filtered by the deblocking filter 562, the SAO filter 564, and the ALF 566 are stored in the memory 570. When all blocks in one image are reconstructed, the reconstructed image can be used as a reference image for inter-frame prediction of blocks within an image to be subsequently encoded.

[0099] In some embodiments, the present invention relates to encoding and decoding video images as described above. More specifically, the present invention provides a video encoding and decoding method and apparatus using geometric motion prediction in inter-frame prediction of a current block. Here, the geometric motion prediction divides the current block into an arbitrary shape, performs different motion predictions on each partitioned block to generate prediction factors, and mixes the prediction factors based on different weights of predefined pixel units to generate a final inter-frame prediction factor.

[0100] The following embodiments may be performed by the inter-frame predictor 124 in the video encoding apparatus. The following embodiments may also be performed by the inter-frame predictor 544 in the video decoding apparatus.

[0101] The video encoding device can generate signaling information associated with the present embodiment from the perspective of optimizing rate distortion when predicting the current block. The video encoding device can encode the signaling information using the entropy encoder 155 and send the encoded signaling information to the video decoding device. The video decoding device can decode the signaling information associated with the prediction of the current block from the bitstream using the entropy decoder 510.

[0102] In the following description, the term 'target block' may be used interchangeably with a current block or a coding unit (CU), or may refer to some areas of a coding unit.

[0103] In addition, a flag value of true indicates a case where the flag is set to 1. In addition, a flag value of false indicates a case where the flag is set to 0.

[0104] I. Merge / skip mode for inter-frame prediction

[0105] The following embodiments are described based on the inter-frame predictor 124 in the video encoding apparatus, but may also be performed by the inter-frame predictor 544 in the video decoding apparatus as described above.

[0106] Hereinafter, a method of configuring a merge candidate list of motion information in a merge / skip mode of inter prediction is described. To support the merge / skip mode, the inter predictor 124 in the video encoding apparatus may configure a merge candidate list by selecting a preset number (eg, 6) of merge candidates.

[0107] The inter-frame predictor 124 searches for spatial merging candidates. The inter-frame predictor 124 searches for spatial merging candidates from neighboring blocks, such as Figure 4 As shown, up to 4 spatial merging candidates can be selected.

[0108] The inter-frame predictor 124 searches for a temporal merge candidate. The inter-frame predictor 124 may add a block (co-located block) located at the same position as the current block in the reference picture (which may be the same as or different from the reference picture used to predict the current block) instead of the current picture where the target block is located as a temporal merge candidate. One temporal merge candidate may be selected.

[0109] The inter-frame predictor 124 searches for history-based motion vector predictor (HMVP) candidates. The inter-frame predictor 124 may store the motion vectors of the previous h (here, h is a natural number) CUs in a table and use the motion vectors as merge candidates. The size of the table is 6, and the motion vectors of the previous CUs are stored according to a first-in first-out (FIFO) method. This indicates that up to 6 HMVP candidates are stored in the table. The inter-frame predictor 124 may set the latest motion vector among the HMVP candidates stored in the table as a merge candidate.

[0110] The inter predictor 124 searches for a pairwise average MVP (PAMVP) candidate. The inter predictor 124 may set an average of motion vectors of the first and second candidates in the merge candidate list as a merge candidate.

[0111] If the merge candidate list cannot be filled up even after performing all the above-mentioned search processes (ie, if the preset number is not filled up), the inter predictor 124 adds a zero motion vector as a merge candidate.

[0112] From the perspective of optimizing coding efficiency, the inter-frame predictor 124 may determine a merge index indicating one of the candidates in the merge candidate list. The inter-frame predictor 124 may derive a motion vector predictor (MVP) from the merge candidate list by using the merge index, and then may determine the MVP as the motion vector of the current block. In addition, the video encoding device may signal the merge index to the video decoding device.

[0113] In case of the skip mode, the video encoding apparatus uses the same motion vector transmission method as the merge mode, but does not transmit a residual block corresponding to a difference between a current block and a predicted block.

[0114] The method of configuring the above-mentioned merge candidate list can be performed in the same manner by the inter-frame predictor 544 in the video decoding device. The video decoding device can decode the merge index. The inter-frame predictor 544 can derive the MVP from the merge candidate list by using the merge index, and then can determine the MVP as the motion vector of the current block.

[0115] II. AMVP Mode for Inter-frame Prediction

[0116] The following embodiments are described based on the inter-frame predictor 124 in the video encoding apparatus, but may also be performed by the inter-frame predictor 544 in the video decoding apparatus as described above.

[0117] Hereinafter, a method of configuring a candidate list of motion information in the AMVP mode of inter prediction is described. To support the AMVP mode, the inter predictor 124 in the video encoding apparatus may select a preset number (eg, 2) of candidates to configure the candidate list.

[0118] The inter-frame predictor 124 searches for spatial candidates. The inter-frame predictor 124 searches for spatial candidates from neighboring blocks, such as Figure 4 As shown. Up to two spatial candidates can be selected.

[0119] The inter-frame predictor 124 searches for temporal candidates. The inter-frame predictor 124 may add a block located at the same position as the current block in a reference picture (which may be the same as or different from the reference picture used to predict the current block) (rather than the current picture where the target block is located) as a temporal candidate. One temporal candidate may be selected.

[0120] If the candidate list cannot be filled up even after performing all the above-mentioned search processes (ie, if the preset number cannot be filled up), the inter predictor 124 adds a zero motion vector as a candidate.

[0121] From the perspective of optimizing codec efficiency, the inter-frame predictor 124 may determine a candidate index indicating one of the candidates in the candidate list. The inter-frame predictor 124 may derive the MVP from the candidate list by using the candidate index. In addition, from the perspective of optimizing codec efficiency, the inter-frame predictor 124 calculates a motion vector difference (MVD) by subtracting the MVP from the motion vector after determining the motion vector. In addition, the video encoding device may signal the candidate index and the MVD to the video decoding device.

[0122] The method of configuring the above-mentioned AMVP candidate list can be performed in the same manner by the inter-frame predictor 544 in the video decoding device. The video decoding device can decode the candidate index and the MVD. The inter-frame predictor 544 can derive the MVP from the candidate list by using the candidate index. The inter-frame predictor 544 can restore the motion vector of the current block by adding the MVD and the MVP.

[0123] In the following, the merge candidate list and the merge list can be used interchangeably. In addition, the candidate list of the AMVP mode and the AMVP list can be used interchangeably.

[0124] III. Embodiments according to the present invention

[0125] Hereinafter, an embodiment according to the present invention is described based on a video decoding apparatus, but can be similarly applied to a video encoding apparatus.

[0126] Figure 6 is a schematic diagram illustrating block partitioning for geometric motion prediction according to an embodiment of the present invention.

[0127] The inter-frame predictor 544 according to the present invention bisects the current block based on a straight line orthogonal to a line segment having a constant angle θ and a constant distance ρ based on the center of the block. Hereinafter, the two bisected blocks are referred to as a first block partition and a second block partition. In addition, the straight line that bisects the current block is referred to as a bisector.

[0128] Here, the center of the block represents a virtual position where a position at half the height of the block and a position at half the width of the block intersect with respect to the current block before partitioning. Angle θ represents the angle of a line segment orthogonal to the bisector, rotated counterclockwise from an imaginary horizontal axis passing through the center of the block. Distance ρ represents the distance between the center of the block and the bisector.

[0129] As described above, the bisector divides the current block into two different block partitions. The inter-frame predictor 544 according to the present invention divides the current block into two blocks, namely the first block partition and the second block partition for performing prediction respectively, based on the bisector by utilizing the above-mentioned geometric partition.

[0130] Specifically, the GPM of the related art converts information about the bisector (e.g., Figure 6 The video decoding device may decode the geometric block partition of the current block by using the information about the resolved straight line. Here, the information about the straight line may include an angle θ and a distance ρ based on the center of the block.

[0131] Figure 7a and Figure 7b is a schematic diagram showing a straight line bisecting a block according to an embodiment of the present invention.

[0132] On the other hand, according to Figure 6 The examples in this section use different methods to signal angle and distance, such as Figure 7a , Figure 7b As shown in Table 1, an index mapped by combining an angle and a distance for partitioning a current block into a first block partition and a second block partition may be signaled from a video encoding apparatus to a video decoding apparatus.

[0133] As described above, the geometric partition shape is based on a straight line representing a bisection of the block. The information of the straight line may include an index distanceIdx representing the distance p from the center of the block to the straight line and an index angleIdx representing the angle θ of a line segment orthogonal to the straight line. Figure 7a The index representing the angle of the line segment orthogonal to the straight line can be set as shown. Figure 7b The setup shown partitions the shape into 64 cells based on angle and distance.

[0134] 64 geometric partition shapes may be signaled using the merge_gpm_partition_idx syntax, which is an index indicating the geometric partition shape, as shown in Table 1. In other words, the shape of partitioning the current block into the first block partition and the second block partition according to various angles and various distances may be effectively signaled using one index.

[0135] [Table 1]

[0136] merge_gpm_partition_idx 0 1 2 3 4 5 6 7 angleIdx 0 0 2 2 2 2 3 3 distanceIdx 1 3 0 1 2 3 0 1 merge_gpm_partition_idx 8 9 10 11 12 13 14 15 angleIdx 3 3 4 4 4 4 5 5 distanceIdx 2 3 0 1 2 3 0 1 merge_gpm_partition_idx 16 17 18 19 20 21 22 23 angleIdx 5 5 8 8 11 11 11 11 distanceIdx 2 3 1 3 0 1 2 3 merge_gpm_partition_idx 24 25 26 27 28 29 30 31 angleIdx 12 12 12 12 13 13 13 13 distanceIdx 0 1 2 3 0 1 2 3 merge_gpm_partition_idx 32 33 34 35 36 37 38 39 angleIdx 14 14 14 14 16 16 18 18 distanceIdx 0 1 2 3 1 3 1 2 merge_gpm_partition_idx 40 41 42 43 44 45 46 47 angleIdx 18 19 19 19 20 20 20 21 distanceIdx 3 1 2 3 1 2 3 1 merge_gpm_partition_idx 48 49 50 51 52 53 54 55 angleIdx 21 21 24 24 27 27 27 28 distanceIdx 2 3 1 3 1 2 3 1 merge_gpm_partition_idx 56 57 58 59 60 61 62 63 angleIdx 28 28 29 29 29 30 30 30 distanceIdx 2 3 1 2 3 1 2 3

[0137] from Figure 7b The index distanceIdx derived from the example of is a value excluding the size of the current block. Therefore, the actual distance between the pixel in the current block and the straight line can be calculated by using the size information of the current block, the index angleIdx indicating the angle, and the index distanceIdx indicating the distance. Here, the actual distance is a value expressed in pixel units.

[0138] On the other hand, the actual distance may be used to calculate the weight for each pixel in the current block. For example, for a pixel in the first block partition, as the actual distance between the pixel and the straight line increases, the weight of the prediction factor of the first block partition as described above increases, and the weight of the prediction factor of the second block partition may decrease. For pixels located on the bisecting straight line, the two prediction factors may use weights with the same value. At this time, the sum of the weights of the two prediction factors for one pixel remains 1.

[0139] Figure 8 is a schematic diagram conceptually illustrating inter-frame prediction using geometric partitioning.

[0140] As in Figure 8 In the example of , for two different block partitions of the current block within the current picture, the inter-frame predictor 542 performs prediction using each motion vector (mv0 or mv1). For the prediction block for the first block partition and the prediction block for the second block partition, the inter-frame predictor 542 performs a weighted sum-based blending process to generate a final prediction block for the current block. Figure 8 In the example of , PART_0 represents the first block partition, and PART_1 represents the second block partition. In addition, the reference block of PART_0 represents the prediction block for the first block partition, and the reference block of PART_1 represents the prediction block for the second block partition.

[0141] In order to execute Figure 8 The video decoding apparatus obtains a first prediction block using a motion vector for a first block partition and obtains a second prediction block using a motion vector for a second block partition. At this time, in the process of obtaining a prediction block for each block partition, the video decoding apparatus may obtain a prediction block multiplied by different weights according to pixel positions as described above. In addition, the video decoding apparatus may use a shift operation and a cropping operation in the process of generating a final prediction block from the prediction blocks multiplied by the weights.

[0142] Fig. 9 is a schematic diagram showing a GPM merge list for geometric motion prediction.

[0143] In execution Figure 8 When the geometric motion prediction is shown, the inter-frame predictor 544 can be as shown in Fig. 9 The motion information for motion prediction is selected from the merge list as in the example of , and then the selected motion information can be used.

[0144] However, unlike the existing block-unit motion prediction techniques, the inter-frame predictor 544 is used as in Fig. 9 unidirectional prediction for one block partition is performed by limiting the prediction directionality as in the example for geometric motion prediction of . This is because, compared with motion prediction based on block units, when motion prediction is performed according to bidirectional prediction for each block partition in geometric motion prediction, the memory bandwidth for prediction increases by two times. Therefore, in order to effectively solve the above-mentioned memory bandwidth increase problem, a technique for limiting the directionality of prediction for each block partition may be applied.

[0145] In the case where the directionality of prediction is restricted for each block partition in geometric motion prediction, a GPM merge list for geometric motion prediction can be generated using an existing merge list. In order to generate a GPM merge list, the video decoding device first configures the existing merge list. Thereafter, the video decoding device can generate a GPM merge list for geometric motion prediction from the merge list based on the prediction directionality and the order in the list. At this time, the video decoding device adds the motion information in the L0 direction to the GPM merge list to generate a merge candidate for geometric motion prediction of the first block partition. In addition, the video decoding device can add the motion information in the L1 direction to the GPM merge list to generate a merge candidate for geometric motion prediction of the second block partition. In other words, the video decoding device can derive unidirectional motion information in one direction from the existing bidirectional motion information, and can then add the derived motion information to the GPM merge list.

[0146] like Fig. 9 As shown in the example of , in the existing geometric motion prediction method, the video decoding device can use the merge candidate in the GPM merge list as the motion information for motion prediction of the first block partition and the second block partition. Alternatively, the motion information for motion prediction of the first block partition and the second block partition can be derived.

[0147] Fig.10a and Fig.10b is a schematic diagram showing an AMVP list and a GPM merge list for geometric motion prediction according to an embodiment of the present invention.

[0148] In execution Figure 8 When performing geometric motion prediction as shown, the inter-frame predictor 544 can use the AMVP mode and the merge mode together, such as Fig.10a and Fig.10b as shown in the example.

[0149] For example, as in Fig.10a In the example of , the inter-frame predictor 544 performs motion prediction by using the motion information obtained from the AMVP list based on the AMVP mode for one of the first block partition and the second block partition. In addition, the inter-frame predictor 544 can perform motion prediction by using the motion information obtained from the GPM merge list based on the merge mode for the other block partition.

[0150] exist Fig.10a In the example of , the video decoding device uses the existing AMVP list to apply the AMVP mode. At this time, the information about the prediction direction can be determined as follows. For example, as the information about the prediction direction, a flag indicating the prediction direction of the candidate included in the AMVP list can be notified to the video decoding device by a signal from the video encoding device. Alternatively, a prediction direction predefined between the video encoding device and the video decoding device can be used. Fig.10a In the example, the L0 direction is applied as the prediction direction of the AMVP list, and the L1 direction is applied as the prediction direction of the GPM merge list.

[0151] As another example, in Fig.10b In the example of , the inter predictor 544 may perform motion prediction by using motion information obtained from the GPM AMVP list based on the AMVP mode for one of the first block partition and the second block partition.

[0152] At this time, the GPM AMVP list can be configured in a manner similar to the method of generating a GPM merge list from an existing merge list. In order to generate the GPM AMVP list, the video decoding device first configures the existing AMVP list. Thereafter, the video decoding device generates a GPM AMVP list from the existing AMVP list to include unidirectional motion information. At this time, the information about the prediction direction can be determined as follows. For example, as the information about the prediction direction, a flag indicating the prediction direction of the candidates included in the GPM AMVP list can be notified to the video decoding device by a signal from the video encoding device. Alternatively, a predefined prediction direction can be used between the video encoding device and the video decoding device. Therefore, the GPM AMVP list can be configured to include only motion information corresponding to unidirectional motion. Fig.10b In the example of , the GPM AMVP list includes motion information in the L0 direction.

[0153] Compared with the existing geometric motion prediction method using only the existing merge mode, the geometric motion prediction using the AMVP mode and the merge mode according to the present invention can generate more accurate motion information by using the AMVP mode for at least one direction. Accordingly, compared with the geometric motion prediction method using only the existing merge mode, the prediction accuracy can be improved. In addition, since accurate motion information can be signaled using the AMVP mode, the block partition shape can be determined based on the corresponding motion information using the same predefined arithmetic operation between the video encoding device and the video decoding device. In other words, the signaling of additional information about the geometric partition shape of the current block can be omitted. Therefore, since the block can be partitioned by using the block partition shape derived from the motion information of the AMVP mode, the signaling overhead for the block partition shape can be reduced, thereby improving the encoding and decoding efficiency.

[0154] Fig.11 is a schematic diagram illustrating geometric motion prediction using both the AMVP mode and the merge mode according to an embodiment of the present invention.

[0155] As in Fig.11 In the example of , the current block can be partitioned into a first block partition and a second block partition based on a bisector line. At this time, the bisector line of the current block can be derived from the motion information of the first block partition. In the geometric motion prediction according to the present invention, the inter-frame predictor 544 derives the geometric block partition shape of the current block by using a reference block referenced by the motion information of the first block partition. The inter-frame predictor 544 obtains the motion information of the first block partition based on the motion vector predictor and the motion vector difference obtained from the GPM AMVP list. The inter-frame predictor 544 can specify a specific position of the reference picture using the obtained motion information. The inter-frame predictor 544 can set the same reference block as the current block based on the specific position, and then can perform a process of deriving the geometric partition shape within the region of the reference block. The inter-frame predictor 544 performs an operation of searching for the boundary area within the block when deriving the geometric partition shape, thereby deriving the bisector line of the reference block.

[0156] As an example, a directional derivation method based on a Sobel filter can be used to search for a boundary area within a block. At this time, a directional prediction operation using a 3×3 kernel can be performed as an example of the above-mentioned operation based on a Sobel filter. In addition, the inter-frame predictor 544 can select one of the predefined geometric partition shapes when deriving the geometric partition shape of the block using the Sobel filter.

[0157] On the other hand, when determining the geometric partition shape, in addition to the method of partitioning the block using a single straight line, the inter-frame predictor 544 can derive an arbitrary partition shape that partitions the block into an arbitrary shape based on the boundary of the object present in the reference block. The method of performing prediction using the above-mentioned arbitrary partition shape is also included in the scope of the present invention. In addition, such an arbitrary partition shape can be expressed using a bisecting curve. In the following, the bisecting straight line and the bisecting curve are generally expressed as a bisecting edge.

[0158] In order to perform geometric motion prediction using the AMVP mode and the MERGE mode according to the present invention, the video decoding device derives motion information of the first block partition by using the GPM AMVP list, and then selects the first reference block by using the derived motion information of the first block partition. The video decoding device derives a geometric partition shape from the selected first reference block. The video decoding device derives motion information of the second block partition by using the GPM merge list, and then selects the second reference block by using the derived motion information of the second block partition. The video decoding device generates a final prediction block by using a hybrid process based on the geometric partition shape derived from the first reference block of the first reference block and the second reference block.

[0159] In the following, for the use of Fig. 9 The geometric motion prediction of the merge mode shown in , uses the motion information to signal as follows.

[0160] The video decoding device first decodes the geometric partition information from the bitstream. At this time, the video decoding device can decode the index mapped to the single representative value for the shape of the block partition. In other words, the video decoding device can decode merge_gpm_partition_idx (which is an index indicating the partition shape) as the geometric partition information. At this time, the index can indicate a combination of an angle and a distance for partitioning the current block into a first block partition and a second block partition according to the shape of the block partition.

[0161] Thereafter, the video decoding apparatus decodes the motion information of the first block partition from the bitstream, and then decodes the motion information of the second block partition. Here, the motion information of the first block partition includes merge_gpm_idx0, which is an index indicating the first candidate in the GPM merge list. In addition, the motion information of the second block partition includes merge_gpm_idx1, which is an index indicating the second candidate in the GPM merge list.

[0162] In the case where the directionality of geometric motion prediction is restricted to one direction, the video decoding apparatus may reconfigure the GPM merge list by processing the existing merge list as unidirectional motion information to reduce the memory bandwidth for prediction.

[0163] The video decoding device obtains the first reference block (i.e., the first prediction block) by using the motion information of the first block partition. In addition, the video decoding device obtains the second reference block (i.e., the second prediction block) by using the motion information of the second block partition. Finally, the video decoding device can mix the first reference block and the second reference block by using the decoded geometric block partition shape to generate a final prediction block of the current block.

[0164] In the following, as shown in Figures 10 and Fig.11 As shown, a method for signaling motion information for geometric motion prediction using the AMVP mode and the merge mode is described.

[0165] Fig.12 is a flowchart illustrating a method of decoding motion information by a video decoding apparatus according to another embodiment of the present invention.

[0166] As described above, the block partition shape may be determined based on the AMVP mode. Therefore, in the case of geometric motion prediction using both the AMVP mode and the merge mode, decoding of the geometric partition information may be omitted.

[0167] In the case of using both AMVP mode and merge mode, the motion information for the first block partition may include geometric prediction direction information, motion vector predictor information, reference picture index and motion vector difference. In addition, the motion information for the second block partition may include motion information derived from the GPM merge list.

[0168] The video decoding apparatus decodes geometric prediction direction information of the first block partition from the bitstream (S1200). Here, the geometric prediction direction information of the first block partition indicates which reference picture list of L0 and L1 is used. At this time, the information of the reference picture list may be a flag having a value of 0 or 1. Hereinafter, as the geometric prediction direction information, the flag indicating the L0 or L1 reference picture list is expressed as gpm_dir_flag.

[0169] The video decoding apparatus decodes the motion vector predictor information of the first block partition from the bitstream (S1202). If the AMVP list includes Fig.10a In the example of two candidates, the motion vector predictor information may be a flag indicating a candidate index of one of the AMVP candidates. In the following, as motion vector predictor information, a flag indicating one of the candidates is expressed as gpm_mvp_flag. The video decoding device may select a candidate according to the value of gpm_mvp_flag. Thereafter, based on the geometric prediction direction of the first block partition, the unidirectional motion vector predictor of the first block partition may be set from the (one or more) motion vector predictors of the selected candidate.

[0170] As another example, if the GPM AMVP list includes Fig.10b In the example of , the motion vector predictor information may be a flag indicating one of the GPM AMVP candidates. Similarly, as the motion vector predictor information, the flag indicating one of the candidates is expressed as gpm_mvp_flag. The video decoding device may set the motion vector of the candidate selected according to the value of gpm_mvp_flag as the motion vector predictor of the first block partition. As described above, in order to match the geometric prediction direction information of the first block partition, the GPM AMVP list may include only motion information corresponding to one direction.

[0171] The video decoding apparatus decodes a reference picture index of the first block partition from the bitstream (S1204). The reference picture index may be an index indicating a reference picture included in the L0 or L1 reference picture list based on the geometric prediction direction information of the first block partition. In the following, the index indicating the reference picture for the first block partition is expressed as gpm_ref_idx.

[0172] The video decoding apparatus decodes the motion vector difference of the first block partition from the bitstream (S1206).

[0173] Here, the motion vector difference of the first block partition represents the difference between the motion vector predictor of the first block partition and the motion vector of the first block partition. At this time, the motion vector difference can be parsed into the x, y coordinate values ​​of the motion vector. Alternatively, the motion vector difference can be parsed into predefined direction information and distance information from the origin of the corresponding direction. In the following, the motion vector difference is expressed as gpm_mvd_coding.

[0174] After decoding all information about the AMVP mode for the first block partition, the video decoding device obtains the first reference block (i.e., the first prediction block) of the current block by using the motion information of the first block partition. In addition, as described above, the video decoding device derives the geometric block partition shape of the current block by using the first reference block.

[0175] The video decoding apparatus decodes motion information for the second block partition from the bitstream (S1208).

[0176] Here, the motion information of the second block partition may be an index indicating a merge candidate in the GPM merge list. Hereinafter, the motion information of the second block partition is expressed as merge_gpm_idx. The video decoding apparatus performs motion prediction by using the motion information of the merge candidate for the second block partition as is, thereby obtaining a second reference block (i.e., a second prediction block) of the current block.

[0177] Finally, the video decoding apparatus may generate a final prediction block of the current block by mixing the first reference block and the second reference block based on the derived geometric block partition shape.

[0178] On the other hand, if a predefined direction is used as the geometric prediction direction of the first block partition between the video encoding device and the video decoding device, the operation of decoding the information of the geometric prediction direction of the first block partition can be omitted. In other words, the signaling of the information of the geometric prediction direction can be omitted.

[0179] As an example, the geometric prediction direction of the first block partition may be limited to L0. Accordingly, if the geometric prediction direction of the first block partition is limited to L0, the information of the motion vector predictor of the first block partition, the reference picture index of the first block partition, and the information of the motion vector difference of the first block partition all represent information limited to the L0 direction. In addition, for the second block partition, the merge list may be configured by limitedly utilizing the motion information in the L1 direction.

[0180] In the following, reference Fig.13 and Fig.14 A method for predicting a current block according to geometric motion prediction utilizing both AMVP mode and merge mode is described.

[0181] Fig.13 is a flowchart illustrating a method for a video encoding apparatus to predict a current block according to an embodiment of the present invention.

[0182] The video encoding apparatus determines motion information of a first block partition for a current block (S1300). Here, the motion information of the first block partition may include geometric prediction direction information, motion vector predictor information, reference picture index, and motion vector difference.

[0183] The video encoding apparatus determines a flag indicating the L0 or L1 reference picture list as the geometric prediction direction information.

[0184] In addition, the video encoding apparatus determines a unidirectional motion vector of the first block partition according to the geometric prediction direction information.

[0185] The video encoding device generates an AMVP list of the current block. The video encoding device determines a flag indicating one of the candidates in the AMVP list of the current block as motion vector predictor information. The video encoding device determines a unidirectional motion vector predictor by using geometric prediction direction information, motion vector predictor information, and the AMVP list.

[0186] The video encoding apparatus determines, as a reference picture index, an index indicating a reference picture included in the L0 or L1 reference picture list based on the geometric prediction direction information.

[0187] The video encoding apparatus subtracts the motion vector predictor from the motion vector to generate a motion vector difference for the first block partition.

[0188] In terms of rate-distortion optimization, the video encoding apparatus may determine geometric prediction direction information, a motion vector of a first block partition, motion vector predictor information, and a reference picture index.

[0189] At this time, when a predefined direction is used as the geometric prediction direction of the first block partition, the video encoding apparatus may omit determining the geometric prediction direction information.

[0190] The video encoding apparatus generates a first prediction block of a current block by using motion information of a first block partition (S1302).

[0191] The video encoding apparatus may generate a first prediction block by using a motion vector and a reference picture index.

[0192] The video encoding apparatus determines a merge candidate index for a second block partition of the current block ( S1304 ).

[0193] In terms of rate-distortion optimization, the video encoding apparatus may determine a merge candidate index.

[0194] The video encoding apparatus generates a second prediction block of the current block by using the merge candidate index (S1306). The video encoding apparatus may obtain motion information of the second block partition from the GPM merge list according to the merge candidate index, and then generates a second prediction block by using the motion information of the second block partition.

[0195] In order to generate a GPM merge list, the video encoding apparatus first generates a merge list of the current block. The video encoding apparatus generates a GPM merge list from the merge list, and each candidate in the GPM merge list includes unidirectional motion information.

[0196] The video encoding apparatus derives a geometric block partition shape for partitioning a current block into a first block partition and a second block partition by using the first prediction block ( S1308 ).

[0197] The video encoding device performs an arithmetic operation to search for a boundary area within the area of ​​the first prediction block, thereby deriving a bisection boundary for dividing the first prediction block into the first block partition and the second block partition. Thereafter, the video encoding device may select one of the predefined geometric block partition shapes based on the bisection boundary.

[0198] The video encoding apparatus generates a final prediction block of the current block by using a geometric block partition shape (S1310). The video encoding apparatus may generate the final prediction block by performing weighted sum-based blending on the first prediction block and the second prediction block by using the geometric block partition shape.

[0199] The video encoding apparatus encodes motion information for the first block partition ( S1312 ).

[0200] On the other hand, when a predefined direction is used as the geometric prediction direction of the first block partition, the video encoding apparatus may omit encoding of the geometric prediction direction information in the motion information for the first block partition.

[0201] The video encoding apparatus encodes a merge candidate index for the second block partition ( S1314 ).

[0202] Fig.14 is a flowchart illustrating a method for predicting a current block by a video decoding device according to an embodiment of the present invention.

[0203] The video decoding apparatus decodes motion information of a first block partition for a current block from a bitstream (S1400). Here, the motion information of the first block partition includes geometric prediction direction information, motion vector predictor information, reference picture index, and motion vector difference.

[0204] according to Fig.12 In the order shown, the video decoding device can decode the motion information of the first block partition. In other words, the video decoding device decodes a flag indicating the L0 or L1 reference picture list as geometric prediction direction information. The video decoding device decodes a flag indicating one of the candidates in the AMVP list of the current block as motion vector predictor information. The video decoding device decodes an index indicating a reference picture included in the L0 or L1 reference picture list as a reference picture index based on the geometric prediction direction information. In addition, the video decoding device decodes a motion vector difference.

[0205] On the other hand, when a predefined direction is used as the geometric prediction direction of the first block partition, the video decoding apparatus may omit decoding the geometric prediction direction information.

[0206] The video decoding apparatus generates a first prediction block of the current block by using motion information of the first block partition (S1402).

[0207] The video decoding device generates an AMVP list. The video decoding device obtains a unidirectional motion vector predictor by using geometric prediction direction information, motion vector predictor information, and the AMVP list. The video decoding device generates a motion vector for the first block partition by adding the motion vector predictor to the motion vector difference. The video decoding device can generate a first prediction block by using the motion vector and the reference picture index.

[0208] The video decoding apparatus decodes a merge candidate index for a second block partition of the current block from a bitstream ( S1404 ).

[0209] The video decoding apparatus generates a second prediction block of the current block by using the merge candidate index (S1406). The video decoding apparatus may obtain motion information of the second block partition from the GPM merge list according to the merge candidate index, and then generates a second prediction block by using the motion information of the second block partition.

[0210] In order to generate a GPM merge list, the video decoding apparatus first generates a merge list of the current block. The video decoding apparatus generates a GPM merge list from the merge list, and each candidate in the GPM merge list includes unidirectional motion information.

[0211] The video decoding apparatus derives a geometric block partition shape for partitioning a current block into a first block partition and a second block partition by using the first prediction block ( S1408 ).

[0212] The video decoding apparatus performs an arithmetic operation to search for a boundary region within the region of the first prediction block to derive a bisection boundary for dividing the first prediction block into the first block partition and the second block partition. Thereafter, the video decoding apparatus may select one of the predefined geometric block partition shapes based on the bisection boundary.

[0213] The video decoding apparatus generates a final prediction block of the current block by using a geometric block partition shape (S1410). The video decoding apparatus may generate the final prediction block by performing weighted sum-based blending on the first prediction block and the second prediction block by using the geometric block partition shape.

[0214] Although the steps in the various flowcharts are described as being performed sequentially, these steps merely illustrate the technical ideas of some embodiments of the present invention. Therefore, a person of ordinary skill in the art to which the present invention belongs can perform the steps by changing the order described in the various figures or by performing two or more steps in parallel. Therefore, the steps in the various flowcharts are not limited to the order shown in the order in which they occur.

[0215] It should be understood that the above description presents illustrative embodiments that can be implemented in various other ways. The functions described in some embodiments can be implemented by hardware, software, firmware and / or their combination. It should also be understood that the functional components described in the present invention are marked as "... units" to highlight the possibility of their independent implementation.

[0216] On the other hand, the various methods or functions described in some embodiments can be implemented as instructions stored in a non-volatile recording medium, which can be read and executed by one or more processors. Non-volatile recording media can include various types of recording devices that store data in a form that is readable by a computer system. For example, non-volatile recording media can include storage media, such as erasable programmable read-only memory (EPROM), flash drives, optical disk drives, magnetic hard drives, and solid-state drives (SSDs), etc.

[0217] Although exemplary embodiments of the present invention are described for illustrative purposes, it will be appreciated by those skilled in the art that various modifications, additions and substitutions may be made without departing from the spirit and scope of the present invention. Therefore, embodiments of the present invention are described for simplicity and clarity. The scope of the technical ideas of the embodiments of the present invention are not limited by the examples. Accordingly, it will be appreciated by those skilled in the art that the scope of the present invention should not be limited by the embodiments clearly described above, but by the claims and their equivalents.

[0218] Reference numerals:

[0219] 124: Inter-frame predictor

[0220] 155: Entropy Encoder

[0221] 510: Entropy Decoder

[0222] 544: Inter-frame predictor.

[0223] CROSS-REFERENCE TO RELATED APPLICATIONS

[0224] This application claims priority to and the benefit of Korean Patent Application No. 10-2022-0059398, filed on May 16, 2022, and Korean Patent Application No. 10-2023-0051381, filed on April 19, 2023, the entire contents of each of which are incorporated herein by reference.

Claims

1. A method for decoding a current block performed by a video decoding device, the method comprising: decoding motion information for a first block partition of a current block from a bitstream; generating a first prediction block of the current block by using motion information of the first block partition; decoding a merge candidate index for a second block partition of the current block from a bitstream; acquiring motion information of a second block partition from a geometric partition mode (GPM) merge list based on a merge candidate index, and then generating a second prediction block of the current block by using the motion information of the second block partition; deriving a geometric block partition shape for partitioning a current block into a first block partition and a second block partition by using the first prediction block; as well as The first prediction block and the second prediction block are mixed by using a geometric block partition shape to generate a final prediction block of the current block.

2. The method according to claim 1, wherein: The motion information of the first block partition includes geometric prediction direction information, motion vector prediction factor information, reference picture index and motion vector difference.

3. The method according to claim 2, wherein: Decoding the motion information for the first block partition includes: decoding a flag indicating an L0 or L1 reference picture list as geometric prediction direction information; and A flag indicating one of the candidates in an advanced motion vector prediction (AMVP) list of the current block is decoded as the motion vector predictor information.

4. The method according to claim 3, wherein: Decoding the motion information for the first block partition includes: decoding, based on the geometric prediction direction information, an index indicating a reference picture included in the L0 or L1 reference picture list as a reference picture index; and Decode motion vector differences from the bitstream.

5. The method according to claim 3, wherein: When a predefined direction is used as the geometric prediction direction of the first block partition, decoding of the geometric prediction direction information is omitted.

6. The method according to claim 4, wherein: Generating a first prediction block includes: Generate an AMVP list; and The unidirectional motion vector predictor is obtained by utilizing the geometric prediction direction information, the motion vector predictor information and the AMVP list.

7. The method according to claim 6, wherein: Generating a first prediction block includes: generating a motion vector by adding a motion vector predictor and a motion vector difference; and A first prediction block is generated by using a motion vector and a reference picture index.

8. The method according to claim 1, further comprising: Generate a merge list of the current block; as well as Generate a GPM merge list from the merge list, Among them, each candidate in the GPM merge list includes unidirectional motion information.

9. The method according to claim 1, wherein: Derivation of the cell partition shape includes: performing an arithmetic operation of searching a boundary region within a region of the first prediction block to derive a bisection boundary for partitioning the first prediction block into a first block partition and a second block partition; and Select one of the predefined cell partition shapes based on the bisecting boundary.

10. The method according to claim 2, wherein: Based on the geometric prediction direction information, the motion information of the second block partition has a prediction direction different from the motion information for the first block partition.

11. A method for predicting a current block performed by a video encoding device, the method comprising: determining motion information for a first block partition of the current block; generating a first prediction block of the current block by using motion information of the first block partition; determining a merge candidate index for a second block partition of the current block; acquiring motion information of a second block partition from a geometric partition mode (GPM) merge list based on a merge candidate index, and then generating a second prediction block of the current block by using the motion information of the second block partition; deriving a geometric block partition shape for partitioning a current block into a first block partition and a second block partition by using the first prediction block; as well as The first prediction block and the second prediction block are mixed by using a geometric block partition shape to generate a final prediction block of the current block.

12. The method according to claim 11, wherein: The motion information of the first block partition includes geometric prediction direction information, motion vector prediction factor information, reference picture index, and motion vector difference.

13. The method according to claim 12, further comprising: encoding motion information for a first block partition; as well as Encodes the merge candidate index for the second chunk partition.

14. The method according to claim 13, wherein: Encoding the motion information for the first block partition includes: encoding a flag indicating the L0 or L1 reference picture list as geometric prediction direction information; and A flag indicating one of the candidates in an advanced motion vector prediction (AMVP) list of the current block is encoded as the motion vector predictor information.

15. The method according to claim 14, wherein: Encoding the motion information for the first block partition includes: Encoding, based on the geometric prediction direction information, an index indicating a reference picture included in the L0 or L1 reference picture list as a reference picture index; Encoded motion vector difference.

16. The method according to claim 14, wherein: When a predefined direction is used as the geometric prediction direction of the first block partition, determining and encoding the geometric prediction direction information is omitted.

17. A computer-readable recording medium storing a bit stream generated by a video encoding method, the video encoding method comprising: determining motion information for a first block partition of the current block; generating a first prediction block of the current block by using motion information of the first block partition; determining a merge candidate index for a second block partition of the current block; acquiring motion information of a second block partition from a geometric partition mode (GPM) merge list based on a merge candidate index, and then generating a second prediction block of the current block by using the motion information of the second block partition; deriving a geometric block partition shape for partitioning a current block into a first block partition and a second block partition by using the first prediction block; The first prediction block and the second prediction block are mixed by using a geometric block partition shape to generate a final prediction block of the current block.

Images