Geometric Partitioning Mode with Motion Vector Refinement

Abstract

A method and device are provided for decoding a video block with a GPM, the method including: partitioning the video block into a first and a second geometric partition, receiving a first template matching (TM) enable flag for the first geometric partition and a second TM enable flag for the second geometric partition, the first TM enable flag indicating whether unidirectional motion of the first partition is refined by TM and the second TM enable flag indicating whether unidirectional motion of the second partition is refined by TM, receiving a first merged GPM index for the first geometric partition and a second merged GPM index for the second geometric partition, building a unidirectional motion vector (MV) candidate list for the GPM, and generating a unidirectional MV for the first geometric partition and a unidirectional MV for the second geometric partition.

Description

Technical Field

[0001] The present disclosure relates to video coding and compression. More specifically, the present disclosure relates to methods and apparatuses for improving the coding efficiency of a geometric partition mode (GPM), also known as an angular weighted prediction (AWP) mode.

Background Art

[0002] To compress video data, various video coding techniques can be used. Video coding is performed according to one or more video coding standards. For example, recently, well-known video coding standards include Versatile Video Coding (VVC), High Efficiency Video Coding (HEVC, also known as H.265 or MPEG-H Part 2), Advanced Video Coding (AVC, also known as H.264 or MPEG-4 Part 10), which are jointly developed by ISO / IEC MPEG and ITU-T VECG. AOMedia Video 1 (AV1) was developed by the Alliance for Open Media (AOM) as a successor to the previous standard VP9. Audio Video Coding (AVS) refers to digital audio and digital video compression standards and is another series of video compression standards developed by the Audio and Video Coding Standard Workgroup in China. Most of the existing video coding standards are constructed based on well-known hybrid video coding frameworks, that is, block-based prediction methods (e.g., inter prediction, intra prediction) are used to reduce the redundancy present in video images or sequences, and transform coding is used to compress the energy of the prediction error. An important goal of video coding technology is to compress video data into a form that uses a lower bitrate while avoiding or minimizing the degradation of video quality.

Summary of the Invention

[0003] Examples of the present disclosure provide a method and apparatus for video coding, and a non-transitory computer-readable storage medium.

[0004] According to a first aspect of the present disclosure, a method for decoding a video block by GPM is provided. The method may include dividing the video block into a first Geometric division and a second geometric partition. The method includes receiving a first template matching (TM) enable flag for the first geometric partition and receiving a second TM enable flag for the second geometric partition, where the first TM enable flag indicates whether the uni-directional motion of the first Geometric partition is refined by TM, and the second TM enable flag indicates whether the uni-directional motion of the second Geometric partition is refined by TM. The method may include receiving a first merge GPM index for the first geometric partition and a second merge GPM index for the second geometric partition. The method may include constructing a uni-directional motion Vector (MV) candidate list for GPM. The method may include generating a uni-directional MV for the first geometric partition and a uni-directional MV for the second geometric partition.

[0005] According to a second aspect of the present disclosure, a method for decoding a video block by GPM is provided. The method may include dividing the video block into a first Geometric division and a second geometric partition. The method may include constructing a uni-directional MV candidate list for GPM. The method may include receiving a first merge GPM index for the first geometric partition and a second merge GPM index for the second geometric partition. The method may include updating the uni-directional MV candidate list based on the first merge GPM index and the second merge GPM index, where the first merge GPM index and the second merge GPM index indicate whether a single uni-directional MV is refined by template matching (TM).

[0006] According to a third aspect of the present disclosure, an apparatus for video decoding is provided. The apparatus may include one or more processors and a non-transitory computer-readable storage medium. The non-transitory computer-readable storage medium is configured to store instructions executable by the one or more processors. The one or more processors are configured to execute the method in the first or second aspect when the instructions are executed.

[0007] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium is provided. The non-transitory computer-readable storage medium may store computer-executable instructions that, when executed by one or more computer processors, cause the one or more computer processors to execute the method in the first or second aspect.

[0008] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate examples consistent with the present disclosure and, together with the specification, serve to explain the principles of the present disclosure.

Brief Description of the Drawings

[0009]

Figure 1

Figure 2

Figure 3A

Figure 3B

Figure 3C

Figure 3D

Figure 3E

Figure 4

Figure 5

Figure 6A

Figure 6B

Figure 7

Figure 8

Figure 9

Figure 10

[0010] Here, the embodiments are referred to in detail. Examples thereof are shown in the accompanying drawings. The following description refers to the accompanying drawings in which the same numbers in different drawings represent the same or similar elements, unless otherwise indicated. The embodiments described in the following description of implementation do not represent all implementations consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with aspects related to the present disclosure described in the appended claims.

[0011] The terms used in this disclosure are for the purpose of describing particular embodiments only and are not intended to limit the disclosure. As used in this disclosure and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly dictates otherwise. It should also be understood that the term "and / or" used herein means any and all possible combinations of one or more of the associated listed items and is intended to include them.

[0012] Although the terms "first", "second", "third", etc. can be used herein to describe various information, it should be understood that the information should not be limited by these terms. These terms are only used to distinguish one category of information from another. For example, without departing from the scope of this disclosure, the first information can be referred to as the second information, and similarly, the second information can be referred to as the first information. As used herein, the term "if" can be understood to mean "when" or "upon" or "in response to a judgment", depending on the context.

[0013] The first-generation AVS standards include the Chinese national standards "Advanced Audio Video Coding, Part 2: Video" (known as AVS1) and "Information Technology, Advanced Audio Video Coding Part 16: Radio Television Video" (known as AVS+). Compared with the MPEG-2 standard, it can provide about 50% bitrate savings at the same perceptual quality. The video part of the AVS1 standard was published as a Chinese national standard in February 2006. The second-generation AVS standards include a series of Chinese national standards "Information Technology, Efficient Multimedia Coding" (known as AVS2), which is mainly targeted at the transmission of ultra-high-definition TV programs. The coding efficiency of AVS2 is twice that of AVS+. In May 2016, AVS2 was published as a Chinese national standard. On the other hand, the video part of the AVS2 standard was submitted by the IEEE (Institute of Electrical and Electronics Engineers) as one of the international standards for applications. The AVS3 standard is a new generation of video coding standard for UHD video applications, aiming to exceed the coding efficiency of the latest international standard HEVC. In March 2019, at the 68th AVS meeting, the AVS3-P2 baseline was completed, saving about 30% bitrate compared with the HEVC standard. Currently, a reference software called High Performance Model (HPM) is maintained by the AVS group, demonstrating the reference implementation of the AVS3 standard.

[0014] Similar to HEVC, the AVS3 standard is built on a block-based hybrid video coding framework.

[0015] FIG. 1 shows a schematic diagram of a block-based video encoder for VVC. Specifically, FIG. 1 shows a typical encoder 100. The encoder 100 includes a video input 110, motion compensation 112, motion estimation 114, intra / inter mode decision 116, block predictor 140, summer 128, transform 130, quantization 132, prediction-related information 142, intra prediction 118, picture buffer 120, inverse quantization 134, inverse transform 136, adder 126, memory 124, in-loop filter 122, entropy coding 138, and bitstream 144.

[0016] In encoder 100, a video frame is divided into a plurality of video blocks for processing. For each given video block, a prediction is formed based on either an inter prediction approach or an intra prediction approach.

[0017] A prediction residual representing the difference between the current video block, which is part of the video input 110, and its predictor, which is part of the block predictor 140, is sent from adder 128 to transform 130. Next, the transform coefficients are sent from transform 130 to quantization 132 for entropy reduction. Then, the quantized coefficients are supplied to entropy coding 138 to generate a compressed video bitstream. As shown in FIG. 1, prediction-related information 142 from intra / inter mode decision 116, such as video block partition information, motion vector (MV), reference picture index, and intra prediction mode, is also supplied through entropy coding 138 and stored in the compressed bitstream 144. The compressed bitstream 144 includes the video bitstream.

[0018] In encoder 100, decoder-related circuitry is also required to reconstruct pixels for prediction purposes. First, the prediction residual is reconstructed by inverse quantization 134 and inverse transform 136. This reconstructed prediction residual is combined with the block predictor 140 to generate the non-filtered reconstructed pixels for the current video block.

[0019] Spatial prediction (or "intra prediction") predicts the current video block using samples from already-coded adjacent blocks (referred to as reference samples) within the same video frame as the current video block.

[0020] Temporal prediction (also referred to as "inter prediction") predicts the current video block using reconstructed pixels from already-encoded video pictures. Temporal prediction reduces the temporal redundancy inherent in the video signal. The temporal prediction signal for a given coding unit (CU) or coding block is typically signaled by one or more motion vectors (MVs) indicating the amount and direction of motion between the current CU and its temporal reference. Also, if multiple reference pictures are supported, an additional reference picture index is transmitted. This is used to identify from which reference picture in the reference picture store the temporal prediction signal comes.

[0021] Motion estimator 114 takes in signals from video input 110 and picture buffer 120 and outputs a motion estimation signal to motion compensator 112. Motion compensator 112 takes in the video input 110, the signal from picture buffer 120, and the motion estimation signal from motion estimator 114 and outputs a motion compensation signal to intra / inter mode decision 116.

[0022] After spatial prediction and / or temporal prediction is performed, the intra / inter mode decision 116 within the encoder 100 selects an optimal prediction mode, for example, based on a rate distortion optimization method. Next, the block predictor 140 is subtracted from the current video block, and the resulting prediction residual is decorrelated using the transform 130 and quantization 132. The resulting quantized residual coefficients are inverse quantized by the inverse quantization 134 and inverse transformed by the inverse transform 136 to form the reconstructed residual, and then the reconstructed residual is added back to the prediction block to form the reconstructed signal of the CU. Further, in-loop filtering 122, such as a deblocking filter, sample adaptive offset (SAO), and / or adaptive loop filter (ALF), is applied on the reconstructed CU before being put into the reference picture store of the picture buffer 120 and can be used to code future video blocks. To form the output video bitstream 144, all of the coding mode (inter or intra), prediction mode information, motion information, and quantized residual coefficients are sent to the entropy coding unit 138 to form the bitstream, which is further compressed and packed.

[0023] FIG. 1 shows a block diagram of a general block-based hybrid video coding system. The input video signal is processed block by block (referred to as a coding unit (CU)). Different from HEVC which divides blocks based only on quad-trees, in AVS3, one coding tree unit (CTU) is split into CUs to adapt to varying local characteristics based on a quad / binary / extended quad-tree. In addition, the concept of multiple split unit types in HEVC is removed, that is, the distinction between CU, prediction unit (PU), and transform unit (TU) does not exist in AVS3. Instead, each CU is always used as the basic unit for both prediction and transform without further splitting. In the tree splitting structure of AVS3, one CTU is first split based on a quad-tree structure. Next, the leaf nodes of each quad-tree can be further split based on binary and extended quad-tree structures.

[0024] As shown in FIGS. 3A, 3B, 3C, 3D, and 3E, there are five split types: four-way split, horizontal binary split, vertical binary split, horizontal extended quad-tree split, and vertical extended quad-tree split.

[0025] FIG. 3A is a diagram showing a four-way block split in a multi-type tree structure according to the present disclosure.

[0026] FIG. 3B is a diagram showing a vertical binary block split in a multi-type tree structure according to the present disclosure.

[0027] FIG. 3C is a diagram showing a horizontal binary block split in a multi-type tree structure according to the present disclosure.

[0028] FIG. 3D is a diagram showing a vertical ternary block split in a multi-type tree structure according to the present disclosure.

[0029] FIG. 3E is a diagram showing a horizontal ternary block split in a multi-type tree structure according to the present disclosure.

[0030] In FIG. 1, spatial prediction and / or temporal prediction can be performed. Spatial prediction (or "intra prediction") uses pixels from samples of already-coded adjacent blocks (referred to as reference samples) in the same video picture / slice to predict the current video block. Spatial prediction reduces the spatial redundancy inherent in the video signal. Temporal prediction (also referred to as "inter prediction" or "motion-compensated prediction") uses reconstructed pixels from already-coded video pictures to predict the current video block. Temporal prediction reduces the temporal redundancy inherent in the video signal. The temporal prediction signal for a given CU is typically signaled by one or more motion vectors (MVs) indicating the amount and direction of motion between the current CU and its temporal reference. Also, when multiple reference pictures are supported, an additional reference picture index is transmitted to identify from which reference picture in the reference picture store the temporal prediction signal is coming. After spatial and / or temporal prediction, the mode decision block in the encoder selects the optimal prediction mode, for example, based on a rate-distortion optimization method. Then, the prediction block is subtracted from the current video block, and the prediction residual is decorrelated using transformation and quantization. The quantized residual coefficients are inverse quantized and inverse transformed to form the reconstructed residual, and the reconstructed residual is added back to the prediction block to form the reconstructed signal of the CU. Further, in-loop filtering such as deblocking filter, sample adaptive offset (SAO), and adaptive loop filter (ALF) may be applied to the reconstructed CU before it is put into the reference picture store and used for coding future video blocks. To form the output video bitstream, the coding mode (inter or intra), prediction mode information, motion information, and quantized residual coefficients are all sent to the entropy coding unit to form the bitstream, which is further compressed and packed.

[0031] Figure 2 shows a schematic block diagram of a video decoder for VVC. Specifically, Figure 2 shows a block diagram of a typical decoder 200. The decoder 200 has a bitstream 210, entropy decoding 212, inverse quantization 214, inverse transform 216, adder 218, intra / inter mode selection 220, intra prediction 222, memory 230, in-loop filter 228, motion compensation 224, picture buffer 226, prediction-related information 234, and video output 232.

[0032] The decoder 200 is similar to the reconstruction-related section in the encoder 100 of Figure 1. In the decoder 200, the incoming video bitstream 210 is first decoded by the entropy decoding 212 to derive the quantized coefficient levels and prediction-related information. Then, the quantized coefficient levels are processed through inverse quantization 214 and inverse transform 216 to obtain the reconstructed prediction residuals. The block predictor mechanism implemented in the intra / inter mode selector 220 is configured to perform either intra prediction 222 or motion compensation 224 based on the decoded prediction information. The set of unfiltered reconstructed pixels is obtained by using the summer 218 to sum the reconstructed prediction residuals from the inverse transform 216 and the prediction output generated by the block predictor mechanism.

[0033] The reconstructed blocks may further pass through the in-loop filter 228 before being stored in the picture buffer 226 that functions as a reference picture store. The reconstructed video in the picture buffer 226 may be transmitted to drive a display device and may also be used to predict future video blocks. In the situation where the in-loop filter 228 is on, a filtering operation is performed on these reconstructed pixels to derive the final reconstructed video output 232.

[0034] FIG. 2 shows a schematic block diagram of a block-based video decoder. The video bitstream is first entropy decoded by an entropy decoding unit. The coding mode and prediction information are sent to either a spatial prediction unit (when intra-coded) or a temporal prediction unit (when inter-coded) to generate a prediction block. The residual transform coefficients are sent to an inverse quantization unit and an inverse transform unit to reconstruct the residual block. Then, the prediction block and the residual block are summed. The reconstructed block may further pass through in-loop filtering before being stored in the reference picture buffer. Then, the reconstructed video in the reference picture buffer is sent for display and also used to predict future video blocks.

[0035] The focus of the disclosure is to improve the coding performance of the Geometric Partitioning Mode (GPM) used in both the VVC and AVS3 standards. In AVS3, this tool is also known as Angular Weighted Prediction (AWP) which follows the same design spirit of GPM, but there are subtle differences in specific design details. To facilitate the description of the disclosure, hereinafter, the existing GPM design in the VVC standard is used as an example to explain the main aspects of the GPM / AWP tool. On the other hand, another existing inter-prediction technique called the Motion Vector Merging Mode for Difference (MMVD) which is applicable to both the VVC and AVS3 standards is also briefly considered in view of its close relation to the techniques proposed in the present disclosure. Then, some drawbacks of the current GPM / AWP design are identified. Finally, the proposed method is shown in detail. Although the existing GPM design of the VVC standard is used as an example throughout the disclosure, it should be noted by those skilled in the art of modern video coding technology that the proposed techniques can also be applied to other GPM / AWP designs or other coding tools having the same or similar design spirit.

[0036] Geometric Partitioning Mode (GPM)

[0037] In VVC, geometric partitioning modes are supported for inter prediction. The geometric partitioning mode is signaled by one CU-level flag as one special merge mode. In the current GPM design, a total of 64 partitions are supported in the GPM mode for each possible CU size where the width and height are not less than 8 and not greater than 64, except for 8×64 and 64×8.

[0038] When this mode is used, as shown in Figure 4 (the description will be given below), the CU is geometrically divided into two parts by a straight line. The position of the dividing line is mathematically derived from the angle and offset parameters of a specific partition. Each part of the geometric partition in the CU is inter-predicted by its own motion; only one-direction prediction is allowed for each partition, that is, each part has one motion vector and one reference index. The one-direction prediction motion constraint is applied to ensure that only two motion compensation predictions are required for each CU, similar to the conventional bi-directional prediction. When the geometric partitioning mode is used in the current CU, a geometric partitioning index indicating the partitioning mode of the geometric partition (angle and offset), and two merge indices (one for each partition) are further signaled. The number of maximum GPM candidate sizes is explicitly signaled at the sequence level.

[0039] Figure 4 shows the permitted GPM partitions, and the split direction is the same for each picture.

[0040] One-direction prediction candidate list construction

[0041] To derive the one-direction prediction motion vector for one geometric partition, one one-direction prediction candidate list is first directly derived by the normal merge candidate list generation process. In the geometric one-direction prediction candidate list, the index of the one-direction prediction motion is denoted by n. The LX motion vector of the n-th merge candidate (X is equal to the parity of n) is used as the n-th one-direction prediction motion vector of the geometric partitioning mode.

[0042] These motion vectors are marked with an "x" in Fig. 5 (described later). If the corresponding LX motion vector for the n-th extended merge candidate does not exist, the L(1-X) motion vector of the same candidate is used instead as the unidirectional prediction motion vector for the geometric partitioning mode.

[0043] Fig. 5 shows the selection of the unidirectional prediction motion vector from the motion vectors of the GPM merge candidate list.

[0044] Blending along the geometric partitioning edge

[0045] After each geometric partition has obtained its own motion, blending is applied to the two unidirectional prediction signals to derive the samples around the geometric partition edge. The blend weighting for each position of the CU is derived based on the distance from each individual sample position to the corresponding partition edge.

[0046] GPM signaling design

[0047] According to the current GPM design, the use of GPM is indicated by signaling one flag at the CU level. This flag is signaled only if the current CU is coded in merge mode or skip mode. Specifically, if the flag is equal to 1, it indicates that the current CU is predicted by GPM. Otherwise (the flag is equal to 0), the CU is coded in another merge mode, such as the normal merge mode, the merge mode with motion vector difference, the combination of inter prediction and intra prediction. If GPM is valid for the current CU, one syntax element, namely merge_gpm_partition_idx, is further signaled to indicate the applied geometric partitioning mode (specifying the direction and offset of the straight line from the CU center that divides the CU into two partitions, as shown in Fig. 4). Then, two syntax elements merge_gpm_idx0 to merge_gpm_idx1 are First GPM divisionIt is signaled to indicate the index of the uni - directional prediction merge candidates used for the first and second GPM partitions. Specifically, as described in the "Uni - directional Prediction Merge List Construction" section, these two syntax elements are used to determine the uni - directional MVs for the two GPM partitions from the uni - directional prediction merge list. According to the current GPM design, in order to make the two uni - directional MVs more different, the two indexes cannot be the same. Based on such prior knowledge, the uni - directional prediction merge index of the first GPM partition is signaled first and used as a predictor to reduce the signaling overhead of the uni - directional prediction merge index of the second GPM partition. Specifically, if the second uni - directional prediction merge index is smaller than the first uni - directional prediction merge index, the original value is directly signaled. Otherwise (if the second uni - directional prediction merge index is larger than the first uni - directional prediction merge index), the value is subtracted by 1 and then signaled to the bitstream. On the decoder side, the first uni - directional prediction merge index is initially the decoder. Next, for the decoding of the second uni - directional prediction merge index, if the parsed value is smaller than the first uni - directional prediction merge index, the second uni - directional prediction merge index is set to the same as the parsed value. Otherwise (if the parsed value is the same as or larger than the first uni - directional prediction merge index), the second uni - directional prediction merge index is set to the value obtained by adding 1 to the parsed value. Table 1 shows the existing syntax elements used in the GPM mode in the current VVC specification.

Table 1

[0048] On the one hand, in the current GPM design, truncated unary codes are used for the binary conversion of two one-way prediction merge indexes, namely merge_gpm_idx0 to merge_gpm_idx1. Furthermore, since the two one-way prediction merge indexes cannot be the same, different maximum values are used to truncate the codewords of the two one-way prediction merge indexes. These merge indexes are set to MaxGPMMergeCand-1 and MaxGPMMergeCand-2 for merge_gpm_idx0 and merge_gpm_idx1 respectively. MaxGPMMergeCand is the number of candidates in the one-way prediction merge list.

[0049] When the GPM / AWP mode is applied, two different binary conversion methods are applied and the merge_gpm_partition_idx syntax is converted into a string of binary bits. Specifically, the syntax elements are binary-converted by fixed-length codes and truncated binary codes in the VVC and AVS3 standards respectively. On the other hand, in the AWP mode of AVS3, different maximum values are used for the binary conversion of the values of the syntax elements. Specifically, in AVS3, the number of permitted GPM / AWP split modes is 56 (i.e., the maximum value of merge_gpm_partition_idx is 55), while in VVC it increases to 64 (i.e., the maximum value of merge_gpm_partition_idx is 63).

[0050] Differential motion vector merge mode (MMVD)

[0051] In addition to the conventional merge mode that derives the motion information of one current block from its spatial / temporal neighbor, the MMVD / UMVE mode is introduced in both the VVC and AVS standards as one special merge mode. Specifically, in both VVC and AVS3, the mode is signaled by one MMVD flag at the coding block level. In the MMVD mode, the first two candidates in the merge list of the normal merge mode are selected as the two basic merge candidates of MMVD. After one basic merge candidate is selected and signaled, additional syntax elements are signaled to indicate the motion vector difference (MVD) added to the motion of the selected merge candidate. The MMVD syntax elements include a merge candidate flag for selecting the basic merge candidate, a distance index for specifying the MVD magnitude, and a direction index for indicating the MVD direction.

[0052] In the existing MMVD design, the distance index specifies the magnitude of the MVD, and the MVD magnitude is defined based on a set of predefined offsets from the starting point. As shown in FIGS. 6A and 6B, the offset is added to either the horizontal or vertical component of the starting MV (i.e., the MV of the selected basic merge candidate).

[0053] FIG. 6A shows the MMVD mode with L0 reference. FIG. 6B shows the MMVD mode with L1 reference.

[0054] Table 2 shows the MVD offsets applied to AVS3, respectively.

Table 2

[0055] As shown in Table 3, the direction index is used to specify the sign of the signaling MVD. Note that the meaning of the MVD sign may vary depending on the starting MV. When the starting MV is a unidirectional prediction MV or a bidirectional prediction MV with MVs pointing to two reference pictures whose POCs are both greater than the POC of the current picture or less than the POC of the current picture, the signaled sign is the sign of the MVD added to the starting MV. When the starting MV is a bidirectional prediction MV pointing to two reference pictures where the POC of one picture is greater than the current picture and the POC of the other picture is less than the current picture, the signaled sign is applied to the L0 MVD, and the inverse value of the signaled sign is applied to the L1 MVD. [Table 3]

[0056] Motion Signaling in Regular Inter Mode

[0057] Similar to the HEVC standard, in addition to the merge / skip mode, in both VVC and AVS3, motion information within the bitstream can be explicitly specified for one CU. Overall, the motion information signaling in both VVC and AVS3 is kept the same as that in the HEVC standard. Specifically, one inter-prediction syntax, i.e., inter_pred_idc, is first signaled to indicate the prediction syntax from list L0, L1, or both. For each reference list used, the corresponding reference picture is identified by signaling one reference picture index ref_idx_lx (X = 0, 1) of the corresponding reference list, the corresponding MV is represented by one MVP index mvp_lx_flag (X = 0, 1) used to select the motion vector predictor (MVP), and the motion vector difference (MVD) between the target MV and the selected MVP follows. Further, in the VVC standard, one control flag mvd_l1_zero_flag is signaled at the slice level. When mvd_l1_zero_flag is equal to 0, the L1 MVD is signaled in the bitstream. Otherwise (when mvd_l1_zero_flag is equal to 1), the L1 MVD is not signaled and its value is always assumed to be 0 at the encoder and decoder.

[0058] Bi-directional prediction by CU-level weighting

[0059] In the previous standards of VVC and AVS3, when weighted prediction (WP) is not applied, the bi-directional prediction signal is generated by averaging the uni-directional prediction signals obtained from two reference pictures. In VVC, one tool coding, i.e., bi-directional prediction by CU-level weighting (BCW), is introduced to improve the efficiency of bi-directional prediction. Specifically, the bi-directional prediction of BCW is extended by enabling weighted averaging of two prediction signals as follows instead of simple averaging.

Number

[0060] In VVC, when the current picture is a single low-delay picture, the weight of one BCW coding block can be selected from a set of predefined weight values w ∈ {-2, 3, 4, 5, 10}, and a weight of 4 represents the case of conventional bidirectional prediction where two single-direction prediction signals are equally weighted. For low latency, only three weights w ∈ {3, 4, 5} are allowed. Generally, there are some design similarities between WP and BCW, but the two coding tools aim to solve the illumination change problem at different granularities. However, since the interaction between WP and BCW can complicate the VVC design, the two tools cannot be enabled simultaneously. Specifically, when WP is effective for a single slice, the BCW weights of all bidirectional prediction CUs within the slice are not signaled and are assumed to be 4 (i.e., equal weights are applied).

[0061] Template matching

[0062] Template matching (TM) is a decoder-side MV derivation method for refining the motion information of the current CU by finding the optimal matching between a template consisting of reconstructed samples adjacent to the current CU above and to the left, and a reference block (i.e., the same size as the template) within the reference picture. As shown in Figure 7, one MV is searched around the initial motion vector of the current CU within a search range of [-8, +8] pels. The optimal matching may be defined as the MV that achieves the minimum matching cost, such as the sum of absolute differences (SAD), the sum of absolute transformed differences (SATD), etc., between the current template and the reference template. There are two methods for applying the TM mode to inter-coding.

[0063] In the AMVP mode, the MVP candidate is determined based on the difference of template matching to pick up the one with the minimum difference between the current block template and the reference block template, and the TM is executed only for this specific MVP candidate for MV refinement. The TM uses an iterative diamond search to narrow down this MVP candidate from the full-pel MVD accuracy within the search range of [-8, +8] pels (or 4 pels in the case of 4-pel AMVR mode). The AMVP candidate is further narrowed down using a cross-search with full-pel MVD accuracy (or 4 pels in the case of 4-pel AMVR mode), and then may be narrowed down using 1 / 2-pel and 1 / 4-pel cross-searches according to the AMVR mode specified in Table 13 below. This search process guarantees that the MVP candidate maintains the same MV accuracy as that indicated in the AMVR mode after the TM process.

Table 3-1

[0064] In the merge mode, a similar search method is applied to the merge candidate indicated by the merge index. As shown in the above table, the TM may be fully executed up to 1 / 8-pel MVD accuracy or skip those exceeding 1 / 2-pel MVD accuracy depending on whether to use an alternative interpolation filter (used when AMVR is in 1 / 2-pel mode) according to the merged motion information.

[0065] As described above, the uni-directional motion used to generate the prediction samples for the two GPM partitions is directly obtained from the normal merge candidates. If there is no strong correlation between the MVs of the spatial / temporal adjacent blocks, the uni-directional MVs derived from the merge candidates may not be accurate enough to capture the true motion of each GPM partition. Motion estimation can provide more accurate motion, but at the cost of an inescapable signaling overhead for any motion refinement that can be applied to the existing uni-directional MVs. On the other hand, the MVMD mode is used in both the VVC and AVS3 standards, which has been proven to be an efficient signaling mechanism for reducing the MVD signaling overhead. Therefore, it is also beneficial to combine GPM with the MMVD mode. Such a combination can potentially improve the overall coding efficiency of the GPM tool by providing more accurate MVs to capture the individual motions of each GPM partition.

[0066] As described above, in both the VVC and AVS3 standards, the GPM mode is only applied to the merge / skip mode. Considering that all non-merge inter CUs cannot benefit from the flexible non-rectangular partitioning of GPM, such a design may not be optimal in terms of coding efficiency. On the other hand, for the same reason as above, the uni-directional prediction motion candidates derived from the normal merge / skip mode are not necessarily accurate enough to capture the true motion of the two geometric partitions. Based on such analysis, by reasonably extending the GPM mode to the non-merge inter mode, additional coding gain can be expected (i.e., the CUs that explicitly signal their motion information in the bitstream). However, the improvement in MV accuracy comes at the cost of an increase in signaling overhead. Therefore, in order to efficiently apply the GPM mode to the explicit inter mode, it is important to identify an effective signaling scheme that can minimize the signaling cost while providing more accurate MVs for the two geometric partitions.

[0067] Proposed method

[0068] In the present disclosure, a method is proposed to further improve the coding efficiency of GPM by applying motion refinement to the existing unidirectional MVs applied to each GPM partition. The proposed method is named as Geometric Partitioning Mode with Motion Vector Refinement (GPM-MVR). Further, in the proposed scheme, the motion refinement is signaled based on a set of predefined MVD magnitudes and directions of motion refinement, which is a similar method to one of the existing MMVD designs.

[0069] In another aspect of the disclosure, a solution for extending the GPM mode to an explicit inter-mode is provided. For ease of description, these schemes are named as Geometric Partitioning Mode with Explicit Motion Signaling (GPM-EMS). Specifically, in the proposed GPM-EMS scheme, to achieve better harmony with the normal inter-mode, the existing motion signaling mechanism, i.e., MVP+MVD, is utilized to specify the corresponding unidirectional MVs of two geometric partitions.

[0070] Geometric Partitioning Mode with Separated Motion Vector Refinement

[0071] To improve the coding efficiency of GPM, in this section, an improved geometric partitioning mode with separate motion vector refinement is proposed. Specifically, given a GPM partition, the proposed method first uses the existing syntax merge_gpm_idx0tomerge_gpm_idx1 to identify the unidirectional MVs of two GPM partitions from the existing unidirectional prediction merge candidate list and uses them as the base MVs. After the two base MVs are determined, a new set of two syntax elements is introduced to specify the values of motion refinement applied separately to the base MVs of the two GPM partitions. Specifically, two flags, i.e., gpm_MVR_partIdx0_enable_flagtogpm_MVR_partIdx1_enable_flag, initially indicate whether GPM-MVR is applied to the first GPM divisionand a signal indicating whether it is applied to the second GPM split. When the flag of one GPM split is equal to 1, the corresponding value of the MVR applied to the base MV of the split is signaled in MMVD style. That is, one distance index (indicated by the syntax elements gpm_mvr_partIdx0_distance_idx and gpm_mvr_partIdx1_distance_idx) specifies the MVR magnitude, and one direction index (indicated by the syntax elements gpm_mvr_partIdx0_direction_idx and gpm_mvr_partIdx1_distance_idx) specifies the direction of the MVR. Table 4 shows the syntax elements introduced by the proposed GPM-MVR method.

Table 4

[0072] Based on the proposed syntax elements as shown in Table 4, in the decoder, the final MV used to generate the unidirectional prediction samples for each GPM split is equal to the sum of the signaled motion vector refinement and the corresponding base MV. In practice, different sets of MVR magnitudes and directions can be predefined and applied to the proposed GPM-MVR scheme, which can provide various trade-offs between motion vector accuracy and signaling overhead. In one specific example, it is proposed to reuse the eight MVD offsets (i.e., 1 / 4 pel, 1 / 2 pel, 1 pel, 2 pel, 4 pel, 8 pel, 16 pel, and 32 pel) and four MVD directions (i.e., + / -x-axis and y-axis) used in the VVC standard for the proposed GPM-MVR scheme. In another example, the existing five MVD offsets {1 / 4 pel, 1 / 2 pel, 1 pel, 2 pel, and 4 pel} and four MVD directions (i.e., + / -x-axis and y-axis) used in the AVS3 standard are applied to the proposed GPM-MVR scheme.

[0073] As described in the "GPM Signaling Design" section, since the unidirectional MVs used for the two GPM partitions cannot be identical, in the existing GPM design, one constraint that forces the two unidirectional prediction merge indexes to be different is applied. However, in the proposed GPM-MVR scheme, further motion refinement is applied to the existing GPM unidirectional MVs. Therefore, even if the base MVs of the two GPM partitions are the same, the final unidirectional MVs used to predict the two partitions may still be different unless the values of the two motion vector refinements are the same. Based on the above considerations, when the proposed GPM-MVR scheme is applied, the constraint (restricting the two unidirectional prediction merge indexes to be different) is removed. Furthermore, since it is allowed for the two unidirectional prediction merge indexes to be identical, the same maximum value MaxGPMMergeCand-1 is used for the binarization of both merg_gpm_idx0 and merge_gpm_idx1. MaxGPMMergeCand is the number of candidates in the unidirectional prediction merge list.

[0074] As analyzed above, when the unidirectional prediction merge indexes of the two GPM partitions (i.e., merge_gpm_idx0 and merge_gpm_idx1) are identical, in order to ensure that the final MVs used for the two partitions are different, the values of the two motion vector refinements cannot be made the same. Based on such conditions, in one embodiment of the disclosure, when the unidirectional prediction merge indexes of the two GPM partitions are the same (i.e., merge_gpm_idx0 is equal to merge_gpm_idx1), one signaling redundancy removal method is proposed to reduce the signaling overhead of the MVR of the second GPM partition by using the MVR of the first GPM partition. In one example, the following signaling conditions are applied:

[0075] First, when the flag gpm_MVR_partIdx0_enable_flag is equal to 0 (i.e., GPM-MVR is not applied to the first GPM division), the flag of gpm_MVR_partIdx1_enable_flag is not signaled and is presumed to be 1 (i.e., GPM-MVR is applied to the second GPM division).

[0076] Next, when both flags gpm_MVR_partIdx0_enable_flag and gpm_MVR_partIdx1_enable_flag are equal to 1 (i.e., GPM-MVR is applied to two GPM partitions) and gpm_mvr_partIdx0_direction_idx is equal to gpm_mvr_partIdx1_direction_idx (i.e., the MVRs of the two GPM partitions have the same direction), the MVR magnitude of the first GPM partition (i.e., gpm_mvr_partIdx0_distance_idx) is used to predict the magnitude of the MVR of the second GPM partition (i.e., gpm_mvr_partIdx1_distance_idx). Specifically, if gpm_mvr_partIdx1_distance_idx is smaller than gpm_mvr_partIdx0_distance_idx, the original value is directly signaled. Otherwise (if gpm_mvr_partIdx1_distance_idx is larger than gpm_mvr_partIdx0_distance_idx), the value is decremented by 1 before being signaled in the bitstream. On the decoder side, to decode the value of gpm_mvr_partIdx1_distance_idx, if the parsed value is smaller than gpm_mvr_partIdx0_distance_idx, gpm_mvr_partIdx1_distance_idx is set to the parsed value. Otherwise (if the parsed value is equal to or larger than gpm_mvr_partIdx0_distance_idx), gpm_mvr_partIdx1_distance_idx is set to the value obtained by adding 1 to the parsed value. In such a case, to further reduce overhead, different maximum values MaxGPMMVRDistance - 1 and MaxGPMMVRDistance - 2 can be used for the binarization of gpm_mvr_partIdx0_distance_idx and gpm_mvr_partIdx1_distance_idx. MaxGPMMVRDistance is the number of magnitudes allowed for motion vector refinement.

[0077] In another embodiment, it is proposed to switch the signaling order to gpm_mvr_partIdx0_direction_idx / gpm_mvr_partIdx1_direction_idx and gpm_mvr_partIdx0_distance_idx / gpm_mvr_partIdx1_distance_idx so that the MVR magnitude is signaled before the MVR magnitude. In this way, similar to the logic described above, the encoder / decoder can condition the signaling of the MVR direction of the second GPM split using the MVR direction of the first GPM split. In another embodiment, it is proposed to first signal the MVR magnitude and direction of the second GPM split and use them to condition the signaling of the MVR magnitude and direction of the second GPM split.

[0078] In another embodiment, it is proposed to signal GPM-MVR related syntax elements before signaling existing GPM syntax elements. Specifically, in such a design, two flags, gpm_MVR_partIdx0_enable_flag and gpm_MVR_partIdx1_enable_flag, are initially signals indicating whether GPM-MVR is applied to the first GPM division and the second GPM split, respectively. If the flag of one GPM split is equal to 1, the distance index (indicated by the syntax elements gpm_mvr_partIdx0_distance_idx and gpm_mvr_partIdx1_distance_idx) and the direction index (indicated by the syntax elements gpm_mvr_partIdx0_direction_idx and gpm_mvr_partIdx1_distance_idx) are signaled to specify the direction of MVR. Then, the existing syntax merge_gpm_idx0 and merge_gpm_idx1 are signaled to identify the one-way MV of the two GPM splits, i.e., the base MV. Table 5 shows the proposed GPM-MVR signaling scheme.

Table 5

[0079] Similar to the signaling method in Table 4, when applying the GPM-MVR signaling method in Table 5 to ensure that the resulting MVs used for predicting two GPM splits are not the same, specific conditions may be applied. Specifically, the signaling of the unidirectional prediction merge indices merge_gpm_idx0 and merge_gpm_idx1 is GPM division constrained according to the values of the MVRs applied to the first

[0080] First, when the values of both gpm_MVR_partIdx0_enable_flag and gpm_MVR_partIdx1_enable_flag are equal to 0 (i.e., GPM-MVR is invalid for both GPM splits), the values of merge_gpm_idx0 and merge_gpm_idx1 will not be the same;

[0081] Second, when gpm_MVR_partIdx0_enable_flag is equal to 1 (i.e., GPM-MVR is valid for the first GPM split) and gpm_MVR_partIdx1_enable_flag is equal to 0 (i.e., GPM-MVR is invalid for the second GPM split), the values of merge_gpm_idx0 and merge_gpm_idx1 are allowed to be the same;

[0082] Third, when gpm_MVR_partIdx0_enable_flag is equal to 0 (i.e., GPM-MVR is invalid for the first GPM split) and gpm_MVR_partIdx1_enable_flag is equal to 1 (i.e., GPM-MVR is valid for the second GPM split), the values of merge_gpm_idx0 and merge_gpm_idx1 are allowed to be the same;

[0083] Fourth, when the values of both gpm_MVR_partIdx0_enable_flag and gpm_MVR_partIdx1_enable_flag are equal to 1 (i.e., GPM-MVR is valid for both of the two GPM partitions), the determination of whether the values of merge_gpm_idx0 and merge_gpm_idx1 are allowed to be the same depends on the values of MVR applied to the two GPM partitions (indicated by gpm_mvr_partIdx0_direction_idx and gpm_mvr_partIdx0_distance_idx, and gpm_mvr_partIdx1_direction_idx and gpm_mvr_partIdx1_distance_idx). When the values of the two MVRs are equal, merge_gpm_idx0 and merge_gpm_idx1 are not the same. Otherwise (when the values of the two MVRs are not equal), the values of merge_gpm_idx0 and merge_gpm_idx1 are allowed to be the same.

[0084] In the above four cases, when the values of merge_gpm_idx0 and merge_gpm_idx1 are not allowed to be the same, the index value of one split can be used as a predictor for the index value of the other split. In one method, it is proposed to first signal merge_gpm_idx0 and use its value to predict merge_gpm_idx1. Specifically, if merge_gpm_idx1 is greater than merge_gpm_idx0 in the encoder, the value of merge_gpm_idx1 sent to the decoder decreases by 1 each time. In the decoder, if the received value of merge_gpm_idx1 is equal to or greater than the received value of merge_gpm_idx0, the value of merge_gpm_idx1 increases by 1. In another method, it is proposed to first signal merge_gpm_idx1 and use its value to predict merge_gpm_idx0. Therefore, if merge_gpm_idx0 is greater than merge_gpm_idx1 in the encoder, the value of merge_gpm_idx0 sent to the decoder decreases by 1. In the decoder, if the received value of merge_gpm_idx0 is equal to or greater than the received value of merge_gpm_idx1, the value of merge_gpm_idx0 increases by 1. In addition, similar to the existing GPM signaling design, different maximum values of MaxGPMMergeCand-1 and MaxGPMMergeCand-2 can be used for the binarization of the first and second index values respectively according to the signaling order. On the other hand, since there is no correlation between the two index values, when the values of merge_gpm_idx0 and merge_gpm_idx1 are allowed to be the same, the same maximum value MaxGPMMergeCand-1 is used for the binary values of both index values.

[0085] In the above method, different maximum values can be applied to the binarization of merge_gpm_idx0 and merge_gpm_idx1 in order to reduce the signaling cost. The selection of the corresponding maximum value depends on the decoded values of MVR (as indicated by gpm_mvr_partIdx0_enable, gpm_mvr_partIdx1_enable, gpm_mvr_partIdx0_direction_idx, gpm_mvr_partIdx1_direction_idx, gpm_mvr_partIdx0_distance_idx and gpm_mvr_partIdx1_distance_idx_). Such a design introduces undesirable syntax analysis dependencies between different GPM syntax elements, which may affect the entire analysis. To solve such problems, in one embodiment, one same maximum value (e.g., MaxGPMMergeCand-1) is always proposed for analyzing the values of merge_gpm_idx0 and merge_gpm_idx1. When using such a method, one bitstream compatibility constraint may be used so that the two decoded MVs of the two GPM splits do not become the same. In another method, such a non-identity constraint can also be removed so that the decoded MVs of the two GPM splits are allowed to be the same. On the other hand, when such a method is applied (i.e., when the same maximum value is used for merge_gpm_idx0 and merge_gpm_idx1), there is no analysis dependency between merge_gpm_idx0 / merge_gpm_idx1 and other GPM-MVR syntax elements. Therefore, the order of signaling these syntax elements is no longer important. As an example, it is proposed to move the signaling of merge_gpm_idx0 / merge_gpm_idx1 before the signaling of gpm_mvr_partIdx0_enable, gpm_mvr_partIdx1_enable, gpm_mvr_partIdx0_direction_idx, gpm_mvr_partIdx1_direction_idx, gpm_mvr_partIdx0_distance_idx and gpm_mvr_partIdx1_distance_idx.

[0086] Geometric Partitioning Mode with Symmetric Motion Vector Refinement

[0087] In the above GPM-MVR method, two separate MVR values are applied and signaled, and one of the MVR values is applied to improve the base MV for only one GPM partition. Such a method is efficient in terms of improving prediction accuracy by enabling independent motion refinement for each GPM partition. However, considering that such flexible motion refinement requires two different sets of GMP-MVR syntax elements to be sent from the encoder to the decoder, there is a cost of increased signaling overhead. In this section, to reduce the signaling overhead, one geometric partitioning mode with symmetric motion vector refinement is proposed. Specifically, in this method, one single MVR value is signaled for one GPM CU and is used for both GPM partitions according to the symmetric relationship between the picture order count (POC) value of the current picture and the reference pictures associated with the two GPM partitions. Table 6 shows the syntax elements when the proposed method is applied.

Table 6

[0088] As shown in Table 6, after two GPM split base MVs are selected (based on merge_gpm_idx0 and merge_gpm_idx1), one flag gpm_mvr_enable_flag is signaled to indicate whether the GPM-MVR mode is applied to the current GPM CU. When the flag is equal to 1, it indicates that motion refinement is applied to enhance the two GPM split base MVs. Otherwise (when the flag is equal to 0), it indicates that motion refinement is not applied to either of the two splits. When the GPM-MVR mode is enabled, additional syntax elements are further signaled by the direction index gpm_mvr_direction_idx and the magnitude index gpm_mvr_distance_idx to specify the value of the applied MVR. In addition, similar to the MMVD mode, the meaning of the MVR sign can vary depending on the relationship between the POC of the current picture and the two reference pictures of the GPM split. Specifically, when the POCs of both of the two reference pictures are larger or smaller than the POC of the current picture, the signaled sign is the sign of the MVR added to both of the two base MVs. Otherwise (when the POC of one reference picture is larger than the current picture and the POC of the other reference picture is smaller than the current picture), the signaled sign is applied to the MVR of the first GPM split, and the opposite sign is applied to the second GPM split. In Table 6, it is allowed that the values of merge_gpm_idx0 and merge_gpm_idx1 are the same.

[0089] In another embodiment, it is proposed to signal to two different flags to separately control the enabling / disabling of the GPM-MVR mode for two GPM splits. However, when the GPM-MVR mode is enabled, only one MVR is signaled based on the syntax elements gpm_mvr_direction_idx and gpm_mvr_distance_idx. The corresponding syntax table for such a signaling method is shown in Table 7.

Table 7

[0090] When the signaling method of Table 7 is applied, the values of merge_gpm_idx0 and merge_gpm_idx1 are allowed to be the same. However, to confirm that the resulting MVs applied to the two GPM partitions are not redundant, when the flag gpm_MVR_partIdx0_enable_flag is equal to 0 (i.e., GPM-MVR is not applied to the first GPM partition), the flag gpm_MVR_partIdx1_enable_flag is not signaled and is presumed to be 1 (i.e., GPM-MVR is applied to the second GPM partition).

[0091] Allowed MVR Adaptation for GPM-MVR

[0092] In the above GPM-MVR method, one fixed group of MVR values is used for GPM CUs in both the encoder and decoder within one video sequence. Such a design is not optimal for video content with high resolution or intense motion. In these cases, since the MVs tend to be very large, the fixed MVR values may not be optimal for capturing the actual motion of those blocks. To further improve the coding performance of the GPM-MVR mode, the present disclosure proposes to support the adaptation of MVR values that are allowed to be selected by the GPM-MVR mode at various coding levels such as sequence level, picture / slice picture, coding block group level, etc. For example, multiple MVR sets and corresponding codewords can be derived offline according to the specific motion characteristics of various video sequences. The encoder may select the optimal MVR set and signal the corresponding index of the selected set to the decoder.

[0093] Encoder Speedup Logic for GPM-MVR Rate-Distortion Optimization

[0094] For the proposed GPM-MVR scheme, in order to determine the optimal MVR for each GPM split, the encoder may need to change the applied MVR value and test the rate-distortion cost of each GPM split multiple times. This can significantly increase the complexity of GPM mode encoding. To address the issue of encoding complexity, the following fast encoding logic is proposed in this section.

[0095] First, due to the quad / binary / ternary tree block splitting structures applied in VVC and AVS3, one same coding block can be checked during rate-distortion optimization (RDO) processing when it is split through one different split path. In the implementation of the current VTM / HPM encoder, the GPM and GPM-MVR modes are always tested together with other inter and intra coding modes every time one same CU is obtained by different combinations of block splitting. Generally, for different split paths, only the adjacent blocks of one CU may be different. However, the impact on the optimal coding mode selected by one CU should be relatively small. Based on such considerations, to reduce the total number of applied GPM RDOs, it is proposed to save the decision on whether the GPM mode is selected when the RD cost of one CU is first checked. Then, when the same CU is checked again in the RDO process (by a different split path), the RD cost of GPM (including GPM-MVR) is only checked if GPM was selected for the CU initially. If GPM is not selected for the initial RD check of one CU, when the same CU is achieved through a different split path, only GPM (without GPM-MVR) is tested. In another way, if GPM is not selected for the first RD check of one CU, when the same CU is achieved through a different split path, neither GPM nor GPM-MVR is tested.

[0096] Second, to reduce the number of GPM partitions for the GPM-MVR mode, when initially checking the RD cost of one CU, it is proposed to maintain the first M GPM partition modes without the minimum RD cost. Then, when the same CU is checked again by RDO processing (by a different partition path), only these M GPM partition modes are tested in the GPM-MVR mode.

[0097] Third, to reduce the number of GPM partitions tested for the initial RDO process of 1, when using different unidirectional prediction merge candidates for two GPM partitions, it is proposed to first calculate the sum of the absolute difference (Sum Absolute Difference: SAD) values. Next, for each GPM partition of one specific partition mode, select the optimal unidirectional prediction merge candidate with the minimum SAD value, and calculate the corresponding SAD value of the partition mode equal to the sum of the SAD values of the optimal unidirectional prediction merge candidates of the two GPM partitions. Next, in the next RD process, only the first N partition modes with the optimal SAD value from the previous step are tested in the GPM-MVR mode.

[0098] Geometric partitioning using explicit motion signaling

[0099] In this section, several methods are proposed to extend the GPM mode to the normal inter-mode bidirectional prediction where the two unidirectional MVs of the GPM mode are explicitly signaled from the encoder to the decoder.

[0100] In the first solution (Solution 1), it is proposed to fully reuse the existing motion signals for bi - directional prediction in order to signal two uni - directional MVs in GPM mode. Table 8 shows the modified syntax table of the proposed scheme where the newly added syntax elements are in italic bold. As shown in Table 8, in this solution, all existing syntax elements of the signal L0 and L1 motion information are fully reused respectively to indicate the two GPM - split uni - directional MVs. Furthermore, it is assumed that the L0 MV is always associated with the first GPM split and the L1 MV is always associated with the second GPM split. On the other hand, in Table 8, the inter - prediction syntax, i.e., inter_pred_idc, is signaled before the GPM flag (i.e., gpm_flag), and the value of inter_pred_idc can be used to condition the presence of the gpm_flag. Specifically, the flag gpm_flag needs to be signaled only when inter_pred_idc is equal to PRED_BI (i.e., bi - directional prediction) and both inter_affine_flag and sym_mvd_flag are equal to 0 (i.e., the CU is not coded by either affine mode or SMVD mode). If the flag gpm_flag is not signaled, its value is always assumed to be 0 (i.e., the GPM mode is invalid). When gpm_flag is 1, another syntax element gpm_partition_idx is further signaled to indicate the selected GPM mode of the current CU (out of a total of 64 GPM splits).

Table 8

[0101] In another way, it is proposed to place the signaling of the flag gpm_flag before other inter-signaling syntax elements so that the value of gpm_flag can be used to determine whether other inter-syntax elements need to exist. Table 9 shows the corresponding syntax table when such a method is applied when the newly added syntax elements are in italic bold. As shown, gpm_flag is signaled first in Table 9. When gpm_flag is 1, the signaling of the corresponding inter_pred_idc, inter_affine_flag, and sym_mvd_flag can be bypassed. Instead, the corresponding values of the three syntax elements can be inferred as PRED_BI, 0, and 0 respectively. [Table 9]

[0102] In both Table 8 and Table 9, the SMVD mode cannot be combined with the GPM mode. In another example, it is proposed to allow the SMVD mode when the current CU is coded by the GPM mode. If such a combination is allowed, assuming that the MVDs of the two GPM partitions are symmetric by following the same design of SMVD, only the MVD of the first GPM partition needs to be signaled, and the MVD of the second GPM partition is always symmetric to the first MVD. By applying such a method, the corresponding signaling condition of sym_mvd_flag of gpm_flag can be removed.

[0103] As described above, in the first solution, it is always assumed that L0 MV is used for the first GPM partition and L1 MV is used for the second GPM partition. Such a design may not be optimal in the sense that it prohibits this method from obtaining the MVs of the two GPM partitions from one and the same prediction list (L0 or L1). To solve such a problem, Solution 2, which is another GPM-EMS scheme, is proposed with a signal design as shown in Table 10. In Table 10, the newly added syntax elements are in italic bold. As shown in Table 10, the flag gpm_flag is signaled first. If the flag is equal to 1 (GPM is valid), the syntax gpm_partition_idx is signaled to specify the selected GPM mode. Next, one additional flag gpm_pred_dir_flag0 is signaled to indicate the corresponding prediction list from which the MV of the first GPM partition is sourced. If the flag gpm_pred_dir_flag0 is equal to 1, it indicates that the MV of the first GPM partition comes from L1. Otherwise (if the flag is equal to 0), it indicates that the MV of the first GPM partition comes from L0. Thereafter, the existing syntax elements ref_idx_l0, mvp_l0_flag, and mvd_coding() are used to signal the reference picture index, mvp index, and the value of the MVD of the first GPM partition. On the other hand, similar to the first Geometric partition, another syntax element gpm_pred_dir_flag1 is introduced to select the corresponding prediction list of the second GPM partition, followed by the existing syntax elements ref_idx_l1, mvp_l1_flag, and mvd_coding() that are used to derive the MV of the second GPM partition.

Table 10

[0104] Also, finally, considering that the GPM mode consists of two unidirectional prediction partitions (excluding the blending samples on the partition edge), existing coding tools of VVC and AVS3, especially designed for bidirectional prediction such as bidirectional optical flow, decoder-side motion vector refinement (DMVR), and CU-weighted bidirectional prediction (BCW), may be automatically bypassed when the proposed GPM-EMS scheme is effective for one inter-CU. For example, when one of the proposed GPM-EMSs is effective for one CU, considering that BCW cannot be applied to the GPM mode, there is no need to further signal the corresponding BCW weight for the CU to reduce the signaling overhead.

[0105] Combination of GPM-MVR and GPM-EMS

[0106] In this section, it is proposed to combine GPM-MVR and GPM-EMS for one CU with geometric partitioning. Specifically, in the proposed scheme, unlike GPM-MVR or GPM-EMS where only one of merge-based motion signaling or explicit signaling can be applied to the signals of the unidirectional prediction MVs of the two GPM partitions, 1) one partition using GPM-MVR-based motion signaling and another partition using GPM-EMS-based motion signaling, or 2) two partitions using GPM-MVR-based motion signaling, or 3) two partitions using GPM-EMS-based motion signaling are allowed. Using the GPM-MVR signaling in Table 4 and GPM-EMS in Table 10, Table 11 shows the corresponding syntax table after combining the proposed GPM-MVR and GPM-EMS. In Table 11, the newly added syntax elements are in italic bold. As shown in Table 11, two additional syntax elements gpm_merge_flag0 and gpm_merge_flag1 are introduced for partitions #1 and #2 respectively, which specify that the corresponding partition uses GPM-MVR-based merge signaling or GPM-EMS-based explicit signaling. When the flag is 1, it means that GPM-MVR-based signaling is valid for the partition where the GPM unidirectional prediction motion is signaled by merge_gpm_idxX, gpm_mvr_partIdxX_enabled_flag, gpm_mvr_partIdxX_direction_idx, and gpm_mvr_partIdxX_distance_idx (X = 0, 1). Otherwise, when the flag is 0, it means that the unidirectional prediction motion of the partition is explicitly signaled by the GPM-EMS method using the syntax elements gpm_pred_dir_flagX, ref_idx_lX, mvp_lX_flag, and mvd_lX (X = 0, 1).

Table 11

[0107] Combination of GPM-MVR and Template Matching

[0108] In this section, various solutions for combining GPM-MVR and template matching are provided.

[0109] In Method 1, when one CU is coded in GPM mode, it is proposed to signal two separate flags for two GPM partitions, each flag indicating whether the unidirectional motion of the corresponding partition is further refined by template matching. When the flag is enabled, a template is generated using the reconstructed samples adjacent to the left and upper sides of the current CU. Then, following the same procedure introduced in the "Template Matching" section, the unidirectional motion of the partition is narrowed down by minimizing the difference between the template and its reference sample. Otherwise (when the flag is disabled), template matching is not applied to the partition, and GPM-MVR may be further applied. Taking the GPM-MVR signaling method in Table 5 as an example, Table 11 shows the corresponding syntax table when GPM-MVR and template matching are combined. [Table 11-1]

[0110] As shown in Table 11, in the proposed scheme, two additional flags, gpm_tm_enable_flag0 and gpm_tm_enable_flag1, are first signaled to indicate whether the movement is refined for each of the two GPM partitions. When the flag is 1, it indicates that TM is applied to refine the one-way MV of one partition. When the flag is 0, one more flag (gpm_MVR_partIdx0_enable_flag or gpm_MVR_partIdx1_enable_flag) indicating whether GPM-MVR is applied to the GPM partition is further signaled. When the flag of one GPM partition is equal to 1, the distance index (indicated by the syntax elements gpm_mvr_partIdx0_distance_idx and gpm_mvr_partIdx1_distance_idx) and the direction index (indicated by the syntax elements gpm_mvr_partIdx0_direction_idx and gpm_mvr_partIdx1_distance_idx) are signaled to specify the direction of MVR. Then, the existing syntax elements merge_gpm_idx0 and merge_gpm_idx1 are signaled to identify the one-way MVs of the two GPM partitions. On the other hand, similar to the signaling conditions applied to Table 5, the following conditions can be applied to ensure that the resulting MVs used for prediction of the two GPM partitions are not the same.

[0111] First, when both values of gpm_tm_enable_flag0 and gpm_tm_enable_flag1 are 1 (i.e., TM is enabled for both of the two GPM partitions), the values of merge_gpm_idx0 and merge_gpm_idx1 cannot be the same.

[0112] Second, when one of gpm_tm_enable_flag0 and gpm_tm_enable_flag1 is 1 and the other is 0, the values of merge_gpm_idx0 and merge_gpm_idx1 are allowed to be the same.

[0113] Otherwise, that is, when both gpm_tm_enable_flag0 and gpm_tm_enable_flag1 are equal to 1: First, when both values of gpm_MVR_partIdx0_enable_flag and gpm_MVR_partIdx1_enable_flag are equal to 0 (that is, GPM-MVR is invalid for both of the two GPM partitions), the values of merge_gpm_idx0 and merge_gpm_idx1 will not be the same; Second, when gpm_MVR_partIdx0_enable_flag is equal to 1 (that is, GPM-MVR is valid for the first GPM partition) and gpm_MVR_partIdx1_enable_flag is equal to 0 (that is, GPM-MVR is invalid for the second GPM partition), the values of merge_gpm_idx0 and merge_gpm_idx1 are allowed to be the same; Third, when gpm_MVR_partIdx0_enable_flag is equal to 0 (that is, GPM-MVR is invalid for the first GPM partition) and gpm_MVR_partIdx1_enable_flag is equal to 1 (that is, GPM-MVR is valid for the second GPM partition), the values of merge_gpm_idx0 and merge_gpm_idx1 are allowed to be the same; Fourth, when both values of gpm_MVR_partIdx0_enable_flag and gpm_MVR_partIdx1_enable_flag are equal to 1 (that is, GPM-MVR is valid for both of the two GPM partitions), the determination of whether the values of merge_gpm_idx0 and merge_gpm_idx1 are allowed to be the same depends on the values of MVR applied to the two GPM partitions (indicated by gpm_mvr_partIdx0_direction_idx and gpm_mvr_partIdx0_distance_idx, and gpm_mvr_partIdx1_direction_idx and gpm_mvr_partIdx1_distance_idx). When the values of the two MVRs are equal, merge_gpm_idx0 and merge_gpm_idx1 are not the same.When not so (when the values of the two MVRs are not equal), the values of merge_gpm_idx0 and merge_gpm_idx1 are allowed to be the same.

[0114] In the above method 1, TM and MVR are exclusively applied to GPM. In such a scheme, it is prohibited to further apply MVR to the refined MV in TM mode. Therefore, in order to provide more MV candidates for GPM, method 2 is proposed in which an MVR offset can be applied to the TM refined MV. Table 12 shows the corresponding syntax table when GPM-MVR is combined with template matching.

Table 12

[0115] As shown in Table 12, different from Table 11, the signaling conditions of gpm_MVR_partIdx0_enable_flag and gpm_MVR_partIdx1_enable_flag at gpm_tm_enable_flag0 and gpm_tm_enable_flag1 are removed. Therefore, regardless of whether TM is applied to refine the one-way movement of one GPM split, MV refinement is always allowed to be applied to the MV of the GPM split. Similar to the above, in order to confirm that the MVs obtained from the two GPM splits are not the same, the following conditions need to be applied.

[0116] First, when one of gpm_tm_enable_flag0 and gpm_tm_enable_flag1 is 1 and the other is 0, the values of merge_gpm_idx0 and merge_gpm_idx1 are allowed to be the same.

[0117] Otherwise, that is, when both gpm_tm_enable_flag0 and gpm_tm_enable_flag1 are equal to 1 or both flags are equal to 0: First, when both values of gpm_MVR_partIdx0_enable_flag and gpm_MVR_partIdx1_enable_flag are equal to 0 (i.e., GPM-MVR is invalid for both of the two GPM partitions), the values of merge_gpm_idx0 and merge_gpm_idx1 will not be the same; Second, when gpm_MVR_partIdx0_enable_flag is equal to 1 (i.e., GPM-MVR is valid for the first GPM partition) and gpm_MVR_partIdx1_enable_flag is equal to 0 (i.e., GPM-MVR is invalid for the second GPM partition), the values of merge_gpm_idx0 and merge_gpm_idx1 are allowed to be the same; Third, when gpm_MVR_partIdx0_enable_flag is equal to 0 (i.e., GPM-MVR is invalid for the first GPM partition) and gpm_MVR_partIdx1_enable_flag is equal to 1 (i.e., GPM-MVR is valid for the second GPM partition), the values of merge_gpm_idx0 and merge_gpm_idx1 are allowed to be the same; Fourth, when both values of gpm_MVR_partIdx0_enable_flag and gpm_MVR_partIdx1_enable_flag are equal to 1 (i.e., GPM-MVR is valid for both of the two GPM partitions), the determination of whether the values of merge_gpm_idx0 and merge_gpm_idx1 are allowed to be the same depends on the values of MVR applied to the two GPM partitions (indicated by gpm_mvr_partIdx0_direction_idx and gpm_mvr_partIdx0_distance_idx, and gpm_mvr_partIdx1_direction_idx and gpm_mvr_partIdx1_distance_idx). When the values of the two MVRs are equal, merge_gpm_idx0 and merge_gpm_idx1 are not the same.When not (when the values of the two MVRs are not equal), the values of merge_gpm_idx0 and merge_gpm_idx1 are allowed to be the same.

[0118] In the above two methods, in order to indicate whether the TM is applied to each GPM split, it is necessary to signal two individual flags. The additional signaling can reduce the overall coding efficiency due to additional overhead, especially at low bitrates. To reduce the signaling overhead, instead of introducing additional signaling, Method 3 is proposed to insert the TM-based unidirectional MV into the unidirectional MV candidate list in the GPM mode. The TM-based unidirectional MV is generated using the original unidirectional MV of the GPM as the initial MV according to the same TM process as described in the "Template Matching" section. With such a scheme, there is no need to further send a signal of an additional control flag from the encoder to the decoder. Instead, the decoder can identify whether one MV is refined by the TM via the corresponding merge index (i.e., merge_gpm_idx0 and merge_gpm_idx1) received from the bitstream. There can be various ways to arrange the normal GPM MV candidates (i.e., non-TM) and the TM-based MV candidates. In one method, it is proposed to place the TM-based MV candidates at the beginning of the MV candidate list, followed by the non-TM-based MV candidates. In another method, it is proposed to first place the non-TM-based MV candidates first, followed by the TM-based candidates. In another method, it is proposed to arrange the TM-based MV candidates and the non-TM-based MV candidates in an interleaved manner. For example, first place N non-TM-based candidates, then place all the TM-based candidates, and finally place the remaining non-TM-based candidates. In another example, first place N TM-based candidates, then place all the non-TM-based candidates, and finally place the remaining TM-based candidates. In another example, it is proposed to arrange the non-TM-based candidates and the TM-based candidates one by one in order, i.e., one non-TM-based candidate, one TM-based candidate, etc.

[0119] The above method may be implemented using an apparatus including one or more circuits, including application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components. The apparatus may use the circuits in combination with other hardware or software components to perform the above method. Each of the above disclosed modules, sub-modules, units, or sub-units may be implemented using at least in part one or more circuits.

[0120] FIG. 10 shows a computing environment (or computing device) 1010 coupled to a user interface 1060. The computing environment 1010 can be part of a data processing server. In some embodiments, the computing device 1010 can perform any of the various methods or processes (such as encoding / decoding methods or processes) as described above, according to the various examples of the present disclosure. The computing environment 1010 may include a processor 1020, a memory 1040, and an I / O interface 1050.

[0121] The processor 1020 typically controls the overall operation of the computing environment 1010, such as operations related to display, data collection, data communication, and image processing. The processor 1020 may include one or more processors for executing instructions to perform all or some of the steps of the above-described methods. Further, the processor 1020 may include one or more modules that facilitate interaction between the processor 1020 and other components. The processor can be a central processing unit (CPU), a microprocessor, a single chip machine, a GPU, etc.

[0122] Memory 1040 is configured to store various types of data to support the operation of the computing environment 1010. Memory 1040 may include a predetermined software 1042. Examples of such data include instructions for any application or method operating in the computing environment 1010, video data sets, image data, and the like. Memory 1130 can be implemented by using any type of volatile or non-volatile memory device such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, optical disk, or a combination thereof.

[0123] The I / O interface 1050 provides an interface between the processor 1020 and peripheral interface modules such as a keyboard, click wheel, buttons, etc. The buttons may include, but are not limited to, a home button, a scan start button, and a scan stop button. The I / O interface 1050 can be coupled to an encoder and a decoder.

[0124] In some embodiments, a non-transitory computer-readable storage medium containing a plurality of programs executable by the processor 1020 within the computing environment 1010 is also provided for performing the methods described above. For example, the non-transitory computer-readable storage medium may be a ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, or the like.

[0125] The non-transitory computer-readable storage medium stores a plurality of programs for execution by a computing device having one or more processors, and when the plurality of programs are executed by the one or more processors, causes the computing device to execute the method for the above operation prediction.

[0126] In some embodiments, the computing environment 1010 may be implemented using one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), graphics processing units (GPUs), controllers, microcontrollers, microprocessors, or other electronic components to execute the above methods.

[0127] FIG. 8 is a flowchart illustrating a method for decoding a video block with GPM according to an example of the present disclosure.

[0128] In step 801, the processor 1020 can divide the video block into first Geometric division and second geometric partitions.

[0129] In step 802, the processor 1020 can receive a first TM enable flag for the first geometric partition and obtain a second TM enable flag for the second geometric partition.

[0130] In some examples, the first TM enable flag indicates whether the one - way motion of the first Geometric partition is refined by TM, and the second TM enable flag indicates whether the one - way motion of the second Geometric partition is refined by TM. For example, the first TM enable flag becomes flag gpm_tm_enable_flag0 as shown in Table 11 or Table 12, and the second TM enable flag becomes flag gpm_tm_enable_flag1 as shown in Table 11 or Table 12.

[0131] In step 803, the processor 1020 may receive a first merge GPM index for the first geometric partition and a second merge GPM index for the second geometric partition.

[0132] In some examples, the first merge GPM index identifies the unidirectional MV of the first geometric partition, and the second merge GPM index identifies the unidirectional MV of the second geometric partition.

[0133] In some examples, the first merge GPM index may be the syntax element merge_gpm_idx0 as shown in Table 11 or 12, and the second merge GPM index may be the syntax element merge_gpm_idx1 as shown in Table 11 or 12.

[0134] In step 804, the processor 1020 can construct a unidirectional MV candidate list for GPM.

[0135] In step 805, the processor 1020 may generate a unidirectional MV for the first geometric partition and a unidirectional MV for the second geometric partition.

[0136] In some examples, as shown in Table 11, the processor 1020 may apply TM to refine the unidirectional MV of the first geometric partition in response to a determination that the first TM enable flag is equal to 1, and may determine that MVR is not applied to the first geometric partition.

[0137] In some examples, as shown in Table 11, the processor 1020 can receive a first GPM having a first GPM (GPM-MVR) enable flag for the first geometric partition in response to a determination that the first TM enable flag is equal to 0, and the first GPM-MVR enable flag can indicate whether GPM-MVR is applied to the first geometric partition. The first GPM-MVR enable flag can be gpm_tm_enable_flag0 as shown in Table 11.

[0138] In some examples, as shown in Table 11, the processor 1020 can receive a first GPM-MVR distance index and a first GPM-MVR direction index for a first geometric partition in response to a determination that the first GPM-MVR enable flag is equal to 1. As shown in Table 11, the first GPM-MVR distance index and the first GPM-MVR direction index can each specify the direction of MVR of the first geometric partition.

[0139] In some examples, as shown in Table 11, the processor 1020 may determine that TM is applied to refine the one-way MV of a second geometric partition and MVR is not applied to the second geometric partition in response to a determination that the second TM enable flag is equal to 1.

[0140] In some examples, as shown in Table 11, the processor 1020 can receive a second GPM having a motion vector refinement (GPM-MVR) enable flag for a second geometric partition in response to a determination that the second TM enable flag is equal to 0. The second GPM-MVR enable flag can indicate whether GPM-MVR is applied to the second geometric partition.

[0141] In some examples, as shown in Table 11, the processor 1020 can receive a second GPM-MVR distance index and a first GPM-MVR direction index for a first geometric partition in response to a determination that the second GPM-MVR enable flag is equal to 1. The second GPM-MVR distance index and the second GPM-MVR direction index may each specify the direction of MVR for the second geometric partition.

[0142] In some examples, the processor 1020 may further constrain a first merge GPM index and a second merge GPM index based on the first TM enable flag and the second TM enable flag.

[0143] In some examples, in response to determining that the first TM enable flag and the second TM enable flag are each equal to 1, the processor 1020 may determine that the first merge GPM index and the second merge GPM index are different.

[0144] In some examples, in response to determining that one of the first TM enable flag and the second TM enable flag is equal to 1 and the other is equal to 0, the processor 1020 may determine that it is acceptable for the first merge GPM index and the second merge GPM index to be the same. In some examples, when it is acceptable for the first merge GPM index and the second merge GPM index to be the same, the first merge GPM index and the second merge GPM index may be the same or different.

[0145] For example, in some examples, when one of gpm_tm_enable_flag0 and gpm_tm_enable_flag1 is 1 and the other is 0, the values of merge_gpm_idx0 and merge_gpm_idx1 may be the same. In some examples, when one of gpm_tm_enable_flag0 and gpm_tm_enable_flag1 is 1 and the other is 0, the values of merge_gpm_idx0 and merge_gpm_idx1 may be different.

[0146] In some examples, in response to determining that the first TM enable flag and the second TM enable flag are each equal to 1, the processor 1020 may further constrain the first merge GPM index and the second merge GPM index based on the first GPM-MVR enable flag and the second GPM-MVR enable flag.

[0147] For example, in response to determining that the first GPM-MVR enable flag and the second GPM-MVR enable flag are each equal to 0, the processor 1020 may determine that the first merge GPM index and the second merge GPM index are different.

[0148] For example, the processor 1020 may determine that it is acceptable for the first merged GPM index and the second merged GPM index to be the same in response to a determination that the first GPM-MVR enable flag is equal to 1 and the second GPM-MVR enable flag is equal to 0, or in response to a determination that the first GPM-MVR enable flag is equal to 0 and the second GPM-MVR enable flag is equal to 1. In some examples, when it is acceptable for the first merged GPM index and the second merged GPM index to be the same, the first merged GPM index and the second merged GPM index may be the same or different.

[0149] For example, the processor 1020 may determine the first merged GPM index and the second merged GPM index based on the first MVR of the first geometric division and the second MVR of the second geometric division in response to a determination that the first GPM-MVR enable flag and the second GPM-MVR enable flag are each equal to 1.

[0150] In some examples, the processor 1020 may determine that the first merged GPM index and the second merged GPM index are different in response to a determination that the first MVR is equal to the second MVR.

[0151] In some examples, the processor 1020 may determine that it is acceptable for the first merged GPM index and the second merged GPM index to be the same in response to a determination that the first MVR is not equal to the second MVR. In some examples, when it is acceptable for the first merged GPM index and the second merged GPM index to be the same, the first merged GPM index and the second merged GPM index may be the same or different.

[0152] In some examples, as shown in Table 12, the processor 1020 may further constrain the first merge GPM index and the second merge GPM index based on the first TM enable flag and the second TM enable flag.

[0153] For example, the processor 1020 may determine that it is acceptable for the first merge GPM index and the second merge GPM index to be the same in response to a determination that one of the first TM enable flag and the second TM enable flag is equal to 1 and the other is equal to 0.

[0154] For example, the processor 1020 may constrain the first merge GPM index and the second merge GPM index based on the first GPM motion vector refinement (GPM-MVR) enable flag of the first geometric division and the second GPM-MVR enable flag of the second geometric division in response to a determination that both the first TM enable flag and the second TM enable flag are equal to 0 or both the first TM enable flag and the second TM enable flag are equal to 1. The first GPM-MVR enable flag can indicate whether GPM-MVR is applied to the first geometric division, and the second GPM-MVR enable flag can indicate whether GPM-MVR is applied to the second geometric division.

[0155] In some examples, the processor 1020 may determine that the first merge GPM index and the second merge GPM index are different in response to a determination that the first GPM-MVR enable flag and the second GPM-MVR enable flag are 0.

[0156] In some examples, the processor 1020 may determine that it is acceptable for the first merge GPM index and the second merge GPM index to be the same in response to a determination that the first GPM-MVR enable flag is equal to 1 and the second GPM-MVR enable flag is equal to 0, or in response to a determination that the first GPM-MVR enable flag is equal to 0 and the second GPM-MVR enable flag is equal to 1. If it is acceptable for the first merge GPM index and the second merge GPM index to be the same, the first merge GPM index and the second merge GPM index may be the same or different.

[0157] In some examples, the processor 1020 GPM-MVR enable flag and the second GPM-MVR enable flag are each equal to 1, the first merge GPM index and the second merge GPM index may be determined based on the first motion vector refinement (MVR) of the first geometric partition and the second MVR of the second geometric partition.

[0158] In some examples, the processor 1020 may determine that the first merge GPM index and the second merge GPM index are different in response to a determination that the first MVR is equal to the second MVR. Further, the processor 1020 may determine that it is acceptable for the first merge GPM index and the second merge GPM index to be the same in response to a determination that the first MVR is not equal to the second MVR.

[0159] FIG. 9 is a flowchart illustrating a method for decoding a video block with GPM according to an example of the present disclosure.

[0160] In step 901, the processor 1020 may divide the video block into a first Geometric division and a second geometric partition.

[0161] In step 902, the processor 1020 may construct a unidirectional MV candidate list for GPM.

[0162] In step 903, the processor 1020 may receive a first merge GPM index of the first geometric division and a second merge GPM index of the second geometric division.

[0163] In some examples, the first merge GPM index may be the syntax element merge_gpm_idx0 as shown in Table 11 or 12, and the second merge GPM index may be the syntax element merge_gpm_idx1 as shown in Table 11 or 12.

[0164] In step 904, the processor 1020 may update the unidirectional MV candidate list based on the first merge GPM index and the second merge GPM index.

[0165] In some examples, the first merge GPM index and the second merge GPM index indicate whether a single unidirectional MV is refined.

[0166] In some examples, the processor 1020 may generate a TM-based unidirectional MV based on the original unidirectional MV of the GPM as the initial MV, and obtain an updated unidirectional MV candidate list by adding the TM-based unidirectional MV to the unidirectional MV candidate list. The first merge GPM index and the second merge GPM index may indicate the TM-based unidirectional MV.

[0167] In some examples, the processor 1020 may add the TM-based unidirectional MV to the unidirectional MV candidate list by one of the following operations: adding the TM-based unidirectional MV to the head of the unidirectional MV candidate list so that the non-TM-based unidirectional MV follows the TM-based unidirectional MV; adding the TM-based unidirectional MV to the unidirectional MV candidate list so that the TM-based unidirectional MV follows the non-TM-based unidirectional MV within the unidirectional MV candidate list; adding the TM-based unidirectional MV to the unidirectional MV candidate list so that the TM-based unidirectional MV and the non-TM-based unidirectional MV are arranged in an interleaved manner in the unidirectional MV candidate list.

[0168] In some examples, an apparatus for decoding a video block with GPM is provided. The apparatus includes a processor 1020 and a memory 1040 configured to store instructions executable by the processor, and the processor is configured to execute the method shown in FIG. 8 or FIG. 9 when the instructions are executed.

[0169] In some other examples, a non-transitory computer-readable storage medium storing instructions is provided. When the instructions are executed by the processor 1020, the instructions cause the processor to execute a method as shown in FIG. 8 or FIG. 9.

[0170] Other examples of the present disclosure will be apparent to those skilled in the art from the specification and consideration of the embodiments disclosed herein. This application is intended to cover any variations, uses, or adaptations of the disclosure following its general principles, including departures from the present disclosure that are known or customary in the art. The specification and examples are intended to be considered exemplary only.

[0171] It will be understood that the present disclosure is not limited only to the examples described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope.

Claims

1. Dividing a video block into a first geometric division and a second geometric division, Receiving a first TM enable flag for the first geometric division indicating whether the unidirectional motion of the first geometric division is refined by template matching, i.e., TM, and a second TM enable flag for the second geometric division indicating whether the unidirectional motion of the second geometric division is refined by the TM, Receiving a first merge GPM index for the first geometric division and a second merge GPM index for the second geometric division, Constructing a unidirectional motion vector (MV) candidate list for the geometric partitioning mode, i.e., GPM, Generating a unidirectional MV for the first geometric division and a unidirectional MV for the second geometric division, A method comprising: The method further comprises: Constraining the first merge GPM index and the second merge GPM index based on the first TM enable flag and the second TM enable flag, a method for decoding a video block in a geometric partitioning mode (GPM).

2. In response to a determination that the first TM enable flag is equal to 1, the TM is applied to refine the unidirectional MV of the first geometric division, and it is determined that motion vector refinement, i.e., MVR, is not applied to the first geometric division, In response to a determination that the first TM enable flag is equal to 0, receiving a first GPM having a first GPM-MVR enable flag indicating whether GPM-MVR, i.e., MVR for the first geometric division, is applied to the first geometric division, The method according to claim 1, further comprising.

3. In response to a determination that the first GPM-MVR enable flag is equal to 1, receiving a first GPM-MVR distance index and a first GPM-MVR direction index for the first geometric division, wherein the first GPM-MVR distance index and the first GPM-MVR direction index further comprise specifying the direction of the MVR for the first geometric division, the method according to claim 2.

4. In response to the determination that the second TM enable flag is equal to 1, the TM is applied to refine the one-directional MV of the second geometric division, and it is determined that the motion vector refinement (MVR) is not applied to the second geometric division. In response to the determination that the second TM enable flag is equal to 0, a second GPM (GPM-MVR) enable flag with motion vector refinement for the second geometric division is received, and the second GPM-MVR enable flag indicates whether the GPM-MVR is applied to the second geometric division. The method according to claim 3, further comprising.

5. In response to the determination that the second GPM-MVR enable flag is equal to 1, receiving a second GPM-MVR distance index and a second GPM-MVR direction index for the first geometric division, wherein the second GPM-MVR distance index and the second GPM-MVR direction index specify the direction of the MVR for the second geometric division. The method according to claim 4.

6. Constraining the first merge GPM index and the second merge GPM index based on the first TM enable flag and the second TM enable flag is In response to the determination that the first TM enable flag and the second TM enable flag are each equal to 1, determining that the first merge GPM index and the second merge GPM index are different. In response to the determination that one of the first TM enable flag and the second TM enable flag is equal to 1 and the other is equal to 0, determining that it is acceptable for the first merge GPM index and the second merge GPM index to be the same. The method according to claim 5, comprising.

7. The method according to claim 5, further comprising constraining the first merge GPM index and the second merge GPM index based on the first GPM-MVR enable flag and the second GPM-MVR enable flag in response to the determination that the first TM enable flag and the second TM enable flag are each equal to 1.

8. Constraining the first merged GPM index and the second merged GPM index based on the first GPM-MVR enable flag and the second GPM-MVR enable flag is determining that the first merged GPM index and the second merged GPM index are different in response to a determination that the first GPM-MVR enable flag and the second GPM-MVR enable flag are each equal to 0; determining that it is allowed that the first merged GPM index and the second merged GPM index are the same in response to a determination that the first GPM-MVR enable flag is equal to 1 and the second GPM-MVR enable flag is equal to 0; determining that it is allowed that the first merged GPM index and the second merged GPM index are the same in response to a determination that the first GPM-MVR enable flag is equal to 0 and the second GPM-MVR enable flag is equal to 1; determining the first merged GPM index and the second merged GPM index based on the first motion vector refinement of the first geometric partition, that is, the first MVR, and the second MVR of the second geometric partition in response to a determination that the first GPM-MVR enable flag and the second GPM-MVR enable flag are each equal to 1; The method according to claim 7, comprising

9. determining that the first merged GPM index and the second merged GPM index are different in response to a determination that the first MVR is equal to the second MVR; determining that it is allowed that the first merged GPM index and the second merged GPM index are the same in response to a determination that the first MVR is not equal to the second MVR; The method according to claim 8, further comprising

10. Constraining the first merged GPM index and the second merged GPM index based on the first TM enable flag and the second TM enable flag is Determine that it is acceptable for the first merge GPM index and the second merge GPM index to be the same in response to a determination that one of the first TM enable flag and the second TM enable flag is equal to 1 and the other is equal to 0. In response to a determination that both the first TM enable flag and the second TM enable flag are 0 or both are equal to 1, constrain the first merge GPM index and the second merge GPM index based on the first GPM motion vector refinement enable flag of the first geometric division, i.e., the first GPM-MVR enable flag, indicating whether GPM-MVR is applied to the first geometric division, and the second GPM motion vector refinement enable flag of the second geometric division, i.e., the second GPM-MVR enable flag, indicating whether GPM-MVR is applied to the second geometric division. The method according to claim 1, comprising the above.

11. Constraining the first merge GPM index and the second merge GPM index based on the first GPM-MVR enable flag of the first geometric division and the second GPM-MVR enable flag of the second geometric division means determining that the first merge GPM index and the second merge GPM index are different in response to a determination that the first GPM-MVR enable flag and the second GPM-MVR enable flag are 0. determining that it is acceptable for the first merge GPM index and the second merge GPM index to be the same in response to a determination that the first GPM-MVR enable flag is equal to 1 and the second GPM-MVR enable flag is equal to 0. determining that it is acceptable for the first merge GPM index and the second merge GPM index to be the same in response to a determination that the first GPM-MVR enable flag is equal to 0 and the second GPM-MVR enable flag is equal to 1. In response to the determination that the first GPM-MVR enable flag and the second GPM-MVR enable flag are each equal to 1, determining the first merge GPM index and the second merge GPM index based on the first motion vector refinement of the first geometric partition, i.e., the first MVR, and the second motion vector refinement of the second geometric partition, i.e., the second MVR; The method according to claim 10, comprising: **Claim 12** In response to the determination that the first MVR is equal to the second MVR, determining that the first merge GPM index is different from the second merge GPM index; In response to the determination that the first MVR is not equal to the second MVR, determining that it is acceptable that the first merge GPM index and the second merge GPM index are the same; The method according to claim 11, further comprising: **Claim 13** Dividing a video block into a first geometric partition and a second geometric partition; Constructing a geometric partition mode, i.e., a GPM one-way motion vector candidate list, i.e., a one-way MV candidate list; Receiving a first merge GPM index of the first geometric partition and a second merge GPM index of the second geometric partition; Updating the one-way MV candidate list based on the first merge GPM index and the second merge GPM index, which indicate whether a single one-way MV is refined by template matching (TM); A method for decoding a video block in a geometric partition mode (GPM), comprising: **Claim 14** Generating a TM-based one-way MV based on the original one-way MV of GPM as an initial MV, wherein the first merge GPM index and the second merge GPM index indicate the TM-based one-way MV; Obtaining an updated one-way MV candidate list by adding the TM-based one-way MV to the one-way MV candidate list; The method according to claim 13, further comprising: **Claim 15** Adding the TM-based one-way MV to the one-way MV candidate list Adding the TM-based one-way MV to the head of the one-way MV candidate list such that the non-TM-based one-way MV follows the TM-based one-way MV; Adding the TM-based uni-directional MV to the uni-directional MV candidate list such that the TM-based uni-directional MV follows the non-TM-based uni-directional MVs in the uni-directional MV candidate list; Adding the TM-based uni-directional MV to the uni-directional MV candidate list such that the TM-based uni-directional MV and the non-TM-based uni-directional MVs are arranged in an interleaved manner in the uni-directional MV candidate list; The method according to claim 14, comprising one of the operations above.

16. One or more processors; A non-transitory computer-readable storage medium configured to store instructions executable by the one or more processors, wherein the one or more processors are configured to execute the method according to any one of claims 1 to 15 when the instructions are executed; a non-transitory computer-readable storage medium; An apparatus for video coding, comprising the above.

17. A method for storing a bitstream decoded by a method of decoding a video block in a geometric partitioning mode (GPM) according to any one of claims 1 to 15.

18. A method for receiving a bitstream, wherein the bitstream is decoded by a method of decoding a video block in a geometric partitioning mode (GPM) according to any one of claims 1 to 15.

19. A storage medium storing a computer-readable program for storing a bitstream containing video data and causing a decoding device to execute the method according to any one of claims 1 to 15.

Images