Geometric Segmentation with Explicit Motion Signaling

Abstract

A method, apparatus, and non-transitory computer-readable storage medium are provided for decoding a video block in a geometric partition mode (GPM). The method includes partitioning the video block into first and second geometric partitions, obtaining a first prediction list for the first geometric partition, obtaining a second prediction list for the second geometric partition, obtaining motion vectors (MVs) for the first and second geometric partitions by applying GPM with explicit motion signaling (EMS) to the first and second geometric partitions based on the first and second prediction lists, and obtaining prediction samples for the first and second geometric partitions based on the MVs.

Description

[Technical Field]

[0001] [CROSS-REFERENCE TO RELATED APPLICATIONS] This application is based on and claims priority to U.S. Provisional Patent Application No. 63 / 173,303, filed April 9, 2021, the disclosure of which is incorporated herein by reference in its entirety for all purposes.

[0002] This disclosure relates to video coding and compression, and more particularly to methods and apparatus for improving coding efficiency of geometric partitioning (GPM) mode. [Background technology]

[0003] Various video coding techniques can be used to compress video data. Video coding is performed according to one or more video coding standards. For example, currently, some well-known video coding standards include Versatile Video Coding (VCC), High Efficiency Video Coding (HEVC, also known as H.265 or MPEG-H Part 2), and Advanced Video Coding (AVC, also known as H.264 or MPEG-4 Part 10), which are jointly developed by ISO / IEC MPEG and ITU-T VECG. AOMedia Video 1 (AV1) was developed by the Alliance for Open Media (AOM) as a successor to its predecessor, VP9. Audio Video Coding (AVS), which refers to digital audio and digital video compression standards, is another video compression standard series developed by the China Audio and Video Coding Standards Workgroup. Most existing video coding standards are built on the well-known hybrid video coding framework, which uses block-based prediction methods (e.g., inter-prediction, intra-prediction) to reduce redundancy in a video image or sequence and transform coding to compress the energy of the prediction error. An important goal of video coding techniques is to compress video data into a format that uses a lower bit rate while avoiding or minimizing degradation of video quality. Summary of the Invention [Means for solving the problem]

[0004] The present disclosure provides a method and apparatus for video coding, and a non-transitory computer-readable storage medium thereof.

[0005] A method for decoding a video block in geometric partitioning mode (GPM) is provided. The method may include partitioning the video block into a first geometric partition and a second geometric partition. The method may include obtaining a first motion vector refinement (MVR) for the first geometric partition and a second MVR for the second geometric partition. The method may include obtaining motion vectors (MVs) for the first and second geometric partitions by applying GPM with explicit motion signaling (EMS) to the first and second geometric partitions based on the first and second prediction lists. The method may include obtaining prediction samples for the first and second geometric partitions based on the MVs.

[0006] According to a second aspect of the present disclosure, a method for decoding a video block in a GPM is provided. The method may include partitioning the video block into first and second geometric partitions and obtaining a first motion vector for the first geometric partition and a second motion vector for the second geometric partition. The method may include obtaining the motion vectors for the first and second geometric partitions by applying a GPM with MVR or a GPM with EMS to the first and second geometric partitions based on the first and second prediction lists. The method may include obtaining prediction samples for the first and second geometric partitions based on the motion vectors.

[0007] According to a third aspect of the present disclosure, there is provided an apparatus for video coding. The apparatus may include one or more processors and a non-transitory computer-readable storage medium. The non-transitory computer-readable storage medium is configured to store instructions executable by the one or more processors. The one or more processors, upon execution of the instructions, are configured to perform the method of the first or second aspect.

[0008] According to a fourth aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium, which may store computer-executable instructions that, when executed by one or more computer processors, cause one or more computer processors to perform the method of the first or second aspect. [Brief explanation of the drawings]

[0009] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the detailed description, serve to explain the principles of the disclosure. [Figure 1] FIG. 1 is a block diagram of an encoder according to one embodiment of the present disclosure. [Figure 2] FIG. 2 is a block diagram of a decoder according to one embodiment of the present disclosure. [Figure 3A] FIG. 3A is a diagram illustrating block division in a multi-type tree structure according to one embodiment of the present disclosure. [Figure 3B] FIG. 3B is a diagram illustrating block division in a multi-type tree structure according to one embodiment of the present disclosure. [Figure 3C] FIG. 3C is a diagram illustrating block division in a multi-type tree structure according to one embodiment of the present disclosure. [Figure 3D] FIG. 3D is a diagram illustrating block division in a multi-type tree structure according to one embodiment of the present disclosure. [Figure 3E] FIG. 3E is a diagram illustrating block division in a multi-type tree structure according to one embodiment of the present disclosure. [Figure 4] FIG. 4 is a diagram of permitted geometric partitioning (GPM) according to one embodiment of the present disclosure. [Figure 5] FIG. 5 is a table illustrating unidirectional motion vector selection according to one embodiment of the present disclosure. [Figure 6A] FIG. 6A is a diagram of a motion vector differential (MMVD) mode according to one embodiment of the present disclosure. [Figure 6B]FIG. 6B is a diagram of an MMVD mode according to one embodiment of the present disclosure. [Figure 7] FIG. 7 is a method for decoding a video block in a GPM according to one embodiment of this disclosure. [Figure 8] FIG. 8 is a method for decoding a video block in a GPM according to one embodiment of this disclosure. [Figure 9] FIG. 9 is a diagram illustrating a computing environment coupled with a user interface according to one embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0010] Reference will now be made in detail to embodiments, examples of which are illustrated in the accompanying drawings. In the following description, reference will be made to the accompanying drawings, in which the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The specific examples set forth in the following description of embodiments do not represent all specific examples consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with aspects related to the present disclosure, as set forth in the appended claims.

[0011] The terms used in this disclosure are for the purpose of describing particular embodiments only and are not intended to limit the disclosure. As used in this disclosure and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. The term "and / or," as used herein, should also be understood to imply and include any and all possible combinations of one or more of the associated listed items.

[0012] While terms such as "first," "second," and "third" may be used herein to describe various pieces of information, it should be understood that the information should not be limited by these terms. These terms are used only to distinguish one category of information from another. For example, first information could be referred to as second information, and similarly, second information could be referred to as first information, without departing from the scope of this disclosure. As used herein, the term "if" can be understood to mean "when," "upon," or "in response to a judgment," depending on the context.

[0013] The first generation of AVS standards includes the Chinese national standards "Information Technology, Advanced Audio-Video Coding, Part 2: Video" (known as AVS1) and "Information Technology, Advanced Audio-Video Coding, Part 16: Radio-Television Video" (known as AVS+). Compared to the MPEG-2 standard, this provides approximately 50% bitrate savings at the same perceptual quality. The AVS1 video part was published as a Chinese national standard in February 2006. The second generation of AVS standards includes the Chinese national standard "Information Technology, Efficient Multimedia Coding" (known as AVS2), which primarily targets the transmission of additional HD television programs. AVS2's coding efficiency is twice that of AVS+. AVS2 was published as a Chinese national standard in May 2016. Meanwhile, the AVS2 video part was submitted by the Institute of Electrical and Electronics Engineers (IEEE) as an international standard for practical use. The AVS3 standard is a new generation of video coding standard for UHD video applications that aims to exceed the coding efficiency of the latest international standard, HEVC. At the 68th AVS Conference in March 2019, the AVS3-P2 baseline was completed, enabling approximately 30% bitrate reduction over the HEVC standard. Currently, the AVS group maintains a single reference software called the High Performance Model (HPM) to demonstrate the reference implementation of the AVS3 standard.

[0014] Like HEVC, the AVS3 standard is built on a block-based hybrid video coding framework.

[0015] Figure 1 shows a general diagram of a block-based video encoder for VVC. Specifically, Figure 1 shows a typical encoder 100. The encoder 100 includes a video input 110, motion compensation 112, motion estimation 114, intra / inter mode decision 116, a block predictor 140, an adder 128, a transform 130, quantization 132, prediction-related information 142, intra prediction 118, a picture buffer 120, inverse quantization 134, an inverse transform 136, an adder 126, a memory 124, an in-loop filter 122, entropy coding 138, and a bitstream 144.

[0016] At encoder 100, a video frame is divided into video blocks for processing. For each given video block, a prediction is formed based on either an inter-prediction technique or an intra-prediction technique.

[0017] A prediction residual, representing the difference between the current video block, a portion of video input 110, and its predictor, which is part of block predictor 140, is sent from summer 128 to transform 130. The transform coefficients are then sent from transform 130 to quantization 132 for entropy reduction. The quantized coefficients are then provided to entropy coding 138 to generate a compressed video bitstream. As shown in FIG. 1, prediction-related information 142 from intra / inter mode decision 116, such as video block partition information, motion vectors (MVs), reference picture indexes, and intra prediction modes, is also provided through entropy coding 138 and stored in compressed bitstream 144. Compressed bitstream 144 comprises the video bitstream.

[0018] Decoder-related circuitry is also required in encoder 100 to reconstruct pixels for prediction purposes. First, a prediction residual is reconstructed by inverse quantization 134 and inverse transform 136. This reconstructed prediction residual is combined with block predictor 140 to generate unfiltered reconstructed pixels for the current video block.

[0019] Spatial prediction (or "intra prediction") predicts the current video block using pixels from samples of already coded neighboring blocks (called reference samples) in the same video frame as the current video block.

[0020] Temporal prediction (also called "inter-prediction") predicts a current video block using reconstructed pixels from an already coded video picture. Temporal prediction reduces the temporal redundancy inherent in video signals. Temporal prediction reduces the temporal redundancy inherent in video signals. The temporal prediction signal for a given coding unit (CU), or coding block, is typically signaled by one or more MVs that indicate the amount and direction of motion between the current CU and its temporal reference. Furthermore, if multiple reference pictures are supported, a reference picture index is additionally sent, which is used to identify which reference picture in the reference picture store the temporal prediction signal comes from.

[0021] Motion estimation 114 takes signals from video input 110 and picture buffer 120 and outputs a motion estimation signal to motion compensation 112. Motion estimation 114 takes signals from video input 110 and picture buffer 120 and outputs a motion estimation signal to motion compensation 112. Motion compensation 112 takes signals from video input 110, picture buffer 120, and the motion estimation signal from motion estimation 114 and outputs a motion compensation signal to intra / inter mode decision 116.

[0022] After spatial and / or temporal prediction is performed, intra / inter mode decision 116 in encoder 100 selects the best prediction mode, for example, based on a rate-distortion optimization method. Block predictor 140 is then subtracted from the current video block, and the resulting prediction residual is decorrelated using transform 130 and quantization 132. The resulting quantized residual coefficients are inversely quantized by inverse quantization 134 and inversely transformed by inverse transform 136 to form a reconstructed residual, which is then added back to the prediction block to form a reconstructed signal for the CU. Further in-loop filtering 122, such as a deblocking filter, sample adaptive offset (SAO), and / or adaptive in-loop filter (ALF), may be applied to the reconstructed CU before it is placed into a reference picture store in picture buffer 120 and used to code future video blocks. To form the output video bitstream 144, the coding mode (inter or intra), prediction mode information, motion information, and quantized residual coefficients are all sent to the entropy coding unit 138 for further compression and packing to form the bitstream.

[0023] Figure 1 shows a block diagram of a typical block-based hybrid video coding system. An input video signal is processed block by block (called a coding unit (CU)). Unlike HEVC, which divides blocks solely based on a quadtree, AVS3 divides a single coding tree unit (CTU) into CUs, adapting them to various local characteristics based on a quad, binary, or extended quadtree. Furthermore, the concept of multiple division unit types in HEVC is eliminated. That is, in AVS3, there is no separation between CUs, prediction units (PUs), and transform units (TUs). Instead, each CU is always used as the basic unit for both prediction and transformation without further division. In the AVS3 tree division structure, a single CTU is first divided based on a quadtree structure. Then, each quadtree leaf node can be further divided based on a binary or extended quadtree structure.

[0024] As shown in Figures 3A, 3B, 3C, 3D, and 3e, there are five types: 4-way partitioning, horizontal 2-way partitioning, vertical 2-way partitioning, horizontally extended quadtree partitioning, and vertically extended quadtree partitioning.

[0025] FIG. 3A shows a diagram illustrating block quad division in a multi-type tree structure according to the present disclosure.

[0026] FIG. 3B shows a diagram illustrating block vertical bisection in a multi-type tree structure according to the present disclosure.

[0027] FIG. 3C shows a diagram illustrating horizontal bisection of blocks in a multi-type tree structure according to the present disclosure.

[0028] FIG. 3D shows a diagram illustrating a vertical 3-part division of blocks in a multi-type tree structure according to the present disclosure.

[0029] FIG. 3E shows a diagram illustrating a horizontal 3-part division of blocks in a multi-type tree structure according to the present disclosure.

[0030] In FIG. 1, spatial prediction and / or temporal prediction may be performed. Spatial prediction (or "intra prediction") predicts a current video block using pixels from samples of previously coded neighboring blocks (called reference samples) in the same video picture / slice. Spatial prediction reduces spatial redundancy inherent in video signals. Temporal prediction (also called "inter prediction" or "motion-compensated prediction") predicts a current video block using pixels reconstructed from previously coded video pictures. Temporal prediction reduces temporal redundancy inherent in video signals. The temporal prediction signal for a given CU is typically signaled by one or more motion vectors (MVs), which indicate the amount and direction of motion between the current CU and its temporal reference. If multiple reference pictures are supported, a reference picture index is additionally sent, which is used to identify which reference picture in the reference picture store the temporal prediction signal comes from. After spatial and / or temporal prediction, a mode decision block in the encoder selects the best prediction mode, for example, based on a rate-distortion optimization method. The prediction block is then subtracted from the current video block, and the prediction residual is decorrelated using a transform and then quantized. The quantized residual coefficients are inverse quantized and inverse transformed to form a reconstructed residual, which is then added back to the prediction block to form a reconstructed signal for the CU. Further in-loop filtering, such as a deblocking filter, sample adaptive offset (SAO), and adaptive in-loop filter (ALF), may be applied to the reconstructed CU before it is placed in a reference picture store and used as a reference for coding future video blocks. The coding mode (inter or intra), prediction mode information, motion information, and quantized residual coefficients are all sent to an entropy coding unit for further compression and packing to form an output video bitstream.

[0031] Figure 2 shows a general block diagram of a video decoder for VVC. Specifically, Figure 2 shows a block diagram of an exemplary decoder 200. The decoder 200 includes a bitstream 210, an entropy decoding 212, an inverse quantization 214, an inverse transform 216, an adder 218, an intra / inter mode selection 220, an intra prediction 222, a memory 230, an in-loop filter 228, a motion compensation 224, a picture buffer 226, prediction-related information 234, and a video output 232.

[0032] The decoder 200 is similar to the reconstruction-related section present in the encoder 100 of Figure 1. In the decoder 200, an input video bitstream 210 is first decoded through entropy decoding 212 to derive quantized coefficient levels and prediction-related information. The quantized coefficient levels are then processed through inverse quantization 214 and an inverse transform 216 to obtain a reconstructed prediction residual. A block predictor implemented in intra / inter mode selection 220 is configured to perform either intra prediction 222 or motion compensation 224 based on the decoded prediction information. A set of unfiltered reconstructed pixels is obtained by summing the reconstructed prediction residual from the inverse transform 216 and the prediction output generated by the block predictor using an adder 218.

[0033] The reconstructed blocks may further pass through an in-loop filter 228 before being stored in a picture buffer 226, which serves as a reference picture store. The reconstructed video in the picture buffer 226 may be transmitted to drive a display device, as well as used to predict future video blocks. In situations where the in-loop filter 228 is turned on, a filtering operation is performed on these reconstructed pixels to derive a final reconstructed video output 232.

[0034] Figure 2 shows a general block diagram of a block-based video decoder. A video bitstream is first entropy decoded in an entropy decoding unit. Coding mode and prediction information are sent to either a spatial prediction unit (if intra-coded) or a temporal prediction unit (if inter-coded) to form a prediction block. Residual transform coefficients are sent to an inverse quantization unit and an inverse transform unit to reconstruct a residual block. The prediction block and the residual block are then summed. The reconstructed block may further undergo in-loop filtering before being stored in a reference picture store. The reconstructed video in the reference picture store is then sent for display and used to predict future video blocks.

[0035] The focus of this disclosure is to improve the coding performance of the geometric partitioning mode (GPM) used in both the VVC and AVS3 standards. In AVS3, the implementation is also known as angle weighted prediction (AWP), which follows the same design spirit of GPM but with slight differences in specific design details. To facilitate the explanation of this disclosure, the following uses the existing GPM design in the VVC standard as an example to explain the main aspects of the GPM / AWP implementation. Meanwhile, another conventional inter-prediction technique called merge mode with motion vector differential (MMVD), which is applied to both the VVC and AVS3 standards, will also be briefly described, considering that it is closely related to the technology proposed in this invention. Then, some drawbacks of the current GPM / AWP design are identified. Finally, the proposed method is presented in detail. Throughout this disclosure, the existing GPM design in the VVC standard is used as an example. However, those skilled in the art of modern video coding technology should note that the proposed technology can also be applied to other GPM / AWP designs or other coding implementations with the same or similar design spirit.

[0036] Geometric Division Mode (GPM) VVC supports geometric partitioning mode for inter prediction. Geometric partitioning mode is signaled by a CU-level flag as a special merge mode. In the current GPM design, a total of 64 partitions are supported in GPM mode for each possible CU size where both width and height are between 8 and 64, excluding 8x64 and 64x8.

[0037] When using this mode, a CU is divided into two parts by a geometrically located line, as shown in FIG. 4 (described below). The location of the division line is mathematically derived from the angle and offset parameters of the specific division. Each part of the geometric division within a CU is inter-predicted using its own motion, but only uni-prediction is allowed for each division. That is, each part has one motion vector and one reference index. A uni-prediction motion constraint is applied to ensure the same as traditional bi-prediction, and only two motion-compensated predictions are required for each CU. When the geometric division mode is used for the current CU, a geometric division index indicating the division mode (angle and offset) of the geometric division and two merge indices (one for each division) are further signaled. The number of maximum GPM candidate sizes is explicitly signaled at the sequence level.

[0038] FIG. 4 shows the allowed GPM partitions, where the partitions in each picture have one and the same partition direction.

[0039] [One-sided prediction candidate list configuration] To derive a uni-predictive motion vector for one geometric partition, first, one uni-predictive candidate list is derived directly from the standard merge candidate list generation process. Let n be the index of the uni-predictive motion vector in the geometric uni-predictive candidate list. The LX motion vector (X equals the parity of n) of the nth merge candidate is used as the nth uni-predictive motion vector for the geometric partition mode.

[0040] These motion vectors are marked with "x" in Figure 5 (described below). If the corresponding LX motion vector of an n extended merge candidate does not exist, the L(1-X) motion vector of the same candidate is used instead as the single-predictor motion vector for the geometric partitioning mode.

[0041] FIG. 5 shows a uni-predictive motion vector selection from the motion vectors in the merge candidate list for GPM.

[0042] Blend along geometric division edges After each geometric partition is obtained using its own motion, blending is applied to the two uni-predicted signals to derive samples around the geometric partition edge. The blending weights for each position of the CU are derived based on the distance from each individual sample position to the corresponding partition edge.

[0043] [GPM Signaling Design] According to the current GPM design, the use of GPM is indicated by signaling one flag at the CU level. The flag is signaled only when the current CU is coded by either merge mode or skip mode. Specifically, when the flag is equal to 1, it indicates that the current CU is predicted by GPM. Otherwise (the flag is equal to 0), the CU is coded by another merge mode, such as standard merge mode, merge mode with motion vector differential, or a combination of inter and intra prediction. When GPM is enabled for the current CU, one syntax element, namely, merge_gpm_partition_idx, is further signaled to indicate the applied geometric partition mode (specifying the direction and offset of the line from the CU center that divides the CU into two partitions, as shown in FIG. 4). Then, two syntax elements, merge_gpm_idx0 and merge_gpm_idx1, are signaled to indicate the indexes of uni-predictive merge candidates used for the first and second GPM partitions. More specifically, these two syntax elements are used to determine the unidirectional MVs of two GPM partitions from the unidirectional merge list, as described in the "Unidirectional Merge List Configuration" section. According to current GPM design, the two indices cannot be the same to make the two unidirectional MVs more different. Based on such prior knowledge, the unidirectional merge index of the first GPM partition is first signaled and used as a predictor to reduce the signaling overhead of the unidirectional merge index of the second GPM partition. In particular, if the second unidirectional merge index is smaller than the first unidirectional merge index, its original value is directly signaled. Otherwise (if the second unidirectional merge index is larger than the first unidirectional merge index), its value is subtracted by 1 before being signaled to the bitstream. At the decoder side, the first unidirectional merge index is decoded first.Then, for decoding the second uni-predictive merge index, if the parsed value is less than the first uni-predictive merge index, the second uni-predictive merge index is set equal to the parsed value. Otherwise (if the parsed value is greater than or equal to the first uni-predictive merge index), the second uni-predictive merge index is set equal to the parsed value plus 1. Table 1 shows the existing syntax elements used for GPM mode in the current VVC specification. [Table 1]

[0044] On the other hand, in the current GPM design, a truncated unary code is used for binarization of the two uni-predictive merge indexes, i.e., merge_gpm_idx0 and merge_gpm_idx1. Furthermore, since the two uni-predictive merge indexes cannot be the same, different maximum values ​​are used to truncate the codewords of the two uni-predictive merge indexes, which are set equal to MaxGPMMergeCand-1 and MaxGPMMergeCand-2 for merge_gpm_idx0 and merge_gpm_idx1, respectively. MaxGPMMergeCand is the number of candidates in the uni-predictive merge list.

[0045] When the GPM / AWP mode is applied, two different binarization methods are applied to convert the syntax element merge_gpm_partition_idx into a string of binary bits. Specifically, the syntax element is binarized by a fixed-length code and a truncated binary code in the VVC standard and the AVS3 standard, respectively. Meanwhile, in the AWP mode in AVS3, a different maximum value is used to binarize the value of the syntax element. Specifically, in AVS3, the number of GPM / AWP partition modes allowed in VVC is 56 (i.e., the maximum value of merge_gpm_partition_idx is 55), while that number is increased to 64 (i.e., the maximum value of merge_gpm_partition_idx is 63).

[0046] [Merge with motion vector difference mode (MMVD)] In addition to the conventional merge mode that derives the motion information of a current block from its spatial / temporal neighbors, the MMVD / UMVE mode is introduced as a special merge mode in both the VVC and AVS standards. Specifically, in both VVC and AVS3, the mode is signaled by an MMVD flag at the coding block level. In the MMVD mode, the first two candidates in the merge list for the standard merge mode are selected as two base merge candidates for MMVD. After a base merge candidate is selected and signaled, an additional syntax element is signaled to indicate the motion vector differentials (MVDs) to be added to the motion of the selected merge candidate. The MMVD syntax element includes a merge candidate flag for selecting the base merge candidate, a distance index for specifying the MVD size, and a direction index for indicating the MVD direction.

[0047] In existing MMVD designs, a distance index specifies the size of the MVD, which is defined based on a set of predetermined offsets from the starting point. The offsets are added to either the horizontal or vertical components of the starting MV (i.e., the MVs of the selected base merge candidate).

[0048] Figure 6A shows the MMVD mode for the L0 standard, and Figure 6B shows the MMVD mode for the L1 standard.

[0049] Table 2 shows the MVD offsets applied in AVS3, respectively. [Table 2]

[0050] As shown in Table 3, the direction index is used to specify the signal of the signaled MVD. Note that the meaning of the MVD code can change depending on the starting MV. When the starting MV is a uni-predictive MV or a bi-predictive MV with MVs pointing to two reference pictures whose POCs are both greater than or less than the POC of the current picture, the signaled code is the code of the MVD added to the starting MV. When the starting MV is a bi-predictive MV pointing to two reference pictures whose POCs are one greater than the current picture and the other less than the current picture, the signaled code is applied to the L0 MVD, and the opposite value of the signaled code is applied to the L1 MVD. [Table 3]

[0051] [Motion signaling for standard inter-mode] Similar to the HEVC standard, in addition to merge / skip modes, both VVC and AVS3 allow one inter CU to explicitly specify its motion information in the bitstream. Overall, motion information signaling in both VVC and AVS3 remains the same as in the HEVC standard. Specifically, one inter prediction syntax, namely, inter_pred_idc, is signaled first to indicate whether the prediction signal is from list L0, L1, or both. For each used reference list, the corresponding reference picture is identified by signaling one reference picture index ref_idx_lx (x=0,1) for the corresponding reference list, and the corresponding MV is used to select an MV predictor (MVP) and is represented by one MVP index mvp_lx_flag (x=0,1) followed by the motion vector difference (MVD) between the target MV and the selected MVP. Furthermore, in the VVC standard, one control flag mvd_l1_zero_flag is signaled at the slice level. When mvd_l1_zero_flag is equal to 0, the L1 MVD is signaled in the bitstream; otherwise (when mvd_L1_zero_flag is equal to 1), the L1 MVD is not signaled and its value is always predicted to be 0 in the encoder and decoder.

[0052] [CU-level weighted bi-prediction] In previous standards prior to VVC and AVS3, when weighted prediction (WP) is not applied, bi-predictive signals are generated by averaging uni-predictive signals obtained from two reference pictures. VVC introduces a coding method, namely bi-prediction with CU level weighting (BCW), to improve the efficiency of bi-prediction. Specifically, instead of simple averaging, bi-prediction in BCW is extended by allowing weighted averaging of two predicted signals, as shown below:

number

[0053] In VVC, when the current picture is a low-latency picture, the weight of one BCW coding block is allowed to be selected from a set of predetermined weight values ​​w∈{-2, 3, 4, 5, 10}, with a weight of 4 representing the traditional bi-predictive case in which two uni-predictive signals are equally weighted. For low-latency cases, only three weights w∈{3, 4, 5} are allowed. Generally speaking, there are some design similarities between WP and BCW, but the two coding methods aim to solve the brightness change problem at different granularities. However, because the interaction between WP and BCW can potentially complicate VVC design, the two tools are not allowed to be enabled simultaneously. Specifically, when WP is enabled for a slice, the BCW weights for all bi-predictive CUs in the slice are not signaled and are assumed to be 4 (i.e., equal weights are applied).

[0054] [Improvements to Geometric Partitioning Mode (GPM)] As mentioned above, the unidirectional motion used to generate the predicted samples for the two GPM partitions is obtained directly from the regular merge candidate. If there is no strong correlation between the MVs of spatially / temporally adjacent blocks, the unidirectional MVs derived from the merge candidate may not be accurate enough to capture the true motion of each GPM partition. Motion estimation can provide more accurate motion, but this comes at the expense of significant signaling overhead due to any motion refinement that may be applied on top of the existing unidirectional MVs. Meanwhile, MVD mode, utilized in both the VVC and AVS3 standards, has proven to be an efficient signaling mechanism for reducing MVD signaling overhead. Therefore, combining GPM with MMVD mode may also be beneficial. Such a combination can potentially improve the overall coding efficiency of the GPM tool by providing more accurate MVs, capturing the individual motion of each GPM partition.

[0055] As mentioned above, in both the VVC and AVS3 standards, the GPM mode is only applied to the merge / skip mode. Such a design may be suboptimal in terms of coding efficiency, assuming that all non-merge inter CUs cannot benefit from the flexible non-rectangular partitioning of the GPM. Meanwhile, for the same reasons as above, the unidirectionally predicted motion candidates derived from the regular merge / skip mode are not necessarily accurate enough to capture the true motion of the two geometric partitions. Based on such analysis, extra coding gains can be expected by reasonable extension of the GPM mode to non-merge inter modes (i.e., CUs that explicitly signal their motion information in the bitstream). However, improved MV accuracy comes at the expense of increased signaling overhead. Therefore, to efficiently apply the GPM mode to explicit inter modes, it is important to identify an effective signaling scheme that can minimize signaling costs while providing more accurate MVs for the two geometric partitions.

[0056] [Proposed method] This disclosure proposes a method to further improve the coding efficiency of GPM by applying additional motion refinement on top of the existing unidirectional MVD applied to each GPM partition. The proposed method is named Geometric Partitioning Mode with Motion Vector Refinement (GPM-MVR). Furthermore, in the proposed scheme, motion refinement is signaled in a similar way to one of the existing MMVD designs, i.e., based on a set of predetermined MVD magnitudes and directions of motion refinement.

[0057] In one aspect of the present disclosure, we provide a solution for extending GPM mode to explicit inter mode. For ease of explanation, we name these schemes as geometric partition mode with explicit motion signaling (GPM-EMS). Specifically, to achieve better integration with regular inter mode, the existing motion signaling mechanism, i.e., MVP+MVD, is utilized in the proposed GPM-EMS scheme to specify corresponding unidirectional MVs of two geometric partitions.

[0058] [Geometric partitioning mode with separate motion vector refinement] To improve the coding efficiency of GPM, this section proposes an improved geometric partitioning mode with separate motion vector refinement. Specifically, given a GPM partition, the proposed method first uses the existing syntax elements merge_gpm_idx0 and merge_gpm_idx1 to identify unidirectional motion vectors for two GPM partitions from the existing unipredictive merge candidate list and uses them as base motion vectors. After the two base motion vectors are determined, a set of two new syntax elements is introduced to specify the motion refinement values ​​to be applied separately on the base motion vectors of the two GPM partitions. Specifically, two flags, namely, gpm_mvr_partIdx0_enable_flag and gpm_mvr_partIdx1_enable_flag, initially indicate whether GPM-MVR is applied to the first and second GPM partitions, respectively. When the flag of one GPM partition is equal to 1, the corresponding value of the MVR applied to the base MV of the partition is signaled in MMVD style, i.e., one distance index (indicated by the syntax elements gpm_mvr_partIdx0_distance_idx and gpm_mvr_partIdx1_distance_idx) to specify the size of the MVR and one direction index (indicated by the syntax elements gpm_mvr_partIdx0_direction_idx and gpm_mvr_partIdx1_distance_idx) to specify the direction of the MVR. Table 4 shows the syntax elements introduced by the proposed GPM-MVR method. In Table 4, newly added syntax elements are in italic and bold. [Table 4]

[0059] Based on the proposed syntax elements as shown in Table 4, at the decoder, the final MV used to generate the single-predicted sample for each GPM partition is equal to the sum of the signaled motion vector refinement and the corresponding base MV. In practice, different sets of MVR magnitudes and directions can be predefined and applied to the proposed GPM-MVR scheme, which may provide various tradeoffs between motion vector accuracy and signaling overhead. In one specific example, we propose reusing the eight MVD offsets (i.e., 1 / 4, 1 / 2, 1, 2, 4, 8, 16, and 32 pels) and four MVD directions (i.e., + / -x and y axes) used in the VVC standard for the proposed GPM-MVR scheme. In another example, we apply the existing five MVD offsets (i.e., 1 / 4-, 1 / 2-, 1-, 2-, and 4-pels) and four MVD directions (i.e., + / -x and y axes) used in the AVS3 standard to the proposed GPM-MVR scheme.

[0060] As explained in the "GPM Signaling Design" section, the unidirectional MVs used for two GPM partitions cannot be identical, so one constraint is applied to the existing GPM design, forcing the two unidirectional merge indices to be different. However, in the proposed GPM-MVR scheme, an additional motion refinement is applied on top of the existing GPM unidirectional MV. Therefore, even when the base MVs of the two GPM partitions are identical, the final unidirectional MVs used to predict the two partitions may still be different unless the two motion vector refinement values ​​are the same. Based on the above considerations, when the proposed GPM-MVR scheme is applied, the constraint (which restricts the two unidirectional merge indices to be different) is removed. Furthermore, since the two unidirectional merge indices are allowed to be identical, the same maximum value MaxGPMMergeCand-1 is used for the binarization of both merge_gpm_idx0 and merge_gpm_idx1, where MaxGPMMergeCand is the number of candidates in the unidirectional merge list.

[0061] As analyzed above, when the uni-predictive merge indices (i.e., merge_gpm_idx0 and merge_gpm_idx1) of two GPM partitions are identical, the values ​​of the two motion vector refinements cannot be the same to ensure that the final MVs used for the two partitions are different. Based on such conditions, in one embodiment of the present disclosure, when the uni-predictive merge indices of two GPM partitions are the same (i.e., merge_gpm_idx0 is equal to merge_gpm_idx1), a signaling redundancy elimination method is proposed for using the MVR of the first GPM partition to reduce the signaling overhead of the MVR of the second GPM partition. In one embodiment, the following signaling conditions apply:

[0062] Initially, when the flag gpm_mvr_partIdx0_enable_flag is equal to 0 (i.e., GPM-MVR is not applied to the first GPM split), the flag gpm_mvr_partIdx1_enable_flag is not signaled and is inferred to be 1 (i.e., GPM-MVR is applied to the second GPM split).

[0063] Then, when both flags gpm_mvr_partIdx0_enable_flag and gpm_mvr_partIdx1_enable_flag are equal to 1 (i.e., GPM-MVR applies to two GPM partitions) and gpm_mvr_partIdx0_direction_idx is equal to gpm_mvr_partIdx1_direction_idx (i.e., the MVRs of the two GPM partitions have the same direction), the MVR magnitude of the first GPM partition (i.e., gpm_mvr_partIdx0_distance_idx) is used to predict the MVR magnitude of the second GPM partition (i.e., gpm_mvr_partIdx1_distance_idx). Specifically, if gpm_mvr_partIdx1_distance_idx is smaller than gpm_mvr_partIdx0_distance_idx, its original value is signaled directly. Otherwise (gpm_mvr_partIdx1_distance_idx is greater than gpm_mvr_partIdx0_distance_idx), the value is decremented by 1 before being signaled in the bitstream. On the decoder side, to decode the value of gpm_mvr_partIdx1_distance_idx, if the parsed value is less than gpm_mvr_partIdx0_distance_idx, gpm_mvr_partIdx1_distance_idx is set equal to the parsed value, otherwise (parsed value is greater than or equal to gpm_mvr_partIdx0_distance_idx), gpm_mvr_partIdx1_distance_idx is set to the parsed value plus 1. In such cases, to further reduce overhead, different maximum values ​​MaxGPMMVRDistance-1 and MaxGPMMVRDistance-2 can be used for binarization of gpm_mvr_partIdx0_distance_idx and gpm_mvr_partIdx1_distance_idx, where MaxGPMMVRDistance is the number of magnitude allowed for motion vector refinement.

[0064] In another example, it is proposed to switch the signaling order to gpm_mvr_partIdx0_direction_idx / gpm_mvr_partIdx1_direction_idx and gpm_mvr_partIdx0_distance_idx / gpm_mvr_partIdx1_distance_idx so that the MVR magnitude is signaled before the MVR magnitude. In this way, following the same logic as above, the encoder / decoder can use the MVR direction of the first GPM partition to adjust the signaling of the MVR direction of the second GPM partition. In another embodiment, it is proposed to signal the MVR magnitude and direction of the second GPM partition first and use them to adjust the signaling of the MVR magnitude and direction of the second GPM partition.

[0065] In another embodiment, it is proposed to signal GPM-MVR-related syntax elements before signaling existing GPM syntax elements. Specifically, in such a design, two flags, gpm_mvr_partIdx0_enable_flag and gpm_mvr_partIdx1_enable_flag, are signaled first to indicate whether GPM-MVR applies to the first and second GPM partitions, respectively. When the flag for one GPM partition is equal to 1, the distance index (indicated by the syntax elements gpm_mvr_partIdx0_distance_idx and gpm_mvr_partIdx1_distance_idx) and direction index (indicated by the syntax elements gpm_mvr_partIdx0_direction_idx and gpm_mvr_partIdx1_distance_idx) specify the direction of MVR. Then, the existing syntax elements merge_gpm_idx0 and merge_gpm_idx1 are signaled to identify the two GPM splits, i.e., the unidirectional MV for the base MV. Table 5 illustrates the proposed GPM-MVR signaling scheme. In Table 5, the newly added syntax elements are in italic bold. [Table 5]

[0066] Similar to the signaling method of Table 4, when the GPM-MVR signaling method of Table 5 is applied, certain conditions may apply to ensure that the resulting MVs used for prediction of the two GPM partitions are not identical. Specifically, the following conditions are proposed to constrain the signaling of the uni-predictive merge indices merge_gpm_idx0 and merge_gpm_idx1 depending on the values ​​of MVR applied to the first and second GPM partitions:

[0067] First, when the values ​​of both gpm_mvr_partIdx0_enable_flag and gpm_mvr_partIdx1_enable_flag are equal to 0 (i.e., GPM-MVR is disabled for both GPM splits), the values ​​of merge_gpm_idx0 and merge_gpm_idx1 cannot be the same;

[0068] Second, when gpm_mvr_partIdx0_enable_flag is equal to 1 (i.e., GPM-MVR is enabled for the first GPM split) and gpm_mvr_partIdx1_enable_flag is equal to 0 (i.e., GPM-MVR is disabled for the second GPM split), the values ​​of merge_gpm_idx0 and merge_gpm_idx1 are allowed to be identical.

[0069] Third, when gpm_mvr_partIdx0_enable_flag is equal to 0 (i.e., GPM-MVR is disabled for the first GPM split) and gpm_mvr_partIdx1_enable_flag is equal to 1 (i.e., GPM-MVR is enabled for the second GPM split), the values ​​of merge_gpm_idx0 and merge_gpm_idx1 are allowed to be identical.

[0070] Fourth, when the values ​​of both gpm_mvr_partIdx0_enable_flag and gpm_mvr_partIdx1_enable_flag are equal to 1 (i.e., GPM-MVR is disabled for both of the two GPM splits), the determination of whether the values ​​of merge_gpm_idx0 and merge_gpm_idx1 are identical depends on the values ​​of the MVR (as indicated by gpm_mvr_partIdx0_direction_idx and gpm_mvr_partIdx0_distance_idx, and gpm_mvr_partIdx1_direction_idx and gpm_mvr_partIdx1_distance_idx) that apply to the two GPM splits. If the values ​​of the two MVRs are equal, then merge_gpm_idx0 and merge_gpm_idx1 are not allowed to be identical. Otherwise (if the two MVR values ​​are not equal), the values ​​of merge_gpm_idx0 and merge_gpm_idx1 will be the same.

[0071] In the above four cases, when the values ​​of merge_gpm_idx0 and merge_gpm_idx1 are not allowed to be identical, the index value of one partition can be used as a predictor of the index value of the other partition. One method proposes to signal merge_gpm_idx0 first and use its value to predict merge_gpm_idx1. Specifically, at the encoder, when merge_gpm_idx1 is greater than merge_gpm_idx0, the value of merge_gpm_idx1 sent to the decoder is reduced by 1. At the decoder, when the received value of merge_gpm_idx1 is greater than or equal to the received value of merge_gpm_idx0, the value of merge_gpm_idx1 is increased by 1. Another method proposes to signal merge_gpm_idx1 first and use its value to predict merge_gpm_idx0. Thus, in such a case, at the encoder, when merge_gpm_idx0 is greater than merge_gpm_idx1, the value of merge_gpm_idx0 sent to the decoder is reduced by 1. At the decoder, when the received value of merge_gpm_idx0 is greater than or equal to the received value of merge_gpm_idx1, the value of merge_gpm_idx0 is increased by 1. Furthermore, similar to existing GPM signaling designs, different maximum values ​​MaxGPMMergeCand-1 and MaxGPMMergeCand-2 can be used for binarization of the first and second index values ​​according to the signaling order, respectively. On the other hand, when the values ​​of merge_gpm_idx0 and merge_gpm_idx1 are allowed to be the same because there is no correlation between the two index values, the same maximum value MaxGPMMergeCand-1 is used to binarize the two index values.

[0072] In the above method, different maximum values ​​can be applied to the binarization of merge_gpm_idx0 and merge_gpm_idx1 to reduce the signaling cost. The selection of the corresponding maximum value depends on the decoded value of MVR (as indicated by gpm_mvr_partIdx0_enable, gpm_mvr_partIdx1_enable, gpm_mvr_partIdx0_direction_idx, gpm_mvr_partIdx1_direction_idx, gpm_mvr_partIdx0_distance_idx, and gpm_mvr_partIdx1_distance_idx). Such a design introduces undesirable parsing dependencies between different GPM syntax elements, which may affect the overall parsing. To solve this problem, in one embodiment, it is proposed to always use one and the same maximum value (e.g., MaxGPMMergeCand-1) to parse the values ​​of merge_gpm_idx0 and merge_gpm_idx1. When such a method is used, one bitstream compatibility constraint may be used to prevent two decoded MVs of two GPM partitions from being the same. Alternatively, such a non-identity constraint may be removed so that the decoded MVs of two GPM partitions are allowed to be the same. On the other hand, when such a method is applied (i.e., when the same maximum value is used for merge_gpm_idx0 and merge_gpm_idx1), there is no syntax dependency between merge_gpm_idx0 / merge_gpm_idx1 and other GPM-MVR syntax elements. Therefore, the order in which these syntax elements are signaled is no longer important.In one example, it is proposed to move the merge_gpm_idx0 / merge_gpm_idx1 signaling before the gpm_mvr_partIdx0_enable, gpm_mvr_partIdx1_enable, gpm_mvr_partIdx0_direction_idx, gpm_mvr_partIdx1_direction_idx, gpm_mvr_partIdx0_distance_idx, and gpm_mvr_partIdx1_distance_idx signaling.

[0073] [Geometric division mode with symmetric motion vector refinement] In the GPM-MVR method described above, two separate MVR values ​​are signaled, with one MVR value applied to refine the base MV for only one GPM partition. Such a method can be efficient in terms of improving prediction accuracy by enabling independent motion refinement for each GPM partition. However, such flexible motion refinement comes at the expense of increased signaling overhead, given that two different sets of GMP-MVR syntax elements need to be transmitted from the encoder to the decoder. To reduce signaling overhead, this section proposes a geometric partitioning mode with symmetric motion vector refinement. Specifically, in this method, one single MVR value is signaled for one GPM CU and is used for both two GPM partitions according to the symmetric relationship between the Picture Order Count (POC) values ​​of the current picture and the reference pictures associated with the two GPM partitions. Table 6 shows the syntax elements when the proposed method is applied. In Table 6, newly added syntax elements are in bold italics. [Table 6]

[0074] As shown in Table 6, after the base MVs of the two GPM partitions are selected (based on merge_gpm_idx0 and merge_gpm_idx1), one flag, gpm_mvr_enable_flag, is signaled to indicate whether the GPM-MVR mode is applied to the current GPM CU. When the flag is equal to 1, it indicates that motion refinement is applied to enhance the base MVs of the two GPM partitions. Otherwise (when the flag is equal to zero), it indicates that motion refinement is not applied to either of the two partitions. If the GPM-MVR mode is disabled, additional syntax elements are further signaled to specify the value of the applied MVR by the direction index, gpm_mvr_direction_idx, and the magnitude index, gpm_mvr_distance_idx. Furthermore, similar to the MMVD mode, the meaning of the MVR code may change according to the relationship between the POC of the current picture and the two reference pictures of the GPM partition. Specifically, when both POCs of two reference pictures are greater or less than the POC of the current picture, the signaled code is the code of the MVR added to both two base MVs. Otherwise (when the POC of one reference picture is greater than the current picture and the POC of the other reference picture is less than the current picture), the signaled code is applied to the MVR of the first GPM partition, and the opposite code is applied to the MVR of the second GPM partition. In Table 6, the values ​​of merge_gpm_idx0 and merge_gpm_idx1 are allowed to be the same.

[0075] In another embodiment, it is proposed to signal two different flags to separately control the enabling / disabling of GPM-MVR mode for the two GPM splits. However, when GPM-MVR mode is disabled, only one MVR is signaled based on the syntax elements gpm_mvr_direction_idx and gpm_mvr_distance_idx. The corresponding syntax table for such a signaling method is shown in Table 7. In Table 7, newly added syntax elements are in italic bold. [Table 7]

[0076] The values ​​of merge_gpm_idx0 and merge_gpm_idx1 are allowed to be identical when the signaling method of Table 7 is applied. However, to ensure that the resulting MVs applied to the two GPM partitions are not redundant, when flag gpm_mvr_partIdx0_enable_flag is equal to 0 (i.e., GPM-MVR is not applied to the first GPM partition), flag gpm_mvr_partIdx1_enable_flag is not signaled and is inferred to be 1 (i.e., GPM-MVR is applied to the second GPM partition).

[0077] [Indications for acceptable MVR for GPM-MVR] In the above-described GPM-MVR method, a fixed group of MVR values ​​is used for GPM CUs in both the encoder and decoder within a video sequence. This design is not optimal for video content with high resolution or intense motion. In these cases, the MV tends to be much larger, so that the fixed MVR value may not be optimal for capturing the actual motion of these blocks. To further improve the coding efficiency of the GPM-MVR mode, this disclosure proposes supporting adaptation of the MVR values ​​allowed to be selected by the GPM-MVR mode at various coding levels, such as the sequence level, picture / slice picture level, and coding block group level. For example, multiple MVR sets and corresponding codewords can be derived offline according to the specific motion characteristics of different video sequences. The encoder can select the best MVR set and signal the corresponding index of the selected set to the decoder.

[0078] [GPM-MVR Encoder speed-up logic for velocity distortion optimization] For the proposed GPM-MVR scheme, to determine the optimal MVR for each GPM partition, the encoder may need to test the rate-distortion cost of each GPM partition multiple times, varying the applied MVR value respectively. This may significantly increase the coding complexity of the GPM mode. To address the coding complexity issue, this section proposes the following fast coding logic:

[0079] First, due to the quad / binary / ternary block partitioning structure applied to VVC and AVS3, the same coding block can be checked during the rate-distortion optimization (RDO) process, each of which is partitioned through a different partitioning path. In current VTM / HPM encoder implementations, whenever the same CU is obtained through different block partitioning combinations, GPM and GPM-MVR modes are always tested along with other inter- and intra-coding modes. Generally speaking, for different partitioning paths, only neighboring blocks of a CU may differ, which should have a relatively small impact on the optimal coding mode selected by a CU. Based on this consideration, to reduce the total number of GPM RDOs applied, it is proposed to memorize the decision on whether GPM mode is selected when the RD cost of a CU is checked for the first time. Subsequently, when the same CU is checked again (through a different partitioning path) by the RDO process, the RD cost of GPM (including GPM-MVR) is checked only if GPM was initially selected for the CU. When a GPM is not selected for the initial RD check of one CU, and the same CU is achieved via another split path, only the GPM (without GPM-MVR) is tested. Alternatively, when a GPM is not selected for the initial RD check of one CU, both the GPM and GPM-MVR are not tested when the same CU is achieved via another split path.

[0080] In another method, a soft-decision method is applied to skip the GPM (including GPM-MVR) RD check when the same CU is coded multiple times. For example, when a CU is coded for the first time, the encoder can compare the RD cost of the GPM mode (including GPM-MVR) with the cost of the best mode. Then, when the same CU is coded through another split pass, the RD check of the GPM mode (including GPM-MVR) is performed only if the GPM RD cost is not worse than the RD cost of the best mode multiplied by a threshold in the first pass.

[0081] Second, to reduce the number of GPM splits for GPM-MVR mode, when the RD cost of one CU is checked for the first time, it is proposed to keep the first M GPM split modes without the smallest RD cost. Then, when the same CU is checked by the RDO process (by another split path), only those M GPM split modes are tested for GPM-MVR mode.

[0082] Third, to reduce the number of GPM partitions tested for the first RDO process, it is proposed to first calculate the total absolute difference when using various uni-predictive merge candidates for two GPM partitions. Then, for each GPM partition under one specific partition mode, select the best uni-predictive merge candidate with the smallest SAD value, and calculate the corresponding SAD value of the partition mode, which is equal to the sum of the SAD values ​​of the best uni-predictive merge candidates for the two GPM partitions. Then, for the next RD process, only the first N partition modes with the best SAD values ​​for the previous step are tested for the GPM-MVR mode.

[0083] [Geometric segmentation using explicit motor signaling] In this part, several methods are proposed to extend the GPM mode to regular inter-mode bidirectional prediction, where the two unidirectional MVs of the GPM mode are explicitly signaled from the encoder to the decoder.

[0084] In the first solution (Solution 1), it is proposed to fully reuse the existing motion signaling of bidirectional prediction to signal two unidirectional MVs in GPM mode. Table 8 shows a modified syntax table of the proposed scheme, with newly added syntax elements in bold italics. As shown in Table 8, in this solution, all existing syntax elements for signaling L0 and L1 motion information are fully reused to indicate the unidirectional MVs of the two GPM partitions, respectively. Furthermore, it is assumed that the L0 MV is always associated with the first GPM partition, and the L1 MV is always associated with the second GPM partition. Meanwhile, in Table 8, the inter prediction syntax, i.e., inter_pred_idc, is signaled before the GPM flag (i.e., gpm_flag), and the value of inter_pred_idc can be used to condition the presence of gpm_flag. Specifically, the flag gpm_flag only needs to be signaled when inter_pred_idc is equal to PRED_bi (i.e., bi-prediction) and both inter_affine_flag and sym_mvd_flag are equal to 0 (i.e., the CU is not coded in either affine or SMVD mode). When the flag gpm_flag is not signaled, its value is always inferred to be 0 (i.e., GPM mode is disabled). When gpm_flag is 1, another syntax element gpm_partition_idx is further signaled to indicate the selected GPM mode (out of a total of 64 GPM partitions) for the current CU. [Table 8]

[0085] Another method is proposed to place the signaling of the flag gpm_flag before other inter-signaling syntax elements, so that the value of gpm_flag can be used to determine whether other inter-syntax elements need to be present or not. Table 9 shows the corresponding syntax table when such a method is applied, where the newly added syntax elements are in italic bold. As can be seen, gpm_flag is signaled first in Table 9. When gpm_flag is equal to 1, the corresponding signaling of inter_pred_idc, inter_affine_flag, and sym_mvd_flag can be bypassed. Alternatively or additionally, the corresponding values ​​of the three syntax elements can be inferred as PRED_BI, 0, and 0, respectively. [Table 9]

[0086] In both Table 8 and Table 9, SMVD mode and GPM mode cannot be combined. Another example is proposed to enable SMVD mode when the current CU is coded by GPM mode. When such a combination is allowed, by following the same design of SMVD, the MVDs of the two GPM partitions are assumed to be symmetric, such that only the MVD of the first GPM partition needs to be signaled and the MVD of the second GPM partition is always symmetric with respect to the first MVD. If such a method is applied, the corresponding signaling condition of sym_mvd_flag on gpm_flag can be removed.

[0087] As mentioned above, the first solution always assumes that L0 MVs are used for the first GPM partition and L1 MVs are used for the second GPM partition. Such a design may not be optimal in the sense that it prohibits MVs of two GPM partitions from coming from the same prediction list (either L0 or L1). To solve this problem, an alternative GPM-EMS scheme, Solution 2, is proposed using a signaling design as shown in Table 10. In Table 10, newly added syntax elements are in bold italics. As shown in Table 10, the flag gpm_flag is signaled first. When the flag is equal to 1 (i.e., GPM is enabled), the syntax gpm_partition_idx is signaled to specify the selected GPM mode. Then, one additional flag gpm_pred_dir_flag0 is signaled to indicate the corresponding prediction list from which the MVs of the first GPM partition come. When the flag gpm_pred_dir_flag0 is equal to 1, it indicates that the MVs of the first GPM partition come from L1; otherwise (the flag is equal to 0), it indicates that the MVs of the first GPM partition come from L0. Then, the existing syntax elements ref_idx_L0 MVp_l0_flag and mvd_coding() are utilized to signal the values ​​of the reference picture index, mvp index, and MVD of the first GPM partition. Meanwhile, similar to the first partition, another syntax element gpm_pred_dir_flag1 is introduced to select the corresponding prediction list of the second GPM partition, and then the existing syntax elements ref_idx_L1 MVp_l1_flag and mvd_coding() are used to derive the MVs of the second GPM partition. [Table 10]

[0088] Finally, considering that the GPM mode consists of two unidirectional splits (excluding blending samples on the split edges), it should be mentioned that some existing coding tools in VVC and AVS3 that are specifically designed for bidirectional prediction, such as bidirectional optical flow, decoder-side motion vector refinement (DMVR), and CU-weighted bidirectional prediction (BCW), can be automatically bypassed when the proposed GPM-EMS scheme is disabled for one inter-CU. For example, when one of the proposed GPM-EMS is enabled for one CU, if BCW cannot be applied to the GPM mode, the corresponding BCW weight does not need to be further signaled for the CU to reduce signaling overhead.

[0089] [Combination of GPM-MVR and GPM-EMS] In this section, we propose combining GPM-MVR and GPM-EMS for one CU with geometric partitioning. Specifically, unlike either GPM-MVR or GPM-EMS, in which only one of merge-based motion signaling or explicit signaling can be applied to signal the unidirectionally predicted MVs of two GPM partitions, the proposed scheme allows 1) one partition using GPM-MVR-based motion signaling and the other partition using GPM-EMS-based motion signaling, 2) two partitions using GPM-MVR-based motion signaling, or 3) two partitions using GPM-EMS-based motion signaling. Using the GPM-MVR signaling in Table 4 and the GPM-EMS in Table 10, Table 11 shows the corresponding syntax table after the proposed GPM-MVR and GPM-EMS are combined. In Table 11, newly added syntax elements are in bold italics. As shown in Table 11, two additional syntax elements, gpm_merge_flag0 and gpm_merge_flag1, are introduced for partitions #1 and #2, respectively, which specify whether the corresponding partition uses GPM-MVR-based merge signaling or GPM-EMS-based explicit signaling. When the flag is 1, it means that GPM-MVR-based signaling is enabled for the partition whose GPM-part predicted motion is signaled via merge_gpm_idxX, gpm_mvr_partIdxX_enabled_flag, gpm_mvr_partIdxX_direction_idx, and gpm_mvr_partIdxX_distance_idx, where X=0, 1. Otherwise, if the flag is 0, it means that the uni-predictive motion of the partition is explicitly signaled by the GPM-EMS scheme using the syntax elements gpm_pred_dir_flagX, ref_idx_lX, mvp_lX_flag and mvd_lX, where X=0, 1. [Table 11]

[0090] The above methods may be implemented using an apparatus including one or more circuits, including an application specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field programmable gate array (FPGA), a controller, a microcontroller, a microprocessor, or other electronic components. The apparatus may use the circuits in combination with other hardware or software components to perform the above methods. Each module, sub-module, unit, or sub-unit disclosed above may be implemented at least in part using one or more circuits.

[0091] 9 illustrates a computing environment (or computing device) 910 coupled to a user interface 960. The computing environment 910 may be part of a data processing server. In some embodiments, the computing device 910 may perform any of the various methods or processes (e.g., encoding / decoding methods or processes) described above in accordance with various examples of this disclosure. The computing environment 910 may include a processor 920, a memory 940, and an I / O interface 950.

[0092] The processor 920 typically controls the overall operation of the computing environment 910, such as operations related to display, data acquisition, data communication, and image processing. The processor 920 may include one or more processors for executing instructions to perform all or some of the steps in the methods described above. Additionally, the processor 920 may include one or more modules that facilitate interaction between the processor 920 and other components. The processor may be a central processing unit (CPU), a microprocessor, a single-chip machine, a GPU, etc.

[0093] The memory 940 is configured to store various types of data to support the operation of the computing environment 910. The memory 940 may include predefined software 942. Examples of such data include instructions for any applications or methods operating on the computing environment 910, video data sets, image data, etc. The memory 940 may be implemented using any type of volatile or non-volatile memory device, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disk, or a combination thereof.

[0094] The I / O interface 950 provides an interface between the processor 920 and a peripheral interface module, such as a keyboard, a click wheel, buttons, etc. The buttons may include, but are not limited to, a home button, a start scan button, and a stop scan button. The I / O interface 950 may be coupled to an encoder and a decoder.

[0095] In some embodiments, a non-transitory computer-readable storage medium is also provided comprising a plurality of programs, such as contained in memory 940, executable by processor 920 in computing environment 910 for performing the methods described above. For example, the non-transitory computer-readable storage medium may be ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc.

[0096] A non-transitory computer-readable storage medium has stored therein a plurality of programs for execution by a computing device having one or more processors, the plurality of programs, when executed by the one or more processors, causing the computing device to perform the above-described method for motion prediction.

[0097] In some embodiments, the computing environment 910 may be implemented using one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), graphical processing units (GPUs), controllers, microcontrollers, microprocessors, or other electronic components to perform the methods described above.

[0098] FIG. 7 is a flowchart illustrating a method for decoding video blocks in a GPM according to an example of this disclosure.

[0099] In step 701, the processor 920 may divide the video block into first and second geometric divisions.

[0100] In step 702, processor 920 may obtain a first predictive list for a first geometric division and a second predictive list for a second geometric division.

[0101] In some embodiments, the first predictive list may be list L0 and the second predictive list may be list L1.

[0102] In step 703, the processor 920 may obtain MVs for the first and second geometric divisions by applying GPM with EMS to the first and second geometric divisions based on the first and second prediction lists.

[0103] In some embodiments, applying GPM with EMS to the first and second geometric divisions may be performed according to Solution Method 2, as shown in Table 10.

[0104] In some embodiments, processor 920 may receive one or more first syntax elements for the first geometric partition. The one or more first syntax elements may include a reference picture index syntax element ref_idx_l0, an MVP index flag mvp_l0_flag, and a first MVD coding syntax element mvd_coding, where the reference picture index ref_idx_l0 may indicate a reference picture index of the first geometric partition, the MVP index flag mvp_l0_flag may indicate an MVP index of the first geometric partition, and the first MVD coding syntax element mvd_coding may indicate an MVD of the first geometric partition.

[0105] In some embodiments, processor 920 may receive one or more second syntax elements for the second geometric partition. The one or more second syntax elements may include a reference picture index syntax element ref_idx_l1, an MVP index flag mvp_l1_flag, and a second MVD coding syntax element mvd_coding, where the reference picture index ref_idx_l1 may indicate a reference picture index of the second geometric partition, the MVP index flag mvp_l1_flag may indicate an MVP index of the second geometric partition, and the second MVD coding syntax element mvd_coding may indicate an MVD of the second geometric partition.

[0106] In some embodiments, the processor 920 may obtain a first prediction list and a second prediction list by receiving a first GPM prediction direction flag gpm_pred_dir_flag0 and a second GPM prediction direction flag gpm_pred_dir_flag1, where the first GPM prediction direction flag gpm_pred_dir_flag0 may indicate the corresponding prediction list from which the MV of the first geometric division is derived, and the second GPM prediction direction flag gpm_pred_dir_flag1 may indicate the corresponding prediction list from which the MV of the second geometric division is derived.

[0107] In some embodiments, in response to processor 920 determining that the first GPM prediction direction flag gpm_pred_dir_flag0 is equal to 1, it may determine that the MV of the first geometric partition is from list L1, and in response to determining that the first GPM prediction direction flag gpm_pred_dir_flag0 is equal to 0, it may determine that the MV of the first geometric partition is from list L0.

[0108] In some embodiments, in response to the processor 920 determining that the second GPM prediction direction flag gpm_pred_dir_flag1 is equal to 1, it may determine that the MV of the second geometric partition is from list L0, and in response to determining that the second GPM prediction direction flag gpm_pred_dir_flag1 is equal to 0, it may determine that the MV of the second geometric partition is from list L1.

[0109] In some embodiments, applying GPM with EMS to the first and second geometric partitions may be performed according to Solution 1 (Option 1) shown in Table 8 and Solution 1 (Option 2) shown in Table 9.

[0110] In some embodiments, processor 920 may receive one or more syntax elements that condition a GPM flag, gpm_flag, which may indicate whether GPM mode is disabled for the first geometric division or the second geometric division.

[0111] In some embodiments, the one or more syntax elements may include an inter-prediction syntax element inter_pred_idc that indicates whether bi-prediction is applied and an affine mode syntax element inter_affine_flag that indicates whether the video block is coded in affine mode. Processor 920 may receive a GPM flag gpm_flag in response to determining that the inter-prediction syntax element inter_pred_idc indicates that bi-prediction is applied and the affine mode syntax element inter_affine_flag indicates that the video block is not coded in affine mode.

[0112] In some embodiments, the one or more syntax elements may further include an SMVD syntax element sym_mvd_flag indicating whether the video block is coded in SMVD mode. In response to determining that the inter prediction syntax element inter_pred_idc indicates that bi-prediction is applied, the processor 920 may receive the GPM flag gpm_flag, the affine mode syntax element inter_affine_flag indicating that the video block is not coded in affine mode, and the SMVD syntax element sym_mvd_flag indicating that the video block is not coded in SMVD mode.

[0113] In some embodiments, in response to determining that the GPM flag gpm_flag is equal to 1, the processor 920 may receive a GPM partition syntax element gpm_partition_idx that indicates a selected GPM mode for the video block.

[0114] In some embodiments, in response to determining that the GMP flag gpm_flag is not signaled, the processor 920 may infer that the value of the GMP flag gpm_flag is equal to 0.

[0115] In some embodiments, processor 920 may receive a GPM flag gpm_flag to condition one or more syntax elements, where the GPM flag gpm_flag indicates whether the GPM mode is valid for the first geometric partition or the second geometric partition. The one or more syntax elements may include an inter prediction syntax element inter_pred_idc that indicates whether bi-prediction is applied and an affine mode syntax element inter_affine_flag that indicates whether the video block is coded in affine mode.

[0116] In some embodiments, in response to processor 920 determining that the GPM flag gpm_flag is equal to one, processor 920 may bypass the inter prediction syntax element inter_pred_idc and the affine mode syntax element inter_affine_flag.

[0117] In some embodiments, the one or more syntax elements may further include an SMVD syntax element sym_mvd_flag that indicates whether the video block is coded in SMVD mode. In response to determining that the GPM flag gpm_flag is equal to 1, processor 920 may bypass the inter prediction syntax element inter_pred_idc, the affine mode syntax element inter_affine_flag, and the SMVD syntax element sym_mvd_flag.

[0118] In some embodiments, the processor 920 may infer the inter prediction syntax element inter_pred_idc as PRED_bi indicating that bi-prediction is applied, the affine mode syntax element inter_affine_flag as 0 indicating that the video block is not coded in affine mode, and in response to determining that the GMP flag is equal to 1, the processor 920 may infer the SMVD syntax element sym_mvd_flag as 0 indicating that the video block is not coded in SMVD mode.

[0119] In step 704, the processor 920 may obtain predicted samples for the first and second geometric divisions based on the MV.

[0120] FIG. 8 is a flowchart illustrating a method for decoding a video block in a GPM according to an example of this disclosure.

[0121] In step 801, the processor 920 may divide the video block into first and second geometric divisions.

[0122] In step 802, processor 920 may obtain a first predictive list for a first geometric division and a second predictive list for a second geometric division.

[0123] In some embodiments, the first predictive list may be list L0 and the second predictive list may be list L1.

[0124] In step 803, the processor 920 may obtain MVs for the first and second geometric divisions by applying GPM with MVR or GPM with EMS to the first and second geometric divisions based on the first and second prediction lists.

[0125] In some embodiments, the processor 920 may obtain the MVs for the first and second geometric partitions by applying a GPM with MVR to the first and second geometric partitions based on the first and second prediction lists, applying a GPM with EMS to the first and second geometric partitions based on the first and second prediction lists, or applying a GPM with MVR to either the first or second geometric partitions and applying a GPM with EMS to either the second or first geometric partitions based on the first and second prediction lists.

[0126] In some embodiments, the processor 920 further receives a first GPM merge flag gpm_merge_flag0 for the first geometric partition, specifying a GPM with MVR or a GPM with EMS to be applied to the first geometric partition, and receives a second GPM merge flag gpm_merge_flag1 for the second geometric partition, specifying a GPM with MVR or a GPM with EMS to be applied to the second geometric partition.

[0127] In some embodiments, processor 920 may further enable the GPM having the MVR for the first geometric partition and receive a plurality of first syntax elements for the first geometric partition in response to determining that the first GPM merge flag gpm_merge_flag0 is equal to 1, where the plurality of first syntax elements may include merge_gpm_idx0, gpm_mvr_partIdx0_enabled_flag, gpm_mvr_partIdx0_direction_idx, and gpm_mvr_partIdx0_distance_id.

[0128] In some embodiments, the processor 920 may further enable GPM with EMS for the first geometric partition and explicitly receive a plurality of second syntax elements for the first geometric partition in response to determining that the first GPM merge flag gpm_merge_flag0 is equal to 0, where the plurality of second syntax elements may include a GPM prediction direction flag gpm_pred_dir_flag0, a reference picture index ref_idx_l0, an MVP index flag mvp_l0_flag, and mvd_l0.

[0129] In some embodiments, the processor 920 may further enable a GPM having an MVR for the second geometric partition and, in response to determining that the second GPM merge flag gpm_merge_flag1 is equal to 1, receive a plurality of first syntax elements for the second geometric partition, where the plurality of first syntax elements may include merge_gpm_idx1, gpm_mvr_partIdx1_enabled_flag, gpm_mvr_partIdx1_direction_idx, and gpm_mvr_partIdx1_distance.

[0130] In some embodiments, the processor 920 may further enable a GPM having an EMS for the second geometric partition and, in response to determining that the second GPM merge flag gpm_merge_flag1 is equal to 0, explicitly receive a plurality of second syntax elements for the second geometric partition, where the plurality of second syntax elements may include a GPM prediction direction flag gpm_pred_dir_flag1, a reference image index ref_idx_L1 MVP index flag mvp_l1_flag, and mvd_l1.

[0131] In step 804, the processor 920 may obtain predicted samples for the first and second geometric divisions based on the MV.

[0132] In some embodiments, an apparatus for decoding video blocks in a GPM is provided, the apparatus including a processor 920 and a memory 940 configured to store instructions executable by the processor, the processor being configured, upon execution of the instructions, to perform the method shown in FIG.

[0133] In some other embodiments, a non-transitory computer-readable storage medium is provided having instructions stored therein that, when executed by processor 920, cause the processor to perform a method such as that shown in FIG.

[0134] Other embodiments of the present disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure in accordance with its general principles, including departures from the present disclosure as come within known or customary practice in the art. It is intended that the specification and examples be considered as illustrative only.

[0135] It will be understood that the present disclosure is not limited to the exact examples set forth and illustrated in the accompanying drawings, and that various modifications and changes can be made without departing from the scope thereof.

Claims

1. Partitioning the video block into first and second geometric partitions; obtaining a first predictive list for a first geometric division and a second predictive list for a second geometric division; obtaining motion vectors (MVs) for first and second geometric partitions by applying a GPM with explicit motion signaling (EMS) to the first and second geometric partitions based on the first prediction list and the second prediction list; obtaining prediction samples for the first and second geometric partitions based on MVs; 1. A method for decoding a video block in geometric partition mode (GPM), comprising:

2. receiving one or more first syntax elements for a first geometric partition, the first syntax element comprising a reference picture index syntax element ref_idx_l0, a motion vector prediction (MVP) index flag mvp_l0_flag, and a first motion vector differential (MVD) coding syntax element mvd_coding, wherein the reference picture index ref_idx_l0 indicates a reference picture index for the first geometric partition, the MVP index flag mvp_l0_flag indicates an MVP index for the first geometric partition, and the first MVD coding syntax element mvd_coding indicates an MVD for the first geometric partition; 2. The method of claim 1, further comprising: receiving one or more second syntax elements for a second geometric partition, the second syntax elements comprising a reference picture index syntax element ref_idx_l1, a motion vector prediction (MVP) index flag mvp_l1_flag, and a second MVD coding syntax element mvd_coding; and further applying the GPM by an EMS, wherein the reference picture index ref_idx_l1 indicates a reference picture index for the second geometric partition, the MVP index flag mvp_l1_flag indicates an MVP index for the second geometric partition, and the second MVD coding syntax element mvd_coding indicates an MVD for the second geometric partition.

3. The method of claim 1 , wherein the first predictive list is list L0 and the second predictive list is list L1.

4. obtaining the first predictive list for the first geometric division and obtaining the second predictive list for the second geometric division; receiving a first GPM prediction direction flag gpm_pred_dir_flag0 indicating that the MVs of the first geometric partition are from a corresponding prediction list; and receiving a second GPM prediction direction flag gpm_pred_dir_flag1 indicating that MVs of the second geometric partition are from a corresponding prediction list.

5. In response to determining that the first GPM prediction direction flag gpm_pred_dir_flag0 is equal to 1, determining that the MVs of the first geometric partition are from list L1; In response to determining that the first GPM prediction direction flag gpm_pred_dir_flag0 is equal to 0, determining that the MVs of the first geometric partition are from list L0; The method of claim 4 further comprising:

6. in response to determining that the second GPM prediction direction flag gpm_pred_dir_flag1 is equal to 1, determining that the MVs of the second geometric partition are from list L0; In response to determining that the second GPM prediction direction flag gpm_pred_dir_flag1 is equal to 0, determining that the MVs of the second geometric partition are from list L1; The method of claim 4 further comprising:

7. The method of claim 1 , further comprising receiving one or more syntax elements that condition a GPM flag gpm_flag that indicates whether GPM mode is enabled for the first geometric division or the second geometric division.

8. the one or more syntax elements include an inter prediction syntax element inter_pred_idc that indicates whether bi-prediction is applied, and an affine mode syntax element inter_affine_flag that indicates whether the video block is coded in affine mode; 8. The method of claim 7, further comprising receiving a GPM flag gpm_flag in response to determining that the inter prediction syntax element inter_pred_idc indicates that bi-prediction is applied and the affine mode syntax element inter_affine_flag indicates that the video block is not coded in affine mode.

9. the one or more syntax elements further comprise a symmetric motion vector differential (SMVD) syntax element, sym_mvd_flag, that indicates whether the video block is coded in SMVD mode; 9. The method of claim 8, further comprising receiving a GPM flag gpm_flag in response to determining that the inter prediction syntax element inter_pred_idc indicates that bi-prediction is applied, the affine mode syntax element inter_affine_flag indicates that the video block is not coded in affine mode, and the SMVD syntax element sym_mvd_flag indicates that the video block is not coded in SMVD mode.

10. 10. The method of claim 9, further comprising: in response to determining that the GPM flag gpm_flag is equal to one, receiving a GPM partition syntax element gpm_partition_idx indicating a selected GPM mode for the video block.

11. 10. The method of claim 9, further comprising inferring a value of the GMP flag gpm_flag equal to 0 in response to determining not to signal the GMP flag gpm_flag.

12. 2. The method of claim 1, further comprising receiving a GPM flag gpm_flag that conditions one or more syntax elements, the GPM flag gpm_flag indicating whether the GPM mode is valid for the first geometric partition or the second geometric partition.

13. the one or more syntax elements include an inter prediction syntax element inter_pred_idc that indicates whether bi-prediction is applied, and an affine mode syntax element inter_affine_flag that indicates whether the video block is coded in affine mode; 13. The method of claim 12, the method further comprising, in response to determining that the GPM flag gpm_flag is equal to one, bypassing the inter prediction syntax element inter_pred_idc and the affine mode syntax element inter_affine_flag.

14. the one or more syntax elements further comprise a symmetric motion vector differential (SMVD) syntax element, sym_mvd_flag, that indicates whether the video block is coded in SMVD mode; 14. The method of claim 13, wherein the method further comprises, in response to determining that the GPM flag gpm_flag is equal to one, bypassing the inter prediction syntax element inter_pred_idc and the affine mode syntax element inter_affine_flag, and the SMVD syntax element sym_mvd_flag.

15. 15. The method of claim 14, further comprising: in response to determining that the GMP flag gpm_flag is equal to 1, inferring an inter prediction syntax element inter_pred_idc as PRED_BI, indicating that bi-prediction is applied; inferring an affine mode syntax element inter_affine_flag as 0, indicating that the video block is not coded in affine mode; and inferring an SMVD syntax element sym_mvd_flag as 0, indicating that the video block is not coded in SMVD mode.

16. Partitioning the video block into first and second geometric partitions; obtaining a first predictive list for a first geometric division and a second predictive list for a second geometric division; obtaining motion vectors (MVs) for the first geometric partition and the second geometric partition by applying a GPM with explicit motion signaling (EMS) to the first geometric partition and the second geometric partition based on the first prediction list and the second prediction list; obtaining prediction samples for the first and second geometric partitions based on MVs; 1. A method for decoding a video block in geometric partition mode (GPM), comprising:

17. one or more processors; a non-transitory computer-readable storage medium configured to store instructions executable by one or more processors; Apparatus for video coding, wherein the one or more processors are configured to, upon execution of the instructions, perform the method of any one of claims 1 to 16.

Images