Location-dependent storage of motion information

Abstract

In one embodiment, a motion candidate list is inserted into a bitstream representation of a video block, the bitstream representation of the video block being coded using a geometry partitioning mode, and the motion candidate list is inserted into a bitstream representation of the video block.

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS Under applicable patent laws and / or regulations under the Paris Convention, this application is intended to timely claim priority to and the benefit of International Patent Application No. PCT / CN2018 / 114057, filed November 6, 2018, International Patent Application No. PCT / CN2019 / 087372, filed May 17, 2019, International Patent Application No. PCT / CN2019 / 090127, filed June 5, 2019, and International Patent Application No. PCT / CN2019 / 092151, filed June 20, 2019. For all purposes under U.S. law, the entire disclosures of the above applications are incorporated by reference as part of the disclosure herein.

[0002] The present specification relates to video encoding techniques, systems and devices. [Background technology]

[0003] Despite advances in video compression, digital video remains the largest bandwidth user on the Internet and other digital communications networks. The bandwidth demands for digital video use are expected to continue to grow as the number of connected user devices capable of receiving and displaying video increases. Summary of the Invention

[0004] Devices, systems, and methods related to digital video coding, including geometry partitioning, are described. The described methods may be applied to both existing video coding standards (e.g., High Efficiency Video Coding (HEVC) and / or Universal Video Coding (VVC)) and future video coding standards or video codecs.

[0005] In one exemplary aspect, the disclosed techniques may be used to provide a method of video processing that includes determining an order for inserting motion candidates into a motion candidate list based on a priority rule for converting between a current block of a video and a bitstream representation of the video, the bitstream representation being encoded using a geometry partitioning mode, and performing the conversion based on the determination and the motion candidate list.

[0006] In another representative aspect, the disclosed techniques may be used to provide a method of video processing that includes inserting one or more averaged uni-predictive motion candidates from List0 and List1, derived from a regular motion candidate, into a motion candidate list for converting between a current video block and a bitstream representation of a video, the current block being coded using a geometry partitioning mode, and performing the conversion based on the motion candidate list.

[0007] In another representative aspect, the disclosed techniques may be used to provide a method of video processing that includes inserting one or more averaged motion candidates with uni-prediction derived from regular motion candidates with uni-prediction into a motion candidate list for converting between a current video block and a bitstream representation of a video, the current block being coded using a geometry partitioning mode, and performing the conversion based on the motion candidate list.

[0008] In another representative aspect, the disclosed techniques may be used to provide a method of video processing that includes inserting one or more virtual motion candidates derived from available motion candidates into a motion candidate list for converting between a current video block and a bitstream representation of the video, where the current block is coded using a geometry partitioning mode, and performing the conversion based on the motion candidate list.

[0009] In another representative aspect, the disclosed techniques may be used to provide a method of video processing that includes determining, based on a priority rule, an order of insertion of motion candidates into a motion candidate list for converting between a current block of video encoded using a geometry partitioning mode and a bitstream representation of the video, the priority rule being based on a position from which the motion candidates were derived or one or more reference picture lists associated with the motion candidates, and performing the conversion based on the determination and the motion candidate list.

[0010] In another representative aspect, the disclosed techniques may be used to provide a method of video processing that includes determining an order for inserting motion candidates into a motion candidate list based on a priority rule for converting between a current video block and a bitstream representation of the video, where the current block is coded using a geometry partitioning mode, and the order or insertion can be changed across sequences, pictures, slices, tile groups, or from the current block to a next block of the video, and performing the conversion based on the determination and the motion candidate list.

[0011] In another representative aspect, the disclosed techniques may be used to provide a method of video processing that includes partitioning a current video block coded using a geometry partitioning mode into a plurality of partitions, constructing a plurality of motion candidate lists corresponding to each of the plurality of partitions, and converting between the current block and a bitstream representation of the video based on the plurality of motion candidate lists.

[0012] In another representative aspect, the disclosed techniques may be used to provide a method of video processing that includes partitioning a current video block coded using a geometry partitioning mode into a plurality of partitions, constructing a plurality of motion candidate lists, each of the plurality of partitions configured to select a same motion candidate list from the plurality of motion candidate lists, and converting between the current block and a bitstream representation of the video based on the same selected motion candidate list.

[0013] In another representative aspect, the disclosed techniques may be used to provide a method of video processing that includes converting between a current video block in a video unit and a bitstream representation of the video, the bitstream representation including a field indicating a maximum number of allowed motion candidates in a motion candidate list for a geometry partitioning mode that is enabled in the video region.

[0014] In another representative aspect, the disclosed techniques may be used to provide a method of video processing that includes selecting, for a current video block divided into a plurality of divisions using a geometry partitioning mode, a set of weighting factors from a plurality of sets of weighting factors selected based on at least one width or height of the current block, and applying the set of weighting factors to samples along a common boundary of at least two of the plurality of divisions as part of a conversion between the current block and a bitstream representation of the video.

[0015] In another representative aspect, the disclosed techniques may be used to provide a method of video processing that includes determining, for samples in a current video block divided into a plurality of divisions using a geometry partitioning mode, at least one weighting factor based on an angle of a common boundary of at least two of the plurality of divisions, and converting between the current block and a bitstream representation of the video based on the at least one weighting factor.

[0016] In another representative aspect, the disclosed techniques may be used to provide a method of video processing that includes performing a motion compensation operation on a current video block partitioned into a plurality of partitions using a geometry partitioning mode for samples along a common boundary of at least two of the plurality of partitions, the sample size being different from a 4×4 sample size, and converting between the current block and a bitstream representation of the video based on the motion compensation operation.

[0017] In another representative aspect, the disclosed techniques may be used to provide a method of video processing that includes partitioning a current video block into multiple partitions using a geometry partitioning mode, storing a set of motion information for a K×L region in the current block derived from motion information associated with the multiple partitions, and converting between the current block and a bitstream representation of the video using at least one set of motion information associated with at least one of the multiple partitions.

[0018] In another representative aspect, the disclosed techniques may be used to provide a method of video processing that includes partitioning a current video block into a plurality of partitions and converting between the current block encoded using a geometry partition mode and a bitstream representation of the video, where a set of motion information is stored for each M×N block unit of the current block, the set of motion information being derived from motion information associated with each of the plurality of partitions, where either M or N is not equal to four, and where each sample in the M×N block unit shares the same motion information.

[0019] In another representative aspect, the disclosed techniques may be used to provide a method of video processing that includes dividing a current video block into a plurality of divisions and performing a motion compensation operation on a first sub-region of the current block based on first motion information that differs from second motion information associated with the first sub-region to be stored as part of a conversion between the current block and a bitstream representation of the video.

[0020] In another representative aspect, the disclosed techniques may be used to provide a method of video processing that includes partitioning a current video block into multiple partitions using a geometry partition mode, storing motion information of a first sample set and a second sample set based on partition directions or decoded merge indexes or merge candidate lists associated with the multiple partitions, where the first sample set is on a common boundary of at least two of the multiple partitions and the second sample set is within one of at least two of the multiple partitions, and converting between the current block and a bitstream representation of the video or between a subsequent block and a bitstream representation of the video based on the stored motion information.

[0021] In another representative aspect, the disclosed techniques may be used to provide a method of video processing that includes partitioning a current video block into a plurality of partitions, storing hypothetical bi-predictive motion information based on motion information of at least two partitions of the plurality of partitions, and converting between the current block and a bitstream representation of the video based on the hypothetical bi-predictive motion information.

[0022] In another representative aspect, the disclosed techniques may be used to provide a method of video processing that includes partitioning a current video block into a plurality of partitions, storing motion information of a first partition (MvInfo1) of the plurality of partitions and motion information of a second partition (MvInfo2) of the plurality of partitions based on a low latency check flag, where the low latency check flag indicates all reference pictures having a Picture Order Count (POC) value less than or equal to a POC value of a current picture that constitutes the current block, and converting between the current block and a bitstream representation of the video based on the stored motion information.

[0023] In yet another exemplary embodiment, the method is embodied in a processor executable code and stored on a computer readable program medium.

[0024] In yet another exemplary aspect, a device configured or operable to perform the above-described method is disclosed. The device may include a processor programmed to implement the method.

[0025] In yet another representative aspect, a video decoder device may implement the methods as described herein.

[0026] These and other aspects and features of the disclosed technology are explained in more detail in the drawings, description and claims. [Brief description of the drawings]

[0027] [Figure 1] 13 is an example of a process for deriving a merge candidate list construction. [Diagram 2] 1 shows examples of spatial merge candidate locations. [Diagram 3] 1 shows examples of candidate pairs that are considered for redundancy checking of spatial merge candidates. [Figure 4A] 13 shows exemplary positions for the second prediction unit (PU) of N×2N and 2N×N partitions. [Figure 4B] 13 shows exemplary positions for the second prediction unit (PU) of N×2N and 2N×N partitions. [Diagram 5] 13 is an example of scaling of motion vectors for temporal merging candidates. [Figure 6] Examples of candidate locations for temporal merging candidates, C0 and C1, are shown. [Figure 7] 1 illustrates an example of combined bi-predictive merge candidates. [Figure 8] 13 shows an example of a process for deriving motion vector prediction candidates. [Figure 9] 1 is an example of motion vector scaling for spatial motion vector candidates. [Figure 10] 13 illustrates an example of alternative temporal motion vector prediction (ATMVP) motion prediction for CU. [Figure 11] An example of one CU having four sub-blocks (AD) and its neighboring blocks (ad) is shown. [Figure 12] 13 is a flowchart of an example of encoding with different MV precision. [Figure 13A] 135 degree division type (division from the upper left corner to the lower right corner) and 45 degree division pattern are shown. FIG. ... [Figure 13B] 135 degree division type (division from the upper left corner to the lower right corner) and 45 degree division pattern are shown. FIG. ... [Figure 14] An example of the location of neighboring blocks is shown below. [Figure 15A] 2 shows an example of a motion vector storage device. [Figure 15B] 2 shows an example of a motion vector storage device. [Figure 16A] 1 illustrates an example of triangular prediction mode (TPM) signaling. [Figure 16B] 1 illustrates an example of triangular prediction mode (TPM) signaling. [Figure 17]1 shows an example of neighboring blocks (A and L) used for context selection in TPM flag encoding. [Figure 18A] 1 shows an example of a sub-block to which overlapped block motion compensation (OBMC) is applied. [Figure 18B] 1 shows an example of a sub-block to which overlapped block motion compensation (OBMC) is applied. [Figure 18C] 13 shows an example of uni-predictive MV selection for triangular prediction mode. [Figure 19] 1 shows an example of nearby samples used to derive IC parameters. [Figure 20A] Simplified affine motion models, 4-parameter affine and 6-parameter affine models are presented. [Figure 20B] Simplified affine motion models, 4-parameter affine and 6-parameter affine models are presented. [Figure 21] 1 shows an example of an affine motion vector field (MVF) for each subblock. [Figure 22A] Examples of a four-parameter affine model and a six-parameter affine model are shown. [Figure 22B] Examples of a four-parameter affine model and a six-parameter affine model are shown. [Diagram 23] An example of a motion vector predictor MVP for AF_INTER is shown below. [Figure 24A] Here are some examples of AF_MERGE candidates: [Figure 24B] Here are some examples of AF_MERGE candidates: [Diagram 25] 13 shows examples of candidate positions for the affine merge mode. [Figure 26] 1 shows an example of an optical flow trajectory. [Figure 27A] Shows an example of BIO w / o block extension, a) access location outside the block, b) using padding to avoid extra memory accesses and computations. [Figure 27B]Shows an example of BIO w / o block extension, a) access location outside the block, b) using padding to avoid extra memory accesses and computations. [Figure 28] 1 shows an example of decoder-side motion vector refinement (DMVR) based on bilateral template matching. [Figure 29] 4 shows an example of blocks used in the motion candidate list building process for different partitions. [Figure 30A] Here are three examples of MV storage when triangleDir=0: [Figure 30B] Here are three examples of MV storage when triangleDir=0: [Figure 30C] Here are three examples of MV storage when triangleDir=0: [Diagram 31] 1 is a flowchart illustrating an example of a video processing method. [Diagram 32] 1 is a flowchart illustrating an example of a video processing method. [Diagram 33] 1 is a flowchart illustrating an example of a video processing method. [Diagram 34] 1 is a flowchart illustrating an example of a video processing method. [Diagram 35] 1 is a flowchart illustrating an example of a video processing method. [Diagram 36] 1 is a flowchart illustrating an example of a video processing method. [Figure 37] 1 is a flowchart illustrating an example of a video processing method. [Figure 38] 1 is a flowchart illustrating an example of a video processing method. [Figure 39] 1 is a flowchart illustrating an example of a video processing method. [Diagram 40] 1 is a flowchart illustrating an example of a video processing method. [Diagram 41] 1 is a flowchart illustrating an example of a video processing method. [Diagram 42] 1 is a flowchart illustrating an example of a video processing method. [Diagram 43] 1 is a flowchart illustrating an example of a video processing method. [Diagram 44] 1 is a flowchart illustrating an example of a video processing method. [Diagram 45] 1 is a flowchart illustrating an example of a video processing method. [Figure 46] 1 is a flowchart illustrating an example of a video processing method. [Figure 47] 1 is a flowchart illustrating an example of a video processing method. [Figure 48] 1 is a flowchart illustrating an example of a video processing method. [Figure 49] FIG. 2 is a block diagram illustrating an example of a hardware platform for implementing the visual media decoding or visual media encoding techniques described herein. [Figure 50] FIG. 1 is a block diagram of an exemplary video processing system in which the disclosed techniques may be implemented. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0028] This specification provides various techniques that can be used by a decoder of a video bitstream to improve the quality of the decompressed or decoded digital video. Furthermore, a video encoder may implement these techniques during the encoding process to reconstruct the decoded frames for use in further encoding.

[0029] Section headings are used herein for ease of understanding, but are not intended to limit the embodiments and techniques to the corresponding section, and thus, embodiments from one section may be combined with examples from other sections.

[0030] 1. Overview

[0031] This patent specification relates to video coding technology. In particular, the present invention relates to motion vector coding and signaling in geometry partitioning of video coding. The present invention may be applied to existing video coding standards such as HEVC or standards to be finalized (e.g., Generalized Video Coding: VVC). The present invention may also be applied to future video coding standards or video codecs.

[0032] 2. Introductory comments

[0033] Video coding standards have evolved primarily through the development of well-known ITU-T and ISO / IEC standards. ITU-T has produced H.261 and H.263, ISO / IEC MPEG-1 and MPEG-4 Visual, and the two organizations have jointly produced H.262 / MPEG-2Video, H.264 / MPEG-4 Advanced Video Coding (AVC), and H.265 / HEVC standards. The video coding standard, H.262, is based on a hybrid video coding structure in which temporal prediction plus transform coding is utilized. In 2015, VCEG and MPEG jointly established the Joint Video Exploration Team (JVET) to explore future video coding technologies beyond HEVC. Since then, many new methods have been adopted by the JVET and incorporated into reference software called the Joint Exploration Model (JEM). In April 2018, the Joint Video Exploration Team (JVET) was formed between VCEG (Q6 / 16) and ISO / IEC JTC1 SC29 / WG11 (MPEG) to work on a generic video coding (VVC) standard with the goal of reducing the bitrate by 50% compared to HEVC.

[0034] The latest version of the VVC draft, namely Generic Video Coding (Draft 2), can be found at:

[0035] http: / / phenix.it-sudparis.eu / jvet / doc_end_user / documents / 11_Ljubljana / wg11 / JVET-K1001-v7.zip

[0036] The latest reference software of the VVC, called VTM, can be found below.

[0037] https: / / vcgit.hhi.fraunhofer.de / jvet / VVCSoftware_VTM / tags / VTM-2.1

[0038] 2.1 Inter Prediction in HEVC / H.265

[0039] Each inter-predicted PU has motion parameters for one or two reference picture lists. The motion parameters include a motion vector and a reference picture index. The use of one of the two reference picture lists may be signaled using inter_pred_idc. The motion vector may be explicitly coded as a delta relative to the predictor.

[0040] If one CU is coded in skip mode, one PU is associated with this CU, there are no significant residual coefficients, no coded motion vector deltas, and no reference picture indexes. A merge mode is specified, which obtains motion parameters for the current PU from neighboring PUs, including spatial and temporal candidates. The merge mode can be applied to any inter-predicted PU, not just for skip mode. An alternative to the merge mode is to explicitly transmit motion parameters, where the motion vector (more precisely, the difference of the motion vector compared to the motion vector predictor), the reference picture index corresponding to each reference picture list, and the usage of the reference picture list are explicitly signaled for each PU. Such a mode is referred to as Advanced Motion Vector Prediction (AMVP) in this disclosure.

[0041] If the signaling indicates to use one of the two reference picture lists, the PU is generated from a block of one sample. This is called "uni-prediction." Uni-prediction is available for both P and B slices.

[0042] If the signaling indicates to use both reference picture lists, generate the PU from a block of two samples. This is called "bi-prediction." Bi-prediction is only available for B slices.

[0043] The inter prediction modes defined in HEVC will be described in detail below. First, the merge mode will be described.

[0044] 2.1.1 Reference Picture List

[0045] In HEVC, the term inter prediction is used to indicate a prediction derived from data elements (e.g., sample values ​​or motion vectors) of reference pictures other than the currently decoded picture. Similar to H.264 / AVC, a picture can be predicted from multiple reference pictures. The reference pictures used for inter prediction are organized into one or more reference picture lists. A reference index identifies which reference picture in the list to use to generate the prediction signal.

[0046] One reference picture list, List0, is used for P slices, and two reference picture lists, List0 and List1, are used for B slices. Note that the reference pictures included in List0 / 1 may be in capture / display order or from past and future pictures.

[0047] 2.1.2 Merge Mode

[0048] 2.1.2.1 Deriving Merge Mode Candidates

[0049] When predicting a PU using merge mode, an index pointing to an entry in the merge candidate list is parsed from the bitstream and used to look up the motion information. The construction of this list is specified in the HEVC standard and can be summarized based on the following sequence of steps: Step 1: Derive initial candidates o Step 1.1: Spatial candidate derivation o Step 1.2: Redundancy check of spatial candidates o Step 1.3: Temporal candidate derivation Step 2: Insert additional candidates o Step 2.1: Creating bi-prediction candidates o Step 2.2: Insertion of zero motion candidates

[0050] These steps are also shown diagrammatically in Fig. 1. For spatial merge candidate derivation, select up to four merge candidates among candidates at five different positions. For temporal merge candidate derivation, select up to one merge candidate among two candidates. Since the decoder side assumes a constant number of candidates per PU, generate additional candidates if the number of candidates obtained in step 1 does not reach the maximum number of merge candidates (MaxNumMergeCand) signaled in the slice header. Since the number of candidates is constant, encode the index of the best merge candidate using a shortened unary binarization (TU). If the size of a CU is equal to 8, all PUs of the current CU share one merge candidate list, which is the same as the merge candidate list of the 2N × 2N prediction units.

[0051] The operations associated with the above steps are described in detail below.

[0052] 2.1.2.2 Spatial candidate derivation

[0053] In the derivation of spatial merge candidates, up to four merge candidates are selected from the candidates at the positions shown in FIG. 2. The order of derivation is A1, B1, B0, A0, B2. Position B2 is considered only if any of the PUs at positions A1, B1, B0, A0 are unavailable (e.g., because they belong to another slice or tile) or are intra-coded. After adding the candidate at position A1, the remaining candidates are subjected to a redundancy check, which ensures that candidates with the same motion information are removed from the list, improving coding efficiency. To reduce computational complexity, the aforementioned redundancy check does not consider all possible candidate pairs. Instead, only the pairs connected by the arrows in FIG. 3 are considered, and a candidate is added to the list only if the corresponding candidate used for the redundancy check does not have the same motion information. Another source of duplicated motion information is a "second PU" associated with a partition different from 2N×2N. As an example, FIG. 4A and FIG. 4B depict the second PU for N×2N and 2N×N cases, respectively. When splitting the current PU into N×2N, the candidate at position A1 is not considered for list construction. In fact, adding this candidate makes the bi-prediction units have the same motion information, and it is redundant to have only one PU in one coding unit. Similarly, when splitting the current PU into 2N×N, position B1 is not considered.

[0054] 2.1.2.3 Temporal candidate derivation

[0055] In this step, only one candidate is added to the list. Specifically, in the derivation of this temporal merge candidate, a scaled motion vector is derived based on the co-located PU belonging to the picture with the smallest POC difference with the current picture in a given reference picture list. In the slice header, the reference picture list used for deriving the co-located PU is explicitly signaled. As shown by the dotted line in Figure 5, the scaled motion vector of the temporal merge candidate is obtained, which is scaled from the motion vector of the co-located PU using the POC distances tb and td. tb is defined as the POC difference between the reference picture of the current picture and the current picture, and td is defined as the POC difference between the reference picture of the co-located PU and the co-located picture. The reference picture index of the temporal merge candidate is set equal to zero. The practical realization of this scaling process is described in the HEVC specification. For B slices, we take two motion vectors, one for reference picture list 0 and one for reference picture list 1, and combine them to form a bi-predictive merge candidate.

[0056] At the same position PU(Y) belonging to the reference frame, select the position of the temporal candidate between candidates C0 and C1 as shown in Figure 6. If the PU at position C0 is not available, is intra-coded, or is outside the current CTU, position C1 is used. Otherwise, position C0 is used to derive the temporal merge candidate.

[0057] 2.1.2.4 Inserting additional candidates

[0058] Besides the spatial-temporal merge candidates, there are two additional types of merge candidates: combined bi-predictive merge candidates and zero merge candidates. The combined bi-predictive merge candidates are generated by utilizing the spatial and temporal merge candidates. The combined bi-predictive merge candidates are used only for B slices. The combined bi-predictive candidate is generated by combining the first reference picture list motion parameters of the first candidate with the second reference picture list motion parameters of another candidate. If these two tuples provide different motion hypotheses, these tuples form a new bi-predictive candidate. As an example, FIG. 7 shows the case where two candidates with mvL0, refIdxL0 or mvL1, refIdxL1 in the original list (left side) are used to generate a combined bi-predictive merge candidate that is added to the final list (right side). There are various rules for the combinations considered to generate these additional merge candidates.

[0059] By inserting zero motion candidates and filling the remaining entries in the merge candidate list, the MaxNumMergeCand capacity is hit. These candidates have a spatial displacement of zero and a reference picture index that starts from zero and increases each time a new zero motion candidate is added to the list. The number of reference frames used by these candidates is one for uni-prediction and two for bi-prediction, respectively. Finally, no redundancy check is performed on these candidates.

[0060] 2.1.2.5 Motion Estimation Regions for Parallel Processing

[0061] To speed up the encoding process, motion estimation can be done in parallel, thereby deriving motion vectors for all prediction units in a given region simultaneously. Derivation of merge candidates from spatial neighborhoods can interfere with parallel processing, since one prediction unit cannot derive motion parameters from neighboring PUs until its associated motion estimation is completed. To mitigate the tradeoff between encoding efficiency and processing latency, HEVC specifies a motion estimation region (MER), the size of which is signaled in the picture parameter set using the "log2_parallel_merge_level_minus2" syntax element. When specifying one MER, merge candidates in the same region are marked as unavailable and therefore not considered in list construction.

[0062] 2.1.3 AMVP

[0063] AMVP exploits the spatio-temporal correlation of motion vectors with neighboring PUs, which is used for unambiguous transmission of motion parameters. For each reference picture list, we first construct a motion vector candidate list by checking the availability of the left temporal neighboring PU positions, removing redundant candidates, and adding zero vectors to keep the length of the candidate list constant. The encoder can then select the best predictor from the candidate list and transmit the corresponding index indicating the selected candidate. Similar to the merge index signaling, the index of the best motion vector candidate is coded using a shortened unary term. The maximum value to be coded in this case is 2 (see Figure 8). The following sections will explain the details of the motion vector prediction candidate derivation process.

[0064] 2.1.3.1 Derivation of AMVP Candidates

[0065] FIG. 8 summarizes the process of deriving motion vector prediction candidates.

[0066] In motion vector prediction, two types of motion vector candidates can be considered: spatial motion vector candidates and temporal motion vector candidates. To derive spatial motion vector candidates, as shown in Figure 2, we finally derive two motion vector candidates based on the motion vectors of each PU at five different positions.

[0067] To derive a temporal motion vector candidate, select one motion vector candidate from two candidates derived based on two different co-located positions. After creating the first spatio-temporal candidate list, remove duplicate motion vector candidates in the list. If the number of candidates is greater than two, remove motion vector candidates whose reference picture index in the associated reference picture list is greater than one from the list. If the number of spatio-temporal motion vector candidates is less than two, add an additional zero motion vector candidate to the list.

[0068] 2.1.3.2 Spatial Motion Vector Candidates

[0069] In the derivation of spatial motion vector candidates, we consider up to two candidates among the five candidates derived from PUs whose positions are the same as the positions of motion merge as shown in Fig. 2. The order of derivation for the left side of the current PU is specified as A0, A1, scaled A0, scaled A1. The order of derivation for the top side of the current PU is specified as B0, B1, B2, scaled B0, scaled B1, scaled B2. So, for each side, there are four cases that can be used as motion vector candidates, namely two cases that do not need to use spatial scaling and two cases that use spatial scaling. The four different cases can be summarized as follows: No spatial scaling -(1) The same reference picture list and the same reference picture index (same POC) - (2) Different reference picture lists, but the same reference picture (same POC) Spatial Scaling - (3) Same reference picture list, but different reference pictures (different POC) (4) Different reference picture lists and different reference pictures (different POCs)

[0070] First check the non-spatial scaling case, then do spatial scaling. Regardless of the reference picture list, if the POC is different between the reference picture of the neighboring PU and the reference picture of the current PU, consider spatial scaling. If all PUs of the left candidate are not available or are intra-coded, scaling of the top motion vector helps in the parallel derivation of the left and top MV candidates. Otherwise, no spatial scaling is allowed for the top motion vector.

[0071] In the spatial scaling process, the motion vectors of neighboring PUs are scaled in the same manner as in temporal scaling, as shown in Figure 9. The main difference is that the reference picture list and index of the current PU are given as input, and the actual scaling process is the same as in temporal scaling.

[0072] 2.1.3.3 Temporal Motion Vector Candidates

[0073] Except for deriving the reference picture index, all the processes for deriving the temporal merge candidates are the same as those for deriving the spatial motion vector candidates (see FIG. 6). The reference picture index is signaled to the decoder.

[0074] 2.2 New Inter Prediction Method in JEM

[0075] 2.2.1 Sub-CU-based motion vector prediction

[0076] In JEM with QTBT, each CU can have at most one motion parameter set for each prediction direction. In the encoder, we consider two sub-CU level motion vector prediction methods by splitting the large CU into sub-CUs and deriving motion information of all sub-CUs of the large CU. The alternative temporal motion vector prediction (ATMVP) method allows each CU to fetch multiple sets of motion information from multiple blocks smaller than the current CU in the aligned reference picture. In the spatio-temporal motion vector prediction (STMVP) method, we use the temporal motion vector predictor and spatial neighborhood motion vectors to recursively derive the motion vectors of the sub-CUs.

[0077] In order to maintain a more accurate motion field for sub-CU motion prediction, reference frame motion compression is currently disabled.

[0078] 2.2.1.1 Alternative Temporal Motion Vector Prediction

[0079] See Figure 10. In alternative temporal motion vector prediction (ATMVP), the temporal motion vector prediction (TMVP) method of motion vector is modified by taking multiple sets of motion information (including motion vectors and reference indexes) from blocks smaller than the current CU. In one example, a sub-CU is a square of NxN blocks (N is set to 4 by default).

[0080] ATMVP predicts the motion vectors of sub-CUs within a CU in two steps. The first step is to identify the corresponding blocks in reference picture 1 with so-called temporal vectors. This reference picture is called the motion source picture. The second step is to split the current CU into sub-CUs and obtain the motion vectors and reference indexes of each sub-CU from the blocks corresponding to each sub-CU.

[0081] In the first step, the reference picture and corresponding block are determined according to the motion information of the spatially neighboring blocks of the current CU. The first merging candidate in the merging candidate list of the current CU is used to avoid the repeated scanning process of the neighboring blocks. The first available motion vector and its associated reference index are set to the temporal vector and the index of the motion source picture. Thus, the ATMVP can identify the corresponding block more accurately than the TMVP, and the corresponding block (sometimes called the aligned block) is always in the lower right or center position with respect to the current CU.

[0082] In the second step, the corresponding block of the sub-CU is identified by its temporal vector in the motion source picture by adding the temporal vector to the coordinates of the current CU. For each sub-CU, the motion information of its corresponding block (the smallest motion grid covering the center sample) is used to derive the motion information of the sub-CU. After identifying the motion information of the corresponding N×N block, it is converted to the motion vector and reference index of the current sub-CU, similar to TMVP in HEVC, and motion scaling and other procedures are applied. For example, the decoder checks whether the low-latency condition (POC of all reference pictures of the current picture is smaller than the POC of the current picture) is met, and possibly calculates the motion vector MV x (the motion vector corresponding to the reference picture list X) is used to calculate the motion vector MV y (X equals 0 or 1, and Y equals 1-X).

[0083] 2.2.1.2 Spatio-temporal Motion Vector Prediction (STMVP)

[0084] In this method, the motion vectors of sub-CUs are derived recursively along the raster scan order. Figure 11 illustrates this concept. Consider an 8x8 CU that contains four 4x4 sub-CUs, A, B, C, and D. The neighboring 4x4 blocks in the current frame are labeled a, b, c, and d.

[0085] The motion derivation of sub-CU A starts by identifying its two spatial neighbors. The first neighbor is the N×N block above sub-CU A (block c). If this block c is not available or is intra-coded, check the other N×N blocks above sub-CU A (starting from block c, going from left to right). The second neighbor is the block to the left of sub-CU A (block b). If block b is not available or is intra-coded, check the other blocks to the left of sub-CU A (starting from top to bottom, starting from block b). Scale the motion information obtained from the neighboring blocks of each list to the first reference frame of a given list. Then, derive the temporal motion vector predictor (TMVP) of sub-block A following the same procedure as the TMVP derivation specified in HEVC. Fetch the motion information of the aligned blocks at location D and scale it accordingly. Finally, after retrieving and scaling the motion information, average all available motion vectors (up to 3) separately for each reference list. Let this averaged motion vector be the motion vector of the current sub-CU.

[0086] 2.2.1.3 Sub CU motion prediction mode signaling

[0087] Sub-CU modes are enabled as additional merge candidates, and no additional syntax elements are required to signal the mode. Add two additional merge candidates to the merge candidate list of each CU to represent ATMVP and STMVP modes. Use up to seven merge candidates if the sequence parameter set indicates that ATMVP and STMVP are enabled. The coding logic for the additional merge candidates is the same as for merge candidates in HM, that is, for each CU in a P or B slice, two or more RD checks are required for the two additional merge candidates.

[0088] In JEM, all bins of the merge index are context coded by CABAC, whereas in HEVC, only the first two values ​​are context coded and the remaining two values ​​are context-bypass coded.

[0089] 2.2.2 Adaptive motion vector difference resolution

[0090] In HEVC, when use_integer_mv_flag is 0 in the slice header, the motion vector difference (MVD) (the difference between the motion vector and the predicted motion vector of the PU) is signaled in units of quarter luma samples. In JEM, locally adaptive motion vector resolution (LAMVR) is introduced. In JEM, MVD can be coded in units of quarter luma samples, integer luma samples or four luma samples. MVD resolution is controlled at the coding unit (CU) level, and the MVD resolution flag is conditionally signaled for each CU that has at least one non-zero MVD module.

[0091] For a CU with at least one non-zero MVD module, a first flag is signaled to indicate whether ¼ luma sample MV precision is used in the CU. If the first flag (equal to 1) indicates that ¼ luma sample MV precision is not used, another flag is signaled to indicate whether integer luma sample MV precision or 4 luma sample MV precision is used.

[0092] If the first MVD resolution flag of a CU is zero or is not coded for the CU (i.e., all MVDs in the CU are zero), then 1 / 4 luma sample MV resolution is used for the CU. If the CU uses integer luma sample MV precision or 4 luma sample MV precision, round the MVPs in the CU's AMVP candidate list to the corresponding precision.

[0093] In the encoder, the RD check at the CU level is used to determine which MVD resolution to use for the CU. That is, the RD check at the CU level is performed three times for each MVD resolution. In order to increase the encoder speed, the following encoding method is applied in JEM. · During the RD check of a CU with a normal 1 / 4 luminance sample MVD resolution, the motion information (integer luminance sample accuracy) of the current CU is stored. During the RD check of the same CU with integer luminance sample and 4 luminance sample MVD resolutions, the stored motion information (after rounding) is used as the starting point for further small-range motion vector refinement, so that the time-consuming motion estimation process does not repeat three times. · Conditionally call the RD check of a CU with a 4 luminance sample MVD resolution. For a CU, if the RD cost of the integer luminance sample MVD resolution is much larger than that of the 1 / 4 luminance sample MVD resolution, the RD check of the 4 luminance sample MVD resolution for the CU is skipped.

[0094] The encoding process is shown in Figure 12. First, test the 1 / 4 pixel MV, calculate the RD cost, and represent it as RDCost0. Next, test the integer MV and represent the RD cost as RDCost1. If RDCost1 < th * RDCost0 (where th is a positive value), test the 4 pixel MV; otherwise, skip the 4 pixel MV. Basically, when checking the integer or 4 pixel MV, the motion information and RD cost, etc. for the 1 / 4 pixel MV are already known, and this can be reused to speed up the encoding process of the integer or 4 pixel MV.

[0095] 2.2.3 Triangular Prediction Mode

[0096] The concept of triangular prediction mode (TPM) is to introduce a new triangular division for motion compensation prediction. As shown in Figures 13A and 13B, a CU is divided into two triangular prediction units in a diagonal or anti-diagonal direction. Each triangular prediction unit in a CU is inter-predicted using its own single-prediction motion vector and reference frame index derived from one single-prediction candidate list. After predicting a triangular prediction unit, adaptive weighting processing is performed on the diagonal edges. Then, transform and quantization processing is performed on the whole CU. Note that this mode is only applicable to merge modes (including skip mode, which is treated as a special merge mode).

[0097] In this application, triangular prediction mode (TPM) is also referred to as triangular partition mode, triangle prediction mode, triangulation mode, geometry partition mode, or geometry partition mode.

[0098] In some embodiments, for non-TPM (also called non-geometry) merge modes, a merge index to a merge list is signaled and a merge list containing multiple regular merge candidates is constructed as follows: spatial merge candidates (derived from up to 5 spatial neighbors), temporal merge candidates (TMVP), HMVP, pairwise averaging candidates, and default merge candidates (zero MV with some assigned reference pictures). In some embodiments, the regular merge candidates are motion candidates derived for blocks of video coded using non-geometry (non-TPM) partition modes.

[0099] In contrast, for a TPM encoded block, two merge indices (with predictive encoding) into the merge list are signaled.

[0100] In one case, the merge list may be constructed in a similar manner to the non-TPM split mode described above, resulting in TPM motion candidates.

[0101] In another case, parity may be used to select the motion information of the single prediction of the two partitions. The two selected merge indexes may be denoted as idx0 and idx1. For each merge candidate, if its reference picture list X=(idx0&idx1) is true, set the motion information of the partition to the merge candidate list X information. Otherwise, set the motion information of the partition to the merge candidate list Y(Y=1-X) information. As a result, the final motion information is obtained with the single prediction as the TPM candidate, and the candidate in the first case is the normal merge candidate (generated using the merge list construction process described for the non-TPM mode coding block).

[0102] 2.2.3.1 TPM Single Prediction Candidate List

[0103] The uni-prediction candidate list, called the TPM motion candidate list, consists of five uni-prediction motion vector candidates. It is derived from seven neighboring blocks of a CU, including five spatially neighboring blocks (1-5) and two temporally co-located blocks (6-7), as shown in FIG. 14. The motion vectors of the seven neighboring blocks are collected and put into the uni-prediction candidate list in the order of uni-prediction motion vector, L0 motion vector of bi-prediction motion vector, L1 motion vector of bi-prediction motion vector, and average motion vector of L0, L1 motion vector of bi-prediction motion vector. If the number of candidates is less than five, add motion vector zero to the list. The motion candidates added to this TPM list are called TPM candidates, and the motion information derived from the spatial / temporal blocks is called regular motion candidates.

[0104] Specifically, the steps include the following: 1) Obtain regular motion candidates from A1, B1, B0, A0, B2, Col, and Col2 without performing any pruning operation (corresponding to blocks 1-7 in Fig. 14). 2) Set the variable numCurrMergeCand=0 3) For each regular motion candidate derived from,A1,,B1,,B0,,A0,,B2,,Col,,Col2,, whose numCurrMergeCand is less than 5, if the regular motion candidate,is uni-predictive (either from List0 or List1), then it is,directly added to the merge list as a TPM candidate with numCurrMergeCand increased by 1.,We name such a TPM candidate as a “candidate that is originally uni-predicted”. Apply full pruning. 4) For each motion candidate derived from A1, B1, B0, B0, A0, B2, Col, and Col2 for which numCurrMergeCand is less than 5, if the regular motion candidate is bi-predictive, add the motion information from List 0 to the TPM merge list as a new TPM candidate (i.e., modify it to be uni-predictive from List 0) and increment numCurrMergeCand by 1. We call such a TPM candidate a "shortened List0 prediction candidate". Apply full pruning. 5) For each motion candidate derived from A1, B1, B0, B0, A0, B2, Col, Col2, with numCurrMergeCand less than 5, if the regular motion candidate is bi-predictive, add the motion information from List1 to the TPM merge list (i.e., modify it to be uni-predictive from List1) and increment numCurrMergeCand by 1. We call such a TPM candidate a "shortened List1-prediction candidate". Apply full pruning. 6) For each motion candidate derived from A1, B1, B0, A0, B2, Col, Col2, where numCurrMergeCand is less than 5, if the canonical motion candidate is bi-predictive, - If List0 reference picture slice QP is smaller than List1 reference picture slice QP, scale the motion information of List1 to List0 reference picture first, and add the average of two MVs (one original from List0, the other scaled MV from List1) to the TPM merge list. We call such a candidate average uniprediction from List0 motion candidate, and increase numCurrMergeCand by 1. - Otherwise, first scale the motion information of List0 to the List1 reference picture, and add the average of two MVs (one original from List1, the other scaled MV from List0) to the TPM merge list. Such a TPM candidate is called average uniprediction from List1 motion candidates, and increase numCurrMergeCand by 1. Apply full pruning. 7) If numCurrMergeCand is less than 5, add a zero motion vector candidate.

[0105] 2.2.3.2 Adaptive Weighting

[0106] After each triangular prediction unit is predicted, an adaptive weighting process is applied to the diagonal edge between two triangular prediction units to derive the final prediction for the entire CU. Two weighting coefficient sets are defined as follows: The first set of weighting factors uses {7 / 8,6 / 8,4 / 8,2 / 8,1 / 8} and {7 / 8,4 / 8,1 / 8} for luminance and chrominance samples respectively. The second set of weighting factors uses {7 / 8,6 / 8,5 / 8,4 / 8,3 / 8,2 / 8,1 / 8} and {6 / 8,4 / 8,2 / 8} for luma and chroma samples respectively.

[0107] A set of weighting factors is selected based on a comparison of the motion vectors of the two triangular prediction units. The second set of weighting factors is used when the reference pictures of the two triangular prediction units are different or the difference between their motion vectors is greater than 16 pixels. Otherwise, the first set of weighting factors is used. Examples are shown in Figures 15A-15B.

[0108] 2.2.3.3 Motion Vector Storage

[0109] The motion vectors of the triangular prediction unit (Mv1, Mv2 in Figs. 16A and 16B) are stored in 4x4 grids. For each 4x4 grid, a uni-predictive or bi-predictive motion vector is stored based on the position of the 4x4 grid in the CU. As shown in Figs. 16A-16B, for a 4x4 grid located in a non-weighted area (i.e., not located on a diagonal edge), either a uni-predictive motion vector Mv1 or Mv2 is stored. Meanwhile, for a 4x4 grid located in a weighted area, a bi-predictive motion vector is stored. A bi-predictive motion vector is derived from Mv1 and Mv2 based on the following rules: 1) If Mv1 and Mv2 have motion vectors in different directions (L0 or L1), then simply combining Mv1 and Mv2 forms a bi-predictive motion vector. 2) If both Mv1 and Mv2 are coming from the same L0 (or L1) direction, - If the reference picture of Mv2 is the same as a picture in the L1 (or L0) reference picture list, Mv2 is scaled to that picture. Mv1 and the scaled Mv2 are combined to form a bi-predictive motion vector. - If the reference picture of Mv1 is the same as a picture in the L1 (or L0) reference picture list, Mv1 is scaled to that picture. The scaled Mv1 and Mv2 are combined to form a bi-predictive motion vector. - Otherwise, only Mv1 is stored for the weighting region.

[0110] 2.2.3.4 Triangle Prediction Mode (TPM) Signaling

[0111] A one bit flag may be signaled first to indicate whether a TPM is used, and then the two split patterns (as shown in Figures 13A, 13B) and the selected merge index for each of the two splits are further signaled.

[0112] 2.2.3.4.1 TPM Flag Signaling

[0113] Let W and H denote the width and height of one luma block, respectively. If W*H<64, triangular prediction mode is disabled.

[0114] When coding a block in affine mode, the triangular prediction mode is also disabled.

[0115] When a block is coded in merge mode, a one-bit flag can be signaled to indicate whether the triangular prediction mode is enabled or disabled for this block.

[0116] This flag is coded in three contexts according to the following formula (see FIG. 17):

[0117] Ctx index=((left block L available && L is coded with TPM?)1:0) +((Above block A available && A is coded with TPM?)1:0);

[0118] 2.2.3.4.2 Signaling an Indication of Two Split Patterns (as Shown in FIGS. 13A-13B) and a Selected Merge Index for Each of the Two Splits

[0119] Note that the split pattern and the merge indexes of the two splits are coded to each other. In one example, two splits are restricted to not use the same reference index. Therefore, there are 2 (split patterns) * N (maximum number of merge candidates) * (N-1) possibilities, where N is set to 5. One indication is coded, and the mapping between the split pattern, the two merge indexes, and the coded instructions is derived from the array defined below. const uint8_t g_TriangleCombination[TRIANGLE_MAX_NUM_CANDS][3]={ {0,1,0},{1,0,1},{1,0,2},{0,0,1},{0,2,0}, {1,0,3},{1,0,4},{1,1,0},{0,3,0},{0,4,0}, {0,0,2},{0,1,2},{1,1,2},{0,0,4},{0,0,3}, {0,1,3},{0,1,4},{1,1,4},{1,1,3},{1,2,1}, {1,2,0},{0,2,1},{0,4,3},{1,3,0},{1,3,2}, {1,3,4},{1,4,0},{1,3,1},{1,2,3},{1,4,1}, {0,4,1},{0,2,3},{1,4,2},{0,3,2},{1,4,3}, {0,3,1},{0,2,4},{1,2,4},{0,4,2},{0,3,4}};

[0120] Division pattern (45° or 135°) = g_TriangleCombination[signaled indication][0];

[0121] Merge index of candidate A = g_TriangleCombination[signaled indication];

[0122] Merge index of candidate B = g_TriangleCombination[signaled indication];

[0123] After deriving two motion candidates A and B, the (PU1, PU2) motion information of the two splits can be set from either A or B, and whether PU1 uses the motion information of merge candidate A or B depends on the prediction direction of the two motion candidates. Table 1 shows the relationship between the two derived motion candidates A and B with two splits.

[0124] [Table 1]

[0125] 2.2.3.4.3 Entropy coding of the representation (denoted by merge_triangle_idx)

[0126] merge_triangle_idx is in the range of [0,39] inclusive. A K-th order Exponential Golomb (EG) code is used to binarize merge_triangle_idx (K is set to 1).

[0127] K-th order EG

[0128] To encode larger numbers with fewer bits (at the expense of using more bits to encode smaller numbers), this can be generalized to use a nonnegative integer parameter k. To encode a nonnegative integer x with an exp-Golomb code of degree k, we use 1. Use the order-0 exp-Golomb code mentioned earlier to find [x / 2 k ] is encoded. Then, 2.x mod 2 k Encode in binary.

[0129] [Table 2]

[0130] 2.2.3.5 Recent Developments in TPM The normal merge candidate list is reused for triangulation merge prediction without extra pruning of motion vectors. For each merge candidate in the normal merge candidate list, only one of its L0 or L1 motion vectors is used for triangulation prediction. And the order of selecting L0 vs. L1 motion vectors is based on their merge index parity. According to this scheme, the normal merge list may be used as is.

[0131] Once the regular merge list is constructed, for each merge candidate in the regular merge candidate list, only one of its L0 or L1 motion vectors is used for triangular prediction. And the order of selecting L0 vs. L1 motion vectors is based on its merge index parity. Specifically, as shown in FIG. 18C, for a candidate with an even-valued index, its L0 motion vector is selected first and used for triangular prediction. If it is not available, its L1 motion vector is used instead. For a candidate with an odd-valued index, its L1 motion vector is selected first and used for triangular prediction. If it is not available, its L0 motion vector is used instead. In FIG. 18C, corresponding to each merge index, the motion vector marked with "x" is selected first for triangular prediction.

[0132] 7.3.7.7 Merging Data Syntax

[0133] [Table 3]

[0134] 2.2.4 Overlapping Block Motion Compensation

[0135] H.263 has used overlapped block motion compensation (OBMC) before. Unlike H.263, in JEM, OBMC can be switched on / off using CU-level syntax. When using OBMC in JEM, OBMC is performed for all motion compensation (MC) block boundaries except the bottom-right boundary of the CU. It also applies to both luma and chroma components. In JEM, one MC block corresponds to one coding block. If the CU is coded in a sub-CU mode (including sub-CU merge, affine, and FRUC modes), each sub-block of the CU is an MC block. In order to handle CU boundaries uniformly, OBMC is performed at the sub-block level for all MC block boundaries, and the sub-block size is set equal to 4×4, as shown in Figures 18A and 18B.

[0136] When OBMC is applied to the current subblock, in addition to the current motion vector, the motion vectors of the four connected neighboring subblocks are also available and used to derive a prediction block for the current subblock if they are not identical to the current motion vector. These multiple prediction blocks based on multiple motion vectors are combined to generate a final prediction signal for the current subblock.

[0137] A predicted block based on the motion vector of a neighboring subblock is P N (N is the index of the neighboring subblocks above, below, left, and right), and let P be the predicted block based on the motion vector of the current subblock. C Let us assume that N If P is based on the motion information of neighboring sub-blocks that contain the same motion information as the current sub-block, then OBMC is N Otherwise, P N All samples of P C Add to the same sample in P N The four rows / columns of P C Add to P N The weighting coefficients for are {1 / 4, 1 / 8, 1 / 16, 1 / 32}, and P CThe exception is for small MC blocks (i.e., when the height or width of the coding block is equal to 4 or one CU is coded in sub-CU mode), in which case the weighting factors for the two rows / columns are N Only P C In this case, P N Weighting factors {1 / 4,1 / 8} are used for P C Weighting factors {3 / 4, 7 / 8} are used for P N In contrast, P N The samples in the same row (column) of are weighted with the same coefficients. C Add to.

[0138] In JEM, for CUs whose size is 256 luma samples or less, a CU level flag is signaled to indicate whether OBMC is applied for the current CU. For CUs whose size is greater than 256 luma samples or not coded in AMVP mode, OBMC is applied by default. In the encoder, if OBMC is applied to a CU, its influence is taken into account during the motion estimation stage. The prediction signal formed by OBMC using the motion information of the upper and left neighboring blocks is used to compensate the upper and left boundaries of the original signal of the current CU, and then the normal motion estimation process is applied.

[0139] 2.2.5 Local illumination compensation

[0140] Local Illumination Compensation (LIC) is based on a linear model of illumination changes, with a scale factor a and an offset b, and is adaptively enabled or disabled for each inter mode coding unit (CU).

[0141] When LIC is applied to a CU, a least square error method is used to derive parameters a and b by using the neighboring samples of the current CU and their corresponding reference samples. Specifically, as shown in Figure 19, the subsampled (2:1 subsampled) neighboring samples of the CU and the corresponding samples in the reference picture (identified by the motion information of the current CU or sub-CU) are used. The IC parameters are derived and applied separately for each prediction direction.

[0142] If the CU is coded in merge mode, the LIC flag is copied from the neighboring block, similar to copying motion information in merge mode; otherwise, an IC flag is signaled to the CU to indicate whether LIC is applied or not.

[0143] When LIC is enabled for a picture, an additional CU-level RD check is required to determine whether LIC applies for a CU. If LIC is enabled for a CU, the mean removed sum of absolute differences (MR-SAD) and mean removed sum of absolute Hadamard transform differences (MR-SATD) are used instead of SAD and SATD for integer pixel motion search and fractional pixel motion search, respectively.

[0144] In order to reduce the coding complexity, the following coding scheme is applied in JEM: · If there is no clear illumination change between the current picture and its reference pictures, LIC is disabled for the entire picture. To identify this situation, in the encoder, we calculate the histograms of the current picture and all its reference pictures. If the histogram difference between the current picture and all its reference pictures is less than a given threshold, we disable LIC for the current picture, otherwise we enable LIC for the current picture.

[0145] 2.2.6 Affine Motion Compensated Prediction

[0146] In HEVC, only the translational motion model is applied for motion compensated prediction (MCP). In the real world, there are various kinds of motions, such as zoom in / out, rotation, perspective motion, and other irregular motions. In JEM, a simple affine transformation motion compensated prediction is applied. As shown in Figures 20A-20B, the affine motion field of a block is represented by two control point motion vectors.

[0147] The motion vector field (MVF) of a block is expressed by the following formula:

[0148]

number

[0149] For the six-parameter affine case,

[0150]

number

[0151] (v 0x ,v 0y ) is the motion vector of the control point in the upper left corner, and (v 1x ,v 1y ) is the motion vector of the control point in the upper right corner, and (v 2x ,v 2y ) is the motion vector of the control point of the lower left corner, and (x, y) represents the coordinates of the representative point relative to the upper left sample in the current block. In VTM, the representative point is set to the center position of a subblock. For example, if the coordinates of the upper left sample of the upper left corner of a subblock in the current block are (xs, ys), the coordinates of the representative point are set to (xs+2, ys+2).

[0152] To further simplify the motion compensation prediction, sub-block-based affine transformation prediction is applied. The size of the sub-block M×N is derived as in Equation 2, MvPre is the fractional precision of the motion vector (1 / 16 in JEM), and (v2x, v2y) is the motion vector of the bottom-left control point calculated according to Equation (1).

[0153]

number

[0154] After being derived by equation (2), M and N should be adjusted downwards as necessary to make them divisors of w and h, respectively.

[0155] To derive a motion vector for each M×N sub-block, we calculate the motion vector of the center sample of each sub-block based on Equation (1) and round it to 1 / 16 fractional precision, as shown in Figures 20A-20B. Then, we apply the motion compensation interpolation filter described in Section 2.2.3 to generate a prediction for each sub-block using the derived motion vector.

[0156] FIG. 21 shows an example of an affine motion vector field (MVF) for each sub-block.

[0157] After the MCP, the high-precision motion vectors for each subblock are rounded and stored with the same precision as the regular motion vectors.

[0158] 2.2.6.1 AF_INTER Mode

[0159] In JEM, there are two affine motion modes: AF_INTER mode and AF_MERGE mode. For CUs with both width and height greater than 8, AF_INTER mode can be applied. A CU-level affine flag is signaled in the bitstream to indicate whether AF_INTER mode is used. In this mode, the motion vector pair {(v0,v1)|v0={vA ,v B ,v C},v1={v D ,v E}}. As shown in FIG. 22A-22B, select v0 from the motion vectors of blocks A, B or C, and scale the motion vectors from the neighboring blocks based on the reference list and the relationship between the POC of the reference to the neighboring blocks, the POC of the reference to the current CU, and the POC of the current CU. Then, the method of selecting v1 from the neighboring blocks D and E is similar. If the number of candidate lists is less than 2, fill the list with motion vector pairs that duplicate each of the AMVP candidates. If the candidate list is greater than 2, first sort the candidates based on the consistency of the neighboring motion vectors (similarity of the two motion vectors in a pair candidate) and keep only the first two candidates. Use RD cost check to determine which motion vector pair candidate to select as the control point motion vector prediction (CPMVP) of the current CU. Then, signal in the bitstream an index indicating the position of the CPMVP in the candidate list. After determining the CPMVP of the current affine CU, apply affine motion estimation to obtain the control point motion vector (CPMV). The difference between CPMV and CPMVP is then signaled in the bitstream.

[0160] FIG. 23 shows an example of a motion vector predictor MVP for AF_INTER.

[0161] In AF_INTER mode, if a 4 / 6 parameter affine mode is used, 2 / 3 control points are needed, and therefore 2 / 3 MVDs need to be coded for these control points, as shown in Figures 22A and 22B. In the existing implementation, it is proposed to derive the MVs as follows: mvd1 and mvd2 are predicted from mvd0.

[0162]

number

[0163] where m ￣ v i ￣ , mvd i , mv1 are the predicted motion vector, the motion vector difference, and the motion vector of the top left pixel (i=0), the top right pixel (i=1), and the bottom left pixel (i=2), respectively, as shown in FIG. 22B. Note that the addition of two motion vectors (e.g., mvA(xA, yA) and mvB(xB, yB)) is equal to the sum of the two components separately, i.e., newMV=mvA+mvB, and the two components of newMV are set to (xA+xB) and (yA+yB), respectively.

[0164] 2.2.6.2 AF_MERGE Mode

[0165] When applying CU in AF_MERGE mode, the CU obtains the first block coded in affine mode from the valid neighboring reconstructed blocks. Then, the selection order of the candidate blocks is left, top, top right, bottom left to top left as shown in FIG. 24A. As shown in FIG. 24B, when the bottom left neighboring block A is coded in affine mode, the motion vectors v2, v3, and v4 of the top left corner, top right corner, and bottom left corner of the CU including block A are derived. Then, the motion vector v0 of the top left corner in the current CU is calculated based on v2, v3, and v4. Then, the motion vector v1 of the top right corner of the current CU is calculated.

[0166] After deriving the CPMV of current CUs v0 and v1, we generate the MVF of the current CU based on the simplified affine motion model Equation 1. To identify whether the current CU is coded in AF_MERGE mode, we signal an affine flag in the bitstream if there is at least one neighboring block coded in affine mode.

[0167] In some implementations, the affine merge candidate list includes the following steps:

[0168] Inserting inherited affine candidates

[0169] An inherited affine candidate means that the candidate is derived from the affine motion models of its valid neighboring affine coded blocks. On a common basis, the scanning order of the candidate positions is A1, B1, B0, A0, B2 as shown in Figures 24A-24B.

[0170] After deriving the candidate, we perform a full pruning process to check whether the same candidate is inserted into the list. If the same candidate exists, we discard the derived candidate.

[0171] Insert constructed affine candidates

[0172] If the number of candidates in the affine merge candidate list is less than MaxNumAffineCand (set to 5 in this contribution), insert the constructed affine candidate into the candidate list. The constructed affine candidate means that we construct a candidate by combining the motion information of the neighborhood of each control point.

[0173] First, derive motion information of the control points from the identified spatial and temporal neighborhoods shown in Figures 24A-24B. CPk (k=1,2,3,4) represents the kth control point. A0, A1, A2, B0, B1, B2, B3 are spatial positions for predicting CPk (k=1,2,3), and T is the temporal position for predicting CP4.

[0174] The coordinates of CP1, CP2, CP3, and CP4 are (0,0), (W,0), (H,0), and (W,H), respectively, where W and H are the width and height of the current block.

[0175] FIG. 25 shows examples of candidate positions for the affine merge mode.

[0176] The motion information of each control point is obtained according to the following priority:

[0177] For CP1, the priority of the check is B2 → B3 → A2. If available, use B2. Otherwise, if B2 is available, use B3. If both B2 and B3 are unavailable, use A2. If all three candidates are not available, the motion information of CP1 cannot be obtained.

[0178] In the case of CP2, the check priority is B1 → B0.

[0179] In the case of CP3, the check priority is A1 → A0.

[0180] Use T for CP4.

[0181] A combination of these control points is then used to construct affine merge candidates.

[0182] To construct a 6-parameter affine candidate, we need motion information of three control points. The three control points can be selected from one of the following four combinations ({CP1,CP2,CP4},{CP1,CP2,CP3},{CP2,CP3,CP4},{CP1,CP3,CP4}). We convert the combinations {CP1,CP2,CP3},{CP2,CP3,CP4},{CP1,CP3,CP4} into a 6-parameter motion model represented by the top-left, top-right, and bottom-left control points.

[0183] To construct a 4-parameter affine candidate, we need motion information of two control points. The two control points can be selected from one of the following six combinations: {CP1,CP4}, {CP2,CP3}, {CP1,CP2}, {CP2,CP4}, {CP1,CP3}, {CP3,CP4}. We convert the combinations {CP1,CP4}, {CP2,CP3}, {CP2,CP4}, {CP1,CP3}, {CP3,CP4} into a 4-parameter motion model represented by the top-left and top-right control points.

[0184] The constructed combinations of affine candidates are inserted into a candidate list in the following order:

[0185] {CP1,CP2,CP3}, {CP1,CP2,CP4}, {CP1,CP3,CP4}, {CP2,CP3,CP4}, {CP1,CP2},{CP1,CP3}, {CP2,CP3}, {CP1,CP4}, {CP2,CP4}, {CP3,CP4}

[0186] For a given combination of reference list X (X is 0 or 1), select the reference index with the highest usage rate in the control point as the reference index in list X, and scale the motion vector point to the differential reference picture.

[0187] After deriving the candidate, we perform a full pruning process to check whether the same candidate is inserted into the list. If the same candidate exists, we discard the derived candidate.

[0188] Motion vector zero padding

[0189] If the number of candidates in the affine merge candidate list is less than five, then insert zero motion vectors with reference index zero into the candidate list until the list is full.

[0190] 2.2.7 Bidirectional Optical Flow

[0191] Bidirectional Optical Flow (BIO) is a sample-by-sample motion refinement that is performed on top of block-by-block motion compensation for bi-prediction. Sample-level motion refinement does not use signaling.

[0192] FIG. 26 shows an example of an optical flow trajectory.

[0193] After motion compensation of the block, I (k) is the brightness value from the reference k (k=0,1), and ∂I (k) / ∂x and ∂I (k) / ∂y, respectively, I (k)Let,be the horizontal and vertical components of the gradient.,Assuming that optical flow is valid, the motion vector field,v, x ,v y ) is given by the following formula:

[0194]

number

[0195] Combining this optical flow equation with Hermite interpolation of the motion trajectory of each specimen, we obtain the function value I (k) and the derivative ∂I (k) / ∂x and ∂I (k) We obtain a unique third-order polynomial that fits both t = 0 and t = 0. The value of this polynomial at t = 0 is the BIO prediction.

[0196]

number

[0197] τ0 and τ1 indicate the distance to the reference frame as shown in Figure 28. The distances τ0 and τ1 are calculated based on the POCs of Ref0 and Ref1, τ0=POC(current)-POC(Ref0), τ1=POC(Ref1)-POC(current). If both predictions are from the same time series (both from the past or both from the future), the BIO with different signs (e.g. τ0·τ1<0) is only applied if the predictions are not from the same temporal moment (i.e. τ0≠τ1), both reference regions have non-zero motion (MVx0,MVy0,MVx1,MVy1≠0), and the motion vector of the block is proportional to the temporal interval MVx0 / MVx1=MVy0 / MVy1=-τ0 / τ1.

[0198] The motion vector field (v_x, v_y) is determined by minimizing the value difference Δ between points A and B (the intersection point of the motion trajectory and the reference frame plane in FIG. 26). The model uses only the first linear term of the local Taylor expansion for Δ.

[0199]

number

[0200] All values ​​in equation (5) depend on the sample position (i',j'), which we have omitted from the previous notation. Assuming that the motion is consistent in the local surrounding area, we minimize Δ inside a (2M+1) × (2M+1) square window centered on the current prediction point (i,j), where M is equal to 2.

[0201]

number

[0202] For this optimization problem, JEM uses a simple approach that first minimizes vertically and then horizontally. The result is

[0203]

number

[0204]

number

[0205] To avoid dividing by zero or getting very small values, we introduce regularization parameters r and m in equations (7) and (8).

[0206]

number

[0207] d is the bit depth of the video samples.

[0208] In order to make the memory access of BIO the same as that of normal biprediction motion compensation, all the prediction values ​​and gradient values ​​I (k) , ∂I (k) / ∂x, ∂I (k) ∂y is calculated for a position within the current block. In Equation 9, a (2M+1)×(2M+1) square window centered on the current prediction point on the boundary of the prediction block needs to access positions outside the block (as shown in Figure 27A). In JEM, I (k) , ∂I (k) / ∂x, ∂I (k) The value of / ∂y is set equal to the closest available value within the block, which may be implemented as padding, for example, as shown in Figure 27B.

[0209] By using BIO, we can refine the motion field on a sample-by-sample basis. To reduce the computational complexity, a block-based BIO design is used in JEM. We calculate the motion refinement based on a 4×4 block. In block-based BIO, we use s in Eq. (9) for all samples in a 4×4 block. n The value of s is integrated and then n The combined value of is used to derive the BIO motion vector offset for the 4x4 block. Specifically, the following equation is used for block-based BIO derivation:

[0210]

number

[0211] b k denotes the set of samples belonging to the k-th 4×4 block of the prediction block. n ((s n ,b k )>>4) and derive the associated motion vector offset.

[0212] In some cases, the MV management of BIO may be unreliable due to noise or irregular motion. Therefore, in BIO, the magnitude of the MV regime is clipped to a threshold thBIO. The threshold is determined based on whether the reference pictures of the current picture are all from one direction. For example, if the reference pictures of the current picture are all from one direction, the threshold is 12×2 14-d otherwise it is set to 12 × 2 13-d is set to.

[0213] Calculate the gradient of BIO simultaneously with the motion compensated interpolation using arithmetic compatible with HEVC motion compensation (2D separable FIR). The input of this 2D separable FIR is the same reference frame sample as for motion compensation and fractional position (fracX, fracY) based on the fractional part of the block motion vector. When the horizontal gradient ∂I / ∂x signal is first vertically interpolated using BIOfilterS corresponding to fractional position fracY with descaling shift d-8, then apply gradient filter BIOfilterG with descaling shift 18-d in the horizontal direction corresponding to fractional position fracX. When the vertical gradient ∂I / ∂y, apply the first gradient filter vertically using BIOfilterG corresponding to fractional position fracY with descaling shift d-8, then move the signal with descaling shift 18-d in the horizontal direction corresponding to fractional position fracX using BIOfilterS. To maintain reasonable complexity, the length of the interpolation filters for gradient calculation BIOfilterG and signal displacement BIOfilterF is shorter (6 taps). Table 4 shows the filters used for gradient calculation for different fractional positions of block motion vectors in BIO. Table 5 shows the interpolation filters used for generation of the prediction signal in BIO.

[0214] [Table 4]

[0215] [Table 5]

[0216] In JEM, BIO can be applied to all bi-predictive blocks when the two predictions are from different reference pictures. If LIC is enabled for a CU, BIO is disabled.

[0217] In JEM, OBMC is applied to blocks after normal MC processing. To reduce computational complexity, BIO is not applied during OBMC processing. That is, BIO is applied in the MC processing of one block if it uses its own MV, and is not applied in the MC processing if it uses the MV of a neighboring block in the OBMC processing.

[0218] 2.2.8 Decoder-side motion vector improvement

[0219] In the bi-prediction operation, for prediction of one block region, the bi-predictive blocks formed using the motion vectors (MV) of list 0 and the motion vectors (MV) of list 1 are respectively combined to form one prediction signal. In the decoder-side motion vector refinement (DMVR) method, the two motion vectors of the bi-prediction are further refined by a bilateral template matching process. To obtain the refined MV without transmitting additional motion information, bilateral template matching is applied in the decoder to perform a distortion-based search between the bilateral template and the reconstructed sample in the reference picture.

[0220] In DMVR, a bilateral template is generated from the first MV0 in list0 and MV1 in list1 as a weighted combination (i.e., average) of bi-predictive blocks, respectively, as shown in FIG. 28. The template matching operation consists of calculating a cost measure between the generated template and a sample region in the reference picture (near the first predicted block). For each of the two reference pictures, the MV with the smallest template cost is considered as the updated MV of that list and replaces the original MV. In JEM, nine MV candidates are searched for each list. The nine MV candidates include the original MV and eight surrounding MVs with one luma sample offset with respect to the original MV in either horizontal or vertical direction or both directions. Finally, two new MVs, namely MV0' and MV1', are used to generate the final bi-predictive result, as shown in FIG. 28. The sum of absolute differences (SAD) is used as the cost measure. Note that when calculating the cost of a predicted block generated by one surrounding MV, a rounded MV (integer pixels) is used to obtain the predicted block instead of the actual MV.

[0221] DMVR is applied to bi-predictive merge modes between one MV from a past reference picture and one MV from a future reference picture without transmitting additional syntax elements. In JEM, DMVR is not applied if LIC, affine motion, FRUC or sub-CU merge candidates are enabled for a CU.

[0222] 2.2.9 Subblock Merge Candidate List

[0223] Note that in addition to the regular merge list of non-subblock merge candidates, it is recommended to put all subblock related motion candidates into a separate merge list.

[0224] Sub-block related motion candidates are put into a separate merge list, called the "sub-block merge candidate list."

[0225] In one example, the sub-block merging candidate list includes affine merging candidates, ATMVP candidates, and / or sub-block based STMVP candidates.

[0226] 2.2.9.1 Other examples of affine merge list construction

[0227] In this contribution, we move the ATMVP merge candidate in the regular merge list to the first position of the affine merge list. All merge candidates in the new list (i.e., the sub-block based merge candidate list) are based on the sub-block coding tool.

[0228] 3. Examples of Problems Overcome by Embodiments

[0229] In the design of triangulation, one block can be divided into two partitions. In order to save memory bandwidth due to motion compensation, it is necessary that the two partitions should be uni-predicted. During the motion compensation process, bi-prediction is used for the diagonal edges and uni-prediction is used for all other parts. If bi-prediction is allowed for each partition, a sample located at a diagonal edge will have four MVs, two of which are from one partition and two from another partition. Such a design has the following problems: 1) The design of the uni-prediction merge list is dependent on the reference picture list, i.e., the MVs associated with reference pictures from reference picture list 0 are given higher priority. That is, the TPM candidates derived from all uni-prediction regular motion candidates are inserted before the TPM candidates derived from all bi-prediction regular motion candidates. However, in general, the positions of spatially neighboring blocks have stronger correlation in terms of similarity of mv information. 2) Although the two splits can select different TPM candidate indices, it is not optimal to utilize the same merge candidate list for both splits since the two splits may have different correlations with the neighboring blocks of the current block. 3) The maximum number of TPM candidates is fixed at 5, which is inconvenient, at least for software design. 4) A set of weighting coefficients is predefined for every type of block. 5) Storing motion information at 4x4 level increases memory capacity. How to compress them is a topic for research. 6) The TPM motion information depends on the parity of the two decoded merge indexes. For a candidate with an even-valued index, first select its L0 motion vector and use it for triangular prediction. If it is not available, its L1 motion vector is used instead. For a candidate with an odd-valued index, first select its L1 motion vector and use it for triangular prediction. If it is not available, its L0 motion vector is used instead. Thus, it is possible for two candidates to be from the same reference picture list.

[0230] 4. Illustrative Embodiments

[0231] The proposed method may be applied to any non-square / non-rectangular partition, such as geometric partition. In the following description, we use "triangular partition mode" as an example of non-square / non-rectangular partition mode (TPM), and refer to the motion vector prediction candidate inserted into the TPM candidate list as "TPM candidate". And, refer to the motion information previously associated with the coding block as "regular motion candidate". Note that other types of partitions are also possible.

[0232] The detailed list of techniques below should be considered as examples to illustrate the general concept. These techniques should not be interpreted in a narrow sense. Moreover, these techniques can be combined in any way.

[0233] Ordered TPM candidate list (AMVP / Merge / Other) construction process 1. Instead of always inserting TPM candidates derived from all unidirectionally predictive regular motion candidates before TPM candidates derived from all bi-predictive regular motion candidates, it is proposed to use the prediction direction as a priority criterion to determine which TPM candidate to add first. In one example, insert all regular motion candidates with single prediction from list X into the candidate list as TPM candidates before those with single prediction from list Y. b. In one example, all TPM candidates derived from bi-predictive regular motion candidates (e.g., shortened List0-prediction candidates, shortened List1-prediction candidates) are given higher priority, i.e., TPM candidates can be inserted before those derived from uni-predictive regular motion candidates. c. In one example, the order is defined as all TPM candidates derived from ListX of bi-predictive regular motion candidates (e.g., shortened ListX-prediction candidates), all TPM candidates derived from ListY of bi-predictive regular motion candidates (e.g., shortened ListY-prediction candidates), and all TPM candidates derived from uni-prediction regular motion candidates (e.g., originally uni-prediction candidates). d. In one example, a TPM candidate derived from one bi-predictive regular motion candidate may be added to the list before a TPM candidate derived from another bi-predictive regular motion candidate. e. In one example, a TPM candidate derived from one bi-predictive regular motion candidate may be added in an interleaved manner with a TPM candidate derived from another bi-predictive regular motion candidate. i. Canonical movement candidate C A , C B If there are two TPMs and four TPM candidates, then A , C B The order of addition to the list may be defined as follows: C A The shortened List0-prediction candidates derived from C B Shortened List1-prediction candidates derived from C A Shortened List1-prediction candidates derived from C BA shortened List0-prediction candidate derived from ii. Alternatively, two canonical motion candidates C A , C B , and if there are four TPM candidates, C A , C B The order of addition to the list may be defined as follows: C A Shortened List1-prediction candidates derived from C B The shortened List0-prediction candidates derived from C A The shortened List0-prediction candidates derived from C B Shortened List1-prediction candidates derived from. 2. Instead of always inserting TPM candidates derived from all unidirectionally predictive regular motion candidates before TPM candidates derived from all bi-predictive regular motion candidates, it is proposed to use the coding mode information associated with the regular motion candidates as a priority criterion to determine which TPM candidate to add first. The encoded information may include AMVP or merge mode. i. In one example, one regular motion candidate C A is derived from block A and another C B If C is derived from block B, block A is coded in AMVP mode, and block B is coded in merge mode, then C A The TPM candidates derived from B Alternatively, C B The TPM candidates derived from A may be added to the list before candidate TPMs derived from b. The encoded information may include a reference index and / or a POC difference. i. In one example, one TPM candidate C' A Another TPM candidate C' B If C' is associated with a reference index that is smaller than A C' B may be added to the list before ii. In one example, one TPM candidate C' A However, another TPM candidate C' B If C' is associated with a smaller POC distance between the reference picture and the current picture compared to A is C' B may be added to the list before c. The coded information may include the picture / slice / tile group quantization parameter (QP) of the reference picture and / or the temporal layer index of the reference picture. i. In one example, one TPM candidate C' A But another TPM C' B If C' is associated with a reference picture that has a smaller QP compared to A is C' B may be added to the list before 3. Instead of always inserting TPM candidates derived from all unidirectionally predictive regular motion candidates before TPM candidates derived from all bidirectionally predictive regular motion candidates, it is proposed to use the position where the motion candidate is derived as a priority criterion to determine which TPM candidate to add first. In one example, one regular motion candidate C A is derived from block A, and after A there is another C B If is derived from block B and B is checked, then C A The TPM motion candidates derived from C B may be added to the list before anything derived from A If is a single prediction, it can be directly inherited or each C A (two TPM candidates copied from a single prediction direction associated with b. Alternatively, both the position and the reference picture list (or prediction direction) may be used as the priority criteria. For example, if one regular motion candidate CA is derived from block A and another CB is derived from block B, and B is checked after A, then C in list X may be used as the priority criteria. A After adding the TPM candidate moves derived from the list, C of list X BThe TPM motion candidates derived from the list Y may be added to the list. A The TPM motion candidates derived from list Y are B where X is equal to 0 or 1 and Y is equal to (1-X). 4. It is proposed to add all shortened List1-prediction candidates before the shortened List0-prediction candidates. a. Alternatively, the order of the shortened List0-prediction candidates and the shortened List1-prediction candidates may be adaptively changed, for example, based on available TPM motion candidates. i. In one example, if the first TPM motion candidate is originally one predicted candidate with a single prediction from List1, a truncated List1-prediction candidate may be added before the shortened List0-prediction candidate. ii. Alternatively, if there are more originally unpredicted candidates than List1, shortened List1-predicted candidates may be added before the shortened List0-predicted candidates. b. Alternatively, all TPM candidates derived from one bi-predictive motion candidate (e.g., one shortened List0-prediction candidate and one shortened List0-prediction candidate) may be added to the list before all TPM candidates derived from another bi-predictive motion candidate. i. Alternatively, for TPM candidates derived from one motion candidate, the order of adding them may further depend on the information of the available TPM motion candidates in the list, eg 4.a.

[0234] Process of building TPM candidate list (AMVP / Merge / Other) with more TPM candidates 5. Multiple average unipredictions from List0 and List1 motion candidates derived from one regular motion candidate may both be added to the TPM motion candidate list. Alternatively, only one of them may be added, which one to add depends on the information of the available TPM candidates in the list, e.g., 4.a. b. Alternatively, only one of the two prediction lists may be added based on the reference picture indexes of the two prediction lists, and the motion vector associated with the smaller reference picture index is kept, and the final prediction direction is associated with such reference picture. Alternatively, only one of these may be added, which depends on the POC distance between the reference pictures in the two prediction lists and the current picture: the motion vector associated with the smaller POC distance is kept, and the final prediction direction is associated with such reference picture. d. In one example, all averaged single predictions from the motion candidates in List0 may be inserted before all averaged single predictions from the motion candidates in List1. Alternatively, all averaged single predictions from the motion candidates in List1 may be inserted before all averaged single predictions from the motion candidates in List0. 6. It is proposed to be able to derive the average uni-predictive TPM candidate using uni-predictive regular motion candidates. a. Uni-predictive regular motion candidates from list LX may be used to generate an averaged uni-predictive candidate for list LX, for example using bi-predictive regular motion candidates or other uni-predictive regular motion candidates from list X. b. A uni-predictive regular motion candidate from list LX may first be scaled to LY (Y=1-X) and this scaled motion vector may be used to generate an averaged uni-predictive candidate in list LY, for example using a bi-predictive regular motion candidate or another uni-predictive regular motion candidate from list X. 7. Virtual TPM motion candidates derived from the available TPM motion candidates may also be added to the TPM candidate list. In one example, the candidate candidates may be derived from the virtual candidates that were added to the merge list after a particular step (eg, after generating the shortened List1 predicted candidates). b. In one example, the virtual candidates may be derived from only the original single prediction candidate. c. In one example, the virtual candidates may be derived from only the original single prediction candidate and the shortened List1 prediction candidate. d. In one example, the virtual candidates may be derived from candidates having a certain reference picture index (i.e., equal to 0) and / or candidates having a range of POC distances between the reference picture and the current picture, or from one reference picture list. e. In one example, the virtual candidates may be derived from only the first K TPM motion candidates, eg, setting K to 1. i. For example, the MVs of two TPM motion candidates can be averaged to obtain a virtual candidate. f. Motion vector is C mv and the reference index is C refidx For TPM motion candidates , the following method may be used to derive virtual vectors. I C mv Add an offset to the horizontal or vertical component of the same C refidx is used. ii.C mv Add offsets to both the horizontal and vertical components of the same C refidx Use. iii.C refidx C does not point to other reference pictures mv Scale the scaled motion vector and C mv Let x be the new TPM motion candidate using the indices of the scaled reference pictures. g. When adding a virtual TPM candidate, pruning may not be applied. i. Alternatively, pruning may be applied to candidates other than the one that generated the virtual TPM candidate.

[0235] TPM candidate list (AMVP / merge / other) construction process with adaptive insertion order 8. The insertion order (priority criteria) of TPM candidates may be changed from sequence to sequence, picture to picture, slice to slice, tile group to tile group, block to block, etc. In one example, it may depend on the block size / block shape / division pattern. b. Alternatively, this order may be signaled from the encoder to the decoder in the VPS / SPS / PPS / slice header / tile group header / tile / CTU / CU.

[0236] Multiple TPM lists for partitioning in TPM encoded blocks 9. For each geometry partition, one may build one separate list of motion candidates instead of using one list for all partitions. a. In one example, for one split, the associated candidate list includes only motion candidates predicted from list X, and for another split, the associated candidate list includes only motion candidates predicted from list Y, where Y is not equal to X. b. In one example, different spatial and / or temporal blocks may be accessed to derive motion candidates to be added to the merge list. c. In one example, the location of spatial and / or temporal blocks in one geometric partition may depend on the location of the partition. d. In one example, the location of spatial and / or temporal blocks in one geometric partition may depend on the partitioning scheme (top left to bottom right (45 degrees) or top right to bottom left (135 degrees)). e. In one example, in a 135 degree split pattern (shown in FIGS. 13A-13B), the blocks above may be checked to build a merge list for PU1. Some examples of the above blocks (marked in grey) are shown in FIG. f. In one example, in a 135 degree split pattern (shown in FIGS. 13A-13B), more left blocks may be checked to build a merge list for PU2. Some examples of left blocks (marked in grey) are shown in FIG. g. In one example, in a 45° split pattern (shown in FIGS. 13A-13B), the leftmost block and the uppermost block may be checked to build a merge list for PU1. Some examples of the upper left block (marked in grey) are shown in FIG. h. In one example, in a 45° division pattern (shown in FIGS. 13A-13B), more temporal blocks may be checked to build the merge list for PU2. i. In one example, an index into the TPM list is signaled for each partition. 10. Multiple TPM lists can be built and shared across all partitions, and each partition can select one of them (e.g., each partition can select the same TPM list from multiple TPM lists). In one example, the index of the TPM list may be signaled first. Then, the index of the candidate TPM may be further signaled. b. In another example, the index of the TPM list and the index of the candidate TPM may be jointly encoded.

[0237] Maximum allowed TPM candidates signaling 11. Maximum allowed TPM candidates for merging / AMVP are signaled in SPS / VPS / PPS / Picture Header / Slice Header / Tile Group Header / Row / Group of LCU. a. Alternatively, the maximum allowed TPM motion candidates is set to that used for non-TPM coded blocks, eg, the non-TPM merge candidate list size, or the sub-block merge candidate list size. b. The maximum number, when signaled, is binarized by unary encoding or truncated unary encoding. c. Instead of signaling the maximum number directly, one may signal (M-maximum number), e.g., M=5 or 6. d. One bit flag can be signaled to indicate whether the maximum number of TPM candidates is the same as the maximum number of regular motion candidates / sub-block motion candidates.

[0238] Weighting coefficients 12. The selection of the weighting coefficients may depend on the width and / or height of the current block. In one example, if the width to height ratio is greater than a threshold, a particular set of weighting factors (eg, set 1) may be selected. b. In one example, if the height to width ratio is greater than a threshold, a particular set of weighting factors (eg, set 1) may be selected. c. A set of multiple weighting coefficient groups may be predefined, and when the width and / or height of the blocks are different, one weighting coefficient group may be selected from one or two of the predefined sets. 13. Weighting factors may be changed on the fly based on the position of the sample. In one example, this angle may depend on the angle of the edge that distinguishes the two divisions. b. In one example, the weighting factors may be defined as a function of the position of the sample. c. In one example, the final predicted block of one TPM encoded block is: It is set as (f(x,y)*P1(x,y)+(2M-f(x,y))*P2(x,y)+offset)>>M, where P1, P2 are the two predicted values ​​of the sample located at (x,y), M is an integer value, the offset can be set to (1<<(M-1)), and f(x,y) is a coefficient that returns the weight applied to the first predicted value. 14. For blocks located in the weighting domain, motion compensation is performed at 8x8 level instead of 4x4 level to reduce memory bandwidth. Alternatively, for blocks located in the weighting region, motion compensation is performed at 8x4 or 4x8 levels instead of 4x4 levels to reduce memory bandwidth. b. Alternatively, if motion compensation based on 8x4 or 4x8 is allowed, the choice of 8x4 or 4x8 may depend on the following: i. Block width and / or height ii. Division patterns (e.g., 45 degrees or 135 degrees).

[0239] Movement information storage device 15. After decoding the TPM encoded block, one or more HMVP tables may be updated. In one example, one or more HMVP tables may be updated based on motion information of one partition to the exclusion of motion information of other partitions. b. In one example, one or more HMVP tables may be updated for weighted regions excluding unweighted regions and / or weighted regions excluding weighted regions from unweighted regions based on the stored motion information. c. In one example, one or more HMVP tables may be updated based on a merge candidate in the merge candidate list used to derive the TPM-encoded block stream motion information. i. In one example, one merge candidate is associated with the merge index (e.g., merge_triangle_idx0 or merge_triangle_idx1) of the decoded merge indexes. ii. In one example, one merge candidate is associated with a merge index equal to k (eg, k=0). 16. The derivation process of triangle motion information for one partition may depend on decoded motion information from other partitions, for example based on the prediction direction of the other partitions. d. In one example, a first reference picture list denoted as LX of one decoded candidate having an index equal to merge_triangle_idx0 is selected (e.g., based on the parity of merge_triangle_idx0 and the availability of LX). Regardless of the parity of the second decoded candidate index, if motion information associated with a second reference picture list denoted as LY (Y=1-X) of the second candidate is available, select it as the triangulation motion information. In the following description, the motion information associated with the reference picture list LX or LY is also referred to as LX motion information or LY motion information. i. Alternatively, if the LY(Y=1-X) motion information of the second candidate having merge_triangle_idx1 is not available, the LX motion information of the second candidate may be used instead. ii. Alternatively, if the LY(Y=1-X) motion information of the second candidate with merge_triangle_idx1 is not available, the LX motion information of the second candidate may be used to derive the LY motion information, and the derived LY motion information may be used for TPM encoding. 1) In one example, this derivation process may involve scaling of motion vectors that refer to reference pictures in LX and refer to reference pictures in LY. e. In one example, a first reference picture list represented as LX of one decoded candidate having an index equal to merge_triangle_idx1 is selected (e.g., based on the parity of merge_triangle_idx1 and the availability of LX). Regardless of the parity of the second decoded candidate index, if LY(Y=1-X) motion information is available, select it as the motion information of the triangulation. i. Alternatively, if LY (Y=1-X) motion information of the candidate with merge_triangle_idx0 is not available, LX motion information may be used instead. ii. Alternatively, if the LY(Y=1-X) motion information of the candidate with merge_triangle_idx0 is not available, the LX motion information of the candidate with merge_triangle_idx0 may be used to derive the LY motion information, and the derived LY motion information may be used for TPM encoding. 1) In one example, this derivation process may involve scaling of motion vectors that refer to reference pictures in LX and refer to reference pictures in LY. 17. Using motion information from the same reference picture list for multiple triangulations is not allowed. f. In one example, one conforming bitstream should satisfy that two triangulations are predicted from different reference picture lists. g. Alternatively, the process of deriving motion information from the parity of the merge index may always output predicting two splits from two different lists, for example using the method disclosed in bullet 15. 18. How to store motion information and / or derive the motion information to be stored for weighted and / or non-weighted regions may depend on the split direction and / or decoded merge index of the triangulation and / or merge candidate list utilized to derive the motion information of the triangulation and / or reference picture in the reference picture list (e.g., all reference pictures have smaller or larger POC values ​​compared to the current picture). In one example, the motion information of the merge candidates associated with the decoded merge index selected for one split may be stored. i. In one example, the selected decoded merge index may be set to one of the decoded merge candidate indexes, for example, merge_triangle_idx0 or merge_triangle_idx1. ii. In one example, the selected decoding merge index is set to a variable k. 1) In one example, k is set to 0. 2) In one example, all information of the kth merge candidate may be inherited and stored. 3) In one example, partial information of the kth merge candidate may be inherited and the remaining information may be modified before being stored. (a) In one example, the indication of the weighting factor used for the generalized bi-prediction (GBi) of the kth merge candidate may not be inherited. Alternatively, the TPM may assign a specific GBi weighting index to the stored motion information (e.g., 0). (b) In one example, the indication of motion vector accuracy of the kth merging candidate may not be inherited. iii. Alternatively, the selected decoding merge index may depend on the split direction. iv. In one example, if the merge candidate with index equal to the selected decoded merge index is bi-predictive, then bi-predictive motion information may be stored. v. In one example, if the merge candidate having an index equal to the selected decoded merge index is uni-predictive, then uni-predictive motion information may be stored. 1) Alternatively, bi-predictive motion information may be stored, and the stored motion information may be derived from merging candidates. vi. Alternatively, a function relating the motion information of the two partitions may be stored. 1) For example, the average of the motion vectors of the two partitions may be stored. b. In one example, if MvInfo1 and MvInfo2 indicate motion information for two partitions, hypothetical bi-predictive motion information may be stored by correcting one prediction direction from LX to LY (Y=1-X) (A is 1 or 2) in MvInfoA. i. In one example, the motion vectors and reference indexes of MvInfoA are kept unchanged. ii. In one example, the reference index of MvInfoA is kept unchanged and the motion vector of MvInfoA is set to the opposite value. iii. The bi-predictive motion information may include modified MvInfoA and unmodified MvInfoB (with B=3-A). iv. In one example, such a correction may be applied only if both MvInfo1 and MvInfo2 are from one and the same prediction direction. c. In one example, if MvInfo1 and MvInfo2 indicate motion information of two partitions, how to store the motion information may depend on a low latency check flag that indicates whether the POC values ​​of all reference pictures are smaller compared to the current picture. i. In one example, the following apply in order: If MvInfo1 and MvInfo2 are different reference picture lists, MvInof0 and MvInfo1 may be combined and stored as bi-predictive motion information. ii. If MvInfo1 and MvInfo2 are in the same LX (X=0 or 1) direction, the following may apply: If the low latency check flag is true (e.g., all reference pictures have POC values ​​smaller or not larger than the current picture), the prediction direction list may be set from LX to LY to modify MvInfoA (e.g., A=2) (Y=1 to X), and the modified MvInfoA is combined with the unmodified MvInfoC (C=3 to A) to form and store virtual bi-predictive motion information. Otherwise, MvInfoB (eg, B=2) is stored, i.e., uni-predictive motion information is stored. iii. In one example, MvInfo1 is associated with PU1 in FIG. 13A and PU1 in FIG. 13B, and MvInfo2 is associated with PU2 in FIG. 13A and PU2 in FIG. 13B. d. In one example, the above method may be applied to storing motion information of a particular sub-block within a block. i. In one example, the particular sub-block may be a sub-block in a weighting domain. ii. In one example, the particular sub-block may be a sub-block that includes a diagonal or anti-diagonal in a block. iii. In one example, the particular sub-block may be the sub-block in the bottom right corner of the block. iv. In one example, the particular sub-block may be the sub-block in the right column or below the block. v. Alternatively, the above method may be applied to storing motion information of all sub-blocks within one block. 19. In TPM mode, the motion information of a sub-region within a block may differ from that used in the motion compensation process for the reconstruction of this sub-region. a. In one example, for an MxN sub-region (e.g., 4x4 in the current coding unit in triangular prediction mode) located in a weighting region, motion compensation may be performed as bi-predictive, but only one set of motion information from the bi-predictive List0 or List1 may be stored. i. Alternatively, for an MxN sub-region located in a weighted region (e.g., 4x4 in the current coding unit in triangular prediction mode), motion compensation may be performed as bi-predictive, but only uni-prediction information from list X may be stored, and the stored information may be different from the motion information from list X utilized in the motion compensation process. ii. Alternatively, for an MxN sub-region (e.g., 4x4 in the current coding unit in triangular prediction mode) located in a weighted region, motion compensation may be performed as bi-predictive, but different bi-predictive information (e.g., different MV and / or different reference pictures) may be stored than that utilized in the motion compensation process. b. In one example, for an MxN sub-region (e.g., 4x4 in the current coding unit in triangular prediction mode) located in a non-weighted area, motion compensation may be performed as uni-predictive, but bi-predictive motion information may be stored. i. Alternatively, for MxN sub-regions located in non-weighted regions (e.g., 4x4 in the current coding unit in triangular prediction mode), motion compensation may be performed as uni-prediction, but the uni-prediction motion information may be stored and may be different from that utilized in the motion compensation process. c. The stored motion information may be utilized for coding of other blocks (eg, merging of neighboring blocks / as spatial motion candidates in AMVP mode). d. The stored motion information may be utilized for encoding future blocks in different pictures (eg, used to derive candidate temporal motion vectors). e. The stored motion information may be utilized for in-loop processing such as deblocking or ALF. f. In one example, motion information of a first sub-region within a block may be different from that used for the motion compensation process of the reconstruction of the first sub-region, while motion information of a second sub-region within the block may be the same as that used for the motion compensation process of the reconstruction of the second sub-region, and the first sub-region and the second sub-region may be the same coding unit in a triangular prediction mode. 20. It is proposed to store the same set of motion information for the whole block, whether it is located in the weighting region or not. Let MvInfo1 and MvInfo2 indicate the motion information of the two splits, respectively (e.g., based on the two decoded / derived merge candidate indexes). a. In one example, uni-predictive motion information (eg, inherited from one of the two partitions or derived from the motion information of the two partitions) may be stored. i. In one example, MvInfo1 may be stored for the entire block. 1. In one example, MvInfo1 may be stored for all sub-regions within a block. ii. In one example, MvInfo2 may be stored for the entire block. 1. In one example, MvInfo1 may be stored for all sub-regions within a block. iii. In one example, which partition's motion information to store may depend on the POC distance between it and the current picture. 1. For example, if the absolute value of the POC distance between the current picture and the reference picture referred to by MvInfo1 is smaller than the value between the current picture and the reference picture referred to by MvInfo2, MvInfo1 may be stored. iv. In one example, which partition's motion information to store may depend on the QP of the reference picture. 1. For example, MvInfo1 may store that the QP of the reference picture that MvInfo1 refers to is smaller than the value of the reference picture that MvInfo2 refers to. v. In one example, which partition's motion information to store may depend on the reference index of the reference picture. 1. For example, MvInfo1 may store that the QP of the reference picture that MvInfo1 refers to is smaller than the value of the reference picture that MvInfo2 refers to. 2. For example, if the reference index of the reference picture referred to by MvInfo1 is smaller than the value of the reference picture referred to by MvInfo2, MvInfo1 may be stored. vi. In one example, which partition's motion information to store may depend on the merge index associated with a partition. 1. For example, if the associated merge index derived from MvInfo1 is less than that derived from MvInfo2, then MvInfo1 may be stored. 2. For example, if merge_triangle_idx0 is smaller than merge_triangle_idx1, then motion information derived from merge_triangle_idx0 may be stored. vii. In one example, a third set of motion information (denoted as MvInfo3) derived from MvInfo1 and MvInfo2 may be stored. 1. In one example, the MV of MvInfo3 may be derived as the average of two MVs, MvInof1 and MvInfo2, or may be derived as a scaled or mapped motion vector from one MV, Mvnof1 and MvInfo2, and averaged with the other MV to generate the MV of MvInfo3. 2. In one example, the reference picture of MvInfo3 may be one of two reference pictures of MvInof1 and MvInfo2. viii. In one example, one motion vector in MvInfoX (X is 0 or 1) may be scaled relative to a reference picture in MvInfoY (Y is 1-X) and then used to derive the motion information to be stored. 1. In one example, the scaled MvInfo1 and MvInfo2 may be used to derive the motion information to be stored, as defined in bullet vii. ix. The above method is only applicable to sub-regions that lie within the weighting region. 1. Alternatively, the above method may be applicable only to sub-regions located in the unweighted region. 2. Alternatively, the above method may be applicable only to specific sub-regions located within the weighting region. For example, the weighting may be applied to the top right and / or bottom left sub-regions located in the region. b. For example, the weighting may be applied to the top left and / or bottom right sub-regions located in the region. c. Which sub-regions apply may depend on the split direction. b. In one example, derive and store bi-predictive motion vectors from MvInfo1 and MvInfo2. i. If MvInfo1 and MvInfo2 have motion vectors in different directions (L0 or L1), then MvInfo1 and MvInfo2 are simply combined to form a bi-predictive motion vector. ii.If both MvInfo1 and MvInfo2 come from the same LX (X=0 or 1) direction, The MV stored for 1.LX may be derived from one motion vector of MvInfo1 and MvInfo2, or from the two of them (eg, by averaging). 2. If LY (Y=1-X) contains a reference picture named MvInfo1 or MvInfo2, simply combine MvInfo1 and MvInfo2 to form a bi-predictive motion vector, and set one of the prediction directions to LY. 3. One motion vector can be scaled to a reference picture in LY (Y=1-X), and the scaled MV and another MV can be combined to form a bi-predictive motion vector. In one example, the object reference picture in LY that the scaled MV refers to may be predefined (e.g., the reference picture index is equal to 0) or signaled. 4. One motion vector can be mapped to a reference picture in LY (Y=1-X), and the mapped MV and other MVs can be combined to form a bi-predictive motion vector. In one example, the object reference picture in LY that the mapped MV refers to can be predefined (e.g., the reference picture index is equal to 0) or signaled. In one example, the mapping process can be performed without scaling. For example, the mapped MV can be equal to or opposite to the original MV. i. The above method is only applicable to sub-regions that lie in the weighting region. 1) Alternatively, the above method may be applicable only to sub-regions located in the unweighted region. 2) Alternatively, the above method may be applicable only to certain sub-regions located within the weighting region. (a) For example, the weighting may be applied to the top right and / or bottom left sub-regions located in the region. (b) For example, the weighting may be applied to the top-left and / or bottom-right sub-regions located in the region. (c) Which sub-regions apply may depend on the split direction. c. Whether to store uni- or bi-predictive motion information or bi-predictive motion information may depend on the decoded motion information of the two splits. i. In one example, if both MvInfo1 and MvInfo2 are included in the same LX reference list, uni-predictive motion information may be stored. ii. In one example, if MvInfo1 and MvInfo2 are from different reference lists (one from L0 and the other one from L1), bi-predictive motion information may be stored. iii. In one example, if both MvInfo1 and MvInfo2 are included in the same LX reference list, but at least one of the reference pictures referenced by MvInfo1 or MvInfo2 is included in the other reference list LY (Y=1-X), bi-predictive motion information may be stored. d. Alternatively, the storage of motion information is still based on whether a sub-block is located in a weighting region. For those located in a weighting region, the stored motion information is derived according to the following rules: i. In one example, if MvInfo1 and MvInfo2 are from the same LX (X=0 or 1) direction, 1. If either MvInfo1 or MvInfo2 reference picture is included in LY (Y=1-X), simply combine MvInfo1 and MvInfo2 to form a bi-predictive motion vector and set one of the prediction directions to LY. 2. One motion vector can be scaled to a reference picture in LY (Y=1-X), and the scaled MV and another MV can be combined to form a bi-predictive motion vector. In one example, the object reference picture in LY that the scaled MV refers to may be predefined (e.g., the reference picture index is equal to 0) or signaled. 3. One motion vector can be mapped to a reference picture in LY (Y=1-X), and the mapped MV and other MVs can be combined to form a bi-predictive motion vector. In one example, the object reference picture in LY that the mapped MV refers to can be predefined (e.g., the reference picture index is equal to 0) or signaled. In one example, the mapping process can be performed without scaling. For example, the mapped MV can be equal to or opposite to the original MV. ii. In one example, if MvInfo1 and MvInfo2 are from the same LX (X=0 or 1) direction, then either MvInfo1 or MvInfo2 is stored, i.e., a single predicted motion vector is stored. iii. In one example, if MvInfo1 and MvInfo2 are from the same LX (X=0 or 1) direction, one motion vector may be scaled relative to the other reference picture and the average or weighted average of the scaled motion vector and the other motion vector may be stored. iv. In one example, if MvInfo1 and MvInfo2 are from the same LX (X=0 or 1) direction and the average or weighted average of the two motion vectors may be stored, the reference picture may be one of the two reference pictures, MvInof1 and MvInfo2. v. In one example, the stored motion information for the entire block may be derived from the motion information in each sub-block, e.g., by weighted summing the MVs in all or some of the sub-blocks to derive the stored MV for the entire block. vi. In the above example, different sub-regions may store different motion information. vii. In the above examples, sub-regions in the weighting area may store uni-predictive motion information or bi-predictive motion information. e. Alternatively, the storage of motion information is still based on the position of the sub-block, however, for each sub-block only uni-predictive motion information may be stored. i. In one example, motion information of sub-regions in the weighting region may be inherited in MvInfo1 and / or MvInfo2 or may be derived from MvInfo1 and / or MvInfo2, and MvInfo2 may be stored. f. In one example, the stored motion information is used in only some modules. i. In one example, the stored motion information is used for temporal motion prediction. ii. Alternatively, the stored motion information is used for spatial motion prediction. iii. Alternatively, the stored motion information is used in a filtering (eg, deblocking) process. g. In one example, the stored motion information may be used in the motion compensation process of the block. h. In one example, one or more HMVP tables may be updated based on the stored motion information. 21. It is proposed that the storage of motion information is based on MxN block units (M and N could be equal to 4 at the same time). Each MxN block shares the same motion information. In one example, M and N are set to 8. b. In one example, for an MxN block, some of which belong to a weighted region, some of which belong to a non-weighted region, and such blocks follow the weighted region rule for motion vector storage. c. Alternatively, for an MxN block, if a first portion thereof belongs to a weighted region but a second portion thereof belongs to a non-weighted region, such a block may store motion information according to the rules for the non-weighted region. 22. Predicting the TPM for two partitions from the same reference picture can be enabled or disabled under conditions such as the motion vectors of the two partitions being sufficiently different. In one example, the reference samples of the two splits must not overlap. In one example, abs(MV0[0]-MV1[0]) is less than TH, where MV0 and MV1 are the motion vectors of two partitions, and MVX[0] and MVX[1] are the horizontal and vertical components of MVX, respectively. The function abs(x) returns the absolute value of x. c. In one example, abs(MV0[1]-MV1[1]) is less than TH. d. In one example, abs(MV0[0]-MV1[0])+abs(MV0[1]-MV1[1]) is less than TH. e. In one example, Max(abs(MV0[0]-MV1[0]),abs(MV0[1]-MV1[1]) must be less than TH, and the function Max(x,y) returns the greater of x and y. f. A compliant bitstream satisfies one or more of the above bullets are true when the TPM is enabled for a block. 23. In the disclosed method, the stored motion information may be used to predict motion information of a subsequent block to be decoded in the current picture or another picture to be decoded. The stored motion information may include: vii. Motion Vectors viii. Reference Index ix. Uni- or bi-predictive display x. Display of inter prediction direction xi. Generalized Bi-Prediction (GBi) Display xii. Motion Vector Resolution xiii. Displaying Affine Predictions

[0240] 5. Exemplary embodiments of the disclosed technology 5.1 Embodiment #1 Motion Vector Storage in Triangle Merge Mode The variables numSbX and numSbY, which specify the number of 4x4 blocks in the horizontal and vertical directions in the current coding block, are equal to numSbX=cbWidth>>2 and numSbY=cbHeight>>2. cbWidth and cbHeigh specify the width and height of the current coding block in luma samples. The variable minSb is set equal to min(numSbX,numSbY)-1. The variable cbRatio is derived as follows: cbRatio=(cbWidth>cbHeight)?(cbWidth / cbHeight):(cbHeight / cbWidth) For each 4x4 sub-block with sub-block index (xSbIdx, ySbIdx), xSbIdx=0..numSbX-1, and ySbIdx=0..numSbY-1, the following applies: The variables xIdx and yIdx are derived as follows: xIdx=(cbWidth>cbHeight)?(xSbIdx / cbRatio):xSbIdx yIdx=(cbWidth>cbHeight)?ySbIdx:(ySbIdx / cbRatio) The variable sType is derived as follows: If -triangleDir is equal to 0, the following applies: sType=(xIdx==yIdx)?2:((xIdx>yIdx)?0:1) - Otherwise (triangleDir=1), the following applies: sType=(xIdx+yIdx==minSb)?2:((xIdx+yIdx <minSb)?0:1) triangleDir specifies the direction of the division. As shown in FIGS. 30A to 30C, sType=0 corresponds to the P1 region, sType=1 corresponds to the P2 region, and sType=2 corresponds to the weighted region. The motion information of the P1 region is set as (Mv1, refIdx1), and the motion information of the P2 region is set as (Mv2, refIdx2). Depending on the -sType value, the following assignments are made: - If sType is equal to 0, the motion information of the 4x4 sub-block is (Mv1, refIdx1). - Otherwise, if sType is equal to 1 or sType is equal to 2, and both Mv1 and Mv2 are from the same reference list, and the current block is located in a slice that has a backward reference picture (slice.getCheckLDC() is false), the motion information of the 4x4 sub-block is (Mv2, refIdx2). - Otherwise (sType equals 2), the following applies: refIdxL0=(predListFlagA==0)?refIdx1:refIdx2 refIdxL1=(predListFlagA==0)?refIdx2:refIdx1 mvL0=(predListFlagA==0)?Mv1:Mv2 mvL1=(predListFlagA==0)?Mv2:Mv1 predListFlagA is the prediction list flag of the P1 region. 5.2 Example #2 For each 4x4 sub-block with sub-block index (xSbIdx, ySbIdx), xSbIdx=0..numSbX-1, and ySbIdx=0..numSbY-1, the following applies: Depending on the -sType value, the following assignments are made: - If sType is equal to 0, the motion information of the 4x4 sub-block is (Mv1, refIdx1). - Otherwise, if sType is equal to 1, the motion information of the 4x4 sub-block is (Mv2, refIdx2). - Otherwise (sType equals 2), the following applies: refIdxL0=(predListFlagA==0)?refIdx1:refIdx2 refIdxL1=(predListFlagA==0)?refIdx2:refIdx1 mvL0=(predListFlagA==0)?Mv1:Mv2 mvL1=(predListFlagA==0)?Mv2:Mv1 predListFlagA is the prediction list flag of the P1 region. 5.3 Example #3 For each 4x4 sub-block with sub-block index (xSbIdx, ySbIdx), xSbIdx=0..numSbX-1, and ySbIdx=0..numSbY-1, the following applies: Depending on the -sType value, the following assignments are made: - If sType is equal to 0, the motion information of the 4x4 sub-block is (Mv1, refIdx1). - Otherwise, if sType is equal to 1 or sType is equal to 2 and both Mv1 and Mv2 are from the same reference list, then the motion information of the 4x4 sub-block is (Mv2, refIdx2). - Otherwise (sType equals 2), the following applies: -If Mv1 and Mv2 are from different reference lists refIdxL0=(predListFlagA==0)?refIdx1:refIdx2 refIdxL1=(predListFlagA==0)?refIdx2:refIdx1 mvL0=(predListFlagA==0)?Mv1:Mv2 mvL1=(predListFlagA==0)?Mv2:Mv1 predListFlagA is the prediction list flag of the P1 region. 5.4 Example #4 For each 4x4 sub-block with sub-block index (xSbIdx, ySbIdx), xSbIdx=0..numSbX-1, and ySbIdx=0..numSbY-1, the following applies: Depending on the -sType value, the following assignments are made: - If sType is equal to 0 or sType is equal to 2, and both Mv1 and Mv2 are from the same reference list, and the current block is located in a slice that has a backward reference picture (slice.getCheckLDC() is false), the motion information of a 4x4 sub-block is (Mv1, refIdx1). - Otherwise, if sType is equal to 1, the motion information of the 4x4 sub-block is (Mv2, refIdx2). - Otherwise (sType equals 2), the following applies: refIdxL0=(predListFlagA==0)?refIdx1:refIdx2 refIdxL1=(predListFlagA==0)?refIdx2:refIdx1 mvL0=(predListFlagA==0)?Mv1:Mv2 mvL1=(predListFlagA==0)?Mv2:Mv1 predListFlagA is the prediction list flag of the P1 region. 5.5 Embodiment #5 For each 4x4 sub-block with sub-block index (xSbIdx, ySbIdx), xSbIdx=0..numSbX-1, and ySbIdx=0..numSbY-1, the following applies: Depending on the -sType value, the following assignments are made: - If sType is equal to 0 or sType is equal to 2 and both Mv1 and Mv2 are from the same reference list, the motion information of a 4x4 sub-block is (Mv1, refIdx1). - Otherwise, if sType is equal to 1, the motion information of the 4x4 sub-block is (Mv2, refIdx2). - Otherwise (sType equals 2), the following applies: -If Mv1 and Mv2 are from different reference lists refIdxL0=(predListFlagA==0)?refIdx1:refIdx2 refIdxL1=(predListFlagA==0)?refIdx2:refIdx1 mvL0=(predListFlagA==0)?Mv1:Mv2 mvL1=(predListFlagA==0)?Mv2:Mv1 predListFlagA is the prediction list flag of the P1 region. 5.6 Example #6 For each 4x4 sub-block with sub-block index (xSbIdx, ySbIdx), xSbIdx=0..numSbX-1, and ySbIdx=0..numSbY-1, the following applies: Depending on the -sType value, the following assignments are made: - If sType is equal to 0, the motion information of the 4x4 sub-block is (Mv1, refIdx1). - Otherwise, if sType is equal to 1, the motion information of the 4x4 sub-block is (Mv2, refIdx2). - Otherwise (sType equals 2), the following applies: -If Mv1 and Mv2 are from different reference lists refIdxL0=(predListFlagA==0)?refIdx1:refIdx2 refIdxL1=(predListFlagA==0)?refIdx2:refIdx1 mvL0=(predListFlagA==0)?Mv1:Mv2 mvL1=(predListFlagA==0)?Mv2:Mv1 predListFlagA is the prediction list flag of the P1 region. - Otherwise, the motion information of the 4x4 sub-block is ((Mv1+Mv2) / 2, refIdx1). 5.7 Example #7 For each 4x4 sub-block with sub-block index (xSbIdx, ySbIdx), xSbIdx=0..numSbX-1, and ySbIdx=0..numSbY-1, the following applies: Depending on the -sType value, the following assignments are made: - If sType is equal to 0, the motion information of the 4x4 sub-block is (Mv1, refIdx1). - Otherwise, if sType is equal to 1, the motion information of the 4x4 sub-block is (Mv2, refIdx2). - Otherwise (when sType is equal to 2), the motion information of the 4×4 sub-block is the motion of the original merge candidate corresponding to merge_triangle_idx0. 5.8 Embodiment #8 For each 4x4 sub-block with sub-block index (xSbIdx, ySbIdx) where xSbIdx = 0..numSbX - 1 and ySbIdx = 0..numSbY - 1, the following applies. - Depending on the value of sType, the following assignments are made. - When sType is equal to 0 or sType is equal to 2, and both Mv1 and Mv2 are from the same reference list, and triangleDir is equal to 0, ySbIdx < numSbY - 1, the motion information of the 4x4 sub-block is (Mv1, refIdx1). - Otherwise, when sType is equal to 1 or sType is equal to 2, and both Mv1 and Mv2 are from the same reference list, and (triangleDir is equal to 0, ySbIdx = numSbY - 1 or triangleDir is equal to 1), the motion information of the 4x4 sub-block is (Mv2, refIdx2). - Otherwise (when sType is equal to 2), the following applies. - When Mv1 and Mv2 are from different reference lists refIdxL0 = (predListFlagA == 0)? refIdx1 : refIdx2 refIdxL1 = (predListFlagA == 0)? refIdx2 : refIdx1 mvL0 = (predListFlagA == 0)? Mv1 : Mv2 mvL1 = (predListFlagA == 0)? Mv2 : Mv1 predListFlagA is the prediction list flag for the P1 region. 5.9 Embodiment #9 For each 4x4 sub-block with sub-block index (xSbIdx, ySbIdx), xSbIdx=0..numSbX-1, and ySbIdx=0..numSbY-1, the following applies: - If both Mv1 and Mv2 are from the same reference list and the current block is located in a slice with a backward reference picture (slice.getCheckLDC() is false), the motion information of a 4x4 sub-block is (Mv2, refIdx2). - If not, the following applies: refIdxL0=(predListFlagA==0)?refIdx1:refIdx2 refIdxL1=(predListFlagA==0)?refIdx2:refIdx1 mvL0=(predListFlagA==0)?Mv1:Mv2 mvL1=(predListFlagA==0)?Mv2:Mv1 predListFlagA is the prediction list flag of the P1 region. 5.10 Example #10 For each 4x4 sub-block with sub-block index (xSbIdx, ySbIdx), xSbIdx=0..numSbX-1, and ySbIdx=0..numSbY-1, the following applies: refIdxL0=(predListFlagA==0)?refIdx1:refIdx2 refIdxL1=(predListFlagA==0)?refIdx2:refIdx1 mvL0=(predListFlagA==0)?Mv1:Mv2 mvL1=(predListFlagA==0)?Mv2:Mv1 predListFlagA is the prediction list flag of the P1 region. 5.11 Example #11 For each 4x4 sub-block with sub-block index (xSbIdx, ySbIdx), xSbIdx=0..numSbX-1, and ySbIdx=0..numSbY-1, the following applies: - If both Mv1 and Mv2 are from the same reference list, the motion information of a 4x4 sub-block is (Mv2, refIdx2). - If not, the following applies: refIdxL0=(predListFlagA==0)?refIdx1:refIdx2 refIdxL1=(predListFlagA==0)?refIdx2:refIdx1 mvL0=(predListFlagA==0)?Mv1:Mv2 mvL1=(predListFlagA==0)?Mv2:Mv1 predListFlagA is the prediction list flag of the P1 region. 5.12 Example #12 For each 4x4 sub-block with sub-block index (xSbIdx, ySbIdx), xSbIdx=0..numSbX-1, and ySbIdx=0..numSbY-1, the following applies: - If both Mv1 and Mv2 are from the same reference list and the current block is located in a slice with a backward reference picture (slice.getCheckLDC() is false), the motion information of a 4x4 sub-block is (Mv1, refIdx1). - If not, the following applies: refIdxL0=(predListFlagA==0)?refIdx1:refIdx2 refIdxL1=(predListFlagA==0)?refIdx2:refIdx1 mvL0=(predListFlagA==0)?Mv1:Mv2 mvL1=(predListFlagA==0)?Mv2:Mv1 predListFlagA is the prediction list flag of the P1 region. 5.13 Embodiment #13 For each 4x4 sub-block with sub-block index (xSbIdx, ySbIdx), xSbIdx=0..numSbX-1, and ySbIdx=0..numSbY-1, the following applies: - If both Mv1 and Mv2 are from the same reference list, the motion information of a 4x4 sub-block is (Mv1, refIdx1). - If not, the following applies: -If Mv1 and Mv2 are from different reference lists refIdxL0=(predListFlagA==0)?refIdx1:refIdx2 refIdxL1=(predListFlagA==0)?refIdx2:refIdx1 mvL0=(predListFlagA==0)?Mv1:Mv2 mvL1=(predListFlagA==0)?Mv2:Mv1 predListFlagA is the prediction list flag of the P1 region. 5.14 Embodiment #14 For each 4x4 sub-block with sub-block index (xSbIdx, ySbIdx), xSbIdx=0..numSbX-1, and ySbIdx=0..numSbY-1, the following applies: -If Mv1 and Mv2 are from different reference lists refIdxL0=(predListFlagA==0)?refIdx1:refIdx2 refIdxL1=(predListFlagA==0)?refIdx2:refIdx1 mvL0=(predListFlagA==0)?Mv1:Mv2 mvL1=(predListFlagA==0)?Mv2:Mv1 predListFlagA is the prediction list flag of the P1 region. - Otherwise, the motion information of the 4x4 sub-block is ((Mv1+Mv2) / 2, refIdx1). 5.15 Embodiment #15 For each 4x4 sub-block of sub-block indices (xSbIdx, ySbIdx) where xSbIdx = 0..numSbX-1 and ySbIdx = 0..numSbY-1, the motion information of the 4x4 sub-block is the motion of the original merge candidate corresponding to merge_triangle_idx0. 5.16 Embodiment #16 For each 4x4 sub-block with sub-block index (xSbIdx, ySbIdx) where xSbIdx = 0..numSbX-1 and ySbIdx = 0..numSbY-1, the following applies. - If both Mv1 and Mv2 are from the same reference list and triangleDir is equal to 0 and ySbIdx < numSbY-1, the motion information of the 4x4 sub-block is (Mv1, refIdx1). - Otherwise, if both Mv1 and Mv2 are from the same reference list (triangleDir is equal to 0, ySbIdx = numSbY-1 or triangleDir is equal to 1), the motion information of the 4x4 sub-block is (Mv2, refIdx2). - Otherwise, the following applies. refIdxL0 = (predListFlagA == 0)? refIdx1 : refIdx2 refIdxL1 = (predListFlagA == 0)? refIdx2 : refIdx1 mvL0 = (predListFlagA == 0)? Mv1 : Mv2 mvL1 = (predListFlagA == 0)? Mv2 : Mv1 predListFlagA is the prediction list flag of the P1 region.

[0241] 31 is a flow chart of a video processing method 3100. The method 3100 includes, at step 3102, determining, based on a priority rule, an order for inserting motion candidates into a motion candidate list for converting between a current video block and a bitstream representation of a video, where the current block is encoded using a geometry partition mode.

[0242] The method 3100 includes, at step 3104, performing a transformation based on the determination and the motion candidate list.

[0243] 32 is a flow chart of a video processing method 3200. The method 3200 includes, at step 3202, inserting one or more averaged uni-predictive motion candidates from List0 and List1 derived from one regular motion candidate into a motion candidate list for converting between a current video block and a bitstream representation of a video, the current block being coded using a geometry partitioning mode.

[0244] The method 3200 includes, at step 3204, performing a transformation based on the motion candidate list.

[0245] 33 is a flow chart of a video processing method 3300. The method 3300 includes, at step 3302, inserting one or more averaged motion candidates with uni-prediction derived from regular motion candidates with uni-prediction into a motion candidate list for converting between a current video block and a bitstream representation of a video, the current block being coded using a geometry partitioning mode.

[0246] The method 3300 includes, at step 3304, performing a transformation based on the motion candidate list.

[0247] 34 is a flow chart of a video processing method 3400. The method 3400 includes, at step 3402, inserting one or more hypothetical motion candidates derived from available motion candidates into a motion candidate list for converting between a current video block and a bitstream representation of a video, the current block being encoded using a geometry partitioning mode.

[0248] The method 3400 includes, at step 3404, performing a transformation based on the motion candidate list.

[0249] 35 is a flow chart of a video processing method 3500. The method 3500 includes, at step 3502, determining an order for inserting motion candidates into a motion candidate list for converting between a current video block encoded using a geometry partitioning mode and a bitstream representation of the video based on a priority rule, the priority rule being based on a position from which to derive one of the motion candidates or one or more reference picture lists associated with the motion candidate.

[0250] The method 3500 includes, at step 3504, performing a transformation based on the determination and the motion candidate list.

[0251] Figure 36 is a flow chart of a video processing method 3600. The method 3600 includes, at step 3602, determining an order for inserting motion candidates into a motion candidate list based on priority rules for converting between a current video block and a bitstream representation of the video, where the current block is coded using a geometry partitioning mode, and the ordering or insertion can be changed across sequences, pictures, slices, tile groups, or from the current block to a next block of the video.

[0252] The method 3600 includes, at step 3604, performing a transformation based on the determination and the motion candidate list.

[0253] 37 is a flow chart of a video processing method 3700. The method 3700 includes, at step 3702, dividing a current video block, which has been coded using a geometry partitioning mode, into a number of partitions.

[0254] The method 3700 includes, at step 3704, constructing a plurality of motion candidate lists, each motion candidate list corresponding to each of the plurality of partitions.

[0255] The method 3700 includes, at step 3706, converting between the current block and a bitstream representation of the video based on the multiple motion candidate lists.

[0256] 38 is a flow chart of a video processing method 3800. The method 3800 includes, at step 3802, dividing a current video block, which has been encoded using a geometry partitioning mode, into a number of partitions.

[0257] The method 3800 includes, at operation 3804, constructing a plurality of motion candidate lists, each of the plurality of divisions configured to select a same motion candidate list from the plurality of motion candidate lists.

[0258] The method 3800 includes, at step 3806, converting between the current block and a bitstream representation of the video based on the same selected motion candidate list.

[0259] 39 is a flow chart of a video processing method 3900. The method 3900 includes, at step 3902, converting between a current video block in a video unit and a bitstream representation of the video, the bitstream representation including a field indicating a maximum number of allowed motion candidates in a motion candidate list for a geometry partitioning mode enabled in the video domain.

[0260] 40 is a flow chart of a video processing method 4000. The method 4000 includes, at step 4002, selecting a set of weighting factors from a plurality of sets of weighting factors for a current video block that is divided into a plurality of divisions utilizing a geometry partitioning mode, selecting the set of weighting factors based on at least a width or height of the current block.

[0261] The method 4000 includes, at operation 4004, applying a set of weighting coefficients to samples along a common boundary of at least two of the multiple divisions as part of a conversion between the current block and a bitstream representation of the video.

[0262] 41 is a flow chart of a video processing method 4100. The method 4100 includes, at operation 4102, determining at least one weighting factor for samples in a current video block that is divided into multiple divisions using a geometry division mode based on an angle of a common boundary of at least two of the multiple divisions.

[0263] The method 4100 includes, at step 4104, converting between the current block and a bitstream representation of the video based on the at least one weighting factor.

[0264] 42 is a flow chart of a video processing method 4200. The method 4200 includes, at operation 4202, performing motion compensation on a current video block divided into a plurality of divisions using a geometry partitioning mode, with a sample size different from the 4×4 sample size on samples along a common boundary of at least two of the plurality of divisions.

[0265] The method 4200 includes, at act 4204, converting between the current block and a bitstream representation of the video based on a motion compensation process.

[0266] 43 is a flow chart of a video processing method 4300. The method 4300 includes, at operation 4302, dividing a current video block into multiple divisions using a geometry division mode.

[0267] The method 4300 includes, at operation 4304, storing a set of motion information for a K×L region in the current block derived from the motion information associated with the multiple partitions.

[0268] The method 4300 includes, at step 4306, converting between the current block and a bitstream representation of the video using at least one set of motion information associated with at least one partition of the plurality of partitions.

[0269] 44 is a flow chart of a video processing method 4400. The method 4400 includes, at operation 4402, dividing a current video block into a number of divisions.

[0270] The method 4400 includes, at operation 4404, converting between a current block encoded using a geometry partitioning mode and a bitstream representation of the video, where a set of motion information is stored for each M×N block unit of the current block, the set of motion information being derived from motion information associated with each of a plurality of partitions, where either M or N is not equal to four, and where each sample in the M×N block unit shares the same motion information.

[0271] 45 is a flow chart of a video processing method 4500. The method 4500 includes, at operation 4502, dividing a current video block into a number of divisions.

[0272] The method 4500 includes, at operation 4504, performing a motion compensation process on a first sub-region of the current block based on first motion information that is different from second motion information associated with the first sub-region to be stored as part of the conversion between the current block and a bitstream representation of the video.

[0273] 46 is a flow chart of a video processing method 4600. The method 4600 includes, at operation 4602, dividing a current video block into multiple divisions using a geometry division mode.

[0274] The method 4600 includes, at operation 4604, storing motion information of the first sample set and the second sample set based on a split direction or a decoded merge index or a merge candidate list associated with the multiple splits, where the first sample set is on a common boundary of at least two of the multiple splits and the second sample set is within at least two of the multiple splits.

[0275] The method 4600 includes, at operation 4606, converting between the current block and a bitstream representation of the picture or converting between the block and a bitstream representation of the subsequent picture based on the stored motion information.

[0276] 47 is a flow chart of a video processing method 4700. The method 4700 includes, at operation 4702, dividing a current video block into a number of divisions.

[0277] The method 4700 includes, at operation 4704, storing hypothetical bi-predictive motion information based on the motion information of at least two partitions of the plurality of partitions.

[0278] The method 4700 includes, at step 4706, converting between the current block and a bitstream representation of the video based on the assumed bi-predictive motion information.

[0279] 48 is a flow chart of a video processing method 4800. The method 4800 includes, at operation 4802, dividing a current video block into a number of divisions.

[0280] The method 4800 includes, at step 4804, storing motion information for a first partition (MvInfo1) of the multiple partitions and motion information for a second partition (MvInfo2) of the multiple partitions based on a low latency check flag, where the low latency check flag indicates all reference pictures having a picture order count (POC) value less than or equal to a POC value of the current picture that constitutes the current block.

[0281] The method 4800 includes, at step 4806, converting between the current block and a bitstream representation of the video based on the stored motion information.

[0282] In some embodiments, the following technical solutions can be implemented. A1. A video processing method comprising: determining, based on a priority rule, an order for inserting motion candidates into a motion candidate list for converting between a current video block and a bitstream representation of the video; encoding the current block using a geometry partitioning mode; and performing the conversion based on the determination and the motion candidate list. A2. The method of plan A1, wherein the priority rule is based on a prediction direction of at least one of the motion candidates. A3. The method of plan A2, wherein the order of insertion includes inserting any regular motion candidates with uni-prediction from list X before inserting any regular motion candidates with uni-prediction from list Y. A4. The method of plan A2, wherein the order of insertion includes inserting a motion candidate with bi-prediction before inserting any motion candidate with uni-prediction. A5. The method described in proposal A2, in which the order of insertion is defined as the motion candidates of list X derived from regular motion candidates with bi-prediction, the motion candidates of list X derived from regular motion candidates with bi-prediction, and the regular motion candidates with uni-prediction, where X=0 or X=1. A6. The method of A2, wherein the insertion order includes bi-predictive insertion of a motion candidate derived from a first regular motion candidate, followed by bi-predictive insertion of a motion candidate derived from a second regular motion candidate, and the insertion order of the current block is the same as the insertion order of video blocks encoded using a non-geometry partitioning mode. A7. The method of plan A2, wherein the order of insertion includes inserting a motion candidate derived from a first bi-predictive regular motion candidate and a motion candidate derived from a second bi-predictive regular motion candidate in an interleaved manner. A8. The method of proposal A7, wherein the interleaving scheme includes inserting a shortened List0 prediction candidate derived from CA, a shortened List1 prediction candidate derived from CB, a shortened List1 prediction candidate derived from CA, and a shortened List0 prediction candidate derived from CB, where CA and CB are two regular motion candidates. A9. The method of proposal A7, wherein the interleaving scheme includes inserting shortened List1 prediction candidates in List1 derived from CA, shortened List0 prediction candidates derived from CB, shortened list1 prediction candidates derived from CA, and shortened List1 derived from CB, where CA and CB are two regular motion candidates. A10. The method of any one of proposals A1-A9, wherein the regular motion candidates are motion candidates derived for blocks of video coded using a non-geometry partitioning mode. A11. The method of proposal A1, wherein the priority rule is based on coding mode information associated with the canonical motion candidates. A12. The method of claim A11, wherein the encoding mode information includes encoding using a merge mode or an advanced motion vector prediction (AMVP) mode. A13. The method of proposal A12, wherein the order of insertion includes inserting motion candidates derived from blocks coded in AMVP mode before inserting motion candidates derived from blocks coded in merge mode. A14. The method of proposal A12, wherein the order of insertion includes inserting motion candidates derived from blocks coded in merge mode before inserting motion candidates derived from blocks coded in AMVP mode. A15. The method of proposal A11, wherein the coding mode information includes a reference index or a picture order count (POC) difference. A16. The method of plan A15, wherein the order of insertion includes inserting a motion candidate associated with a first reference index followed by inserting a motion candidate associated with a second reference index that is greater than the first reference index. A17. The method described in proposal A15, wherein the order of insertion includes inserting a motion candidate associated with a first POC difference followed by inserting a motion candidate associated with a second POC difference reference index greater than the first POC difference. A18. The method of proposal A11, in which the coding mode information includes a quantization parameter (QP) or a temporal layer index of a reference picture associated with the picture, slice, or tile group that contains the current block. A19. The method of plan A18, in which the order of insertion includes inserting a motion candidate associated with a first QP followed by inserting a motion candidate associated with a second QP greater than the first QP. A20. The method of proposal A1, wherein the order of insertion includes inserting all shortened List1-predicted candidates before any shortened List0-predicted candidates. A21. The method of Plan A1, wherein an order of insertion includes inserting one or more truncated List0 prediction candidates and one or more truncated List1 prediction candidates based on available geometric motion candidates. A22. The method of plan A1, wherein the order of insertion includes inserting a first motion candidate with bi-prediction followed by inserting a second motion candidate with bi-prediction based on available geometric motion candidates. A23. A video processing method comprising: inserting one or more averaged single-prediction motion candidates from List0 and List1 derived from one regular motion candidate into a motion candidate list for converting between a current video block and a bitstream representation of the video, the current block being encoded using a geometry partitioning mode; and performing the conversion based on the motion candidate list. A24. The method of plan A23, further including selecting an averaged uni-predictive motion candidate from the one or more averaged uni-predictive motion candidates for insertion into the motion candidate list. A25. The method of proposal A24, wherein the selecting is based on available geometric motion candidates in a motion candidate list. A26. The method of proposal A24, wherein the selection is based on reference picture indexes of List0 and List1. A27. The method of proposal A24, wherein the selecting is based on a picture order count (POC) distance between the reference pictures for List0 and List1 and the current picture. A28. The method of proposal A23, inserting all averaged uni-predictive motion candidates from List0 before inserting any averaged uni-predictive motion candidates from List1. A29. The method of plan A23, inserting all averaged uni-predictive motion candidates from List1 before inserting any averaged uni-predictive motion candidates from List0. A30. A video processing method comprising: inserting one or more average motion candidates with uni-prediction derived from regular motion candidates with uni-prediction into a motion candidate list for converting between a current video block and a bitstream representation of the video, the current block being encoded using a geometry partitioning mode; and performing the conversion based on the motion candidate list. A31. The method of proposal A30, in which the averaged motion candidates with single prediction from list LX are derived from regular motion candidates with single prediction from list LX, where X=0 or X=1. A32. The method of proposal A30, in which average motion candidates with single prediction from list LY are derived from scaled regular motion candidates, and regular motion candidates from list LX are scaled to list LY to generate scaled regular motion candidates, where X=0 or X=1 and Y=1-X. A33. A video processing method comprising: inserting one or more virtual motion candidates derived from available motion candidates into a motion candidate list for converting between a current video block and a bitstream representation of the video, the current block being coded using a geometry partitioning mode; and performing the conversion based on the motion candidate list. A34. The method of plan A33, in which the one or more virtual motion candidates are derived by first generating a shortened list of motion candidates in a motion candidate list. A35. The method of proposal A33, wherein the one or more virtual motion candidates are derived based only on motion candidates having a single prediction. A36. The method of proposal A33, in which the one or more virtual motion candidates are derived based only on the single predicted motion candidate and the shortened list of predicted motion candidates from list 1. A37. The method of proposal A33, deriving one or more hypothetical motion candidates based on motion candidates that have a predetermined reference picture index or a picture order count (POC) distance within a certain range. A38. The method of proposal A33, in which one or more virtual motion candidates are derived without applying a pruning operation. A39. A method according to any one of the schemes A1 to A38, wherein a current block is divided into a plurality of divisions based on a division pattern. A40. The method of any of proposals A1-A39, wherein at least one division of the plurality of divisions is non-square and non-rectangular. A41. A method according to any one of proposals A1 to A39, wherein the geometry division mode includes a triangulation mode. A42. A method according to any one of proposals A1 to A41, in which the transformation generates the current block from a bitstream representation. A43. The method according to any one of Plans A1-A41, wherein the transformation generates a bitstream representation from the current block. A44. An apparatus in a video system including a processing device and a non-transitory memory loaded with instructions, the instructions executed by the processing device causing the processing device to implement a method described in any one of Solutions A1 to A43. A45. A computer program product stored on a non-transitory computer-readable medium, the computer program product comprising program code for executing a method according to any one of proposals A1 to A43.

[0283] In some embodiments, the following technical solutions can be implemented. B1. A video processing method comprising: determining, based on a priority rule, an order of insertion of motion candidates into a motion candidate list for converting between a current block of a video encoded using a geometry partitioning mode and a bitstream representation of the video, the priority rule being based on a position from which the motion candidates of the motion candidates were derived or one or more reference picture lists associated with the motion candidates; and performing the conversion based on the determination and the motion candidate list. B2. The video processing method described in plan B1, wherein if a first motion candidate CA is derived from block A, a second motion candidate CB is derived from block B, and block B is checked after block A, and the order of insertion includes inserting the motion candidate derived from CA into the motion candidate list followed by the motion candidate derived from CB. B3. The method of plan B2, further including checking block B after block A in the regular motion candidate list construction process. B4. The method of plan B2, wherein the order of insertion further includes inserting a motion candidate having a prediction from list X derived from CA after inserting a motion candidate having a prediction from list X derived from CB, where X=0 or X=1. B5. The method described in plan B2, wherein the order of insertion further includes inserting a motion candidate having a prediction from list Y derived from CA after inserting a motion candidate having a prediction from list Y derived from CB, where X=0 or X=1 and Y=(1-X). B6. The method described in plan B2, wherein the order of insertion further includes inserting motion candidates having predictions from list Y derived from CA followed by inserting motion candidates having predictions from list X derived from CB, where X=0 or X=1 and Y=(1-X). B7. The method according to any one of the plans B1-B6, dividing the current block into a plurality of divisions based on a division pattern. B8. A method according to any one of proposals B1 to B6, wherein the geometry division mode includes a triangulation mode. B9. The method of any of plans B1-B6, wherein the current block is divided into multiple divisions having at least one division that is non-square and non-rectangular. B10. The method of any one of plans B1-B9, wherein the transformation generates the current block from a bitstream representation. B11. The method of any one of Plans B1-B9, wherein the transformation generates a bitstream representation from the current block. B12. An apparatus in a video system comprising a processing device and a non-transitory memory loaded with instructions, the instructions executed by the processing device causing the processing device to implement a method described in any one of the proposals B1 to B11. B13. A computer program product stored on a non-transitory computer-readable medium, the computer program product including program code for executing a method according to any one of the methods B1-B11.

[0284] In some embodiments, the following technical solutions can be implemented. C1. A method of processing video including determining, based on a priority rule, an order for inserting motion candidates into a motion candidate list for converting between a current video block and a bitstream representation of the video, where the current block is coded using a geometry partitioning mode, and the order or insertion can be changed across sequences, pictures, slices, tile groups, or from the current block to a next block of the video, and performing the conversion based on the determination and the motion candidate list. C2. The method of plan C1, wherein the order of insertion is based on the current block or a sequence, picture, slice, or tile group that includes the current block. C3. The method of proposal C2, wherein the priority rule is based on at least one of a size, a shape, or a division pattern of the current block. C4. A method according to any one of proposals C1 to C3, in which the order of insertion is signaled from the encoder to the decoder in a sequence parameter set (SPS), a video parameter set (VPS), a picture parameter set (PPS), a slice header, a tile group header, a tile, a coding tree unit (CTU), or a coding unit (CU) in a bitstream representation. C5. A method of processing video, comprising: dividing a current video block encoded using a geometry partitioning mode into a plurality of partitions; constructing a plurality of motion candidate lists corresponding to each of the plurality of partitions; and converting between the current block and a bitstream representation of the video based on the plurality of motion candidate lists. C6. The method described in proposal C5, wherein a first list of the multiple motion candidate lists for a first division of the multiple divisions includes only motion candidates predicted from list X, and a second list of the multiple motion candidate lists for a second division of the multiple divisions includes only motion candidates predicted from list Y, where X=0 or X=1 and Y=(1-X). C7. The method of plan C5, wherein one or more motion candidates inserted into the multiple motion candidate lists are based on motion information associated with different spatial or temporal blocks. C8. The method of proposal C7, wherein the positions of the different spatial or temporal blocks used to derive one or more motion candidates for a first division of the multiple divisions are based on the positions of the first division or a second division of the multiple divisions in the current block. C9. The method of plan C7, in which the locations of the different spatial or temporal blocks used to derive one or more motion candidates for a first division or a second division of the multiple divisions are based on the division pattern of the current block. C10. The method of proposal C9, wherein the division pattern includes a division from the upper right corner to the lower left corner or from the upper left corner to the lower right corner, represented by a 45° division pattern or a 135° division pattern, respectively. C11. The method of plan C10, wherein the division pattern includes a 135 degree division pattern, and the method further includes checking more blocks for a division that includes an upper right corner of the current block. C12. The method of plan C10, wherein the division pattern includes a 135 degree division pattern, and the method further includes checking more left blocks for the partition that includes the lower left corner of the current block. C13. The method of proposal C5, signaling an index for each of a plurality of motion candidate lists corresponding to a plurality of partitions. C14. The method of plan C5, in which a bitstream representation including an index for each of a plurality of motion candidate lists is parsed for transformation. C15. A method for processing a video, comprising: dividing a current video block encoded using a geometry partitioning mode into a plurality of partitions; constructing a plurality of motion candidate lists, each of the plurality of partitions configured to select a same motion candidate list from the plurality of motion candidate lists; and converting between the current block and a bitstream representation of the video based on the same selected motion candidate list. C16. The method of plan C15, wherein the bitstream representation includes an index to a first motion candidate list of the multiple motion candidate lists. C17. The method of plan C16, in which the bitstream representation further includes an index of a motion candidate from a first motion candidate list, the index of the motion candidate being signaled after the index of the first motion candidate list. C18. The method of plan C16, wherein the bitstream representation further includes an index of a motion candidate from the first motion candidate list following an index of the first motion candidate list, and the bitstream representation is parsed for conversion. C19. The method of proposal C15, jointly encoding an index of the first motion candidate list with an index of a motion candidate from the first motion candidate list in the bitstream representation. C20. The method of plan C15, parsing for conversion a bitstream including an index of the first motion candidate list coded in conjunction with an index of a motion candidate from the first motion candidate list. C21. The method of any one of proposals C1-C20, dividing the current block into a plurality of partitions based on at least one division pattern. C22. The method according to any one of proposals C1 to C21, wherein the geometry decomposition mode includes a triangulation mode. C23. The method of any of Plans C1-C21, dividing the current block into a plurality of partitions, at least one partition being non-square and non-rectangular. C24. The method of any one of proposals C1-C23, wherein the conversion generates the current block from a bitstream representation. C25. The method of any one of proposals C1-C23, wherein the conversion generates a bitstream representation from the current block. C26. An apparatus in a video system including a processing device and a non-transitory memory loaded with instructions, the instructions executed by the processing device causing the processing device to implement a method described in any one of the proposals C1 to C25. C27. A computer program product stored on a non-transitory computer-readable medium, the computer program product comprising program code for executing a method according to any one of suggestions C1-C25.

[0285] In some embodiments, the following technical solutions can be implemented. D1. A method of processing an image, comprising: converting between a current image block in a video unit and a bitstream representation of the image, the bitstream representation including a field indicating a maximum number of allowed motion candidates in a motion candidate list for a geometry partitioning mode enabled in the image domain. D2. The method of proposal D1, in which the field includes an explicit signaling of an indication of the maximum number of motion candidates allowed in the bitstream representation. D3. The method of proposal D1, in which the field implicitly indicates that the maximum number of allowed motion candidates is equal to the maximum number of allowed motion candidates in a motion candidate list of another block that is coded using a non-geometry partitioning mode. D4. In the method according to any one of the proposals D1-D3, the current block is further encoded using a merge mode or an advanced motion vector prediction (AMVP) mode. D5. The method of any of plans D1-D3, wherein the video unit includes a sequence, a video, a picture, a slice, a tile group, a row of largest coding units (LCUs), or a group of largest coding units (LCUs). D6. A method according to any of plans D1 to D3, wherein an indication of the maximum number of motion candidates allowed in a field is signaled in a sequence parameter set (SPS), a video parameter set (VPS), a picture parameter set (PPS), a picture header, a slice header, a tile group header, a largest coding unit (LCU) row or a largest coding unit (LCU) group. D7. The method of any of plans D1-D6, wherein the indication of the maximum number of allowed motion candidates in the field is binarized with unary coding or truncated unary coding. D8. The method of plan D2, wherein explicitly signaling an indication of a maximum number of allowed motion candidates in the field includes signaling a difference between M and the maximum number of allowed motion candidates, where M is an integer. D9. The method of proposal D8, wherein the maximum number of allowed motion candidates for a block encoded using the geometry partitioning mode is set to M minus the parsed indicator. D10. The method according to plan D8 or D9, wherein M=5 or M=6. D11. The method of any one of Plans D1-D8, wherein a bitstream representation including an indication of the maximum number of allowed motion candidates in a field is parsed for conversion. D12. The method of proposal D1, in which the bitstream representation includes a single bit flag indicating that the maximum number of motion candidates allowed for a block of a video coded using a geometry partitioning mode is the same as the maximum number of regular motion candidates or sub-block motion candidates allowed. D13. The method of proposal D12, in which a bitstream representation including a single bit flag is parsed for conversion. D14. The method of any one of plans D1-D13, wherein the current block is divided into a plurality of divisions based on a division pattern. D15. A method according to any one of the proposals D1 to D14, wherein the geometry division mode includes a triangulation mode. D16. The method of any of plans D1-D14, dividing the current block into a plurality of divisions, where at least one division of the plurality of divisions is non-square and non-rectangular. D17. The method of any one of plans D1-D16, wherein the conversion generates the current block from a bitstream representation. D18. The method of any one of plans D1-D16, wherein the transformation generates a bitstream representation from the current block. D19. An apparatus in a video system including a processing device and a non-transitory memory loaded with instructions, the instructions executed by the processing device causing the processing device to implement a method described in any one of proposals D1 to D18. D20. A computer program product stored on a non-transitory computer-readable medium, the computer program product comprising program code for executing the method according to any one of suggestions D1-D18.

[0286] In some embodiments, the following technical solutions can be implemented. A video processing method including: selecting a set of weighting factors from a plurality of sets of weighting factors for a current video block partitioned into a plurality of partitions using a geometry partition mode, the set of weighting factors being selected based on at least one width or height of the current block, and applying the set of weighting factors to samples along a common boundary of at least two of the plurality of partitions as part of a conversion between the current block and a bitstream representation of the video. E2. The method of proposal E1, wherein the selecting is based on the ratio between the width and height being greater than a threshold. E3. The method of proposal E1, wherein the selecting is based on the ratio between the height and width being greater than a threshold. E4. The method of proposal E1, wherein the sets of weighting coefficients are predefined based on a width or height of the current block. E5. The method of plan E4, selecting one or two sets of weighting factors from the plurality of sets of weighting factors. E6. A method of processing a video comprising: determining at least one weighting factor for samples in a current video block divided into a plurality of divisions using a geometry partitioning mode based on an angle of a common boundary of at least two of the plurality of divisions; and converting between the current block and a bitstream representation of the video based on the at least one weighting factor. E7. The method of proposal E6, wherein the determining is further based on a location of the sample. E8. The method according to any one of proposals E1-7, deriving a final predicted value for a sample in the current block as a weighted sum of two predicted values ​​based on at least one weighting factor. E9. The method described in proposal E8, in which the final predicted block of the current block is (f(x,y)×P1(x,y)+(2M-f(x,y))×P2(x,y)+offset)>>M, where P1(x,y) and P2(x,y) are the first and second predicted values, respectively, of the sample with coordinates (x,y), M and the offset are integers, and f(x,y) returns the weight applied to the first predicted value. The method according to proposal E9, wherein E10.offset=(1<<(M-1). E11. The method according to any one of proposals E1-E10, wherein the current block is divided into a plurality of divisions based on a division pattern. E12. The method according to any one of proposals E1-E11, wherein the geometry decomposition mode includes a triangulation mode. E13. The method of proposal E12, wherein the geometry partitioning mode includes partitioning one block into two partitions. E12. The method of any of plans E1-E10, wherein the current block is divided into a plurality of partitions, and at least one of the plurality of partitions is non-square and non-rectangular. E13. The method according to any one of proposals E1-E12, wherein the transformation generates the current block from a bitstream representation. E14. The method according to any one of proposals E1-E12, wherein the transformation generates a bitstream representation from the current block. E15. An apparatus in a video system including a processing device and a non-transitory memory loaded with instructions, the instructions executed by the processing device causing the processing device to implement a method described in any one of proposals E1 to E14. E16. A computer program product stored on a non-transitory computer-readable medium, the computer program product comprising program code for executing a method according to any one of suggestions E1 to E14.

[0287] In some embodiments, the following technical solutions can be implemented. F1. A method for processing an image, comprising: performing a motion compensation process on a current image block divided into a plurality of divisions using a geometry partitioning mode, the motion compensation process being performed on samples along common boundaries of at least two of the plurality of divisions, the samples having a sample size different from a 4x4 sample size; and converting between the current block and a bitstream representation of the image based on the motion compensation process. F2. The method according to scheme F1, which performs motion compensation with a sample size of 8x8 to reduce the memory bandwidth required for the transform. F3. The method according to scheme F1, characterized in that the motion compensation process is performed with a sample size of 8x4 or 4x8, reducing the memory bandwidth required for the transformation. F4. The method described in option F3, where the sample size is based on the height or width of the current block. F5. The method according to scheme F3, in which the sample size is based on the partitioning pattern for multiple partitions of the current block. F6. The method of proposal F5, wherein the division pattern includes a division from the upper right corner to the lower left corner or from the upper left corner to the lower right corner, represented by a 45° division pattern or a 135° division pattern, respectively. F7. A method of processing video, comprising: partitioning a current video block into a plurality of partitions using a geometry partitioning mode; storing a set of motion information for a K×L region in the current block derived from motion information associated with the plurality of partitions; and converting between the current block and a bitstream representation of the video using at least one set of motion information associated with at least one of the plurality of partitions. F8. The method described in proposal F7, where MVInfoi represents motion information of the i-th partition, where i=1 and i=2, and when it is determined that MVInfo1 is from a first reference picture list LX and MVInfo2, one set of motion information includes bi-prediction and related motion information of the current block based on a combination of MVInfo1 and MVInfo2, where X=0 or X=1, and Y=(1-X). F9. The method described in proposal F7, wherein MVInfoi represents motion information for the i-th partition, where i=1 and i=2, and when it is determined that both MVInfo1 and MVInfo2 are from reference list LX, one motion information single prediction and a set of associated motion information for the current block based on MVInfoj, where j=1 or j=2, and X=0 or X=1. F10. The method of plan F8, in which a set of motion information including bi-prediction is derived by combining motion vectors and reference picture indexes of MVInfo1 and MVInfo2. F11. The method of scheme F9, in which one set of motion information including a single prediction is based on MVInfo1. F12. The method of scheme F9, in which one set of motion information including a single prediction is based on MVInfo2. F13. A method according to any one of proposals F1 to F12, wherein MVInfo1 includes a set of motion information associated with a partition covering the upper right corner sample, and MVInfo2 includes a set of motion information associated with a partition covering the lower left corner sample if it is determined that the upper right corner sample and the lower left corner sample are in two different partitions. F14. The method according to scheme F13, in which the division direction of the current block is from the upper left corner to the lower right corner. F15. A method according to any one of proposals F1 to F12, wherein MVInfo1 includes a set of motion information associated with a partition covering the upper left corner sample, and MVInfo2 includes a set of motion information associated with a partition covering the lower right corner sample if it is determined that the upper left corner sample and the lower right corner sample are in two different partitions. F16. The method according to scheme F15, in which the division direction of the current block is from the upper right corner to the lower left corner. F17. The method of plan F7, wherein one set of stored motion information is used for one of temporal motion estimation, spatial motion estimation, or filtering of a current block of the image for a subsequent block. F18. The method of plan F7, in which one stored set of motion information is used to process other blocks in different pictures. F19. The method of proposal F17, wherein the filtering process includes deblocking or adaptive loop filtering (ALF). F20. The method of Scheme 7, wherein MVInfoi and MVInfoj represent motion information for the i-th partition and the j-th partition, respectively, both MVInfoi and MVInfoj are from reference picture list LX, where X=0 or X=1, and one set of motion information is based on combining MVInfoi and a scaled motion vector generated by scaling MVInfoj to reference picture list LY, where Y=(1-X). The method of scheme F20, wherein the value of F21.X is predetermined or signaled in the bitstream representation. F22. The method according to proposal F7, wherein MVInfoi and MVInfoj represent motion information for the i-th partition and the j-th partition, respectively, both MVInfoi and MVInfoj are from a reference picture list LX, where X=0 or X=1, and one set of motion information is based on combining MVInfoi and a mapped motion vector generated by mapping MVInfoj to the reference picture list LY, where Y=(1-X). F23. The method of scheme F22, wherein one set of motion information includes bi-predictive motion vectors. F24. The method of proposal F22, in which mapping MVInfoj to reference pictures in list LY excludes a scaling operation. F25. The method of plan F7, wherein storing one set of motion information is based on decoded motion information associated with multiple partitions. F26. The method described in proposal F25, in which MVInfoi and MVInfoj represent motion information for the i-th partition and the j-th partition, respectively, MVInfoi and MVInfoj are both from the same reference picture list LX, X=0 or X=1, and one set of motion information includes uni-predictive motion information. F27. The method described in proposal F25, wherein MVInfoi and MVInfoj represent motion information for the i-th partition and the j-th partition, respectively, MVInfoi is from reference picture lists LX and MVInfoj, X=0 or X=1, Y=(1-X), and one set of motion information includes bi-predictive motion information. F28. The method described in proposal F25, wherein MVInfoi and MVInfoj represent motion information for the i-th and j-th partitions, respectively, both MVInfoi and MVInfoj are from reference picture list LX, X=0 or X=1, reference picture list LY contains MVInfoi or MVInfoj, Y=(1-X), and one set of motion information includes bi-predictive motion information. F29. The method of plan F7, in which storing one set of motion information is based on a sub-region of the current block that is located along a common boundary of at least two of the multiple partitions. F30. The method of scheme F29, in which different sub-regions store different motion information. F31. The method of proposal F29, wherein the subregion stores uni-predictive or bi-predictive motion information. F32. The method of plan F7, in which storing the single set of motion information is based on a position of one sub-block in the current block, and the sub-block stores only single predictive motion information. F33. The method according to any of the plans F1-F32, wherein one set of motion information is used in the motion compensation process applied to the current block. F34. The method described in proposal F7, wherein MVInfoi and MVInfoj represent motion information for the i-th and j-th partitions, respectively, both MVInfoi and MVInfoj are from reference picture list LX, X=0 or X=1, and one set of motion information includes bi-predictive motion vectors for reference picture list LX derived from (a) MVInfoi, (b) MVInfoj, or (c) averaging MVInfoi and MVInfoj. F35. The method described in proposal F7, wherein MVInfoi and MVInfoj represent motion information for the i-th and j-th partitions, respectively, both MVInfoi and MVInfoj are from reference picture list LX, X=0 or X=1, and one set of motion information includes an average or weighted average of MVInfoi and MVInfoj, and reference pictures associated with one set of motion information include either reference pictures of MVInfoi or reference pictures of MVInfoj. F36. The method of any of plans F1-F35, wherein the K×L region includes at least one sample at a common boundary between at least two divisions of the plurality of divisions. F37. A method according to any one of proposals F1 to F36, wherein the geometry division mode includes a triangulation mode. F38. The method of any one of plans F1-F36, wherein at least one division of the plurality of divisions is non-square and non-rectangular. F39. The method of any one of plans F1-F38, wherein the transformation generates the current block from a bitstream representation. F40. The method according to any one of proposals F1-F38, wherein the transformation generates a bitstream representation from the current block. F41. An apparatus in a video system including a processing device and a non-transitory memory loaded with instructions, the instructions executed by the processing device causing the processing device to implement a method described in any one of the proposals F1 to F40. F42. A computer program product stored on a non-transitory computer-readable medium, the computer program product comprising program code for executing a method according to any one of suggestions F1 to F40.

[0288] In some embodiments, the following technical solutions can be implemented. G1. A method of processing video, comprising: partitioning a current video block into a plurality of partitions; and converting between the current block encoded using a geometry partitioning mode and a bitstream representation of the video, wherein one set of motion information is stored for each M×N block unit of the current block, the one set of motion information being derived from motion information associated with each of the plurality of partitions, wherein M or N is not equal to 4, and each sample in the M×N block unit shares the same motion information. G2. The method according to scheme G1, wherein M=8 and N=8. G3. The method of proposal G1, wherein a first portion of the M×N block unit comprises a first sample set that is on a common boundary of at least two of the plurality of partitions, and a second portion of the M×N block unit comprises a second sample set that is within one of at least two of the plurality of partitions. G4. The method described in proposal G3, in which when it is determined that MVInfoi represents motion information of the i-th partition, where i=1 and i=2, MVInfo1 is from the first reference picture list LX, and MVInfo2 is from LY, the same motion information includes bi-prediction and related motion information of the current block based on MVInfoi of multiple partitions, where X=0 or X=1, and Y=(1-X). G5. The method described in proposal G3, in which MVInfoi represents motion information for the i-th partition, where i=1 and i=2, and when it is determined that both MVInfo1 and MVInfo2 are from the reference picture list LX, the same motion information includes single prediction and associated motion information for the current block based on MVInfoj, where j=1 or j=2, and X=0 or X=1. G6. The method described in proposal G3, in which MVInfoi represents motion information for the i-th partition, where i=1, i=2, and the same motion information includes single prediction and associated motion information for the current block based on MVInfoj, where j=1 or j=2. G7. A method of video processing comprising: dividing a current video block into a plurality of divisions; and performing a motion compensation process on a first sub-region of the current block based on first motion information different from second motion information associated with the first sub-region to be stored as part of a conversion between the current block and a bitstream representation of the video. G8. A method as described in proposal G7, wherein the first subregion includes M×N sample sets on a common boundary of at least two of the multiple divisions, the motion compensation process includes bi-prediction, and only one set of motion information from bi-prediction List0 or List1 is stored. G9. The method of proposal G8, in which one set of motion information is used to derive spatial motion candidates for neighboring blocks that are coded using merge mode or advanced motion vector prediction (AMVP) mode. G10. The method described in proposal G8, in which one set of motion information is used to derive temporal motion candidates for a subsequent block in a picture different from the current picture containing the current block. G11. The method according to proposal G8, in which one set of motion information is used for in-loop processing of the current block. G12. The method of proposal G11, wherein the in-loop processing includes deblocking or adaptive loop filtering. G13. A method as described in proposal G7, comprising: a motion compensation process for a second sub-region of the current block based on third motion information associated with the second sub-region; and the first sub-region and the second sub-region are within the same coding unit (CU) including multiple divisions. G14. A method of processing a video, comprising: partitioning a current video block into a plurality of partitions using a geometry partition mode; storing motion information of a first set of samples and a second set of samples based on partition directions or decoded merge indexes or merge candidate lists associated with the plurality of partitions, wherein the first set of samples is on a common boundary of at least two of the plurality of partitions and the second set of samples is within one of at least two of the plurality of partitions; and converting between the current block and a bitstream representation of the video or between a subsequent block of the video and a bitstream representation based on the stored motion information. G15. The method of proposal G14, wherein the stored motion information corresponds to motion information associated with one partition of a plurality of partitions associated with the selected decoding merge index. G16. The method of proposal G14, wherein the stored motion information corresponds to motion candidates in a motion candidate list where the selected merge index set is variable k. G17. The method according to scheme G16, wherein k=0. G18. The method of proposal G16, wherein the stored motion information corresponds to the motion information of the kth merging candidate in one merging candidate list. G19. The method of proposal G15, wherein the selected decoding merge index is based on the split direction. G20. The method of proposal G15, wherein if a merge candidate having an index equal to the selected decoded merge index is determined to be bi-predictive, the stored motion information includes bi-predictive motion information. G21. The method of proposal G15, wherein if it is determined that a merge candidate having an index equal to the selected decoded merge index is based on uni-prediction, the stored motion information includes uni-prediction motion information. G22. The method described in proposal G6, wherein the first sub-region includes M×N samples on a common boundary of at least two of the multiple divisions, the motion compensation process includes bi-prediction, and only one set of motion information from bi-prediction List0 or List1 is stored. G23. The method described in proposal G6, wherein the first sub-region includes M×N samples on a common boundary of at least two of the multiple divisions, the motion compensation process includes bi-prediction, and only single prediction information from ListX is stored, where X=0 and X=1. G24. The method described in proposal G6, wherein the first sub-region includes M×N samples on a common boundary of at least two of the multiple divisions, the motion compensation process includes single prediction, and the second motion information includes single prediction information from a stored ListX, where X=0 and X=1. G25. A method of processing a video comprising: dividing a current video block into a plurality of divisions; storing hypothetical bi-predictive motion information based on motion information of at least two of the plurality of divisions; and converting between the current block and a bitstream representation of the video based on the hypothetical bi-predictive motion information. G26. The method described in proposal G25, wherein MvInfo1 and MvInfo2 are motion information for the first and second partitions, respectively, and the hypothetical bi-predictive motion information is based on modifying the prediction direction of MvInfoA from ListX to ListY, where A=1 or A=2, X=0 or X=1, and Y=(1-X). The method of proposal G26, wherein the motion vectors and reference indexes of MvInfoA are maintained unchanged. The method according to proposal G26, in which the reference index of MvInfoA is not changed and the motion vector of MvInfoA is set to the opposite value. G29. A method according to any one of proposals G26 to G28, wherein MvInfo1 and MvInfo2 are from the same prediction direction. G30. A method for processing a video, comprising: dividing a current video block into a plurality of divisions; storing motion information of a first division (MvInfo1) of the plurality of divisions and motion information of a second division (MvInfo2) of the plurality of divisions based on a low latency check flag, the low latency check flag indicating all reference pictures having picture order count (POC) values ​​less than or equal to a POC value of a current picture that constitutes the current block; and converting between the current block and a bitstream representation of the video based on the stored motion information. G31. The method of proposal G30, wherein MvInfo1 and MvInfo2 are from different prediction directions, and storing the motion information includes storing bi-predictive motion information generated by combining MvInfo1 and MvInfo2. G32. The method of proposal G30, wherein MvInfo1 and MvInfo2 are from the same prediction direction (ListX), X=0 or X=1, and further including modifying MvInfo2 by setting the bi-predictive list of MvInfo2 from ListX to ListY, where Y=(1-X), and storing the motion information includes storing bi-predictive operation generated by combining MvInfo1 and the modified MvInfo2. G33. The method according to any one of plans G25 to G32, wherein storing the motion information includes storing motion information of a sub-block of the current block. G34. The method of proposal G33, in which the sub-block includes samples on a common boundary of at least two of the multiple divisions. G35. The method according to proposal G33, wherein the sub-block comprises a diagonal or anti-diagonal in the current block. G36. The subblock is located in the lower right corner of the current block, as described in scheme G33. G37. The method according to proposal G33, in which the sub-block is located in the right column or below the current block. G38. The method of any one of proposals G1 to G37, wherein at least one division of the plurality of divisions is non-square and non-rectangular. G39. A method according to any one of plans G1 to G38, in which the conversion generates the current block from a bitstream representation. G40. A method according to any one of proposals G1 to G38, wherein the transformation generates a bitstream representation from the current block. G41. An apparatus in a video system including a processing device and a non-transitory memory loaded with instructions, the instructions executed by the processing device causing the processing device to implement a method described in any one of the proposals G1 to G40. G42. A computer program product stored on a non-transitory computer-readable medium, the computer program product comprising program code for executing a method according to any one of proposals G1 to G40.

[0289] FIG. 49 is a block diagram of a video processing device 4900. The device 4900 may be used to implement one or more of the methods described herein. The device 4900 may be implemented in a smartphone, tablet, computer, Internet of Things (IoT) receiver, etc. The device 4900 may include one or more processing devices 4902, one or more memories 4904, and video processing hardware 4906. The one or more processing devices 4902 may be configured to implement one or more of the methods described herein. The memory(s) 4904 may be used to store data and codes used to implement the methods and techniques described herein. The video processing hardware 4906 may be used to implement the techniques described herein in a hardware circuit.

[0290] In some embodiments, the video encoding method may be performed using an apparatus implemented on a hardware platform, such as that described with reference to FIG.

[0291] Some embodiments of the disclosed technology include making a decision or determination to enable a video processing tool or mode. In one example, if a video processing tool or mode is enabled, the encoder uses or implements the tool or mode when processing a single video block, but may not necessarily modify the resulting bitstream based on the use of the tool or mode. That is, the conversion from a block of video to a bitstream representation of video uses the video processing tool or mode if the video processing tool or mode is enabled based on the decision or determination. In another example, if the video processing tool or mode is enabled, the decoder processes the bitstream knowing that the bitstream has been modified based on the video processing tool or mode. That is, the conversion from the bitstream representation of video to a block of video is performed using the video processing tool or mode enabled based on the decision or determination.

[0292] Some embodiments of the disclosed technology include making a decision or determination to disable a video processing tool or mode. In one example, when a video processing tool or mode is disabled, an encoder does not use the tool or mode when converting blocks of a video into a bitstream representation of the video. In another example, when a video processing tool or mode is disabled, a decoder processes the bitstream with the knowledge that the bitstream has not been modified using the video processing tool or mode that was enabled based on the decision or determination.

[0293] FIG. 50 is a block diagram illustrating an example video processing system 5000 in which various techniques disclosed herein may be implemented. Various implementations may include some or all of the modules of the system 5000. The system 5000 may include an input unit 5002 for receiving video content. The video content may be received in a raw or uncompressed format, for example, 8 or 10 bit multi-module pixel values, or may be received in a compressed or encoded format. The input unit 5002 may represent a network interface, a peripheral bus interface, or a storage interface. Examples of network interfaces include wired interfaces such as Ethernet, passive optical network (PON), and wireless interfaces such as Wi-Fi or cellular interfaces.

[0294] The system 5000 may include an encoding module 5004 that may implement various encoding or coding methods described herein. The encoding module 5004 may reduce the average bit rate of the video from the input unit 5002 to the output of the encoding module 5004, generating an encoded representation of the video. This encoding technique may therefore be referred to as a video compression or video transcoding technique. The output of the encoding module 5004 may be stored or transmitted via a connected communication, as represented by module 5006. The bitstream (or encoded) representation of the video received at the input unit 5002, stored or communicated, may be used by module 5008 to generate pixel values ​​or displayable video that are sent to the display interface unit 5010. The process of generating a user-viewable video from the bitstream representation may be referred to as video decompression. Furthermore, although certain video processing operations are referred to as "encoding" operations or tools, it will be understood that the encoding tools or operations are used in an encoder and a corresponding decoder tool, and that the decoding tools or operations that reverse the results of the decoding are performed by a decoder.

[0295] Examples of peripheral bus interface units or display interface units may include Universal Serial Bus (USB) or High-Definition Multimedia Interface (HDMI) or DisplayPort, etc. Examples of storage interfaces include Serial Advanced Technology Attachment (SATA), PCI, IDE interfaces, etc. The techniques described herein may be implemented in various electronic devices such as mobile phones, laptops, smartphones, or other devices capable of digital data processing and / or video display.

[0296] For purposes of the discussion herein, the bitstream representation of the current video block does not necessarily have to be in the form of contiguous bits, but may be based on bits present in different positions in the bitstream syntax (e.g., header fields and parameter sets).

[0297] Furthermore, in some disclosed embodiments, for sub-block coding modes that include ATMVP and affine merge, one merge list construction process is shared, where ATMVP and affine merge candidates may be added in order.

[0298] In some embodiments, in triangular prediction mode, one merge list construction process for two partitions is shared even if the two partitions select their own merge candidate indexes. When constructing this merge list, we check the spatially neighboring blocks and the two temporal blocks of this block. The motion information derived from the spatial neighbors and the temporal blocks is called regular motion candidates in our IDF. These regular motion candidates are further utilized to derive multiple TPM candidates. This conversion may be done at the whole block level, and the two partitions may use different motion vectors to generate their own prediction blocks.

[0299] For the remaining coding blocks, we share one merge list construction process. Spatial / temporal / HMVP, pairwise composite bi-predictive merge candidates, and zero motion candidates may be inserted in order.

[0300] It will be appreciated by those skilled in the art that a video encoding or decoding method (compression / decompression) is disclosed. A geometric (or geometric) partitioning of video blocks may be used to accurately track motion information and improve the compression performance of the encoded video. It will further be appreciated that by using the disclosed method, embodiments can achieve additional coding gain and / or reduce implementation complexity compared to today's video encoding / decoding techniques.

[0301] The solutions, examples, embodiments, modules, and functional operations disclosed herein, including the structures disclosed herein and their structural equivalents, may be implemented in digital electronic circuitry, or computer software, firmware, or hardware, or one or more combinations thereof. The disclosed and other embodiments may be embodied as one or more computer program products, i.e., one or more modules of computer program instructions encoded on a computer-readable medium for implementation by or for controlling the operation of a data processing apparatus. The computer-readable medium may be a machine-readable storage device, a machine-readable storage substrate, a memory device, a composition of matter that provides a machine-readable propagated signal, or one or more combinations thereof. The term "data processing apparatus" includes all apparatus, devices, and machines for processing data, including, for example, a programmable processing apparatus, a computer, or multiple processing apparatuses or computers. The apparatus may include, in addition to hardware, code that creates an environment for the execution of the computer program, such as code that constitutes a processing apparatus firmware, a protocol stack, a database management system, an operating system, or one or more combinations thereof. A propagated signal is an artificially generated signal, for example a machine-generated electrical, optical, or electromagnetic signal, that is generated to encode information for transmission to an appropriate receiving device.

[0302] A computer program (also called a program, software, software application, script, or code) can be written in any form of programming language, including compiled or interpreted languages, and it can be deployed as a stand-alone program or in any form including modules, components, subroutines, or other units suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. A program may be recorded as part of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), may be stored in one file dedicated to the program, or may be stored in multiple coordinating files (e.g., a file that stores one or more modules, subprograms, or code portions). A computer program can be deployed to run on one computer located at one site, or on multiple computers distributed across multiple sites and interconnected by a communication network.

[0303] The processes and logic flows described herein may be performed by one or more programmable processing devices executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows may also be performed by, and devices may be implemented as, special purpose logic circuits, such as an FPGA (field programmable gate array) or an ASIC (application specific integrated circuit).

[0304] Processors suitable for executing a computer program include, for example, both general purpose and special purpose microprocessors, as well as any one or more processors of any kind of digital computer. Typically, a processor receives instructions and data from a read-only memory or a random access memory or both. The essential elements of a computer are a processor for executing instructions and one or more memory devices for storing instructions and data. Typically, a computer may include one or more mass storage devices, e.g., magnetic, magneto-optical, or optical disks, for storing data, or may be operatively coupled to receive data from or transfer data to these mass storage devices. However, a computer need not have such devices. Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including, for example, EPROMs, EEPROMs, flash memory devices, magnetic disks, e.g., internal hard disks or removable disks, magneto-optical disks, and semiconductor memory devices, such as CD-ROM and DVD-ROM disks. The processor and memory may be supplemented by, or incorporated in, special purpose logic circuitry.

[0305] Although this patent specification contains many details, these should not be construed as limiting the scope of any subject matter or what may be claimed, but as descriptions of features that may be specific to particular embodiments of a particular technology. Certain features described in this patent specification in the context of another embodiment may be implemented in combination in one example. Conversely, various features described in the context of one example may be implemented in multiple embodiments separately or in any suitable subcombination. Furthermore, although features may be described above as acting in a particular combination and initially claimed as such, one or more features from a claimed combination may, in some cases, be extracted from the combination, and the claimed combination may be directed to a subcombination or a variation of the subcombination.

[0306] Similarly, although operations are shown in a particular order in the figures, this should not be understood as requiring that such operations be performed in the particular order or sequential order shown, or that all of the operations shown be performed, to achieve desired results, nor should the separation of various system modules in the examples described in this patent specification be understood as requiring such separation in all embodiments.

[0307] Only some implementations and examples have been described; other embodiments, extensions and variations are possible based on what is described and illustrated in this patent specification.

Claims

1. In a conversion between a current block of a video and a bitstream of the video, the conversion includes determining that the current block is coded in a geometry partition mode, and the conversion includes decoding from the bitstream to the current block or encoding the current block to the bitstream; determining a first motion information MVInfo1 and a second motion information MVInfo2; and if both the MVInfo1 and the MVInfo2 are from a first reference picture list LX, storing a first set of single motion information for m×n sized sub-blocks within a weighting region of the current block, regardless of the presence or absence of the reference picture of the MVInfo1 in a second reference picture list L(1-X) or the presence or absence of the reference picture of the MVInfo2 in a second reference picture list L(1-X), wherein the first set of single motion information includes single-prediction motion information, and X=0 or X=1.

2. The method of claim 1 , wherein the first set of single motion information including the single predictive motion information is based on MVInfo1 or MVInfo2.

3. The method of claim 1 or 2, wherein the first single set of motion information including the single predictive motion information is equal to MVInfo1 or MVInfo2.

4. The method according to any one of claims 1 to 3, wherein the stored first single set of motion information is used for at least one of a temporal motion prediction, a spatial motion prediction, or a filtering process.

5. The method according to any one of claims 1 to 3, wherein the stored first single set of motion information is used for processing other blocks in different pictures.

6. 6. The method of claim 1, further comprising: storing a second single set of motion information for m×n sized sub-blocks within the weighted area of ​​the current block in response to MVInfo1 being from the first reference picture list LX and MVInfo2 being from the second reference picture list L(1-X), wherein the second single set of motion information comprises bi-predictive motion information, and X=0 or X=1.

7. The method of claim 6 , wherein the bi-predictive motion information is based on combining the MVInfo1 and the MVInfo2.

8. The method of claim 7 , wherein the bi-predictive motion information is derived by combining the motion vectors and reference picture indexes of the MVInfo1 and the MVInfo2.

9. The method according to any one of claims 1 to 8, further comprising storing a third single set of motion information of size m x n in a non-weighted region, the third single set of motion information including single predicted motion information based on MVInfo1 or MVInfo2.

10. The method according to any one of claims 1 to 9, wherein m=4 or 8 and n=4 or 8.

11. 11. The method of claim 1, wherein the geometry partitioning mode includes multiple partitioning schemes, at least one partitioning scheme splits the current block with a partition boundary into two partitions, at least one of which is non-square and non-rectangular, and in each partition, a number of pixels around the partition boundary are part of the weighting region.

12. 12. The method of claim 11, wherein if the top right corner sample and the bottom left corner sample are in two different partitions, MVInfo1 contains a set of motion information associated with the partition covering the top right corner sample, and MVInfo2 contains a set of motion information associated with the partition covering the bottom left corner sample.

13. The method of claim 12 , wherein the direction of the partition boundary of the current block is from the top left corner to the bottom right corner.

14. 12. The method of claim 11, wherein if the top-left corner sample and the bottom-right corner sample are in two different partitions, MVInfo1 contains a set of motion information associated with the partition covering the top-left corner sample, and MVInfo2 contains a set of motion information associated with the partition covering the bottom-right corner sample.

15. The method of claim 14 , wherein the direction of the partition boundary of the current block is from the top right corner to the bottom left corner.

16. 16. A method according to any one of claims 1 to 15, wherein in a weighting process applied to generate a final prediction for an mxn size sub-block within the weighting region as a weighted sum of prediction samples derived based on MVInfo1 and MVInfo2, none of the weights of prediction samples of the mxn size sub-block within the weighting region are equal to 0.

17. 1. An apparatus for processing video data comprising a processor and a non-transitory memory storing instructions, the instructions executed by the processor causing the processor to: In a conversion between a current block of a video and a bitstream of the video, the conversion includes determining that the current block is coded in a geometry partition mode, and the conversion includes decoding from the bitstream to the current block or encoding the current block to the bitstream; determining a first motion information MVInfo1 and a second motion information MVInfo2; 4. An apparatus for processing video data, comprising: if both MVInfo1 and MVInfo2 are from a first reference picture list LX, storing a first single set of motion information for m×n sized sub-blocks within a weighting region of the current block, regardless of the presence or absence of the reference picture of MVInfo1 in a second reference picture list L(1-X) or the presence or absence of the reference picture of MVInfo2 in a second reference picture list L(1-X), wherein the first single set of motion information includes single-prediction motion information, and X=0 or X=1.

18. A non-transitory computer readable storage medium storing instructions for causing a processor to: In a conversion between a current block of a video and a bitstream of the video, the conversion includes determining that the current block is coded in a geometry partition mode, and the conversion includes decoding from the bitstream to the current block or encoding the current block to the bitstream; determining a first motion information MVInfo1 and a second motion information MVInfo2; a first set of single motion information for m×n sized sub-blocks within a weighted region of the current block, regardless of the presence or absence of the reference picture of the MVInfo1 in a second reference picture list L(1-X) or the presence or absence of the reference picture of the MVInfo2 in a second reference picture list L(1-X), if both the MVInfo1 and the MVInfo2 are from a first reference picture list LX, wherein the first set of single motion information includes single-prediction motion information, and X=0 or X=1.

19. determining that a current block of a video is coded in a geometry partition mode in encoding the current block of the video into a bitstream of the video; determining a first motion information MVInfo1 and a second motion information MVInfo2; generating a bitstream based on said determining; storing a first set of single motion information for m×n sized sub-blocks within a weighting region of the current block if both the MVInfo1 and the MVInfo2 are from a first reference picture list LX, regardless of the presence or absence of the MVInfo1 reference picture in a second reference picture list L(1-X) or the presence or absence of the MVInfo2 reference picture in a second reference picture list L(1-X), where the first set of single motion information includes single-prediction motion information, and X=0 or X=1; 1. A method for storing a video bitstream, comprising: storing the bitstream on a non-transitory computer-readable recording medium.

Images