Ordered motion candidate list generation using geometric partitioning mode

By adopting geometric segmentation mode and optimized motion candidate list construction method in video encoding, the problem of low video encoding efficiency in the prior art is solved, and more efficient video data processing and storage is achieved.

CN111418210BActive Publication Date: 2024-11-22DOUYIN VISION CO LTD +1
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN201980004147.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Priority Date
2019-06-20
Filing Date
2019-11-06
Publication Date
2024-11-22
Estimated Expiration
2039-11-06

AI Technical Summary

Technical Problem

The existing video encoding technology uses bandwidth efficiency when processing digital video, especially in the Internet and digital communication networks, resulting in further improvement in video encoding efficiency to cope with the growing bandwidth demand.

Method used

Using a video encoding method based on geometric segmentation mode, the motion candidate insertion order is determined through priority rules, and multiple motion candidate lists are constructed to optimize the bitstream representation of the video, combining the weighting factor group and the motion compensation process to improve the encoding efficiency.

Benefits of technology

It improves the quality and efficiency of video encoding, reduces the computational complexity and storage requirements during the decoding process, and is suitable for existing HEVC and future video encoding standards.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN111418210B_ABST
    Figure CN111418210B_ABST
Patent Text Reader

Abstract

An exemplary method for video processing includes: making a decision, based on a priority rule, about an order in which motion candidates are inserted into a motion candidate list for conversion between a current block of video and a bitstream representation of the video, wherein the current block is encoded using a geometric partitioning mode; and performing the conversion based on the decision and the motion candidate list.
Need to check novelty before this filing date? Find Prior Art

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application is intended to timely claim the priority and benefits of International Patent Application Nos. PCT / CN2018 / 114057 filed on November 6, 2018, PCT / CN2019 / 087372 filed on May 17, 2019, PCT / CN2019 / 090127 filed on June 5, 2019, and PCT / CN2019 / 092151 filed on June 20, 2019, in accordance with applicable patent law and / or the rules of the Paris Convention. The entire disclosures of the above applications are incorporated by reference as part of the disclosure of this patent document for all purposes under U.S. law. Technical Field

[0003] This document deals with video coding technologies, systems, and devices. Background Art

[0004] Despite advances in video compression, digital video still accounts for the largest use of bandwidth on the Internet and other digital communications networks. As the number of networked user devices capable of receiving and displaying video increases, bandwidth demands for digital video usage are expected to continue to grow. Summary of the invention

[0005] Devices, systems and methods related to digital video coding including geometric partitioning are described. The described methods can be applied to existing video coding standards (e.g., High Efficiency Video Coding (HEVC) and / or Versatile Video Coding (VVC)) as well as future video coding standards or video codecs.

[0006] In one representative aspect, the disclosed technology can be used to provide a method for video processing. The method includes: making a decision about the order of inserting motion candidates into a motion candidate list for conversion between a current block of video and a bitstream representation of the video based on a priority rule, wherein the current block is encoded using a geometric partitioning mode; and performing the conversion based on the decision and the motion candidate list.

[0007] In another representative aspect, the disclosed technology can be used to provide a method for video processing. The method includes: inserting one or more average unidirectional prediction motion candidates from list 0 and list 1 derived from a regular motion candidate into a motion candidate list for conversion between a current block of video and a bitstream representation of the video, wherein the current block is encoded using a geometric partitioning mode; and performing conversion based on the motion candidate list.

[0008] In another representative aspect, the disclosed technology can be used to provide a method for video processing. The method includes: inserting one or more average motion candidates with unidirectional prediction derived from a regular motion candidate with unidirectional prediction into a motion candidate list for conversion between a current block of a video and a bitstream representation of the video, wherein the current block is encoded using a geometric partitioning mode; and performing conversion based on the motion candidate list.

[0009] In another representative aspect, the disclosed technology can be used to provide a method for video processing. The method includes: inserting one or more virtual motion candidates derived from available motion candidates into a motion candidate list for conversion between a current block of a video and a bitstream representation of the video, wherein the current block is encoded using a geometric partitioning mode; and performing conversion based on the motion candidate list.

[0010] In yet another representative aspect, the disclosed technology can be used to provide a method for video processing. The method includes: making a decision about the order in which motion candidates are inserted into a motion candidate list for conversion between a current block of a video encoded using a geometric partitioning mode and a bitstream representation of the video based on a priority rule, wherein the priority rule is based on a position of the motion candidate in the motion candidate from which it is derived or one or more reference picture lists associated with the motion candidate; and performing the conversion based on the decision and the motion candidate list.

[0011] In yet another representative aspect, the disclosed technology can be used to provide a method for video processing. The method includes: making a decision based on a priority rule about the order in which motion candidates are inserted into a motion candidate list for conversion between a current block of a video and a bitstream representation of the video, wherein the current block is encoded using a geometric partitioning mode, wherein the order of insertion is variable from sequence to sequence, picture to picture, slice to slice, slice group to slice group, or from the current block to a subsequent block of the video; and performing the conversion based on the decision and the motion candidate list.

[0012] In another representative aspect, the disclosed technology can be used to provide a method for video processing. The method includes: partitioning a current block of a video encoded using a geometric partitioning mode into a plurality of partitions; constructing a plurality of motion candidate lists, each motion candidate list corresponding to each of the plurality of partitions; and performing conversion between a bitstream representation of the current block and the video based on the plurality of motion candidate lists.

[0013] In another representative aspect, the disclosed technology can be used to provide a method for video processing. The method includes: partitioning a current block of a video encoded using a geometric partitioning mode into a plurality of partitions; constructing a plurality of motion candidate lists, wherein each partition in the plurality of partitions is configured to select a same motion candidate list in the plurality of motion candidate lists; and performing conversion between a bitstream representation of the current block and the video based on the selected same motion candidate list.

[0014] In yet another representative aspect, the disclosed technology can be used to provide a method for video processing. The method includes: performing a conversion between a current block of video in a video unit and a bitstream representation of the video, wherein the bitstream representation includes a field indicating a maximum number of motion candidates allowed in a motion candidate list for a geometric partitioning mode enabled in the video region.

[0015] In yet another representative aspect, the disclosed technology can be used to provide a method for video processing. The method includes: for a current block of a video partitioned into a plurality of partitions using a geometric partitioning pattern, selecting a weighting factor group from a plurality of weighting factor groups, wherein the weighting factor group is selected based on at least a width or a height of the current block; and as part of a conversion between the current block and a bitstream representation of the video, applying the weighting factor group to samples along a common boundary of at least two of the plurality of partitions.

[0016] In another representative aspect, the disclosed technology can be used to provide a method for video processing. The method includes: for a sample point within a current block of a video partitioned into a plurality of partitions using a geometric partitioning pattern, determining at least one weighting factor based on an angle of a common boundary of at least two of the plurality of partitions; and performing conversion between the current block and a bitstream representation of the video based on the at least one weighting factor.

[0017] In another representative aspect, the disclosed technology can be used to provide a method for video processing. The method includes: for a current block of a video partitioned into a plurality of partitions using a geometric partitioning pattern, performing a motion compensation process on samples along a common boundary of at least two of the plurality of partitions, the sample size of which is different from the sample size of 4×4; and performing conversion between the current block and a bitstream representation of the video based on the motion compensation process.

[0018] In yet another representative aspect, the disclosed technology can be used to provide a method for video processing. The method includes: using a geometric partitioning pattern to partition a current block of a video into a plurality of partitions; storing a single motion information set for a K×L region within the current block derived from motion information associated with the plurality of partitions; and performing conversion between the current block and a bitstream representation of the video using at least one motion information set associated with at least one of the plurality of partitions.

[0019] In yet another representative aspect, the disclosed technology can be used to provide a method for video processing. The method includes: partitioning a current block of a video into a plurality of partitions; and performing a conversion between the current block encoded using a geometric partitioning mode and a bitstream representation of the video, wherein a single motion information set is stored for each M×N block unit of the current block, wherein the single motion information set is derived from motion information associated with each of the plurality of partitions, wherein M or N is not equal to 4, and wherein each sample in the M×N block unit shares the same motion information.

[0020] In yet another representative aspect, the disclosed technology can be used to provide a method for video processing. The method includes: partitioning a current block of a video into a plurality of partitions; and as part of a conversion between the current block and a bitstream representation of the video, performing a motion compensation process on a first sub-region of the current block based on first motion information different from second motion information associated with the first sub-region to be stored.

[0021] In another representative aspect, the disclosed technology can be used to provide a method for video processing. The method includes: using a geometric partitioning mode to partition a current block of a video into multiple partitions; based on the partitioning direction or the decoded Merge index or the Merge candidate list associated with the multiple partitions, storing motion information of a first sample point set and a second sample point set, wherein the first sample point set is located on a common boundary of at least two partitions of the multiple partitions, and the second sample point set is located inside one of at least two partitions of the multiple partitions; and based on the stored motion information, performing conversion between the current block and a bitstream representation of the video or conversion between a subsequent block of the video and a bitstream representation.

[0022] In another representative aspect, the disclosed technology can be used to provide a method for video processing. The method includes: partitioning a current block of a video into a plurality of partitions; storing virtual bidirectional prediction motion information based on motion information of at least two of the plurality of partitions; and performing conversion between a bitstream representation of the current block and the video based on the virtual bidirectional prediction motion information.

[0023] In another representative aspect, the disclosed technology can be used to provide a method for video processing. The method includes: dividing a current block of a video into a plurality of partitions; based on a low-latency check flag, storing motion information (MvInfo1) of a first partition of the plurality of partitions and motion information (MvInfo2) of a second partition of the plurality of partitions, wherein the low-latency check flag indicates all reference pictures whose picture order count (POC) values ​​are not greater than the POC value of the current picture including the current block; and performing conversion between a bitstream representation of the current block and the video based on the stored motion information.

[0024] In yet another representative aspect, the above method is embodied in the form of processor executable code and stored in a computer readable program medium.

[0025] In yet another representative aspect, a device configured or operable to perform the above method is disclosed. The device may include a processor programmed to implement the method.

[0026] In yet another representative aspect, a video decoder device may implement a method as described herein.

[0027] The above and other aspects and features of the disclosed technology are described in more detail in the drawings, the description and the claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 is an example of the derivation process used for Merge candidate list construction.

[0029] Figure 2 Example locations of spatial merge candidates are shown.

[0030] Figure 3 An example of candidate pairs considered for redundancy checking of spatial merge candidates is shown.

[0031] Figure 4A and Figure 4B Example locations of the second prediction units (PUs) for N×2N and 2N×N partitions are shown.

[0032] Figure 5 FIG. 4 is an example diagram of motion vector scaling of a temporal merge candidate.

[0033] Figure 6 The time domain Merge candidate C 0 and C 1 Example candidate locations for .

[0034] Figure 7 An example of a combined bi-predictive Merge candidate is shown.

[0035] Figure 8 An example derivation process of motion vector prediction candidates is shown.

[0036] Fig. 9 is an example illustration of motion vector scaling of spatial motion vector candidates.

[0037] Fig.10An example of Alternative Temporal Motion Vector Prediction (ATMVP) motion prediction for a CU (Coding Unit) is shown.

[0038] Fig.11 An example of one CU having four sub-blocks (AD) and its neighboring blocks (ad) is shown.

[0039] Fig.12 This is a flowchart of an example of encoding with different MV (Motion Vector) precisions.

[0040] Fig.13A and Fig. 13B A 135 degree partition type (partitioning from the upper left corner to the lower right corner) and a 45 degree partition mode are shown. Illustration of the partitioning of a CU into two triangular prediction units (two partition modes).

[0041] Fig.14 An example of the positions of neighboring blocks is shown.

[0042] Fig.15A and Fig. 15B An example of motion vector storage is shown.

[0043] Fig.16A and Fig. 16B An example of signaling of Triangular Prediction Mode (TPM) is shown.

[0044] Fig.17 An example of neighboring blocks (A and L) used for context selection in TPM flag encoding is shown.

[0045] Fig.18A and Fig.18B FIG. 5 is an exemplary diagram of a sub-block to which Overlapped Block Motion Compensation (OBMC) is applied.

[0046] Fig.18C An example of uni-prediction MV selection for triangular prediction mode is shown.

[0047] Fig.19 An example of neighboring samples used to derive IC (Illumination Compensation) parameters is shown.

[0048] Fig. 20A and Fig. 20B Simplified affine motion models 4-parameter affine model and 6-parameter affine model are shown.

[0049] Fig.21 An example of the affine motion vector field MVF (Motion Vector Field) of each sub-block is shown.

[0050] Fig.22A and Fig. 22B Examples of a 4-parameter affine model and a 6-parameter affine model are shown.

[0051] Fig.23 An example of the motion vector predictor MVP (Motion Vector Predictor) of AF_INTER is shown.

[0052] Fig.24A and Fig. 24B Example candidates for AF_MERGE are shown.

[0053] Fig.25 An example of candidate positions of the affine merge mode is shown.

[0054] Fig.26 An example of an optical flow trajectory is shown.

[0055] Fig.27A and Fig.27B An example of BIO (Bi-directional Optical flow) without block extension is shown: a) access location outside the block; b) padding is used in order to avoid extra memory access and calculation.

[0056] Fig.28 An example of Decoder-side Motion Vector Refinement (DMVR) based on bilateral template matching is shown.

[0057] Fig.29 Examples of blocks of the motion candidate list construction process for different partitions are shown.

[0058] Figure 30A-Figure 30C Three examples of MV storage areas with a triangle Dir equal to 0 are shown.

[0059] Figure 31-Figure 48 is a flow chart of an example of a video processing method.

[0060] Fig.49 is a block diagram of an example of a hardware platform for implementing the visual media decoding or visual media encoding techniques described in this document.

[0061] Fig.50 is a block diagram of an example video processing system in which the disclosed techniques may be implemented. DETAILED DESCRIPTION

[0062] This document provides various techniques that can be used by decoders of video bitstreams to improve the quality of decompressed or decoded digital video. In addition, video encoders can also implement these techniques during the encoding process to reconstruct decoded frames for further encoding.

[0063] For ease of understanding, section headings are used in this document, and embodiments and techniques are not limited to the corresponding sections. Thus, embodiments from one section can be combined with embodiments from other sections.

[0064] 1. Overview

[0065] This patent document relates to video coding technology. Specifically, it relates to motion vector coding and signaling under geometric segmentation in video coding. It can be applied to existing video coding standards, such as HEVC, or standards that are about to be finalized (e.g., Versatile Video Coding (VVC)). It can also be applied to future video coding standards or video codecs.

[0066] 2. Introductory Notes

[0067] Video coding standards have evolved primarily through the development of the well-known ITU-T and ISO / IEC standards. ITU-T developed H.261 and H.263, ISO / IEC developed MPEG-1 and MPEG-4 Visual, and the two organizations jointly developed the H.262 / MPEG-2 Video and H.264 / MPEG-4 Advanced Video Coding (AVC) and H.265 / HEVC standards. Since H.262, video coding standards have been based on a hybrid video coding structure, where temporal prediction plus transform coding is used. In order to explore future video coding technologies beyond HEVC, VCEG and MPEG jointly established the Joint Video Exploration Team (JVET) in 2015. Since then, JVET has adopted many new methods and put them into a reference software called the Joint Exploration Model (JEM). In April 2018, the Joint Video Experts Group (JVET) between VCEG (Q6 / 16) and ISO / IEC JTC1 SC29 / WG11 (MPEG) was established to work on the Versatile Video Coding (VVC) standard with the goal of a 50% bitrate reduction compared to HEVC.

[0068] The latest version of the VVC draft, Versatile Video Coding (Draft 2), can be found at:

[0069] http: / / phenix.it-sudparis.eu / jvet / doc_end_user / documents / 11_Ljubljana / wg11 / JVET-K1001-v7.zip.

[0070] The latest reference software for VVC is called VTM and can be found at:

[0071] https: / / vcgit.hhi.fraunhofer.de / jvet / VVCSoftware_VTM / tags / VTM-2.1.

[0072] 2.1 Inter-frame prediction of HEVC / H.265

[0073] Each inter-predicted PU has motion parameters for one or two reference picture lists. The motion parameters include motion vectors and reference picture indices. The use of one of the two reference picture lists may also be signaled using inter_pred_idc. The motion vector may be explicitly coded as a delta relative to the predictor.

[0074] When a CU is encoded in skip mode, one PU is associated with the CU and there are no significant residual coefficients, no coded motion vector increments, or reference picture indices. A Merge mode is specified, whereby the motion parameters of the current PU are obtained from neighboring PUs including spatial and temporal candidates. Merge mode can be applied to any inter-predicted PU, not just for skip mode. An alternative to Merge mode is explicit transmission of motion parameters, where each PU explicitly signals the motion vector (more accurately, the motion vector difference compared to the motion vector predictor), the corresponding reference picture index for each reference picture list, and the use of the reference picture list. In the present disclosure, this mode is referred to as Advanced Motion Vector Prediction (AMVP).

[0075] When signaling indicates that one of the two reference picture lists is to be used, a PU is generated from a block of samples. This is called "unidirectional prediction". Unidirectional prediction is available for P slices and B slices.

[0076] When signaling indicates that both reference picture lists are to be used, a PU is generated from two sample blocks. This is called "bi-prediction". Bi-prediction is only available for B slices.

[0077] The following text provides detailed information about the inter prediction modes specified in HEVC. The description will start with the Merge mode.

[0078] 2.1.1 Reference Image List

[0079] In HEVC, the term inter prediction is used to refer to predictions derived from data elements (e.g., sample values ​​or motion vectors) of reference pictures other than the currently decoded picture. As in H.264 / AVC, a picture can be predicted from multiple reference pictures. Reference pictures used for inter prediction are organized in one or more reference picture lists. A reference index identifies which reference pictures in the list should be used to create the prediction signal.

[0080] A single reference picture list (list 0) is used for P slices, and two reference picture lists (list 0 and list 1) are used for B slices. It should be noted that the reference pictures included in lists 0 / 1 can be pictures from both the past and the future in terms of capture / display order.

[0081] 2.1.2Merge Mode

[0082] 2.1.2.1 Merge Mode Candidate Derivation

[0083] When predicting a PU using Merge mode, an index pointing to an entry in the Merge candidate list is parsed from the bitstream and used to retrieve motion information. The construction of this list is specified in the HEVC standard and can be summarized in the following sequence of steps:

[0084] Step 1: Initial candidate derivation

[0085] oStep 1.1: Spatial Candidate Derivation

[0086] oStep 1.2: Redundancy check of spatial domain candidates

[0087] oStep 1.3: Time Domain Candidate Derivation

[0088] Step 2: Additional candidate insertions

[0089] oStep 2.1: Create bidirectional prediction candidates

[0090] oStep 2.2: Insert zero motion candidates

[0091] These steps are also schematically depicted in Figure 1For spatial Merge candidate derivation, a maximum of four Merge candidates are selected from candidates located at five different positions. For temporal Merge candidate derivation, a maximum of one Merge candidate is selected from two candidates. Since a constant number of candidates for each PU is assumed at the decoder, additional candidates are generated when the number of candidates obtained from step 1 does not reach the maximum number of Merge candidates (MaxNumMergeCand) signaled in the slice header. Since the number of candidates is constant, the index of the best Merge candidate is encoded using Truncated Unarybinarization (TU). If the size of the CU is equal to 8, all PUs of the current CU share a single Merge candidate list, which is the same as the Merge candidate list of the 2N×2N prediction unit.

[0092] Hereinafter, operations associated with the above-mentioned steps will be described in detail.

[0093] 2.1.2.2 Spatial Candidate Derivation

[0094] In the derivation of spatial merge candidates, Figure 2 Select up to four Merge candidates from the candidates for the positions depicted in . The order of derivation is A 1 , B 1 , B 0 , A 0 and B 2 Only when position A 1 , B 1 , B 0 , A 0 Position B is considered only when any PU in is unavailable (for example, because it belongs to another slice or part) or is intra-coded. 2 . Add position A 1 After the candidates at , a redundancy check is performed on the remaining candidates, which ensures that candidates with the same motion information are excluded from the list, thereby improving coding efficiency. In order to reduce computational complexity, not all possible candidate pairs are considered in the mentioned redundancy check. Instead, only candidates with Figure 3 The arrows in the table link pairs, and candidates are only added to the list if the corresponding candidates for redundancy checking do not have the same motion information. Another source of repeated motion information is a "second PU" associated with a partition other than 2N×2N. As an example, Figure 4A and Figure 4B The second PU in the N×2N and 2N×N cases is depicted respectively. When the current PU is partitioned into N×2N, position A 1The candidate at position B is not considered for list construction. In fact, by adding this candidate it would result in two prediction units having the same motion information, which is redundant for having only one PU in the coding unit. Similarly, when the current PU is split into 2N×N, position B is not considered. 1 .

[0095] 2.1.2.3 Time Domain Candidate Derivation

[0096] In this step, only one candidate is added to the list. Specifically, in the derivation of the temporal Merge candidate, the scaled motion vector is derived based on the co-located PU belonging to the picture with the smallest POC (Picture Order Count) difference with the current picture in a given reference picture list. The reference picture list to be used for derivation of the co-located PU is explicitly signaled in the slice header. Figure 5 As shown by the dotted line in , a scaled motion vector for the temporal Merge candidate is obtained, which is scaled from the motion vector of the collocated PU using the POC distances tb and td, where tb is defined as the POC difference between the reference picture of the current picture and the current picture, and td is defined as the POC difference between the reference picture of the collocated picture and the collocated picture. The reference picture index of the temporal Merge candidate is set equal to zero. The actual implementation of the scaling process is described in the HEVC specification. For B slices, two motion vectors (one for reference picture list 0 and the other for reference picture list 1) are obtained and combined to generate a bidirectional prediction Merge candidate.

[0097] In the collocated PU(Y) belonging to the reference frame, in the candidate C 0 and C 1 Select the position of the time domain candidate between Figure 6 If position C 0 If the PU at position C is not available, is intra-coded, or is outside the current CTU row, then position C is used. 1 Otherwise, position C is used in the derivation of the time-domain Merge candidate 0 .

[0098] 2.1.2.4 Additional candidate insertion

[0099] In addition to the spatial and temporal Merge candidates, there are two additional types of Merge candidates: combined bidirectional prediction Merge candidates and zero Merge candidates. The combined bidirectional prediction Merge candidate is generated by utilizing the spatial and temporal Merge candidates. The combined bidirectional prediction Merge candidate is only used for B slices. The combined bidirectional prediction candidate is generated by combining the first reference picture list motion parameters of an initial candidate with the second reference picture list motion parameters of another initial candidate. If the two tuples provide different motion hypotheses, they will form a new bidirectional prediction candidate. As an example, Figure 7 Shown is when two candidates in the original list (on the left) (which have mvL0 and refIdxL0 or mvL1 and refIdxL1) are used to create a combined bi-predictive Merge candidate that is added to the final list (on the right). There are many rules for combining that are considered to generate these additional Merge candidates.

[0100] Zero motion candidates are inserted to fill the remaining entries in the Merge candidate list and thus reach the MaxNumMergeCand capacity. These candidates have zero spatial displacement and a reference picture index that starts at zero and increases each time a new zero motion candidate is added to the list. The number of reference frames used by these candidates is 1 and 2 for unidirectional prediction and bidirectional prediction, respectively. Finally, no redundancy check is performed on these candidates.

[0101] 2.1.2.5 Motion Estimation Regions for Parallel Processing

[0102] To speed up the encoding process, motion estimation can be performed in parallel, thereby deriving motion vectors for all prediction units within a given region at the same time. Deriving Merge candidates from spatial neighbors may interfere with parallel processing because one prediction unit cannot derive motion parameters from neighboring PUs until its associated motion estimation is completed. In order to ease the trade-off between coding efficiency and processing latency, HEVC defines a Motion Estimation Region (MER), and uses the "log2_parallel_merge_level_minus2" syntax element to signal the size of the MER in the picture parameter set. When a MER is defined, Merge candidates that fall into the same region are marked as unavailable and are therefore not considered in list construction.

[0103] 2.1.3AMVP

[0104] AMVP exploits the spatiotemporal correlation of motion vectors with neighboring PUs, which is used for explicit transmission of motion parameters. For each reference picture list, a motion vector candidate list is constructed by first checking the availability of the PU positions in the temporal domain adjacent to the left and above, removing redundant candidates and adding zero vectors to make the candidate list a constant length. The encoder can then select the best predictor from the candidate list and send the corresponding index indicating the selected candidate. Similar to Merge index signaling, the index of the best motion vector candidate is encoded using truncated unary. In this case, the maximum value to be encoded is 2 (see Figure 8 ). In the following sections, details about the derivation process of motion vector prediction candidates will be provided.

[0105] 2.1.3.1 Derivation of AMVP Candidates

[0106] Figure 8 The process of deriving motion vector prediction candidates is outlined.

[0107] In motion vector prediction, two types of motion vector candidates are considered: spatial motion vector candidates and temporal motion vector candidates. For spatial motion vector candidate derivation, the final result is based on the location of Figure 2 The motion vectors of each PU are depicted at five different positions to derive two motion vector candidates.

[0108] For temporal motion vector candidate derivation, one motion vector candidate is selected from two candidates derived based on two different collocated positions. After generating the first spatiotemporal candidate list, duplicate motion vector candidates in the list are removed. If the number of potential candidates is greater than two, motion vector candidates whose reference picture index within the associated reference picture list is greater than 1 are removed from the list. If the number of spatiotemporal motion vector candidates is less than two, an additional zero motion vector candidate is added to the list.

[0109] 2.1.3.2 Spatial Motion Vector Candidates

[0110] In the derivation of spatial motion vector candidates, the Figure 2 Consider up to two candidates among the five potential candidates derived in the PU at the depicted position, which are the same as the position of the motion merge. The derivation order to the left of the current PU is defined as A 0 , A 1 , and scale A 0 , Zoom A 1 The derivation order of the upper side of the current PU is defined as B 0 , B 1 , B 2 , Zoom B 0 , Zoom B 1 , Zoom B 2. So for each side, there are four cases that can be used as motion vector candidates, two of which do not require spatial scaling and two of which use spatial scaling. The four different cases are summarized below:

[0111] No airspace scaling

[0112] -(1) Same reference picture list and same reference picture index (same POC)

[0113] -(2) Different reference picture lists, but same reference pictures (same POC)

[0114] Airspace scaling

[0115] -(3) Same reference picture list, but different reference pictures (different POC)

[0116] -(4) Different reference picture lists, and different reference pictures (different POC)

[0117] First check the non-spatial scaling case, then the spatial scaling. Spatial scaling is considered when the POC is different between the reference pictures of the neighboring PU and the reference pictures of the current PU regardless of the reference picture list. If all PUs of the left candidate are unavailable or are intra-coded, then scaling for the upper motion vector is allowed to help parallel derivation of the left and upper MV candidates. Otherwise, spatial scaling is not allowed for the upper motion vector.

[0118] like Fig. 9 As depicted, in the spatial scaling process, the motion vectors of neighboring PUs are scaled in a similar manner as in the temporal scaling. The main difference is that the reference picture list and the index of the current PU are given as input; the actual scaling process is the same as that of the temporal scaling process.

[0119] 2.1.3.3 Temporal Motion Vector Candidates

[0120] Except for the reference picture index derivation, all the processes used to derive the temporal merge candidate are the same as those used to derive the spatial motion vector candidate (see Figure 6 ). The reference picture index is signaled to the decoder.

[0121] 2.2 New inter-frame prediction method in JEM

[0122] 2.2.1 Motion vector prediction based on sub-CU

[0123] In JEM with QTBT (QuadTrees plus Binary Trees), each CU can have at most one motion parameter set for each prediction direction. Two sub-CU level motion vector prediction methods are considered in the encoder by dividing the large CU into sub-CUs and deriving the motion information of all sub-CUs of the large CU. The optional temporal motion vector prediction (ATMVP) method allows each CU to extract multiple sets of motion information from multiple blocks smaller than the current CU in the collocated reference picture. In the Spatial-Temporal Motion Vector Prediction (STMVP) method, the motion vector of the sub-CU is recursively derived by using the temporal motion vector predictor and the spatial neighboring motion vector.

[0124] In order to maintain a more accurate motion field for sub-CU motion prediction, motion compression of reference frames is currently disabled.

[0125] 2.2.1.1 Optional temporal motion vector prediction

[0126] refer to Fig.10 In the optional temporal motion vector prediction (ATMVP) method, the temporal motion vector prediction (TMVP) is modified by extracting multiple sets of motion information (including motion vectors and reference indices) from blocks smaller than the current CU. In the example, the sub-CU is a square N×N block (N is set to 4 by default).

[0127] ATMVP predicts the motion vector of a sub-CU within a CU in two steps. The first step is to identify the corresponding block in the reference picture using the so-called time domain vector. The reference picture is also called the motion source picture. The second step is to divide the current CU into sub-CUs and obtain the motion vector and the reference index of each sub-CU from the block corresponding to each sub-CU.

[0128] In the first step, the reference picture and the corresponding block are determined by the motion information of the spatial neighboring blocks of the current CU. In order to avoid the repeated scanning process of the neighboring blocks, the first Merge candidate in the Merge candidate list of the current CU is used. The first available motion vector and its associated reference index are set to the index of the temporal vector and the motion source picture. In this way, in ATMVP, the corresponding block can be identified more accurately than in TMVP, where the corresponding block (sometimes called the collocated block) is always in the lower right or center position relative to the current CU.

[0129] In the second step, the corresponding block of the sub-CU is identified by the time domain vector in the motion source picture by adding the time domain vector to the coordinates of the current CU. For each sub-CU, the motion information of its corresponding block (the minimum motion grid covering the center sample point) is used to derive the motion information of the sub-CU. After identifying the motion information of the corresponding N×N block, it is converted into the motion vector and reference index of the current sub-CU in the same way as the TMVP of HEVC, where motion scaling and other processes apply. For example, the decoder checks whether the low latency condition is met (that is, the POC of all reference pictures of the current picture is less than the POC of the current picture), and may use the motion vector MV x (e.g., the motion vector corresponding to the reference picture list X) is used to predict the motion vector MV of each sub-CU y (eg, where X equals 0 or 1, and Y equals 1-X).

[0130] 2.2.1.2 Spatiotemporal Motion Vector Prediction (STMVP)

[0131] In this method, the motion vector of a sub-CU is recursively derived in raster scan order. Fig.11 This concept is illustrated in Figure 1. Let us consider an 8×8 CU, which contains four 4×4 sub-CUs: A, B, C, and D. The neighboring 4×4 blocks in the current frame are labeled a, b, c, and d.

[0132] The motion derivation of sub-CU A starts by identifying its two spatial neighbors. The first neighbor is the N×N block (block c) above the sub-CUA. If this block c is not available or is intra-coded, check the other N×N blocks above the sub-CU A (from left to right, starting with block c). The second neighbor is the block (block b) on the left of sub-CU A. If block b is not available or is intra-coded, check the other blocks on the left of sub-CU A (from top to bottom, starting with block b). The motion information obtained from the adjacent blocks in each list is scaled to the first reference frame of the given list. Next, the temporal motion vector predictor (TMVP) of sub-block A is derived by following the same process of TMVP derivation as specified by HEVC. The motion information of the collocated block at position D is extracted and scaled accordingly. Finally, after retrieving and scaling the motion information, all available motion vectors (up to 3) are averaged separately for each reference list. The average motion vector is assigned as the motion vector of the current sub-CU.

[0133] 2.2.1.3 Sub-CU motion prediction mode signaling

[0134] Sub-CU modes are enabled as additional Merge candidates, and no additional syntax elements are required to signal these modes. Two additional Merge candidates are added to the Merge candidate list of each CU to represent ATMVP mode and STMVP mode. If the sequence parameter set indicates that ATMVP and STMVP are enabled, up to seven Merge candidates can be used. The encoding logic of the additional Merge candidates is the same as that of the Merge candidates in HM, which means that for each CU in a P slice or B slice, two RD checks may be required for the two additional Merge candidates.

[0135] In JEM, all bins of the Merge index are context coded by CABAC, whereas in HEVC, only the first bin is context coded and the remaining bins are context bypass coded.

[0136] 2.2.2 Adaptive motion vector difference resolution

[0137] In HEVC, when use_integer_mv_flag in the slice header is equal to 0, the motion vector difference (MVD) (between the PU's motion vector and the predicted motion vector) is signaled in units of quarter luma samples. In JEM, Locally Adaptive Motion Vector Resolution (LAMVR) is introduced. In JEM, MVD can be encoded in units of quarter luma samples, integer luma samples, or four luma samples. The MVD resolution is controlled at the coding unit (CU) level, and the MVD resolution flag is conditionally signaled for each CU with at least one non-zero MVD component.

[0138] For a CU with at least one non-zero MVD component, a first flag is signaled to indicate whether quarter luma sample MV precision is used in the CU. When the first flag (equal to 1) indicates that quarter luma sample MV precision is not used, another flag is signaled to indicate whether integer luma sample MV precision or quad luma sample MV precision is used.

[0139] When the first MVD resolution flag of a CU is zero or is not encoded for the CU (meaning that all MVDs in the CU are zero), a quarter luma sample MV resolution is used for the CU. When the CU uses integer luma sample MV precision or four luma sample MV precision, the MVP in the AMVP candidate list of the CU is rounded to the corresponding precision.

[0140] In the encoder, the RD check at the CU level is used to determine which MVD resolution to use for the CU. That is, for each MVD resolution, the RD check at the CU level is performed three times. To speed up the encoder, the following coding scheme is applied in JEM:

[0141] · During the RD check of a CU with normal quarter-luminance sample MVD resolution, the motion information of the current CU (integer-luminance sample accuracy) is stored. The stored motion information (after rounding) is used as the starting point for further small-range motion vector refinement during the RD check for the same CU with integer-luminance samples and 4-luminance sample MVD resolution, so that the time-consuming motion estimation process is not repeated three times.

[0142] · Conditionally call the RD check of a CU with 4-luminance sample MVD resolution. For a CU, when the RD cost of the integer-luminance sample MVD resolution is much greater than the RD cost of the quarter-luminance sample MVD resolution, skip the RD check of the 4-luminance sample MVD resolution for the CU.

[0143] The encoding process is as Fig.12 shown. First, test the 1 / 4 pixel MV, calculate the RD cost and denote it as RDCost0, then test the integer MV, and the RD cost is denoted as RDCost1. If RDCost1 < th * RDCost0 (where th is a positive value), then test the 4 pixel MV; otherwise, skip the 4 pixel MV. Basically, when checking the integer or 4 pixel MV, the motion information and RD cost, etc. for the 1 / 4 pixel MV are known, and this motion information and RD cost can be reused to accelerate the encoding process of the integer or 4 pixel MV.

[0144] 2.2.3 Triangular Prediction Mode

[0145] The concept of the Triangular Prediction Mode (TPM) is to introduce a new triangular partition for motion compensation prediction. As Fig.13A and Fig. 13B shown, it divides the CU into two triangular prediction units along the diagonal or the opposite diagonal direction. Each triangular prediction unit in the CU is inter-frame predicted using its own unidirectional prediction motion vector and reference frame index, which are derived from a single unidirectional prediction candidate list. After predicting the triangular prediction units, an adaptive weighting process is performed on the diagonal edges. Then, the transform and quantization processes are applied to the entire CU. Note that this mode is only applicable to the Merge mode (including the Skip mode, where the Skip mode is regarded as a special Merge mode).

[0146] In this document, triangle prediction mode (TPM) may be referred to interchangeably as triangle partitioning mode, triangle prediction mode, triangle partitioning mode, geometry partitioning mode, or geometry partitioning mode.

[0147] In some embodiments, for non-TPM (also called non-geometric) Merge mode, the Merge index to the Merge list is signaled. The Merge list including multiple regular Merge candidates is constructed as: spatial Merge candidates (derived from up to five spatial neighbors), temporal Merge candidates (TMVP), HMVP, pairwise average candidates, and default Merge candidates (zero MV with some assigned reference pictures). In some embodiments, the regular Merge candidate is a motion candidate derived for a video block encoded using a non-geometric (non-TPM) partitioning mode.

[0148] In contrast, for a TPM coded block, two Merge indexes (with predictive coding) to the Merge list are signaled.

[0149] o In one case, the Merge list can be constructed in a manner similar to the non-TPM segmentation mode described above, which results in the generation of TPM motion candidates.

[0150] ○ In another case, parity can be used to select the unidirectional prediction motion information of two partitions. The two selected Merge indexes can be represented as idx0 and idx1. For each Merge candidate, if its reference picture list X=(idx0&idx1) is true, the motion information of the partition is set to the list X information of the Merge candidate. Otherwise, the motion information of the partition is set to the Y (Y=1-X) information of the Merge candidate list. This results in the final motion information with unidirectional prediction as a TPM candidate, and the candidate in the first case is a regular Merge candidate (which is generated using the Merge list construction process described for non-TPM mode coding blocks).

[0151] 2.2.3.1 TPM’s One-Way Prediction Candidate List

[0152] The unidirectional prediction candidate list, called the TPM motion candidate list, includes five unidirectional prediction motion vector candidates. Fig.14As shown, it is derived from seven neighboring blocks of the CU, including five spatial neighboring blocks (1 to 5) and two temporal collocated blocks (6 to 7). The motion vectors of the seven neighboring blocks are collected and placed in the unidirectional prediction candidate list in the order of unidirectional prediction motion vector, L0 motion vector of bidirectional prediction motion vector, L1 motion vector of bidirectional prediction motion vector, and average motion vector of L0 motion vector and L1 motion vector of bidirectional prediction motion vector. If the number of candidates is less than five, a zero motion vector is added to the list. The motion candidates added to the TPM list are called TPM candidates, and the motion information derived from the spatial / temporal blocks is called regular motion candidates.

[0153] More specifically, the following steps are involved:

[0154] 1) From A 1 , B 1 , B 0 , A 0 , B 2 , Col and Col2 (corresponding to Fig.14 In blocks 1-7), obtain conventional motion candidates, none No trimming required .

[0155] 2) Set the variable numCurrMergeCand = 0.

[0156] 3) For A 1 , B 1 , B 0 , A 0 , B 2 For each normal motion candidate derived in Col, Col, and Col2 and numCurrMergeCand is less than 5, if the normal motion candidate is uni-predicted (from list 0 or list 1), it is added directly to the Merge list as a TPM candidate and numCurrMergeCand is increased by 1. This TPM candidate is called an "originally uni-predicted candidate".

[0157] application Fully trimmed (full pruning).

[0158] 4) For A 1 , B 1 , B 0 , A 0 , B 2, Col, and Col2, and numCurrMergeCand is less than 5, if the regular motion candidate is bidirectionally predicted, the motion information from List 0 is added to the TPM Merge list as a new TPM candidate (i.e., modified to be unidirectionally predicted from List 0), and numCurrMergeCand is increased by 1. Such a TPM candidate is called a "truncated List0-predicted candidate".

[0159] application Fully trimmed .

[0160] 5) For A 1 , B 1 , B 0 , A 0 , B 2 , Col, and Col2, and numCurrMergeCand is less than 5, if the regular motion candidate is bidirectionally predicted, the motion information from List 1 is added to the TPM Merge list (i.e., modified to be unidirectionally predicted from List 1), and numCurrMergeCand is increased by 1. Such a TPM candidate is called a "truncated List1-predicted candidate".

[0161] application Fully trimmed .

[0162] 6) For A 1 , B 1 , B 0 , A 0 , B 2 , Col and Col2 derived in each motion candidate and numCurrMergeCand is less than 5, if the regular motion candidate is bidirectional prediction,

[0163] - If the slice QP of the list 0 reference picture is less than the slice QP of the list 1 reference picture, the motion information of list 1 is first scaled to the list 0 reference picture, and the average of the two MVs (one from the original list 0 and the other is the scaled MV from list 1) is added to the TPM Merge list, such a candidate is called the averaged uni-prediction from List 0 motion candidate, and numCurrMergeCand is increased by 1.

[0164] Otherwise, first scale the motion information of list 0 to the list 1 reference picture, and add the average of the two MVs (one from the original list 1 and the other is the scaled MV from list 0) to the TPM Merge list, such a TPM candidate is called the averaged uni-prediction from List1motion candidate, and numCurrMergeCand is increased by 1.

[0165] application Fully trimmed .

[0166] 7) If numCurrMergeCand is less than 5, add a zero motion vector candidate.

[0167] 2.2.3.2 Adaptive Weighting Process

[0168] After predicting each triangle prediction unit, an adaptive weighting process is applied to the diagonal edges between two triangle prediction units to derive the final prediction for the entire CU. Two groups of weighting factors are defined as follows:

[0169] The first weighting factor group: {7 / 8, 6 / 8, 4 / 8, 2 / 8, 1 / 8} and {7 / 8, 4 / 8, 1 / 8} are used for luma and chroma samples respectively;

[0170] The second weighting factor group: {7 / 8, 6 / 8, 5 / 8, 4 / 8, 3 / 8, 2 / 8, 1 / 8} and {6 / 8, 4 / 8, 2 / 8} are used for luma and chroma samples respectively.

[0171] The weighting factor set is selected based on the comparison of the motion vectors of the two triangle prediction units. When the reference pictures of the two triangle prediction units are different from each other or the difference of their motion vectors is greater than 16 pixels, the second weighting factor set is used. Otherwise, the first weighting factor set is used. Example Figure 15A-15B shown.

[0172] 2.2.3.3 Motion Vector Storage

[0173] Motion vector of triangle prediction unit ( Fig.16A and Fig. 16B Mv1 and Mv2 in CU are stored in a 4×4 grid. For each 4×4 grid, the unidirectional prediction motion vector or the bidirectional prediction motion vector is stored depending on the position of the 4×4 grid in the CU. Figure 16A-16BAs shown, the unidirectional prediction motion vector Mv1 or Mv2 is stored in a 4×4 grid located in a non-weighted area (i.e. not located at a diagonal edge). On the other hand, the bidirectional prediction motion vector is stored in a 4×4 grid located in a weighted area. The bidirectional prediction motion vector is derived from Mv1 and Mv2 according to the following rules:

[0174] 1) In the case where Mv1 and Mv2 have motion vectors from different directions (L0 or L1), Mv1 and Mv2 are simply combined to form a bidirectional prediction motion vector.

[0175] 2) In the case where both Mv1 and Mv2 come from the same L0 (or L1) direction,

[0176] - If the reference picture of Mv2 is the same as a picture in the L1 (or L0) reference picture list, then Mv2 is scaled to that picture. Mv1 and the scaled Mv2 are combined to form a bidirectional prediction motion vector.

[0177] - If the reference picture of Mv1 is the same as a picture in the L1 (or L0) reference picture list, then Mv1 is scaled to that picture. The scaled Mv1 and Mv2 are combined to form a bidirectional prediction motion vector.

[0178] – Otherwise, only Mv1 is stored for the weighted region.

[0179] 2.2.3.4 Signaling of Triangle Prediction Mode (TPM)

[0180] A bit flag indicating whether the TPM is used may be signaled first. Thereafter, two partitioning modes (such as Fig.13A and Fig. 13B ) as depicted, and the Merge index selected for each of the two partitions.

[0181] 2.2.3.4.1 Signaling of TPM Flags

[0182] Let's denote the width and height of a luma block by W and H respectively. If W*H<64, triangle prediction mode is disabled.

[0183] When a block is encoded in affine mode, triangle prediction mode is also disabled.

[0184] When a block is encoded in Merge mode, a one-bit flag may be signaled to indicate whether triangle prediction mode is enabled or disabled for the block.

[0185] The flag is encoded with 3 contexts (see Fig.17 ).

[0186] Ctx index = ((Left block L is available && L is encoded with TPM?) 1:0)

[0187] +((Block A above is available &&A is encoded with TPM?)1:0);

[0188] 2.2.3.4.2 Two division modes (such as Figure 13A-13B The indication of the depicted) and the signaling of the Merge index selected for each of the two partitions

[0189] Note that the partition mode and the Merge index of the two partitions are jointly encoded. In the example, the two partitions are restricted from using the same reference index. Therefore, there are 2 (partition mode) * N (maximum number of Merge candidates) * (N-1) possibilities, where N is set to 5. An indication is encoded, and the mapping between the partition mode, the two Merge indexes and the encoding indication is derived from the array defined below: const uint8_t g_TriangleCombination[TRIANGLE_MAX_NUM_CANDS][3] = {

[0190] {0,1,0},{1,0,1},{1,0,2},{0,0,1},{0,2,0},

[0191] {1,0,3},{1,0,4},{1,1,0},{0,3,0},{0,4,0},

[0192] {0,0,2},{0,1,2},{1,1,2},{0,0,4},{0,0,3},

[0193] {0,1,3},{0,1,4},{1,1,4},{1,1,3},{1,2,1},

[0194] {1,2,0},{0,2,1},{0,4,3},{1,3,0},{1,3,2},

[0195] {1,3,4},{1,4,0},{1,3,1},{1,2,3},{1,4,1},

[0196] {0,4,1},{0,2,3},{1,4,2},{0,3,2},{1,4,3},

[0197] {0,3,1},{0,2,4},{1,2,4},{0,4,2},{0,3,4}};

[0198] Division mode (45 degrees or 135 degrees) = g_TriangleCombination [signaling indication] [0];

[0199] Merge index of candidate A = g_TriangleCombination [indication of signaling notification];

[0200] Merge index of candidate B = g_TriangleCombination [indication of signaling notification];

[0201] Once two motion candidates A and B are derived, the motion information of the two partitions (PU1 and PU2) can be set according to A or B. Whether PU1 uses the motion information of Merge candidate A or B depends on the prediction direction of the two motion candidates. Table 1 shows the relationship between the two derived motion candidates A and B and the two partitions.

[0202] Table 1: Motion information of partitions derived from two derived Merge candidates (A, B)

[0203] The predicted direction of A The predicted direction of B PU1 motion information PU2 motion information L0 L0 A(L0) B(L0) L1 L1 B(L1) A(L1) L0 L1 A(L0) B(L1) L1 L0 B(L0) A(L1)

[0204] 2.2.3.4.3 Indicated entropy coding (indicated by merge_triangle_idx)

[0205] merge_triangle_idx In the range of [0,39]. K-order Exponential Golomb (EG) code is used for binarization of merge_triangle_idx, where K is set to 1.

[0206] K Tier EG

[0207] To encode larger numbers with fewer bits (at the expense of using more bits to encode smaller numbers), this can be done using Non-negative integer To encode a non-negative integer x using a k-order EGOLomb code:

[0208] 1. Use the above 0th order exponential Golomb code pair Encode, then

[0209] 2. Use binary to solve x mod 2 k Encoding

[0210] Table 2: Exponential Golomb-k encoding examples

[0211]

[0212] 2.2.3.5 Latest Developments in TPM

[0213] The regular Merge candidate list is reused for triangle partition Merge prediction without additional motion vector pruning. For each Merge candidate in the regular Merge candidate list, one and only one of its L0 or L1 motion vectors is used for triangle prediction. In addition, the order of selecting L0 vs. L1 motion vectors is based on their Merge index parity. With this scheme, the regular Merge list can be used directly.

[0214] Once the regular Merge list is constructed, for each Merge candidate in the regular Merge candidate list, one and only one of its L0 or L1 motion vectors is used for triangle prediction. In addition, the order of selecting L0 to L1 motion vectors is based on their Merge index parity. More specifically, as Fig.18C As shown, for candidates with even value indexes, their L0 motion vector is first selected for triangle prediction. If it is not available, their L1 motion vector is used instead. For candidates with odd value indexes, their L1 motion vector is first selected for triangle prediction. If it is not available, their L0 motion vector is used instead. Fig.18C In , corresponding to each Merge index, the motion vector marked with "x" is first selected for triangle prediction.

[0215] 7.3.7.7Merge Data Syntax

[0216]

[0217]

[0218] 2.2.4 Overlapped Block Motion Compensation

[0219] Overlapped block motion compensation (OBMC) was previously used in H.263. In JEM, unlike H.263, OBMC can be turned on and off using CU-level syntax. When OBMC is used in JEM, OBMC is performed on all motion compensation (MC) block boundaries except the right and bottom boundaries of the CU. In addition, it is applied to both luminance and chrominance components. In JEM, an MC block corresponds to a coding block. When a CU is encoded with sub-CU modes, including Sub-CU Merge, Affine, and FRUC (Frame Rate Up Conversion) modes, each sub-block of the CU is an MC block. In order to handle CU boundaries in a unified manner, OBMC is performed on all MC block boundaries at the sub-block level, where the sub-block size is set equal to 4×4, as Figure 18A-18B shown.

[0220] When OBMC is applied to the current sub-block, in addition to the current motion vector, the motion vectors of four connected neighboring sub-blocks (if available and different from the current motion vector) are also used to derive the prediction block of the current sub-block. These multiple prediction blocks based on multiple motion vectors are combined to generate the final prediction signal of the current sub-block.

[0221] The prediction block based on the motion vector of the neighboring sub-block is denoted as P N , where N represents the index for the adjacent upper, lower, left and right sub-blocks, and the prediction block based on the motion vector of the current sub-block is represented as P C When P N When the motion information contains the same motion information of the neighboring sub-block as the current sub-block, OBMC is not from P N Otherwise, P N Each sample point is added to P C In the same point in the same N Four rows / columns are added to P C The weighting factors {1 / 4, 1 / 8, 1 / 16, 1 / 32} are used for P N , and weighting factors {3 / 4, 7 / 8, 15 / 16, 31 / 32} are used for P C The exception is small MC blocks (i.e. when the height or width of the coding block is equal to 4 or the CU is coded in sub-CU mode), for which only P N Two rows / columns are added to P C In this case, the weighting factors {1 / 4,1 / 8} are used for P N , and weighting factors {3 / 4, 7 / 8} are used for P C For P generated based on the motion vector of the vertically (horizontally) adjacent sub-block N , P N The samples in the same row (column) of P are added to P with the same weighting factor. C .

[0222] In JEM, for CUs with a size less than or equal to 256 luma samples, a CU level flag is signaled to indicate whether OBMC is applied for the current CU. For CUs with a size greater than 256 luma samples or not encoded in AMVP mode, OBMC is applied by default. At the encoder, when OBMC is applied to a CU, its impact is taken into account during the motion estimation stage. The prediction signal formed by OBMC using the motion information of the upper and left neighboring blocks is used to compensate the upper and left boundaries of the original signal of the current CU, and then the normal motion estimation process is applied.

[0223] 2.2.5 Local lighting compensation

[0224] Local Illumination Compensation (LIC) is based on a linear model of illumination variation, using a scaling factor a and an offset b. It is adaptively enabled or disabled for each inter-mode coded coding unit (CU).

[0225] When LIC is applied to a CU, the least square error method is used to derive parameters a and b by using the neighboring samples of the current CU and its corresponding reference samples. More specifically, Fig.19 As shown, the subsampling (2:1 subsampling) neighboring samples and corresponding samples (identified by the motion information of the current CU or sub-CU) in the reference picture are used. The IC parameters are derived and applied to each prediction direction separately.

[0226] When the CU is encoded in Merge mode, the LIC flag is copied from the neighboring block in a manner similar to the motion information copying in Merge mode; otherwise, the LIC flag is signaled for the CU to indicate whether LIC is applicable.

[0227] When LIC is enabled for a picture, an additional CU-level RD check is required to determine whether LIC is applied to the CU. When LIC is enabled for a CU, the Mean-Removed Sum of Absolute Difference (MR-SAD) and the Mean-Removed Sum of AbsoluteHadamard-Transformed Difference (MR-SATD) (instead of SAD and SATD) are used for integer pixel motion search and fractional pixel motion search, respectively.

[0228] In order to reduce the coding complexity, the following coding scheme is applied in JEM.

[0229] When there is no significant illumination change between the current picture and its reference pictures, LIC is disabled for the entire picture. To identify this situation, the histogram of the current picture and each reference picture of the current picture is calculated at the encoder. If the histogram difference between the current picture and each reference picture of the current picture is less than a given threshold, LIC is disabled for the current picture; otherwise, LIC is enabled for the current picture.

[0230] 2.2.6 Affine Motion Compensated Prediction

[0231] In HEVC, only the translational motion model is used for motion compensation prediction (MCP). In the real world, there are many kinds of motion, such as zooming in / out, rotation, perspective motion, and other irregular motions. In JEM, a simplified affine transformation motion compensation prediction is applied. Figure 20A-20B As shown, the affine motion field of a block is described by two control point motion vectors.

[0232] The motion vector field (MVF) of a block is described by the following equation:

[0233]

[0234] For 6-parameter affine,

[0235]

[0236] Where (v 0x ,v 0y ) is the motion vector of the upper left control point, and (v 1x ,v 1y ) is the motion vector of the upper right control point, and (v 2x ,v 2y ) is the motion vector of the lower left control point, and (x, y) represents the coordinates of the representative point in the current block relative to the upper left sample point. In VTM, the representative point is defined as the center position of the sub-block. For example, when the coordinates of the upper left corner of the sub-block relative to the upper left sample point in the current block are (xs, ys), the coordinates of the representative point are defined as (xs+2, ys+2).

[0237] To further simplify motion compensated prediction, sub-block based affine transformation prediction is applied. The sub-block size M×N is derived from Equation 2, where MvPre is the motion vector fractional precision (1 / 16 in JEM), (v 2x ,v 2y ) is the motion vector of the lower left control point calculated according to Equation 1.

[0238]

[0239] After being derived from Equation 2, M and N should be adjusted downward, if necessary, to be divisors of w and h, respectively.

[0240] In order to derive the motion vector of each M×N sub-block, Figure 20A-20BAs shown, the motion vector of the center sample of each sub-block is calculated according to Equation 1 and rounded to a fractional precision of 1 / 16. Then, the motion compensated interpolation filter mentioned in Section 2.2.3 is applied to generate the prediction of each sub-block using the derived motion vector.

[0241] Fig.21 An example of the affine motion vector field MVF for each sub-block is shown.

[0242] After MCP, the high-precision motion vector of each sub-block is rounded and saved with the same precision as the normal motion vector.

[0243] 2.2.6.1AF_INTER mode

[0244] In JEM, there are two affine motion modes: AF_INTER mode and AF_MERGE mode. For CUs with width and height greater than 8, AF_INTER mode can be applied. The affine flag at the CU level is signaled in the bitstream to indicate whether AF_INTER mode is used. In this mode, neighboring blocks are used to construct a pair of motion vectors {(v 0 , v 1 )|v 0 = {v A , v B , v C},v 1 = {v D , v F}} candidate list. Figure 22A-22B As shown, v is selected from the motion vectors of block A, block B or block C. 0 . Scale the motion vector from the neighboring block according to the reference list and the relationship between the POC of the neighboring block's reference, the POC of the current block's reference, and the POC of the current CU. And select v from neighboring blocks D and E 1The method is similar. If the number of candidate lists is less than 2, the list is filled with motion vector pairs formed by copying each AMVP candidate. When the candidate list is greater than 2, the candidates are first sorted according to the consistency of adjacent motion vectors (the similarity of the two motion vectors in a pair of candidates), and only the first two candidates are retained. The RD cost check is used to determine which motion vector pair candidate is selected as the control point motion vector prediction (CPMVP) of the current CU. And the index indicating the CPMVP position in the candidate list is signaled in the bitstream. After determining the CPMVP of the current affine CU, affine motion estimation is applied and the control point motion vector (CPMV) is found. Then, the difference between CPMV and CPMVP is signaled in the bitstream.

[0245] Fig.23 An example of a motion vector predictor MVP of AF_INTER is shown.

[0246] In AF_INTER mode, when using the 4 / 6 parameter affine mode, 2 / 3 control points are required, and therefore 2 / 3 MVDs need to be encoded for these control points, such as Figure 22A-22B In the existing implementation, it is proposed to derive MV in the following manner, that is, from mvd 0 Medium predicted mvd 1 and mvd 2 .

[0247]

[0248] in, mvd i and mv 1 are the predicted motion vector, motion vector difference and motion vector of the upper left pixel (i=0), upper right pixel (i=1) or lower left pixel (i=2), respectively. Fig. 22B Note that the addition of two motion vectors (e.g., mvA(xA, yA) and mvB(xB, yB)) is equal to the separate summation of the two components, i.e., newMV = mvA + mvB, and the two components of newMV are set to (xA + xB) and (yA + yB), respectively.

[0249] 2.2.6.2AF_MERGE Mode

[0250] When a CU is applied in AF_MERGE mode, it obtains the first block encoded in affine mode from the valid adjacent reconstructed blocks. And the selection order of candidate blocks is from left, top, top right, bottom left to top left, such as Fig.24AIf the adjacent lower left block A is encoded in affine mode, such as Fig. 24B As shown, the motion vectors v of the upper left corner, upper right corner and lower left corner of the CU containing block A are derived 2 、v 3 and v 4 . And according to v 2 、v 3 and v 4 To calculate the motion vector v of the upper left corner of the current CU 0 . Secondly, calculate the motion vector v of the upper right corner of the current CU 1 .

[0251] In deriving the CPMV v of the current CU 0 and v 1 Afterwards, the MVF of the current CU is generated according to the simplified affine motion model Equation 1. In order to identify whether the current CU is coded in AF_MERGE mode, an affine flag is signaled in the bitstream when there is at least one neighboring block coded in affine mode.

[0252] In some implementations, the affine merge candidate list is constructed by the following steps:

[0253] Insert inheritance affine candidates

[0254] An inherited affine candidate is one that is derived from the affine motion model of its valid neighboring affine coded blocks. In a common basis, such as Figure 24A-24B As shown, the scanning order of candidate positions is: A1, B1, B0, A0 and B2.

[0255] After the candidates are derived, a full pruning process is performed to check if the same candidate has been inserted into the list. If there is an identical candidate, the derived candidate is discarded.

[0256] Insert the constructed affine candidate

[0257] If the number of candidates in the affine merge candidate list is less than MaxNumAffineCand (set to 5 in this draft), the constructed affine candidate is inserted into the candidate list. The constructed affine candidate refers to a candidate constructed by combining the neighboring motion information of each control point.

[0258] The motion information of the control point is first obtained from Figure 24A-24B CPk(k=1,2,3,4) represents the kth control point. A0, A1, A2, B0, B1, B2 and B3 are the spatial locations of predicted CPk(k=1,2,3); T is the temporal location of predicted CP4.

[0259] The coordinates of CP1, CP2, CP3 and CP4 are (0,0), (W,0), (H,0) and (W,H), respectively, where W and H are the width and height of the current block.

[0260] Fig.25 An example of candidate positions for the affine merge mode is shown.

[0261] The motion information for each control point is obtained according to the following priority order:

[0262] For CP1, the inspection priority is B 2 ->B 3 ->A 2 If B 2 If available, use B 2 Otherwise, if B 2 If available, use B 3 If B 2 and B 3 If neither is available, use A 2 If all three candidates are unavailable, the motion information of CP1 cannot be obtained.

[0263] For CP2, the inspection priority is B 1 ->B 0 .

[0264] For CP3, the inspection priority is A 1 ->A 0 .

[0265] For CP4, use T.

[0266] Second, affine merge candidates are constructed using combinations of control points.

[0267] Constructing a 6-parameter affine candidate requires motion information of three control points. The three control points can be selected from one of the following four combinations ({CP1, CP2, CP4}, {CP1, CP2, CP3}, {CP2, CP3, CP4}, {CP1, CP3, CP4}). The combination {CP1, CP2, CP3}, {CP2, CP3, CP4}, {CP1, CP3, CP4} will be converted into a 6-parameter motion model represented by the top left, top right, and bottom left control points.

[0268] Constructing a 4-parameter affine candidate requires motion information of two control points. These two control points can be selected from one of the following six combinations ({CP1,CP4},{CP2,CP3},{CP1,CP2},{CP2,CP4},{CP1,CP3},{CP3,CP4}). The combination {CP1,CP4},{CP2,CP3},{CP2,CP4},{CP1,CP3},{CP3,CP4} will be converted into a 4-parameter motion model represented by the top left and top right control points.

[0269] The combinations of constructed affine candidates are inserted into the candidate list in the following order:

[0270] {CP1,CP2,CP3}, {CP1,CP2,CP4}, {CP1,CP3,CP4}, {CP2,CP3,CP4}, {CP1,CP2}, {CP1,CP3}, {CP2,CP3}, {CP1,CP4}, {CP2,CP4}, {CP3,CP4}.

[0271] For a combined reference list X (X is 0 or 1), the reference index with the highest usage in the control point is selected as the reference index of list X, and the motion vector pointing to the difference reference picture will be scaled.

[0272] After the candidates are derived, a full pruning process is performed to check if the same candidate has been inserted into the list. If there is an identical candidate, the derived candidate will be discarded.

[0273] Fill with zero motion vectors

[0274] If the number of candidates in the affine merge candidate list is less than 5, zero motion vectors with zero reference index are inserted into the candidate list until the list is full.

[0275] 2.2.7 Bidirectional Optical Flow

[0276] Bi-directional optical flow (BIO) is a sample-level motion refinement that is performed on the basis of block-level motion compensation for bi-directional prediction. Sample-level motion refinement does not use signaling.

[0277] Fig.26 An example of an optical flow trajectory is shown.

[0278] Hypothesis I (k) is the luminance value from reference k (k=0,1) after block motion compensation, and I (k) The horizontal and vertical components of the gradient. Assuming that the optical flow is valid, the motion vector field (vx ,v y ) is given by the equation:

[0279]

[0280] Combine the optical flow equation with Hermite interpolation to obtain the motion trajectory of each sample point, and finally get the function value I (k) and derivatives The only third-order polynomial that matches is the BIO prediction:

[0281]

[0282] Here, τ 0 and τ 1 represents the distance to the reference frame, such as Fig.28 As shown. Distance τ 0 and τ 1 Based on Ref 0 and Ref 1 POC calculation: τ 0 =POC(current)-POC(Ref0), τ 1 =POC(Ref1)-POC(current). If the two predictions are from the same time direction (either both from the past or both from the future), then the signals are different (i.e., τ 0 ·τ 1 <0). In this case, the predictions are only valid if they are not from the same moment (i.e., τ 0 ≠τ 1 ), both reference regions have non-zero motion (MVx 0 , MVy 0 , MVx 1 , MVy 1 ≠0), and the block motion vector is proportional to the temporal distance (MVx 0 , MVy 1 =MVy 0 / MVy 1 =-τ 0 / τ 1 ), BIO is only applied when

[0283] By minimizing the distance between points A and B ( Fig.26 The motion vector field (v x ,v y ). The model uses only the first linear term of the local Taylor expansion for Δ:

[0284]

[0285] All values ​​in Equation 5 depend on the sample point position (i′, j′), which has been ignored from the signs so far. Assuming that the motion is consistent in the local surrounding area, we minimize Δ within a (2M+1)×(2M+1) square window Ω centered at the current prediction point (i, j), where M equals 2:

[0286]

[0287] For this optimization problem, JEM uses a simplified approach, first minimizing in the vertical direction and then minimizing in the horizontal direction. This yields:

[0288]

[0289] in,

[0290]

[0291] To avoid division by zero or very small values, regularization parameters r and m are introduced into Equations 7 and 8.

[0292] r=500·4 d-8 (10)

[0293] m=700·4 d-8 (11)

[0294] Here d is the bit depth of the video samples.

[0295] In order to keep the memory access of BIO the same as that of regular bi-predictive motion compensation, all predictions and gradient values ​​I (k) , are only calculated for the positions inside the current block. In Equation 9, the (2M+1)×(2M+1) square window Ω centered at the current prediction point on the boundary of the prediction block needs to access the positions outside the block (such as Fig.27A In JEM, the I (k) , The value of is set equal to the nearest available value inside the block. For example, this can be implemented as padding, such as Fig.27B shown.

[0296] Using BIO, it is possible to refine the motion field for each sample. To reduce the computational complexity, a block-based BIO design is used in JEM. The motion refinement is calculated based on a 4×4 block. In the block-based BIO, s in Equation 9 is aggregated over all samples in the 4×4 block. n value, then s n The aggregated value of is used to derive the BIO motion vector offset for the 4×4 block. More specifically, the following formula is used for block-based BIO derivation:

[0297]

[0298] where b k represents the sample set of the kth 4×4 block belonging to the prediction block. s in equations 7 and 8 n ((s n,bk )>>4) instead to derive the associated motion vector offset.

[0299] In some cases, the MV refinement of BIO may be unreliable due to noise or irregular motion. Therefore, in BIO, the amplitude of MV refinement is clipped to a threshold thBIO. The threshold is determined based on whether the reference pictures of the current picture are all from one direction. If all the reference pictures of the current picture are from one direction, the threshold is set to 12×2 14-d ; otherwise, it is set to 12×2 13-d .

[0300] The gradient of the BIO is calculated simultaneously with the motion compensated interpolation using an operation consistent with the HEVC motion compensation process (2D separable FIR). The input to this 2D separable FIR is the same reference frame samples as the motion compensation process and the fractional positions (fracX, fracY) according to the fractional part of the block motion vector. In the case of , the signal is first interpolated vertically using BIOfilterS corresponding to the fractional position fracY with a descaling offset d-8, and then a gradient filter BIOfilterG corresponding to the fractional position fracX with a descaling offset 18-d is applied in the horizontal direction. In the case of , the gradient filter is first applied vertically using BIOfilterG corresponding to the fractional position fracY with the descaling offset d-8, and then the signal displacement is performed in the horizontal direction using BIOfilterS corresponding to the fractional position fracX with the descaling offset 18-d. The length of the interpolation filter BIOfilterG and the signal displacement BIOfilterF used for the gradient calculation is short (6 taps) in order to maintain a reasonable complexity. Table 3 shows the filters used for the gradient calculation of different fractional positions of the block motion vectors in the BIO. Table 4 shows the interpolation filters used for the prediction signal generation in the BIO.

[0301] Table 3: Filters used for gradient calculation in BIO

[0302] Fractional pixel position Gradient interpolation filter (BIOfilterG) 0 {8,-39,-3,46,-17,5} 1 / 16 {8,-32,-13,50,-18,5} 1 / 8 {7,-27,-20,54,-19,5} 3 / 16 {6,-21,-29,57,-18,5} 1 / 4 {4,-17,-36,60,-15,4} 5 / 16 {3,-9,-44,61,-15,4} 3 / 8 {1,-4,-48,61,-13,3} 7 / 16 {0,1,-54,60,-9,2} 1 / 2 {-1,4,-57,57,-4,1}

[0303] Table 4: Interpolation filters used for prediction signal generation in BIO

[0304] Fractional pixel position Interpolation filter for prediction signal (BIOfilterS) 0 {0,0,64,0,0,0} 1 / 16 {1,-3,64,4,-2,0} 1 / 8 {1,-6,62,9,-3,1} 3 / 16 {2,-8,60,14,-5,1} 1 / 4 {2,-9,57,19,-7,2} 5 / 16 {3,-10,53,24,-8,2} 3 / 8 {3,-11,50,29,-9,2} 7 / 16 {3,-11,44,35,-10,3} 1 / 2 {3,-10,35,44,-11,3}

[0305] In JEM, BIO is applied to all bidirectional prediction blocks when the two predictions come from different reference pictures. BIO is disabled when LIC is enabled for a CU.

[0306] In JEM, OBMC is applied to the block after the normal MC process. To reduce computational complexity, BIO is not applied during the OBMC process. This means that BIO is only applied to the MC process of a block when using its own MV, and BIO is not applied to the MC process when using the MV of a neighboring block during the OBMC process.

[0307] 2.2.8 Decoder-side Motion Vector Refinement

[0308] In the bidirectional prediction operation, for the prediction of a block region, two prediction blocks formed using the motion vectors (MVs) of list 0 and the MVs of list 1, respectively, are combined to form a single prediction signal. In the decoder-side motion vector refinement (DMVR) method, the two motion vectors of the bidirectional prediction are further refined by a bilateral template matching process. Bilateral template matching is applied in the decoder to perform a distortion-based search between the bilateral template and the reconstructed samples in the reference picture to obtain the refined MV without sending additional motion information.

[0309] In DMVR, bilateral templates are generated from the initial MV0 of list 0 and MV1 of list 1 as a weighted combination (i.e., average) of the two prediction blocks, as shown in Fig.28 As shown. The template matching operation includes calculating a cost metric between the generated template and the sample area (around the initial prediction block) in the reference picture. For each of the two reference pictures, the MV that produces the minimum template cost is considered to be the updated MV in the list to replace the original MV. In JEM, 9 MV candidates are searched for each list. The 9 candidate MVs include the original MV and 8 surrounding MVs, where one luminance sample is offset relative to the original MV in the horizontal or vertical direction or both. Finally, the two new MVs (i.e., Fig.28 MV0′ and MV1′ shown) are used to generate the final bidirectional prediction result. The sum of absolute differences (SAD) is used as a cost metric. Note that when calculating the cost of a prediction block generated by one of the surrounding MVs, the rounded MV (rounded to integer pixels) is actually used to obtain the prediction block, not the actual MV.

[0310] DMVR is applied to the Merge mode of bi-directional prediction, where one MV comes from the past reference picture and the other comes from the future reference picture, without transmitting additional syntax elements. In JEM, DMVR is not applied when LIC, affine motion, FRUC or sub-CU Merge candidate is enabled for a CU.

[0311] 2.2.9 Sub-block Merge Candidate List

[0312] It is recommended to put all sub-block related motion candidates except the regular Merge list for non-sub-block Merge candidates into a separate Merge list.

[0313] Sub-block related motion candidates are put into a separate Merge list, which is called "sub-block Merge candidate list".

[0314] In one example, the sub-block Merge candidate list includes affine Merge candidates, ATMVP candidates, and / or sub-block-based STMVP candidates.

[0315] 2.2.9.1 Another example of affine merge list construction

[0316] In this draft, the ATMVP Merge candidate in the normal Merge list is moved to the first position of the affine Merge list, so that all Merge candidates in the new list (i.e., the sub-block-based Merge candidate list) are based on the sub-block coding tool.

[0317] 3. Examples of Problems Overcome by Embodiments

[0318] In the design of triangular partitioning, a block can be divided into two partitions. In order to save the memory bandwidth due to motion compensation, unidirectional prediction is required for both partitions. During the motion compensation process, bidirectional prediction is used for diagonal edges, and unidirectional prediction is used for all the rest. If bidirectional prediction is allowed for each partition, the samples located at the diagonal edge will have four MVs, two from one partition and two from the other partition. This design has the following problems:

[0319] 1) The design of the uni-prediction Merge list depends on the reference picture list, i.e., a higher priority is given to the MV associated with the reference picture from reference picture list 0. That is, the TPM candidates derived from all uni-prediction normal motion candidates are inserted before the TPM candidates derived from all bi-prediction normal motion candidates. However, typically, the positions of spatially adjacent blocks have a stronger correlation in terms of the similarity of MV information.

[0320] 2) Allowing two partitions to select different TPM candidate indexes, however, using the same Merge candidate list for both of them is not optimal because the two partitions may have different relevance to the neighboring blocks of the current block.

[0321] 3) The maximum number of TPM candidates is fixed to 5, which is at least unfriendly to software design.

[0322] 4) The set of weighting factors is predefined for all types of blocks.

[0323] 5) Motion information is stored in 4×4 levels, which increases the memory requirement. How to compress them is a problem that needs to be studied.

[0324] 6) TPM motion information depends on the parity of the two decoded Merge indexes. For candidates with even-valued indexes, first select their L0 motion vector for triangle prediction. If not available, use their L1 motion vector instead. For candidates with odd-valued indexes, first select their L1 motion vector for triangle prediction. If not available, use their L0 motion vector instead. Therefore, the two candidates may come from the same reference picture list.

[0325] 4. Example Embodiments

[0326] The proposed method can be applied to any non-square / non-rectangular partition, e.g., geometric partition. In the following description, we use "triangular partition pattern" to represent an example of a non-square / non-rectangular partition pattern (TPM), and the motion vector prediction candidate inserted into the TPM candidate list is called "TPM candidate". And the motion information associated with any previously coded block is called "regular motion candidate". It should be noted that other types of partitions may also be applicable.

[0327] The detailed list of techniques below should be considered as an example to explain the general concept. These techniques should not be interpreted narrowly. In addition, these techniques can be combined in any way.

[0328] TPM candidate list (AMVP / Merge / Other) construction process with different orders

[0329] 1. Instead of always inserting TPM candidates derived from all uni-directionally predicted normal motion candidates before TPM candidates derived from all bi-directionally predicted normal motion candidates, it is proposed to use the prediction direction as a priority criterion to decide which TPM candidates should be added first.

[0330] a. In one example, all normal motion candidates with unidirectional prediction from list X are inserted into the candidate list as TPM candidates before candidates with unidirectional prediction from list Y.

[0331] b. In one example, all TPM candidates derived from bi-directionally predicted normal motion candidates (eg, truncation list 0 prediction candidates, truncation list 1 prediction candidates) are given higher priority, that is, they can be inserted before TPM candidates derived from uni-directionally predicted normal motion candidates.

[0332] c. In one example, the order is defined as: all TPM candidates derived from bi-directionally predicted normal motion candidate list X (e.g., truncated list X prediction candidates), all TPM candidates derived from bi-directionally predicted normal motion candidate list Y (e.g., truncated list Y prediction candidates), all TPM candidates derived from uni-directionally predicted normal motion candidates (e.g., original uni-directional prediction candidates).

[0333] d. In one example, a TPM candidate derived from one bi-predicted normal motion candidate may be added to the list before a TPM candidate derived from another bi-predicted normal motion candidate.

[0334] e. In one example, TPM candidates derived from one bi-predicted normal motion candidate may be added in an interleaved manner with TPM candidates derived from another bi-predicted normal motion candidate.

[0335] i. When there are two regular motion candidates C A and C B , and four TPM candidates, including C A and C B The derived truncated list of 0 / 1 prediction candidates. The order of adding the list can be defined as: from C A The derived truncated list 0 prediction candidates, from C B The derived truncated list 1 prediction candidate, from C A The derived truncated list 1 prediction candidates, from C B The derived truncated list of 0 prediction candidates.

[0336] ii. Alternatively, when there are two regular motion candidates C A and C B , and four TPM candidates, including C A and C B The derived truncated list of 0 / 1 prediction candidates. The order of adding the list can be defined as: from C A The derived truncated list 1 prediction candidate, from C B The derived truncated list 0 prediction candidates, from C A The derived truncated list 0 prediction candidates, from C B The derived truncated list 1 prediction candidates.

[0337] 2. Instead of always inserting TPM candidates derived from all uni-directionally predicted normal motion candidates before TPM candidates derived from all bi-directionally predicted normal motion candidates, it is proposed to use the coding mode information associated with the normal motion candidates as a priority criterion to decide which TPM candidates should be added first.

[0338] a. The coding information may include AMVP or Merge mode.

[0339] i. In one example, if a regular motion candidate C A is derived from block A, and another C B is derived from block B, and block A is coded in AMVP mode, while block B is coded in Merge mode, then from C A The derived TPM candidate can be found in B The derived TPM candidate is added to the list previously. Alternatively, B The derived TPM candidate can be found in A The derived TPM candidates were previously added to the list.

[0340] b. The coding information may include reference index and / or POC difference.

[0341] i. In one example, with another TPM candidate C' B By comparison, if a TPM candidate C' A is associated with a smaller reference index, then C' A Can be found in C' B Previously added to the list.

[0342] ii. In one example, with another TPM candidate C' B By comparison, if a TPM candidate C' A is associated with a smaller POC distance between the reference picture and the current picture, then C' A Can be found in C' B Previously added to the list.

[0343] c. The coding information may include a picture / slice / slice group quantization parameter (QP) of the reference picture and / or a temporal layer index of the reference picture.

[0344] i. In one example, with another TPM candidate C' B By comparison, if a TPM candidate C' A is associated with a reference picture with a smaller QP, then C' A Can be found in C' B Previously added to the list.

[0345] 3. Instead of always inserting TPM candidates derived from all uni-directionally predicted normal motion candidates before TPM candidates derived from all bi-directionally predicted normal motion candidates, it is proposed to use the position where the motion candidates are derived as a priority criterion to decide which TPM candidates should be added first.

[0346] a. In one example, if a regular motion candidate C A is derived from block A, and another C B is derived from block B, and B is checked after A, then from C A Derivation (for example, if C A is a unidirectional prediction, or two TPM candidates, each candidate is from the same A Those TPM motion candidates that are directly inherited from a prediction direction associated with block C B The derived candidates are previously added to the list.

[0347] b. Alternatively, in addition, both the position and the reference picture list (or prediction direction) can be used as priority criteria. For example, if a regular motion candidate C A is derived from block A, and another C B is derived from block B, and B is checked after A, then C from list X A The derived TPM motion candidates can be found in C from list X B Those TPM motion candidates derived are previously added to the list. A The derived TPM motion candidates can be found in C from list Y B Those TPM motion candidates derived are previously added to the list. X is equal to 0 or 1, and Y is equal to (1-X).

[0348] 4. It is proposed to add all truncated list 1 prediction candidates before truncated list 0 prediction candidates.

[0349] a. Alternatively, the order of truncation list 0 prediction candidates and truncation list 1 prediction candidates may be adaptively changed, for example, based on available TPM motion candidates.

[0350] i. In one example, if the first TPM motion candidate is an original uni-directional prediction candidate with a uni-directional prediction from list 1, the truncated list 1 prediction candidate may be added before the truncated list 0 prediction candidate.

[0351] ii. Alternatively, if the original unidirectional prediction candidates are more from list 1, then the truncated list 1 prediction candidates can be added before the truncated list 0 prediction candidates.

[0352] b. Alternatively, all TPM candidates derived from one bi-predictive motion candidate (eg, one truncation list 0 prediction candidate and one truncation list 0 prediction candidate) may be added to the list before all TPM candidates derived from another bi-predictive motion candidate.

[0353] i. Alternatively, in addition, for those TPM candidates derived from a motion candidate, the order in which they are added may further depend on the information of the TPM motion candidates available in the list, eg, 4.a.

[0354] TPM candidate list (AMVP / Merge / Other) construction process with more TPM candidates

[0355] 5. Multiple average unidirectionally predicted motion candidates from list 0 and list 1 derived from one regular motion candidate can all be added to the TPM motion candidate list.

[0356] a. Alternatively, only one of them may be added, and which one is added depends on the information of the TPM candidates available in the list, for example, 4.a.

[0357] b. Alternatively, only one of them may be added, and which one is added depends on the reference picture indices of the two prediction lists. The motion vector associated with the smaller reference picture index will be kept, and the final prediction direction is associated with such a reference picture.

[0358] c. Alternatively, only one of them can be added, and which one is added depends on the POC distance between the reference pictures of the two prediction lists and the current picture. The motion vector associated with the smaller POC distance will be kept, and the final prediction direction is associated with such a reference picture.

[0359] d. In one example, all motion candidates of average unidirectional prediction from list 0 may be inserted before all motion candidates of average unidirectional prediction from list 1. Alternatively,

[0360] All motion candidates of average uni-directional prediction from list 1 may be inserted before all motion candidates of average uni-directional prediction from list 0 .

[0361] 6. It is proposed that the average unidirectional prediction TPM candidate can be derived using the unidirectional prediction regular motion candidate.

[0362] a. The unidirectionally predicted normal motion candidates from list LX may be used to generate an average unidirectionally predicted candidate of list LX, for example, together with bidirectionally predicted normal motion candidates or with other unidirectionally predicted normal motion candidates from list X to generate an average unidirectionally predicted candidate of list LX.

[0363] b. The unidirectionally predicted normal motion candidates from list LX can be first scaled to LY (Y=1-X), and the scaled motion vector can be used to generate an average unidirectionally predicted candidate of list LY, for example, together with bidirectionally predicted normal motion candidates or together with other unidirectionally predicted normal motion candidates from list X.

[0364] 7. Virtual TPM motion candidates derived from available TPM motion candidates may also be added to the TPM candidate list.

[0365] a. In one example, after a certain step (eg, after generating the truncated list 1 prediction candidates), virtual candidates can be derived from those candidates that have been added to the Merge list.

[0366] b. In one example, virtual candidates can be derived only from the original unidirectional prediction candidates.

[0367] c. In one example, virtual candidates can be derived only from the original unidirectional prediction candidates and the truncated list 1 prediction candidates.

[0368] d. In one example, virtual candidates may be derived only from candidates with a specific reference picture index (ie, equal to 0) and / or with a specific range of POC distances between the reference picture and the current picture or from a reference picture list.

[0369] e. In one example, virtual candidates may be derived from only the first K TPM motion candidates, eg, K is set to 1.

[0370] i. For example, the MVs of two TPM motion candidates may be averaged to obtain a virtual candidate.

[0371] f. For its motion vector C mv Representation, reference index by C refidx The TPM motion candidates represented by can be derived using the following method:

[0372] i. To C mv Add an offset to the horizontal or vertical component and use the same C refidx .

[0373] ii. To C mv Add the offset to the horizontal and vertical components and use the same C refidx .

[0374] iii. C mv Zoom to C refidx Using the scaled motion vector and reference picture (where C mv The index of the ) is scaled to it as the new TPM motion candidate.

[0375] g. When adding a virtual TPM candidate, pruning may not be applied.

[0376] i. Alternatively, pruning may be applied to candidates excluding those from which the virtual TPM candidate was generated.

[0377] TPM candidate list (AMVP / Merge / Other) construction process with adaptive insertion order

[0378] 8. The insertion order (priority criteria) of TPM candidates can be changed from sequence to sequence, picture to picture, slice to slice, slice group to slice group, block to block, etc.

[0379] a. In one example, it may depend on block size / block shape / partitioning mode.

[0380] b. Alternatively, the order may be signaled from the encoder to the decoder in VPS / SPS / PPS / slice header / slice group header / slice / CTU / CU.

[0381] Multiple TPM lists partitioned in TPM encoding blocks

[0382] 9. For each geometric partition, a separate motion candidate list can be built instead of using one list for all partitions.

[0383] a. In one example, for one partition, its associated candidate list contains only motion candidates predicted from list X, while for another partition, its associated candidate list contains only motion candidates predicted from list Y, where Y is not equal to X.

[0384] b. In one example, different spatial and / or temporal blocks may be accessed to derive motion candidates to be added to the Merge list.

[0385] c. In one example, the location of the spatial and / or temporal blocks of a geometric partition may depend on the location of the partition.

[0386] d. In one example, the location of the spatial and / or temporal blocks of a geometric partition may depend on the partitioning method (from upper left to lower right (45 degrees) or from upper right to lower left (135 degrees)).

[0387] e. In one example, for a 135 degree division mode (such as Figure 13A-13B As depicted), more blocks above can be checked to build the Merge list of PU1. Fig.29 Some examples of the upper blocks (marked in grey) are depicted in FIG.

[0388] f. In one example, for a 135 degree division mode (such as Figure 13A-13BAs depicted), more blocks on the left can be checked to build the Merge list of PU2. Fig.29 Some examples of the blocks on the left (marked in grey) are depicted in FIG.

[0389] g. In one example, for a 45 degree division pattern (such as Figure 13A-13B As depicted), more blocks on the left and above can be checked to build the Merge list of PU1. Fig.29 Some examples of blocks on the left and blocks above (marked in grey) are depicted in FIG.

[0390] h. In one example, for a 45 degree division pattern (such as Figure 13A-13B As depicted), more time domain blocks can be checked to construct the Merge list of PU2.

[0391] i. In one example, the index of the TPM list is signaled for each partition.

[0392] 10. Multiple TPM lists can be built and shared for all partitions, and each partition can select one of the TPM lists (eg, each partition can select the same TPM list from multiple TPM lists).

[0393] a. In one example, the index of the TPM list may be signaled first, and then the index of the TPM candidate may be further signaled.

[0394] b. In another example, the index of the TPM list and the index of the TPM candidate may be jointly encoded.

[0395] Signaling of maximum allowed TPM candidates

[0396] 11. The maximum allowed TPM candidates for Merge / AMVP are signaled in SPS / VPS / PPS / picture header / slice header / slice group header / LCU row / LCU group.

[0397] a. Alternatively, the maximum allowed TPM motion candidates are set to for non-TPM coded blocks, e.g., non-TPMMerge candidate list size or sub-block Merge candidate list size.

[0398] b. The maximum number is binarized using unary coding or truncated unary coding during signaling.

[0399] c. Instead of directly signaling the maximum number, (M-maximum number) can be signaled, for example, M=5 or 6.

[0400] d. A one-bit flag may be signaled to indicate whether the maximum number of TPM candidates is the same as the maximum number of regular motion candidates / sub-block motion candidates.

[0401] Weighting factor group

[0402] 12. The selection of the set of weighting factors may depend on the width and / or height of the current block.

[0403] a. In one example, if the ratio between width and height is greater than a threshold, a particular set of weighting factors (eg, set 1) may be selected.

[0404] b. In one example, if the ratio between the height and the width is greater than a threshold, a particular set of weighting factors (eg, set 1) may be selected.

[0405] c. A set of multiple weighting factor groups may be predefined, and for different block widths and / or heights, a weighting factor group may be selected from one or two of them.

[0406] 13. The weighting factors can be changed on-the-fly based on the location of the sample points.

[0407] a. In one example, it may depend on the angle of the edge that separates the two partitions.

[0408] b. In one example, the weighting factor can be defined as a function of the sample location.

[0409] c. In one example, the final prediction block of a TPM encoding block is set to (f(x,y)*P1(x,y)+(2 M –f(x,y))*P2(x,y)+offset)>>M, where P1 and P2 are the two predicted values ​​of the sample point at (x,y), M is an integer value, the offset can be set to (1<<(M-1)), and f(x,y) is a function that returns the weight applied to the first predicted value.

[0410] 14. For blocks located in weighted regions, motion compensation is performed at 8×8 level instead of 4×4 level to reduce memory bandwidth.

[0411] a. Alternatively, for blocks located in the weighted region, motion compensation is performed at 8×4 or 4×8 level instead of 4×4 level to reduce memory bandwidth.

[0412] b. Alternatively, if 8×4 or 4×8 based motion compensation is allowed, the choice of 8×4 or 4×8 may depend on

[0413] i. Block width and / or height.

[0414] ii. Division pattern (eg, 45 degrees or 135 degrees).

[0415] Storage of motion information

[0416] 15. After decoding the TPM encoded block, one or more HMVP tables may be updated.

[0417] a. In one example, one or more HMVP tables may be updated based on the motion information of one partition (excluding the motion information of other partitions).

[0418] b. In one example, one or more HMVP tables may be updated based on stored motion information for weighted regions excluding non-weighted regions or non-weighted regions excluding weighted regions.

[0419] c. In one example, one or more HMVP tables may be updated according to a Merge candidate in a Merge candidate list, where the Merge candidate list is used to derive motion information of a TPM coding block.

[0420] i. In one example, a Merge candidate is associated with a Merge index of a decoded Merge index (eg, merge_triangle_idx0 or merge_triangle_idx1).

[0421] ii. In one example, a Merge candidate is associated with a Merge index equal to k (eg, k=0).

[0422] 16. The derivation of triangular motion information for one partition may depend on the decoded motion information from another partition, for example, based on the prediction direction of another partition.

[0423] d. In one example, assume that a first reference picture list (denoted as LX) of a decoding candidate with an index equal to merge_triangle_idx0 is selected (e.g., based on the parity of merge_triangle_idx0 and the availability of LX). Regardless of the parity of the second decoding candidate index, the motion information associated with the second reference picture list (denoted as LY) (Y=1-X) of the second candidate is selected as the motion information of the triangle partition (if available). In the following discussion, the motion information associated with the reference picture list LX or LY may also be referred to as LX motion information or LY motion information.

[0424] i. Alternatively, further, if the LY (Y=1-X) motion information of the second candidate with merge_triangle_idx1 is not available, the LX motion information of the second candidate may be used instead.

[0425] ii. Alternatively, further, if the LY (Y=1-X) motion information of the second candidate with merge_triangle_idx1 is not available, the LX motion information of the second candidate may be used to derive the LY motion information, and the derived LY motion information may be used for TPM encoding.

[0426] 1) In one example, the derivation process may include scaling the motion vectors of the reference pictures in LX to the reference pictures in LY.

[0427] e. In one example, assume that the first reference picture list (denoted as LX) of a decoding candidate with an index equal to merge_triangle_idx1 is selected (e.g., based on the parity of merge_triangle_idx1 and the availability of LX). Regardless of the parity of the second decoding candidate index, LY (Y=1-X) motion information (if available) is selected as the motion information of the triangle partition.

[0428] i. Alternatively, further, if the candidate LY (Y=1-X) motion information with merge_triangle_idx0 is not available, the LX motion information may be used instead.

[0429] ii. Alternatively, further, if the candidate LY (Y=1-X) motion information with merge_triangle_idx0 is not available, the candidate LX motion information with merge_triangle_idx0 may be used to derive the LY motion information, and the derived LY motion information may be used for TPM encoding.

[0430] 1) In one example, the derivation process may include scaling the motion vectors of the reference pictures in LX to the reference pictures in LY.

[0431] 17. For multiple triangle partitions, it is not allowed to use motion information from the same reference picture list.

[0432] f. In one example, the conformance bitstream should satisfy that two triangle partitions are predicted from different reference picture lists.

[0433] g. Alternatively, the derivation process of deriving motion information based on the parity of the Merge index can always output two partitions predicted from two different lists, for example, using the method disclosed in bullet point 15 for prediction.

[0434] 18. How to store motion information and / or how to derive motion information to be stored for weighted areas and / or non-weighted areas may depend on the partitioning direction of the triangle partition and / or the decoded Merge index, and / or the Merge candidate list for deriving motion information for the triangle partition, and / or the reference pictures in the reference picture list (for example, all reference pictures have a smaller or no larger POC value than the current picture).

[0435] a. In one example, the motion information of the Merge candidate associated with the selected decoded Merge index of a partition may be stored.

[0436] i. In one example, the selected decoding Merge index may be set to one of the decoding Merge candidate indexes, for example, merge_triangle_idx0 or merge_triangle_idx1.

[0437] ii. In one example, the selected decoded Merge index is set to a variable k.

[0438] 1) In one example, k is set to 0.

[0439] 2) In one example, all information of the kth Merge candidate may be inherited and stored.

[0440] 3) In one example, part of the information of the kth Merge candidate may be inherited, and the rest of the information may be modified before storage.

[0441] (a) In one example, the indication of the weighting factor used in the generalized bidirectional prediction (GBi) of the kth Merge candidate may not be inherited. Alternatively, in addition, a specific GBi weighting index may be assigned (eg, 0) to the motion information stored by the TPM.

[0442] (b) In one example, the indication of the motion vector accuracy of the kth Merge candidate may not be inherited.

[0443] iii. Alternatively, furthermore, the selected decoding Merge index may depend on the partitioning direction.

[0444] iv. In one example, if the Merge candidate with an index equal to the selected decoded Merge index is a bidirectional prediction candidate, bidirectional prediction motion information may be stored. v. In one example, if the Merge candidate with an index equal to the selected decoded Merge index is a unidirectional prediction, unidirectional prediction motion information may be stored.

[0445] 1) Alternatively, bi-directional prediction motion information may be stored, and the stored motion information may be derived from the Merge candidate.

[0446] vi. Alternatively, a function may be stored that stores motion information about both partitions.

[0447] 1) For example, the average value of the motion vectors of two partitions may be stored.

[0448] b. In one example, assuming that MvInfo1 and MvInfo2 indicate motion information of two partitions, virtual bidirectional prediction motion information may be stored by modifying one prediction direction from LX in MvInfoA (A is 1 or 2) to LY (Y=1-X).

[0449] i. In one example, the motion vector and reference index of MvInfoA remain unchanged.

[0450] ii. In one example, furthermore, the reference index of MvInfoA remains unchanged, while the motion vector of MvInfoA is set to an opposite value.

[0451] iii. The virtual bi-directional prediction motion information may include modified MvInfoA and unmodified MvInfoB (where B=3-A).

[0452] iv. In one example, this modification may be applied only when both MvInfo1 and MvInfo2 are from the same prediction direction.

[0453] c. In one example, assuming that MvInfo1 and MvInfo2 indicate motion information of two partitions, how to store the motion information may depend on a low latency check flag indicating whether all reference pictures have smaller or no larger POC values ​​than the current picture.

[0454] i. In one example, the following procedures are applied in sequence:

[0455] If MvInfo1 and MvInfo2 are from different reference picture lists, MvInfo0 and MvInfo1 may be combined and stored as bidirectional prediction motion information.

[0456] ii. If MvInfo1 and MvInfo2 are from the same LX (X=0 or 1) direction, the following process can be applied:

[0457] If the low latency check flag is true (e.g., all reference pictures have smaller or no larger POC values ​​than the current picture), MvInfoA (e.g., A=2) can be modified by setting the prediction direction list from LX to LY (Y=1-X), and the modified MvInfoA and the unmodified MVInfoC (C=3-A) can be combined to form virtual bidirectional prediction motion information and stored.

[0458] Otherwise, MvInfoB (eg, B=2) is stored, that is, unidirectional prediction motion information is stored.

[0459] iii. In one example, MvInfo1 and Fig.13A PU1 and Fig. 13B PU1 in MvInfo2 is associated with Fig.13A PU2 and Fig. 13B is associated with PU2 in.

[0460] d. In one example, the above method can be applied to store motion information of certain sub-blocks within a block.

[0461] i. In one example, some of the sub-blocks may be those in the weighted region.

[0462] ii. In one example, some of the sub-blocks may be those sub-blocks in a block that contain diagonal lines or anti-diagonal lines.

[0463] iii. In one example, certain sub-blocks may be those at the bottom right corner of the block.

[0464] iv. In one example, some of the sub-blocks may be those in the right column or bottom row of the block.

[0465] v. Alternatively, the above method can be applied to store motion information of all sub-blocks within a block.

[0466] 19. For the TPM mode, it is proposed that the motion information of a sub-region within a block may be different from the motion information used in the motion compensation process of the reconstruction of the sub-region.

[0467] a. In one example, for an M×N sub-region located in a weighted region (e.g., 4×4 in the current coding unit using triangular prediction mode), motion compensation can be done as bidirectional prediction, but only one motion information set in list 0 or list 1 for bidirectional prediction can be stored.

[0468] i. Alternatively, for an M×N sub-region located in a weighted region (e.g., 4×4 in the current coding unit using triangular prediction mode), motion compensation can be done as bidirectional prediction, but only unidirectional prediction information from list X can be stored, and the stored information is different from the motion information from list X used in the motion compensation process.

[0469] ii. Alternatively, for an M×N sub-region located in a weighted region (e.g., 4×4 in the current coding unit using triangular prediction mode), motion compensation can be performed as bidirectional prediction, but bidirectional prediction information different from the information used in the motion compensation process (e.g., different MV and / or different reference picture) can be stored.

[0470] b. In one example, for M×N sub-regions located in non-weighted regions (e.g., 4×4 in the current coding unit using triangle prediction mode), motion compensation can be done as unidirectional prediction, but bidirectional prediction motion information can be stored.

[0471] i. Alternatively, for M×N sub-regions located in non-weighted regions (e.g., 4×4 in the current coding unit using triangular prediction mode), motion compensation can be done as unidirectional prediction, but the unidirectional prediction motion information can be stored and can be different from the information used in the motion compensation process.

[0472] c. The stored motion information can be used to encode other blocks (for example, as a spatial motion candidate in the Merge / AMVP mode of the neighboring blocks).

[0473] d. The stored motion information may be used to encode future blocks in different pictures (eg, to derive temporal motion vector candidates).

[0474] e. The stored motion information can be used for in-loop processing, such as de-blocking or ALF (adaptive loop filtering).

[0475] f. In one example, motion information of a first sub-region within a block may be different from motion information used in a motion compensation process for reconstructing the first sub-region, while motion information of a second sub-region within the block may be the same as motion information used in a motion compensation process for reconstructing the second sub-region, and the first sub-region and the second sub-region may be located in the same coding unit using a triangular prediction mode.

[0476] 20. It is proposed to store the same motion information set of the entire block, regardless of whether the block is located in a weighted region. Assume that MvInfo1 and MvInfo2 indicate motion information of two partitions respectively (eg, according to two Merge candidate indexes decoded / derived).

[0477] a. In one example, uni-directional prediction motion information (eg, motion information inherited from one of the two partitions or derived from motion information of both partitions) may be stored.

[0478] i. In one example, MvInfo1 may be stored for the entire block.

[0479] 1. In one example, MvInfo1 may be stored for all sub-regions within a block.

[0480] ii. In one example, MvInfo2 may be stored for the entire block.

[0481] 1. In one example, MvInfo1 may be stored for all sub-regions within a block.

[0482] iii. In one example, which partition's motion information is to be stored may depend on the POC distance relative to the current picture.

[0483] 1. For example, if the absolute value of the POC distance between the current picture and the reference picture referenced by MvInfo1 is smaller than the POC distance value between the current picture and the reference picture referenced by MvInfo2, MvInfo1 may be stored.

[0484] iv. In one example, which partition's motion information is to be stored may depend on the QP of the reference picture.

[0485] 1. For example, when the QP of the reference picture referenced by MvInfo1 is smaller than the value of the reference picture referenced by MvInfo2, MvInfo1 may be stored.

[0486] v. In one example, which partition's motion information is to be stored may depend on the reference index of the reference picture.

[0487] 1. For example, when the QP of the reference picture referenced by MvInfo1 is smaller than the value of the reference picture referenced by MvInfo2, MvInfo1 may be stored.

[0488] 2. For example, if the reference index of the reference picture referenced by MvInfo1 is smaller than the reference index value of the reference picture referenced by MvInfo2, MvInfo1 may be stored.

[0489] vi. In one example, which partition's motion information is to be stored may depend on the Merge index associated with a partition.

[0490] 1. For example, if the associated Merge index of derived MvInfo1 is less than the associated Merge index of derived MvInfo2, MvInfo1 may be stored.

[0491] 2. For example, if merge_triangle_idx0 is smaller than merge_triangle_idx1, the motion information derived from merge_triangle_idx0 may be stored.

[0492] vii. In one example, a third motion information set (denoted as MvInfo3) derived from MvInfo1 and MvInfo2 may be stored.

[0493] 1. In one example, the MV of MvInfo3 may be derived as the average of two MVs of MvInof1 and MvInfo2, or the scaled or mapped motion vectors of one MV and the other MV from MvInof1 and MvInfo2 may be averaged to generate the MV of MvInfo3.

[0494] 2. In one example, the reference picture of MvInfo3 may be one of the two reference pictures of MvInfo1 and MvInfo2.

[0495] viii. In one example, a motion vector in MvInfoX (X is 0 or 1) can be scaled to a reference picture in MvInfoY (where Y is 1-X), which can then be used to derive the motion information to be stored.

[0496] 1. In one example, scaled MvInfo1 and MvInfo2 may be used to derive motion information to be stored, such as the motion information specified in bullet vii.

[0497] ix. The above method can be applied only to sub-regions located in the weighted region.

[0498] 1. Alternatively, the above method may be applied only to sub-regions located in the unweighted region.

[0499] 2. Alternatively, the above method may be applied only to certain sub-areas within the weighted area.

[0500] a. For example, they can be applied to sub-areas located at the top right and / or bottom left of the weighted area.

[0501] b. For example, they may be applied to sub-areas located to the upper left and / or lower right of the weighted area.

[0502] c. Which sub-region to apply to may depend on the division direction.

[0503] b. In one example, the bidirectional prediction motion vector is derived from MvInfo1 and MvInfo2 and stored.

[0504] i. In the case where MvInfo1 and MvInfo2 have motion vectors from different directions (L0 or L1), MvInfo1 and MvInfo2 are simply combined to form a bidirectional prediction motion vector.

[0505] ii. In the case where both MvInfo1 and MvInfo2 are from the same LX (X=0 or 1) direction,

[0506] 1. The stored MV for LX may be derived from one motion vector or from both motion vectors of MvInfo1 and MvInfo2 (eg, by averaging).

[0507] 2. If the reference picture of MvInfo1 or MvInfo2 is included in LY (Y=1-X), MvInfo1 and MvInfo2 are simply combined to form a bidirectional prediction motion vector, and one of the prediction directions is set to LY.

[0508] 3. A motion vector can be scaled to a reference picture in LY (Y=1-X), and the scaled MV and another MV are combined to form a bidirectional prediction motion vector. In one example, the target reference picture in LY referenced by the scaled MV can be predefined (e.g., the reference picture index is equal to 0) or can be signaled.

[0509] 4. A motion vector can be mapped to a reference picture in LY (Y=1-X), and the mapped MV and another MV are combined to form a bidirectional prediction motion vector. In one example, the target reference picture in LY referenced by the mapped MV can be predefined (e.g., the reference picture index is equal to 0) or can be signaled. In one example, the mapping process can be completed without scaling. For example, the mapped MV can be equal to or opposite to the original MV.

[0510] i. The above method is only applicable to sub-regions located in the weighted region.

[0511] 1) Alternatively, the above method can be applied only to sub-regions located in the unweighted region

[0512] 2) Alternatively, the above method may be applied only to certain sub-regions within the weighted region.

[0513] (a) For example, they can be applied to sub-regions located at the upper right and / or lower left of the weighted region.

[0514] (b) For example, they may be applied to sub-regions located at the upper left and / or lower right of the weighted region.

[0515] (c) Which sub-region to apply to may depend on the partitioning direction.

[0516] c. Whether to store unidirectional or bidirectional prediction motion information depends on the decoded motion information of the two partitions.

[0517] i. In one example, if both MvInfo1 and MvInfo2 are from the same LX reference list, unidirectional prediction motion information may be stored.

[0518] ii In one example, if MvInfo1 and MvInfo2 are from different reference lists (one from L0 and the other from L1), bidirectional prediction motion information may be stored.

[0519] iii. In one example, if both MvInfo1 and MvInfo2 are from the same LX reference list, but at least one of the reference pictures referenced by MvInfo1 or MvInfo2 is also in another reference list LY (Y=1-X), bidirectional prediction motion information can be stored.

[0520] d. Alternatively, the storage of motion information is still based on whether a sub-block is located in a weighted region.

[0521] For those sub-blocks located in the weighted region, the stored motion information is derived following the following rules:

[0522] i. In one example, if MvInfo1 and MvInfo2 are from the same LX (X=0 or 1) direction,

[0523] 1. If the reference picture of MvInfo1 or MvInfo2 is included in LY (Y=1-X), MvInfo1 and MvInfo2 are simply combined to form a bidirectional prediction motion vector, and one of the prediction directions is set to LY.

[0524] 2. A motion vector can be scaled to a reference picture in LY (Y=1-X), and the scaled MV and another MV are combined to form a bidirectional prediction motion vector. In one example, the target reference picture in LY referenced by the scaled MV can be predefined (e.g., the reference picture index is equal to 0) or can be signaled.

[0525] 3. A motion vector can be mapped to a reference picture in LY (Y=1-X), and the mapped MV and another MV are combined to form a bidirectional prediction motion vector. In one example, the target reference picture in LY referenced by the mapped MV can be predefined (e.g., the reference picture index is equal to 0) or can be signaled. In one example, the mapping process can be completed without scaling. For example, the mapped MV can be equal to or opposite to the original MV.

[0526] ii. In one example, if MvInfo1 or MvInfo2 is from the same LX (X=0 or 1) direction, MvInfo1 or MvInfo2 is stored. That is, a unidirectional prediction motion vector is stored.

[0527] iii. In one example, if MvInfo1 and MvInfo2 are from the same LX (X=0 or 1) direction, one motion vector may be scaled to the reference picture of the other motion vector, and an average or weighted average of the scaled motion vector and the other motion vector may be stored.

[0528] iv. In one example, if MvInfo1 and MvInfo2 are from the same LX (X=0 or 1) direction and the average or weighted average of the two motion vectors can be stored, the reference picture can be one of the two reference pictures of MvInof1 and MvInfo2.

[0529] v. In one example, the stored motion information for the entire block can be derived from the motion information in each sub-block. For example, a weighted sum is performed on the MVs in all or some sub-blocks to derive the stored MV for the entire block.

[0530] vi. For the above example, different sub-regions can store different motion information.

[0531] vii. For the above examples, sub-regions in the weighted region can store uni-directional prediction or bi-directional prediction motion information.

[0532] e. Alternatively, the storage of motion information is still based on the position of the sub-block. However, for each sub-block, only unidirectional prediction motion information can be stored.

[0533] i. In one example, the motion information of the sub-region in the weighted region may be inherited or derived from MvInfo1 and / or MvInfo2, which may be stored.

[0534] f. In one example, the stored motion information is only used in some modules.

[0535] i. In one example, the stored motion information is used for temporal motion prediction.

[0536] ii. Alternatively, in addition, the stored motion information is used for spatial motion prediction.

[0537] iii. Alternatively, in addition, the stored motion information is used for a filtering (eg deblocking) process.

[0538] g. In one example, the stored motion information can be used in the motion compensation process of the block.

[0539] h. In one example, one or more HMVP tables may be updated based on the stored motion information.

[0540] 21. It is proposed that the storage of motion information is based on M×N block units (where M and N cannot be equal to 4 at the same time). Each M×N block shares the same motion information.

[0541] a. In one example, M and N are set to 8.

[0542] b. In one example, for an M×N block, if a part of it belongs to the weighted region and a part of it belongs to the non-weighted region, such a block follows the rule of the weighted region for motion vector storage.

[0543] c. Alternatively, for an M×N block, if its first part belongs to the weighted region, but its second part belongs to the non-weighted region, such a block can follow the rule of the non-weighted region to store motion information.

[0544] 22. Whether to enable or disable TPM for two partitions to be predicted from the same reference picture can be based on conditions such as the motion vectors of the two partitions should be different enough.

[0545] a. In one example, the reference samples of two partitions should not overlap.

[0546] b. In one example, abs(MV0[0]–MV1[0]) should be less than TH, where MV0 and MV1 are the motion vectors of the two partitions, and MVX[0] and MVX[1] are the horizontal and vertical components of MVX, respectively. The function abs(x) returns the absolute value of x.

[0547] c. In one example, abs(MV0[1]–MV1[1]) should be less than TH.

[0548] d. In one example, abs(MV0[0]–MV1[0])+abs(MV0[1]–MV1[1]) should be less than TH.

[0549] e. In an example, Max(abs(MV0[0]–MV1[0]),abs(MV0[1]–MV1[1]) should be less than TH, where the function Max(x,y) returns the larger of x and y.

[0550] f. The consistency bitstream should satisfy when TPM is enabled for a block, and one or more of the above bullet points are true.

[0551] 23. In the disclosed method, the stored motion information can be used to predict the motion information of a subsequently decoded block in the current picture or other pictures to be decoded.

[0552] a. The stored motion information may include:

[0553] vii. Motion Vector

[0554] vii. Reference Index

[0555] ix. Indication of one-way prediction or two-way prediction

[0556] x. Indication of inter-frame prediction direction

[0557] xi. Indication of Generalized Bidirectional Prediction (GBi)

[0558] xii. Motion vector resolution

[0559] xiii. Instructions for affine prediction

[0560] 5. Exemplary Embodiments of the Disclosed Technology

[0561] 5.1 Example #1

[0562] Motion vector storage process of triangle merge mode

[0563] The variables numSbX and numSbY specify that the number of 4×4 blocks in the horizontal and vertical directions in the current coding block is set equal to numSbX=cbWidth>>2 and numSbY=cbHeight>>2.

[0564] Where cbWidth and cbHeight specify the width and height of the current coding block in the luminance sample point.

[0565] The variable minSb is set to min(numSbX,numSbY)-1.

[0566] The variable cbRatio is derived as follows:

[0567] cbRatio=(cbWidth>cbHeight)? (cbWidth / cbHeight):(cbHeight / cbWidth)

[0568] For each 4×4 sub-block at sub-block index (xSbIdx, ySbIdx), where xSbIdx = 0 ... numSbx-1, and ySbIdx = 0 ... numSbY-1, the following applies:

[0569] -The variables xidX and yIdx are derived as follows:

[0570] xIdx=(cbWidth>cbHeight)? (xSbIdx / cbRatio):xSbIdx

[0571] yIdx=(cbWidth>cbHeight)? ySbIdx:(ySbIdx / cbRatio)

[0572] -The variable sType is derived as follows:

[0573] - If triangleDir is equal to 0, the following applies:

[0574] sType=(xIdx==yIdx)? 2((xIdx>yIdx)?0:1)

[0575] - Otherwise (triangleDir is equal to 1), the following applies:

[0576] sType=(xIdx+yIdx==minSb)? 2:((xIdx+yIdx <minSb)?0:1)

[0577] Where triangleDir specifies the division direction.

[0578] like Figure 30A-Figure 30C As shown, sType equals 0 and corresponds to the P1 area; sType equals 1 and corresponds to the P2 area; sType equals 2 and corresponds to the weighted area.

[0579] The motion information of the P1 region is represented as (Mv1, refIdx1); the motion information of the P2 region is represented as (Mv2, refIdx2).

[0580] -Depending on the value of sType, the following assignments are made:

[0581] - If sType is equal to 0, the motion information of the 4×4 sub-block is (Mv1, refIdx1).

[0582] Otherwise, if sType is equal to 1 or sType is equal to 2, and if both Mv1 and Mv2 are from the same reference list and the current block is in a slice with a backward reference picture (slice.getCheckLDC() is false), the motion information of the 4×4 sub-block is (Mv2, refIdx2).

[0583] Otherwise (sType is equal to 2), the following applies:

[0584] refIdxL0=(predListFlagA==0)? refIdx1:refIdx2

[0585] refIdxL1=(predListFlagA==0)? refIdx2:refIdx1

[0586] mvL0=(predListFlagA==0)? Mv1:Mv2

[0587] mvL1=(predListFlagA==0)? Mv2:Mv1

[0588] predListFlagA is the prediction list flag of the P1 region.

[0589] 5.2 Example #2

[0590] For each 4×4 sub-block at sub-block index (xSbIdx, ySbIdx), where xSbIdx = 0 ... numSbx-1, and ySbIdx = 0 ... numSbY-1, the following applies:

[0591] -Depending on the value of sType, the following assignments are made:

[0592] - If sType is equal to 0, the motion information of the 4×4 sub-block is (Mv1, refIdx1).

[0593] - Otherwise, if sType is equal to 1, the motion information of the 4×4 sub-block is (Mv2, refIdx2).

[0594] Otherwise (sType is equal to 2), the following applies:

[0595] refIdxL0=(predListFlagA==0)? refIdx1:refIdx2

[0596] refIdxL1=(predListFlagA==0)? refIdx2:refIdx1

[0597] mvL0=(predListFlagA==0)? Mv1:Mv2

[0598] mvL1=(predListFlagA==0)? Mv2:Mv1

[0599] predListFlagA is the prediction list flag of the P1 region.

[0600] 5.3 Example #3

[0601] For each 4×4 sub-block at sub-block index (xSbIdx, ySbIdx), where xSbIdx = 0 ... numSbx-1, and ySbIdx = 0 ... numSbY-1, the following applies:

[0602] -Depending on the value of sType, the following assignments are made:

[0603] - If sType is equal to 0, the motion information of the 4×4 sub-block is (Mv1, refIdx1).

[0604] Otherwise, if sType is equal to 1 or sType is equal to 2, and if both Mv1 and Mv2 are from the same reference list, the motion information of the 4×4 sub-block is (Mv2, refIdx2).

[0605] Otherwise (sType is equal to 2), the following applies:

[0606] - If Mv1 and Mv2 are from different reference lists, then

[0607] refIdxL0=(predListFlagA==0)? refIdx1:refIdx2

[0608] refIdxL1=(predListFlagA==0)? refIdx2:refIdx1

[0609] mvL0=(predListFlagA==0)? Mv1:Mv2

[0610] mvL1=(predListFlagA==0)? Mv2:Mv1

[0611] predListFlagA is the prediction list flag of the P1 region.

[0612] 5.4 Example #4

[0613] For each 4×4 sub-block at sub-block index (xSbIdx, ySbIdx), where xSbIdx = 0 ... numSbx-1, and ySbIdx = 0 ... numSbY-1, the following applies:

[0614] -Depending on the value of sType, the following assignments are made:

[0615] - If sType is equal to 0 or sType is equal to 2, and if both Mv1 and Mv2 are from the same reference list and the current block is in a slice with a backward reference picture (slice.getCheckLDC() is false), the motion information of the 4×4 sub-block is (Mv1, refIdx1).

[0616] - Otherwise, if sType is equal to 1, the motion information of the 4×4 sub-block is (Mv2, refIdx2).

[0617] Otherwise (sType is equal to 2), the following applies:

[0618] refIdxL0=(predListFlagA==0)? refIdx1:refIdx2

[0619] refIdxL1=(predListFlagA==0)? refIdx2:refIdx1

[0620] mvL0=(predListFlagA==0)? Mv1:Mv2

[0621] mvL1=(predListFlagA==0)? Mv2:Mv1

[0622] predListFlagA is the prediction list flag of the P1 region.

[0623] 5.5 Example #5

[0624] For each 4×4 sub-block at sub-block index (xSbIdx, ySbIdx), where xSbIdx = 0 ... numSbx-1, and ySbIdx = 0 ... numSbY-1, the following applies:

[0625] -Depending on the value of sType, the following assignments are made:

[0626] - If sType is equal to 0 or sType is equal to 2, and if both Mv1 and Mv2 are from the same reference list, the motion information of the 4×4 sub-block is (Mv1, refIdx1).

[0627] - Otherwise, if sType is equal to 1, the motion information of the 4×4 sub-block is (Mv2, refIdx2).

[0628] Otherwise (sType is equal to 2), the following applies:

[0629] - If Mv1 and Mv2 are from different reference lists, then

[0630] refIdxL0=(predListFlagA==0)? refIdx1:refIdx2

[0631] refIdxL1=(predListFlagA==0)? refIdx2:refIdx1

[0632] mvL0=(predListFlagA==0)? Mv1:Mv2

[0633] mvL1=(predListFlagA==0)? Mv2:Mv1

[0634] predListFlagA is the prediction list flag of the P1 region.

[0635] 5.6 Example #6

[0636] For each 4×4 sub-block at sub-block index (xSbIdx, ySbIdx), where xSbIdx = 0 ... numSbx-1, and ySbIdx = 0 ... numSbY-1, the following applies:

[0637] -Depending on the value of sType, the following assignments are made:

[0638] - If sType is equal to 0, the motion information of the 4×4 sub-block is (Mv1, refIdx1).

[0639] - Otherwise, if sType is equal to 1, the motion information of the 4×4 sub-block is (Mv2, refIdx2).

[0640] Otherwise (sType is equal to 2), the following applies:

[0641] - If Mv1 and Mv2 are from different reference lists, then

[0642] refIdxL0=(predListFlagA==0)? refIdx1:refIdx2

[0643] refIdxL1=(predListFlagA==0)? refIdx2:refIdx1

[0644] mvL0 = (predListFlagA == 0)? Mv1 : Mv2

[0645] mvL1 = (predListFlagA == 0)? Mv2 : Mv1

[0646] predListFlagA is the prediction list flag for the P1 region.

[0647] - Otherwise, the motion information for the 4x4 sub-block is ((Mv1 + Mv2) / 2, refIdx1)

[0648] 5.7 Example 7

[0649] For each 4x4 sub-block at sub-block index (xSbIdx, ySbIdx), where xSbIdx = 0...numSbx - 1 and ySbIdx = 0...numSbY - 1, the following applies:

[0650] - Depending on the value of sType, the following assignments are made:

[0651] - If sType is equal to 0, the motion information for the 4x4 sub-block is (Mv1, refIdx1).

[0652] - Otherwise, if sType is equal to 1, the motion information for the 4x4 sub-block is (Mv2, refIdx2).

[0653] - Otherwise (sType is equal to 2), the motion information for the 4x4 sub-block is the motion information of the original Merge candidate corresponding to merge_triangle_idx0.

[0654] 5.8 Example #8

[0655] For each 4x4 sub-block at sub-block index (xSbIdx, ySbIdx), where xSbIdx = 0...numSbx - 1, ySbIdx = 0...numSbY - 1, the following applies:

[0656] - Depending on the value of sType, the following assignments are made:

[0657] - If sType is equal to 0 or sType is equal to 2, and if both Mv1 and Mv2 are from the same reference list, and triangleDir is equal to 0 and ySbIdx < numSbY - 1, the motion information for the 4x4 sub-block is (Mv1, refIdx1).

[0658] Otherwise, if sType is equal to 1 or sType is equal to 2, and if both Mv1 and Mv2 are from the same reference list, and triangleDir is equal to 0 and ySbIdx=numSbY-1 or triangleDir is equal to 1, the motion information of the 4×4 sub-block is (Mv2, refIdx2).

[0659] Otherwise (sType is equal to 2), the following applies:

[0660] - If Mv1 and Mv2 are from different reference lists

[0661] refIdxL0=(predListFlagA==0)? refIdx1:refIdx2

[0662] refIdxL1=(predListFlagA==0)? refIdx2:refIdx1

[0663] mvL0=(predListFlagA==0)? Mv1:Mv2

[0664] mvL1=(predListFlagA==0)? Mv2:Mv1

[0665] predListFlagA is the prediction list flag of the P1 region.

[0666] 5.9 Example #9

[0667] For each 4×4 sub-block at sub-block index (xSbIdx, ySbIdx), where xSbIdx = 0 ... numSbx-1, and ySbIdx = 0 ... numSbY-1, the following applies:

[0668] - If both Mv1 and Mv2 are from the same reference list, and the current block is in a slice with a backward reference picture (slice.getCheckLDC() is false), the motion information of the 4x4 sub-block is (Mv2, refIdx2).

[0669] - Otherwise, the following applies:

[0670] refIdxL0=(predListFlagA==0)? refIdx1:refIdx2

[0671] refIdxL1=(predListFlagA==0)? refIdx2:refIdx1

[0672] mvL0=(predListFlagA==0)? Mv1:Mv2

[0673] mvL1=(predListFlagA==0)? Mv2:Mv1

[0674] predListFlagA is the prediction list flag of the P1 region.

[0675] 5.10 Example #10

[0676] For each 4×4 sub-block at sub-block index (xSbIdx, ySbIdx), where xSbIdx = 0 ... numSbx-1, and ySbIdx = 0 ... numSbY-1, the following applies:

[0677] refIdxL0=(predListFlagA==0)? refIdx1:refIdx2

[0678] refIdxL1=(predListFlagA==0)? refIdx2:refIdx1

[0679] mvL0=(predListFlagA==0)? Mv1:Mv2

[0680] mvL1=(predListFlagA==0)? Mv2:Mv1

[0681] predListFlagA is the prediction list flag of the P1 region.

[0682] 5.11 Example #11

[0683] For each 4×4 sub-block at sub-block index (xSbIdx, ySbIdx), where xSbIdx = 0 ... numSbx-1, and ySbIdx = 0 ... numSbY-1, the following applies:

[0684] - If both Mv1 and Mv2 are from the same reference list, the motion information of the 4×4 sub-block is (Mv2, refIdx2).

[0685] - Otherwise, the following applies:

[0686] refIdxL0=(predListFlagA==0)? refIdx1:refIdx2

[0687] refIdxL1=(predListFlagA==0)? refIdx2:refIdx1

[0688] mvL0=(predListFlagA==0)? Mv1:Mv2

[0689] mvL1=(predListFlagA==0)? Mv2:Mv1

[0690] predListFlagA is the prediction list flag of the P1 region.

[0691] 5.12 Example #12

[0692] For each 4×4 sub-block at sub-block index (xSbIdx, ySbIdx), where xSbIdx = 0 ... numSbx-1, and ySbIdx = 0 ... numSbY-1, the following applies:

[0693] - If both Mv1 and Mv2 are from the same reference list, and the current block is in a slice with a backward reference picture (slice.getCheckLDC() is false), the motion information of the 4x4 sub-block is (Mv1, refIdx1).

[0694] - Otherwise, the following applies:

[0695] refIdxL0=(predListFlagA==0)? refIdx1:refIdx2

[0696] refIdxL1=(predListFlagA==0)? refIdx2:refIdx1

[0697] mvL0=(predListFlagA==0)? Mv1:Mv2

[0698] mvL1=(predListFlagA==0)? Mv2:Mv1

[0699] predListFlagA is the prediction list flag of the P1 region.

[0700] 5.13 Example #13

[0701] For each 4×4 sub-block at sub-block index (xSbIdx, ySbIdx), where xSbIdx = 0 ... numSbx-1, and ySbIdx = 0 ... numSbY-1, the following applies:

[0702] - If both Mv1 and Mv2 are from the same reference list, the motion information of the 4×4 sub-block is (Mv1, refIdx1).

[0703] - Otherwise, the following applies:

[0704] - If Mv1 and Mv2 are from different reference lists, then

[0705] refIdxL0=(predListFlagA==0)? refIdx1:refIdx2

[0706] refIdxL1=(predListFlagA==0)? refIdx2:refIdx1

[0707] mvL0=(predListFlagA==0)? Mv1:Mv2

[0708] mvL1=(predListFlagA==0)? Mv2:Mv1

[0709] predListFlagA is the prediction list flag of the P1 region.

[0710] 5.14 Example #14

[0711] For each 4×4 sub-block at sub-block index (xSbIdx, ySbIdx), where xSbIdx = 0 ... numSbx-1, and ySbIdx = 0 ... numSbY-1, the following applies:

[0712] - If Mv1 and Mv2 are from different reference lists, then

[0713] refIdxL0=(predListFlagA==0)? refIdx1:refIdx2

[0714] refIdxL1=(predListFlagA==0)? refIdx2:refIdx1

[0715] mvL0=(predListFlagA==0)? Mv1:Mv2

[0716] mvL1=(predListFlagA==0)? Mv2:Mv1

[0717] predListFlagA is the prediction list flag of the P1 region.

[0718] Otherwise, the motion information of the 4×4 sub-block is ((Mv1+Mv2) / 2, refIdx1).

[0719] 5.15 Example #15

[0720] For each 4×4 sub-block at sub-block index (xSbIdx, ySbIdx), where xSbIdx = 0…numSbx-1 and ySbIdx = 0…numSbY-1, the motion information of the 4×4 sub-block is the motion information of the original Merge candidate corresponding to merge_triangle_idx0.

[0721] 5.16 Example #16

[0722] For each 4×4 sub-block at sub-block index (xSbIdx, ySbIdx), where xSbIdx = 0…numSbx-1 and ySbIdx = 0…numSbY-1, the following applies:

[0723] - If both Mv1 and Mv2 are from the same reference list, and triangleDir is equal to 0 and ySbIdx < numSbY-1, then the motion information of the 4×4 sub-block is (Mv1, refIdx1).

[0724] - Otherwise, if both Mv1 and Mv2 are from the same reference list, and triangleDir is equal to 0 and ySbIdx = numSbY-1 or triangleDir is equal to 1, then the motion information of the 4×4 sub-block is (Mv2, refIdx2).

[0725] - Otherwise, the following applies:

[0726] refIdxL0 = (predListFlagA == 0)? refIdx1:refIdx2

[0727] refIdxL1 = (predListFlagA == 0)? refIdx2:refIdx1

[0728] mvL0 = (predListFlagA == 0)? Mv1:Mv2

[0729] mvL1 = (predListFlagA == 0)? Mv2:Mv1

[0730] predListFlagA is the prediction list flag of the P1 region.

[0731] Fig.31 is a flowchart of method 3100 for video processing. Method 3100 includes, at operation 3102, making a decision on the order of inserting motion candidates into the motion candidate list for the conversion between the current block for the video and the bitstream representation of the video, based on priority rules, where the current block is encoded using a geometric partitioning mode.

[0732] The method 3100 includes, at operation 3104, performing a conversion based on the determination and the motion candidate list.

[0733] Fig.32 32 is a flow chart of a method 3200 for video processing. The method 3200 includes, at operation 3202, inserting one or more average unidirectional prediction motion candidates from list 0 and list 1 derived from a regular motion candidate into a motion candidate list for conversion between a current block of video and a bitstream representation of the video, wherein the current block is encoded using a geometric partitioning mode.

[0734] The method 3200 includes, at operation 3204, performing a conversion based on the motion candidate list.

[0735] Fig.33 33 is a flow chart of a method 3300 for video processing. The method 3300 includes, at operation 3302, inserting one or more average motion candidates with unidirectional prediction derived from regular motion candidates with unidirectional prediction into a motion candidate list for conversion between a current block of video and a bitstream representation of the video, wherein the current block is encoded using a geometric partitioning mode.

[0736] The method 3300 includes, at operation 3304, performing a conversion based on the motion candidate list.

[0737] Fig.34 34 is a flow chart of a method 3400 for video processing. The method 3400 includes, at operation 3402, inserting one or more virtual motion candidates derived from available motion candidates into a motion candidate list for conversion between a current block of video and a bitstream representation of the video, wherein the current block is encoded using a geometric partitioning mode.

[0738] The method 3400 includes, at operation 3404, performing a conversion based on the motion candidate list.

[0739] Fig.35 35 is a flow chart of a method 3500 for video processing. The method 3500 includes, at operation 3502, making a decision about an order in which motion candidates are inserted into a motion candidate list for conversion between a current block of a video encoded using a geometric partitioning mode and a bitstream representation of the video based on a priority rule, wherein the priority rule is based on a position of the motion candidate in the motion candidate from which it is derived or one or more reference picture lists associated with the motion candidate.

[0740] The method 3500 includes, at operation 3504, performing a conversion based on the determination and the motion candidate list.

[0741] Fig.3636 is a flow chart of a method 3600 for video processing. The method 3600 includes, at operation 3602, making a decision about an order of inserting motion candidates into a motion candidate list for conversion between a current block of a video and a bitstream representation of the video based on a priority rule, wherein the current block is encoded using a geometric partitioning mode, wherein the insertion order is variable between sequence to sequence, picture to picture, slice to slice, slice group to slice group, or from the current block to a subsequent block of the video.

[0742] The method 3600 includes, at operation 3604, performing a conversion based on the determination and the motion candidate list.

[0743] Fig.37 3700 is a flow chart of a method 3700 for video processing. The method 3700 includes, at operation 3702, partitioning a current block of a video encoded using a geometric partitioning mode into a plurality of partitions.

[0744] The method 3700 includes, at operation 3704, constructing a plurality of motion candidate lists, each motion candidate list corresponding to each partition of the plurality of partitions.

[0745] The method 3700 includes, at operation 3706, performing conversion between the current block and a bitstream representation of the video based on the plurality of motion candidate lists.

[0746] Fig.38 38 is a flow chart of a method 3800 for video processing. The method 3800 includes, at operation 3802, partitioning a current block of a video encoded using a geometric partitioning mode into a plurality of partitions.

[0747] The method 3800 includes, at operation 3804 , constructing a plurality of motion candidate lists, wherein each partition of the plurality of partitions is configured to select a same motion candidate list of the plurality of motion candidate lists.

[0748] The method 3800 includes, at operation 3806, performing conversion between the current block and a bitstream representation of the video based on the selected same motion candidate list.

[0749] Fig.39 3900 is a flow chart of a method for video processing. The method 3900 includes, at operation 3902, performing a conversion between a current block of video in a video unit and a bitstream representation of the video, wherein the bitstream representation includes a field indicating a maximum number of motion candidates allowed in a motion candidate list for a geometric partitioning mode enabled in the video region.

[0750] Fig.404000 is a flow chart of a method 4000 for video processing. The method 4000 includes, at operation 4002, for a current block of a video partitioned into a plurality of partitions using a geometric partitioning pattern, selecting a weighting factor group from a plurality of weighting factor groups, wherein the weighting factor group is selected based on at least a width or a height of the current block.

[0751] The method 4000 includes, at operation 4004, applying a set of weighting factors to samples along a common boundary of at least two of the plurality of partitions as part of a conversion between a current block and a bitstream representation of a video.

[0752] Fig.41 4100 is a flowchart of a method 4100 for video processing. The method 4100 includes, at operation 4102, determining at least one weighting factor based on an angle of a common boundary of at least two of the plurality of partitions for a sample within a current block of a video partitioned into a plurality of partitions using a geometric partitioning mode.

[0753] The method 4100 includes, at operation 4104, performing a conversion between the current block and a bitstream representation of the video based on at least one weighting factor.

[0754] Fig.42 4 is a flow chart of a method 4200 for video processing. The method 4200 includes, at operation 4202, for a current block of a video partitioned into a plurality of partitions using a geometric partitioning mode, performing a motion compensation process on samples having a sample size different from a sample size of 4×4 along a common boundary of at least two of the plurality of partitions.

[0755] The method 4200 includes, at operation 4204, performing a conversion between the current block and a bitstream representation of the video based on a motion compensation process.

[0756] Fig.43 is a flow chart of a method 4300 for video processing. The method 4300 includes, at operation 4302, partitioning a current block of a video into a plurality of partitions using a geometric partitioning mode.

[0757] The method 4300 includes, at operation 4304, storing a single set of motion information for a K×L region within the current block derived from motion information associated with a plurality of partitions.

[0758] The method 4300 includes, at operation 4306, performing conversion between the current block and a bitstream representation of the video using at least one motion information set associated with at least one partition of the plurality of partitions.

[0759] Fig.44 44 is a flow chart of a method 4400 for video processing. The method 4400 includes, at operation 4402, partitioning a current block of a video into a plurality of partitions.

[0760] Method 4400 includes, at operation 4404, performing a conversion between a current block encoded using a geometric partitioning mode and a bitstream representation of a video, wherein a single motion information set is stored for each M×N block unit of the current block, wherein the single motion information set is derived from motion information associated with each of a plurality of partitions, wherein M or N is not equal to 4, and wherein each sample in the M×N block unit shares the same motion information.

[0761] Fig.45 45 is a flow chart of a method 4500 for video processing. The method 4500 includes, at operation 4502, partitioning a current block of a video into a plurality of partitions.

[0762] Method 4500 includes, at operation 4504, performing a motion compensation process on a first sub-region of the current block based on first motion information different from second motion information associated with the first sub-region to be stored as part of conversion between the current block and a bitstream representation of the video.

[0763] Fig.46 is a flow chart of a method 4600 for video processing. The method 4600 includes, at operation 4602, partitioning a current block of a video into a plurality of partitions using a geometric partitioning mode.

[0764] The method 4600 includes, at operation 4604, storing motion information of a first sample point set and a second sample point set based on a partition direction or a decoded Merge index or a Merge candidate list associated with a plurality of partitions, wherein the first sample point set is located on a common boundary of at least two partitions among the plurality of partitions, and the second sample point set is located inside one of at least two partitions among the plurality of partitions.

[0765] The method 4600 includes, at operation 4606, performing a conversion between a current block and a bitstream representation of the video or a conversion between a subsequent block and a bitstream representation of the video based on the stored motion information.

[0766] Fig.47 47 is a flow chart of a method 4700 for video processing. The method 4700 includes, at operation 4702, partitioning a current block of a video into a plurality of partitions.

[0767] The method 4700 includes, at operation 4704, storing virtual bi-directional prediction motion information based on motion information of at least two partitions among the plurality of partitions.

[0768] The method 4700 includes, at operation 4706, performing conversion between the current block and a bitstream representation of the video based on the virtual bi-predictive motion information.

[0769] Fig.4848 is a flow chart of a method 4800 for video processing. The method 4800 includes, at operation 4802, partitioning a current block of a video into a plurality of partitions.

[0770] Method 4800 includes, in operation 4804, storing motion information (MvInfo1) of a first partition among multiple partitions and motion information (MvInfo2) of a second partition among multiple partitions based on a low latency check flag, wherein the low latency check flag indicates all reference pictures whose picture order count (POC) values ​​are not greater than the POC value of the current picture including the current block.

[0771] The method 4800 includes, at operation 4806, performing a conversion between the current block and a bitstream representation of the video based on the stored motion information.

[0772] In some embodiments, the following technical solutions may be implemented:

[0773] A1. A method for video processing, comprising: making a decision based on a priority rule about the order in which motion candidates are inserted into a motion candidate list for conversion between a current block of a video and a bitstream representation of the video, wherein the current block is encoded using a geometric partitioning pattern; and performing the conversion based on the decision and the motion candidate list.

[0774] A2. The method according to scheme A1, wherein the priority rule is based on the prediction direction of at least one motion candidate.

[0775] A3. The method of scheme A2, wherein the order of insertion includes inserting normal motion candidates with uni-directional prediction from list X before inserting any normal motion candidates with uni-directional prediction from list Y.

[0776] A4. The method of scheme A2, wherein the order of insertion includes inserting the motion candidates with bi-directional prediction before inserting any motion candidates with uni-directional prediction.

[0777] A5. A method according to scheme A2, wherein the order of insertion is defined as: motion candidates from list X derived from normal motion candidates with bidirectional prediction, followed by motion candidates from list X derived from normal motion candidates with bidirectional prediction, followed by normal motion candidates with unidirectional prediction, where X=0 or X=1.

[0778] A6. A method according to scheme A2, wherein the order of insertion includes inserting a motion candidate derived from a first regular motion candidate with bidirectional prediction before inserting a motion candidate derived from a second regular motion candidate with bidirectional prediction, and wherein the order of insertion of the current block is the same as the order of insertion of video blocks encoded using a non-geometric partitioning mode.

[0779] A7. The method according to scheme A2, wherein the order of insertion includes inserting the motion candidates derived from the first normal motion candidate with bidirectional prediction and the motion candidates derived from the second normal motion candidate with bidirectional prediction in an interleaved manner.

[0780] A8. A method according to scheme A7, wherein the interleaving manner includes inserting a truncated list 0 prediction candidate derived from CA, a truncated list 1 prediction candidate derived from CB, a truncated list 1 prediction candidate derived from CA, and a truncated list 0 prediction candidate derived from CB, wherein CA and CB are two regular motion candidates.

[0781] A9. A method according to scheme A7, wherein the interleaving manner includes inserting a truncated list 1 prediction candidate derived from CA, a truncated list 0 prediction candidate derived from CB, a truncated list 0 prediction candidate derived from CA, and a truncated list 1 prediction candidate derived from CB, wherein CA and CB are two regular motion candidates.

[0782] A10. The method of any one of schemes A1 to A9, wherein the conventional motion candidate is a motion candidate derived for a video block encoded using a non-geometric partitioning mode.

[0783] A11. A method according to scheme A1, wherein the priority rule is based on coding mode information associated with normal motion candidates.

[0784] A12. The method according to scheme A11, wherein the encoding mode information includes encoding using a Merge mode or an Advanced Motion Vector Prediction (AMVP) mode.

[0785] A13. A method according to scheme A12, wherein the order of insertion includes inserting motion candidates derived from blocks encoded in AMVP mode before inserting motion candidates derived from blocks encoded in Merge mode.

[0786] A14. A method according to scheme A12, wherein the order of insertion includes inserting the motion candidates derived from the blocks encoded in Merge mode before inserting the motion candidates derived from the blocks encoded in AMVP mode.

[0787] A15. A method according to scheme A11, wherein the coding mode information includes a reference index or a picture order count (POC) difference.

[0788] A16. A method according to scheme A15, wherein the order of insertion includes inserting the motion candidates associated with the first reference index before inserting the motion candidates associated with the second reference index greater than the first reference index.

[0789] A17. A method according to scheme A15, wherein the order of inserting includes inserting the motion candidates associated with the first POC difference before inserting the motion candidates associated with the reference index of the second POC difference greater than the first POC difference.

[0790] A18. The method according to scheme A11, wherein the coding mode information includes a quantization parameter (QP) or a temporal layer index of a reference picture associated with a picture, a slice, or a slice group including the current block.

[0791] A19. The method of scheme A18, wherein the order of inserting includes inserting the motion candidates associated with the first QP before inserting the motion candidates associated with a second QP greater than the first QP.

[0792] A20. A method according to scheme A1, wherein the order of insertion includes inserting all truncated list 1 prediction candidates before any truncated list 0 prediction candidates.

[0793] A21. A method according to scheme A1, wherein the order of inserting includes inserting one or more truncated list 0 prediction candidates and one or more truncated list 1 prediction candidates based on available geometric motion candidates.

[0794] A22. The method of scheme A1, wherein the order of inserting comprises inserting a first motion candidate with bi-directional prediction based on available geometric motion candidates before inserting a second motion candidate with bi-directional prediction.

[0795] A23. A method for video processing, comprising: inserting one or more average unidirectional prediction motion candidates from list 0 and list 1 derived from a regular motion candidate into a motion candidate list for conversion between a current block of video and a bitstream representation of the video, wherein the current block is encoded using a geometric partitioning pattern; and performing conversion based on the motion candidate list.

[0796] A24. The method according to scheme A23 also includes: selecting a single average uni-directional prediction motion candidate from the one or more average uni-directional prediction motion candidates to insert into the motion candidate list.

[0797] A25. A method according to scheme A24, wherein the selection is based on the available geometric motion candidates in the motion candidate list.

[0798] A26. A method according to scheme A24, wherein the selection is based on the reference picture indexes of list 0 and list 1.

[0799] A27. A method according to scheme A24, wherein the selection is based on a picture order count (POC) distance between the reference pictures of list 0 and list 1 and the current picture.

[0800] A28. The method of scheme A23, wherein all average uni-directionally predicted motion candidates from list 0 are inserted before inserting any average uni-directionally predicted motion candidates from list 1.

[0801] A29. A method according to scheme A23, wherein all average uni-directional predicted motion candidates from list 1 are inserted before inserting any average uni-directional predicted motion candidates from list 0.

[0802] A30. A method for video processing, comprising: inserting one or more average motion candidates with unidirectional prediction, which are derived from regular motion candidates with unidirectional prediction, into a motion candidate list for conversion between a current block of a video and a bitstream representation of the video, wherein the one or more average motion candidates with unidirectional prediction are derived from regular motion candidates with unidirectional prediction, wherein the current block is encoded using a geometric partitioning mode; and performing conversion based on the motion candidate list.

[0803] A31. A method according to scheme A30, wherein the average motion candidate with unidirectional prediction from list LX is derived from the normal motion candidate with unidirectional prediction from list LX, where X=0 or X=1.

[0804] A32. A method according to scheme A30, wherein an average motion candidate with unidirectional prediction from list LY is derived from scaled normal motion candidates, wherein the normal motion candidates from list LX are scaled to list LY to generate scaled normal motion candidates, wherein X=0 or X=1, and wherein Y=1-X.

[0805] A33. A method for video processing, comprising: inserting one or more virtual motion candidates derived from available motion candidates into a motion candidate list for conversion between a current block of a video and a bitstream representation of the video, wherein the current block is encoded using a geometric partitioning pattern; and performing conversion based on the motion candidate list.

[0806] A34. A method according to scheme A33, wherein the one or more virtual motion candidates are derived by first generating a truncated list of motion candidates in the motion candidate list.

[0807] A35. A method according to scheme A33, wherein one or more virtual motion candidates are derived based only on motion candidates with unidirectional prediction.

[0808] A36. A method according to scheme A33, wherein one or more virtual motion candidates are derived based only on the truncated list of motion candidates with unidirectional prediction and motion candidates predicted from list 1.

[0809] A37. A method according to scheme A33, wherein one or more virtual motion candidates are derived based on motion candidates having a predetermined reference picture index or having a picture order count (POC) distance within a specific range.

[0810] A38. A method according to scheme A33, wherein one or more virtual motion candidates are derived without applying a pruning operation.

[0811] A39. A method according to any one of schemes A1 to A38, wherein the current block is divided into multiple partitions based on a partitioning mode.

[0812] A40. A method according to any one of schemes A1 to A39, wherein at least one partition of the plurality of partitions is non-square and non-rectangular.

[0813] A41. A method according to any one of schemes A1 to A39, wherein the geometric segmentation pattern includes a triangular segmentation pattern.

[0814] A42. A method according to any one of schemes A1 to A41, wherein the conversion generates the current block from a bitstream representation.

[0815] A43. A method according to any one of schemes A1 to A41, wherein the conversion generates a bitstream representation from the current block.

[0816] A44. An apparatus in a video system, comprising a processor and a non-transitory memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to implement any one of the methods of embodiments A1 to A43.

[0817] A45. A computer program product stored on a non-transitory computer-readable medium, the computer program product comprising a program code for executing the method of any one of schemes A1 to A43.

[0818] In some embodiments, the following technical solutions may be implemented:

[0819] B1. A method for video processing, comprising: making a decision about the order in which motion candidates are inserted into a motion candidate list for conversion between a current block of a video encoded using a geometric partitioning mode and a bitstream representation of the video based on a priority rule, wherein the priority rule is based on a position of the motion candidate in the motion candidate or one or more reference picture lists associated with the motion candidate from which the motion candidate is derived; and performing the conversion based on the decision and the motion candidate list.

[0820] B2. A method according to scheme B1, wherein when it is determined that the first motion candidate CA is derived from block A, the second motion candidate CB is derived from block B, and block B is checked after block A, the order of insertion includes inserting the motion candidate derived from CB after inserting the motion candidate derived from CA into the motion candidate list.

[0821] B3. The method according to scheme B2 further includes: checking block B after block A during the construction of the regular motion candidate list.

[0822] B4. According to the method of scheme B2, the order of insertion also includes inserting the motion candidate derived from CB with prediction from list X after inserting the motion candidate derived from CA with prediction from list X, where X=0 or X=1.

[0823] B5. A method according to scheme B2, wherein the order of insertion also includes inserting a motion candidate derived from CB with a prediction from list Y after inserting a motion candidate derived from CA with a prediction from list Y, where X=0 or X=1, and where Y=(1-X).

[0824] B6. A method according to scheme B2, wherein the order of insertion further includes inserting a motion candidate derived from CB with a prediction from list X after inserting a motion candidate derived from CA with a prediction from list Y, where X=0 or X=1, and where Y=(1-X).

[0825] B7. A method according to any one of schemes B1 to B6, wherein the current block is divided into multiple partitions based on a partitioning mode.

[0826] B8. A method according to any one of schemes B1 to B6, wherein the geometric segmentation pattern includes a triangular segmentation pattern.

[0827] B9. A method according to any one of schemes B1 to B6, wherein the current block is divided into multiple partitions, wherein at least one partition is non-square and non-rectangular.

[0828] B10. A method according to any one of schemes B1 to B9, wherein the conversion generates the current block from a bitstream representation.

[0829] B11. A method according to any one of schemes B1 to B9, wherein the conversion generates a bitstream representation from the current block.

[0830] B12. An apparatus in a video system, comprising a processor and a non-transitory memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to implement any one of the methods of embodiments B1 to B11.

[0831] B13. A computer program product stored on a non-transitory computer-readable medium, the computer program product comprising a program code for executing the method of any one of schemes B1 to B11.

[0832] In some embodiments, the following technical solutions may be implemented:

[0833] C1. A method of video processing, comprising: making a decision based on a priority rule about the order in which motion candidates are inserted into a motion candidate list for conversion between a current block of a video and a bitstream representation of the video, wherein the current block is encoded using a geometric partitioning pattern, wherein the order of insertion is variable either from sequence to sequence, picture to picture, slice to slice, slice group to slice group, or from the current block to a subsequent block of the video; and performing the conversion based on the decision and the motion candidate list.

[0834] C2. The method according to scheme C1, wherein the order of insertion is based on the current block or on a sequence, picture, slice or slice group including the current block.

[0835] C3. A method according to scheme C2, wherein the priority rule is based on at least one of the size, shape or partitioning mode of the current block.

[0836] C4. A method according to any one of schemes C1 to C3, wherein the order of insertion is signaled from the encoder to the decoder in the bitstream representation in the form of a sequence parameter set (SPS), a video parameter set (VPS), a picture parameter set (PPS), a slice header, a slice group header, a slice, a coding tree unit (CTU) or a coding unit (CU).

[0837] C5. A method for video processing, comprising: partitioning a current block of a video encoded using a geometric partitioning mode into multiple partitions; constructing multiple motion candidate lists, each motion candidate list corresponding to each partition in the multiple partitions; and performing conversion between the current block and the bitstream representation of the video based on the multiple motion candidate lists.

[0838] C6. A method according to scheme C5, wherein a first motion candidate list among multiple motion candidate lists of a first partition among multiple partitions includes only motion candidates predicted from list X, and wherein a second motion candidate list among multiple motion candidate lists of a second partition among multiple partitions includes only motion candidates predicted from list Y, where X=0 or X=1, and where Y=(1-X).

[0839] C7. The method according to scheme C5, wherein one or more motion candidates inserted into the plurality of motion candidate lists are based on motion information associated with different spatial or temporal blocks.

[0840] C8. A method according to scheme C7, wherein the positions of different spatial or temporal blocks used to derive one or more motion candidates for a first partition among multiple partitions are based on the positions of the first partition or the second partition among the multiple partitions in the current block.

[0841] C9. A method according to scheme C7, wherein the positions of different spatial or temporal blocks used to derive one or more motion candidates for a first partition or a second partition in multiple partitions are based on a partitioning mode of the current block.

[0842] C10. A method according to scheme C9, wherein the division pattern includes a division from the upper right corner to the lower left corner or from the upper left corner to the lower right corner, represented by a 45-degree division pattern or a 135-degree division pattern, respectively.

[0843] C11. A method according to scheme C10, wherein the partitioning mode includes a 135 degree partitioning mode, and wherein the method further includes: checking whether more upper blocks have a partition including an upper right corner of the current block.

[0844] C12. The method according to scheme C10, wherein the partitioning mode includes a 135 degree partitioning mode, and wherein the method further includes: checking whether more left blocks have a partition including a lower left corner of the current block.

[0845] C13. The method according to scheme C5, wherein an index of each motion candidate list in a plurality of motion candidate lists corresponding to a plurality of partitions is signaled.

[0846] C14. A method according to scheme C5, wherein a bitstream representation including an index of each motion candidate list in a plurality of motion candidate lists is parsed for the conversion.

[0847] C15. A method for video processing, comprising: partitioning a current block of a video encoded using a geometric partitioning mode into multiple partitions; constructing multiple motion candidate lists, wherein each of the multiple partitions is configured to select the same motion candidate list from the multiple motion candidate lists; and performing conversion between the current block and the bitstream representation of the video based on the selected same motion candidate list.

[0848] C16. A method according to scheme C15, wherein the bitstream representation includes an index of a first motion candidate list among multiple motion candidate lists.

[0849] C17. The method according to scheme C16, wherein the bitstream representation further comprises an index of a motion candidate from the first motion candidate list, and wherein the index of the motion candidate is signaled after the index of the first motion candidate list.

[0850] C18. A method according to scheme C16, wherein the bitstream representation further comprises an index of a motion candidate from the first motion candidate list following the index of the first motion candidate list, and wherein the bitstream representation is parsed for the conversion.

[0851] C19. A method according to scheme C15, wherein the index of the first motion candidate list is jointly encoded with the index of the motion candidate from the first motion candidate list in the bitstream representation.

[0852] C20. The method according to scheme C15, wherein a bitstream representation comprising an index of the first motion candidate list jointly encoded with an index of a motion candidate from the first motion candidate list is parsed for the conversion.

[0853] C21. A method according to any one of schemes C1 to C20, wherein the current block is divided into multiple partitions based on at least a partitioning mode.

[0854] C22. A method according to any one of schemes C1 to C21, wherein the geometric segmentation pattern includes a triangular segmentation pattern.

[0855] C23. A method according to any one of schemes C1 to C21, wherein the current block is divided into a plurality of partitions, wherein at least one partition is non-square and non-rectangular.

[0856] C24. A method according to any one of schemes C1 to C23, wherein the conversion generates the current block from a bitstream representation.

[0857] C25. A method according to any one of schemes C1 to C23, wherein the conversion generates a bitstream representation from the current block.

[0858] C26. An apparatus in a video system, comprising a processor and a non-transitory memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to implement any one of the methods of embodiments C1 to C25.

[0859] C27. A computer program product stored on a non-transitory computer-readable medium, the computer program product comprising a program code for executing the method of any one of schemes C1 to C25.

[0860] In some embodiments, the following technical solutions may be implemented:

[0861] D1. A method of video processing, comprising: performing a conversion between a current block of video in a video unit and a bitstream representation of the video, wherein the bitstream representation includes a field indicating a maximum number of motion candidates allowed in a motion candidate list for a geometric partitioning mode enabled in a video region.

[0862] D2. A method according to scheme D1, wherein the field comprises explicit signaling indicating the maximum number of motion candidates allowed in the bitstream representation.

[0863] D3. A method according to scheme D1, wherein the field implicitly indicates that the maximum number of allowed motion candidates is equal to the maximum number of allowed motion candidates in the motion candidate list of another block encoded using non-geometric partitioning mode.

[0864] D4. A method according to any one of schemes D1 to D3, wherein the current block is further encoded using Merge mode or Advanced Motion Vector Prediction (AMVP) mode.

[0865] D5. A method according to any one of schemes D1 to D3, wherein the video unit includes a sequence, a video, a picture, a slice, a slice group, a largest coding unit (LCU) row or a largest coding unit (LCU) group.

[0866] D6. A method according to any one of schemes D1 to D3, wherein an indication of the maximum number of allowed motion candidates is provided in a signaling notification field in a sequence parameter set (SPS), a video parameter set (VPS), a picture parameter set (PPS), a picture header, a slice header, a slice group header, a maximum coding unit (LCU) row or a maximum coding unit (LCU) group.

[0867] D7. A method according to any one of schemes D1 to D6, wherein the indication of the maximum number of allowed motion candidates in the field is binarized using unary coding or truncated unary coding.

[0868] D8. A method according to scheme D2, wherein explicitly signaling the indication of the maximum number of allowed motion candidates in the field comprises signaling the difference between M and the maximum number of allowed motion candidates, where M is an integer.

[0869] D9. A method according to scheme D8, wherein the maximum number of allowed motion candidates for a block encoded using geometric partitioning mode is set to M minus the resolved indicator.

[0870] D10. The method according to scheme D8 or D9, wherein M=5 or M=6.

[0871] D11. A method according to any of schemes D1 to D8, wherein a bitstream representation including an indication of a maximum number of allowed motion candidates in a field is parsed for the conversion.

[0872] D12. A method according to scheme D1, wherein the bitstream representation includes a bit flag indicating that the maximum number of allowed motion candidates for a video block encoded using a geometric partitioning mode is the same as the maximum number of allowed regular motion candidates or sub-block motion candidates.

[0873] D13. A method according to scheme D12, wherein a bit stream representation including a bit flag is parsed for the conversion.

[0874] D14. A method according to any one of schemes D1 to D13, wherein the current block is divided into multiple partitions according to a partitioning mode.

[0875] D15. A method according to any one of schemes D1 to D14, wherein the geometric segmentation pattern includes a triangular segmentation pattern.

[0876] D16. A method according to any one of schemes D1 to D14, wherein the current block is divided into a plurality of partitions, at least one of the plurality of partitions being non-square and non-rectangular.

[0877] D17. A method according to any one of schemes D1 to D16, wherein the conversion generates the current block from a bitstream representation.

[0878] D18. A method according to any one of schemes D1 to D16, wherein the conversion generates a bitstream representation from the current block.

[0879] D19. An apparatus in a video system, comprising a processor and a non-transitory memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to implement any one of the methods of implementations D1 to D18.

[0880] D20. A computer program product stored on a non-transitory computer-readable medium, the computer program product comprising a program code for executing the method of any one of schemes D1 to D18.

[0881] In some embodiments, the following technical solutions may be implemented:

[0882] E1. A method of video processing, comprising: for a current block of a video partitioned into a plurality of partitions using a geometric partitioning pattern, selecting a weighting factor group from a plurality of weighting factor groups, wherein the weighting factor group is selected based at least on a width or a height of the current block; and as part of a conversion between the current block and a bitstream representation of the video, applying the weighting factor group to samples along a common boundary of at least two of the plurality of partitions.

[0883] E2. A method according to scheme E1, wherein the selection is based on a ratio between the width and the height being greater than a threshold.

[0884] E3. A method according to scheme E1, wherein the selection is based on a ratio between the height and the width being greater than a threshold.

[0885] E4. A method according to scheme E1, wherein the plurality of weighting factor groups are predefined based on a width or a height of the current block.

[0886] E5. A method according to scheme E4, wherein one or two weighting factor groups are selected from a plurality of weighting factor groups.

[0887] E6. A method for video processing, comprising: determining at least one weighting factor for a sample within a current block of a video partitioned into a plurality of partitions using a geometric partitioning pattern based on an angle of a common boundary of at least two of the plurality of partitions; and performing conversion between a bitstream representation of the current block and the video based on the at least one weighting factor.

[0888] E7. The method of scheme E6, wherein the determination is also based on the location of the sample point.

[0889] E8. A method according to any one of schemes E1 to E7, wherein the final prediction value of the sample in the current block is derived as a weighted sum of two prediction values ​​based on at least one weighting factor.

[0890] E9. A method according to scheme E8, wherein the final prediction block of the current block is (f(x,y)×P1(x,y)+(2M–f(x,y))×P2(x,y)+offset)>>M, where P1(x,y) and P2(x,y) are the first prediction value and the second prediction value of the sample point with coordinates (x,y), respectively, where M and the offset are integers, and where f(x,y) is a function that returns the weight applied to the first prediction value.

[0891] E10. A method according to scheme E9, wherein offset = (1<<(M-1)).

[0892] E11. A method according to any one of schemes E1 to E10, wherein the current block is divided into a plurality of partitions based on a partitioning mode.

[0893] E12. A method according to any one of schemes E1 to E11, wherein the geometric segmentation pattern includes a triangular segmentation pattern.

[0894] E13. A method according to scheme E12, wherein the geometric partitioning mode includes dividing a block into two partitions.

[0895] E12. A method according to any one of schemes E1 to E10, wherein the current block is divided into a plurality of partitions, at least one of the plurality of partitions being non-square and non-rectangular.

[0896] E13. A method according to any one of schemes E1 to E12, wherein the conversion generates the current block from a bitstream representation.

[0897] E14. A method according to any one of schemes E1 to E12, wherein the conversion generates a bitstream representation from the current block.

[0898] E15. An apparatus in a video system, comprising a processor and a non-transitory memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to implement the method of any one of embodiments E1 to E14.

[0899] E16. A computer program product stored on a non-transitory computer-readable medium, the computer program product comprising a program code for executing the method of any one of schemes E1 to E14.

[0900] In some embodiments, the following technical solutions may be implemented:

[0901] F1. A method for video processing, comprising: for a current block of a video partitioned into multiple partitions using a geometric partitioning pattern, performing a motion compensation process on samples along a common boundary of at least two of the multiple partitions and having a sample size different from a sample size of 4×4; and based on the motion compensation process, performing conversion between the current block and a bitstream representation of the video.

[0902] F2. A method according to scheme F1, wherein the motion compensation process is performed with a sample size of 8×8 to reduce the memory bandwidth required for conversion.

[0903] F3. A method according to scheme F1, wherein the motion compensation process is performed with a sample size of 8×4 or a sample size of 4×8 to reduce the memory bandwidth required for conversion.

[0904] F4. A method according to scheme F3, wherein the sample size is based on the height or width of the current block.

[0905] F5. A method according to scheme F3, wherein the sample size is based on a partitioning pattern of multiple partitions of the current block.

[0906] F6. A method according to scheme F5, wherein the division pattern includes a division from the upper right corner to the lower left corner or from the upper left corner to the lower right corner, represented by a 45 degree division pattern or a 135 degree division pattern, respectively.

[0907] F7. A method for video processing, comprising: partitioning a current block of a video into multiple partitions using a geometric partitioning pattern; storing a single motion information set for a K×L area within the current block derived from motion information associated with the multiple partitions; and using at least one motion information set associated with at least one of the multiple partitions to perform conversion between the current block and a bitstream representation of the video.

[0908] F8. A method according to scheme F7, wherein MVInfoi represents the motion information of the i-th partition, where i=1 and i=2, wherein when it is determined that MVInfo1 comes from the first reference picture list LX and MVInfo2 comes from the second reference picture list LY, a single motion information set includes bidirectional prediction and associated motion information of the current block based on the combination of MVInfo1 and MVInfo2, where X=0 or X=1, and wherein Y=(1-X).

[0909] F9. A method according to scheme F7, wherein MVInfoi represents the motion information of the i-th partition, where i=1 and i=2, wherein when it is determined that MVInfo1 and MVInfo2 are both from reference list LX, a single motion information set includes a unidirectional prediction and associated motion information of the current block based on MVInfoj, where j=1 or j=2, and wherein X=0 or X=1.

[0910] F10. A method according to scheme F8, wherein a single motion information set including bidirectional prediction is derived by combining the motion vectors and reference picture indices of MVInfo1 and MVInfo2.

[0911] F11. A method according to scheme F9, wherein a single motion information set including unidirectional prediction is based on MVInfo1.

[0912] F12. A method according to scheme F9, wherein a single motion information set including unidirectional prediction is based on MVInfo2.

[0913] F13. A method according to any one of schemes F1 to F12, wherein, when it is determined that the upper right corner sample point and the lower left corner sample point are in two different partitions, MVInfo1 includes a motion information set associated with the partition covering the upper right corner sample point, and MVInfo2 includes a motion information set associated with the partition covering the lower left corner sample point.

[0914] F14. A method according to scheme F13, wherein the partition direction of the current block is from the upper left corner to the lower right corner.

[0915] F15. A method according to any one of schemes F1 to F12, wherein, when it is determined that the upper left corner sample point and the lower right corner sample point are in two different partitions, MVInfo1 includes a motion information set associated with the partition covering the upper left corner sample point, and MVInfo2 includes a motion information set associated with the partition covering the lower right corner sample point.

[0916] F16. A method according to scheme F15, wherein the partition direction of the current block is from the upper right corner to the lower left corner.

[0917] F17. A method according to scheme F7, wherein the stored single motion information set is used for one of temporal motion prediction, spatial motion prediction, or filtering of a current block of a subsequent block of the video.

[0918] F18. A method according to scheme F7, wherein the stored single motion information set is used to process other blocks in different pictures.

[0919] F19. A method according to scheme F17, wherein the filtering process includes deblocking or adaptive loop filtering (ALF).

[0920] F20. A method according to scheme F7, wherein MVInfoi and MVInfoj represent motion information of the i-th partition and the j-th partition respectively, wherein MVInfoi and MVInfoj both come from a reference picture list LX, where X=0 or X=1, and wherein a single motion information set is based on a combination of MVInfoi and a scaled motion vector generated by scaling MVInfoj to a reference picture list LY, where Y=(1-X).

[0921] F21. A method according to scheme F20, wherein the value of X is predetermined or signaled in the bitstream representation.

[0922] F22. A method according to scheme F7, wherein MVInfoi and MVInfoj represent motion information of the i-th partition and the j-th partition respectively, wherein MVInfoi and MVInfoj both come from a reference picture list LX, where X=0 or X=1, and wherein a single motion information set is based on a combination of MVInfoi and a mapped motion vector generated by mapping MVInfoj to a reference picture list LY, where Y=(1-X).

[0923] F23. A method according to scheme F22, wherein the single motion information set includes bidirectional prediction motion vectors.

[0924] F24. A method according to scheme F22, wherein mapping MVInfoj to the reference picture in list LY does not include a scaling operation.

[0925] F25. A method according to scheme F7, wherein storing a single motion information set is based on decoded motion information associated with multiple partitions.

[0926] F26. A method according to scheme F25, wherein MVInfoi and MVInfoj represent motion information of the i-th partition and the j-th partition respectively, wherein MVInfoi and MVInfoj both come from the same reference picture list LX, wherein X=0 or X=1, and wherein a single motion information set includes unidirectional prediction motion information.

[0927] F27. A method according to scheme F25, wherein MVInfoi and MVInfoj represent motion information of the i-th partition and the j-th partition respectively, wherein MVInfoi comes from a reference picture list LX and MVInfoj comes from a reference picture list LY, wherein X=0 or X=1, wherein Y=(1-X), and wherein a single motion information set includes bidirectional prediction motion information.

[0928] F28. A method according to scheme F25, wherein MVInfoi and MVInfoj represent motion information of the i-th partition and the j-th partition respectively, wherein MVInfoi and MVInfoj both come from a reference picture list LX, where X=0 or X=1, wherein the reference picture list LY includes MVInfoi or MVInfoj, where Y=(1-X), and wherein a single motion information set includes bidirectional prediction motion information.

[0929] F29. A method according to scheme F7, wherein storing a single motion information set is based on a sub-region of the current block located along a common boundary of at least two of the multiple partitions.

[0930] F30. A method according to scheme F29, wherein different sub-regions store different motion information.

[0931] F31. A method according to scheme F29, wherein the sub-region stores unidirectional prediction motion information or bidirectional prediction motion information.

[0932] F32. A method according to scheme F7, wherein storing a single motion information set is based on the position of a sub-block in the current block, and wherein the sub-block only stores unidirectional prediction motion information.

[0933] F33. A method according to any one of schemes F1 to F32, wherein a single motion information set is used in the motion compensation process applied to the current block.

[0934] F34. A method according to scheme F7, wherein MVInfoi and MVInfoj represent motion information of the i-th partition and the j-th partition respectively, wherein MVInfoi and MVInfoj both come from a reference picture list LX, where X=0 or X=1, and wherein a single motion information set includes a bidirectional prediction motion vector for the reference picture list LX derived from (a) MVInfoi, (b) MVInfoj, or (c) an average of MVInfoi and MVInfoj.

[0935] F35. A method according to scheme F7, wherein MVInfoi and MVInfoj represent motion information of the i-th partition and the j-th partition respectively, wherein MVInfoi and MVInfoj both come from a reference picture list LX, wherein X=0 or X=1, and wherein a single motion information set comprises an average or weighted average of MVInfoi and MVInfoj, and wherein the reference picture associated with the single motion information set comprises a reference picture of MVInfoi or a reference picture of MVInfoj.

[0936] F36. A method according to any one of schemes F1 to F35, wherein the K×L area contains at least one sample point on a common boundary between at least two partitions of the plurality of partitions.

[0937] F37. A method according to any one of schemes F1 to F36, wherein the geometric segmentation pattern includes a triangular segmentation pattern.

[0938] F38. A method according to any one of schemes F1 to F36, wherein at least one partition of the plurality of partitions is non-square and non-rectangular.

[0939] F39. A method according to any one of schemes F1 to F38, wherein the conversion generates the current block from a bitstream representation.

[0940] F40. A method according to any one of schemes F1 to F38, wherein the conversion generates a bitstream representation from the current block.

[0941] F41. An apparatus in a video system, comprising a processor and a non-transitory memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to implement any one of the methods of embodiments F1 to F40.

[0942] F42. A computer program product stored on a non-transitory computer-readable medium, the computer program product comprising a program code for executing the method of any one of schemes F1 to F40.

[0943] In some embodiments, the following technical solutions may be implemented:

[0944] G1. A method of video processing, comprising: partitioning a current block of a video into a plurality of partitions; and performing conversion between a bitstream representation of the current block and the video encoded using a geometric partitioning pattern, wherein a single motion information set is stored for each M×N block unit of the current block, wherein the single motion information set is derived from motion information associated with each of a plurality of partitions, wherein M or N is not equal to 4, and wherein each sample in the M×N block unit shares the same motion information.

[0945] G2. A method according to scheme G1, wherein M=8 and N=8.

[0946] G3. A method according to scheme G1, wherein the first part of the M×N block unit includes a first set of sample points on a common boundary of at least two partitions among the multiple partitions, and wherein the second part of the M×N block unit includes a second set of sample points inside one of the at least two partitions among the multiple partitions.

[0947] G4. A method according to scheme G3, wherein MVInfoi represents the motion information of the i-th partition, where i=1 and i=2, wherein when it is determined that MVInfo1 is from the first reference picture list LX and MVInfo2 is from the second reference picture list LY, the same motion information includes bidirectional prediction and associated motion information for the current block based on MVInfoi of multiple partitions, where X=0 or X=1, and wherein Y=(1-X).

[0948] G5. A method according to scheme G3, wherein MVInfoi represents the motion information of the i-th partition, where i=1 and i=2, wherein when it is determined that MVInfo1 and MVInfo2 are both from the reference picture list LX, the same motion information includes a unidirectional prediction and associated motion information for the current block based on MVInfoj, where j=1 or j=2, and wherein X=0 or X=1.

[0949] G6. A method according to scheme G3, wherein MVInfoi represents the motion information of the i-th partition, where i=1 and i=2, and wherein the same motion information includes a unidirectional prediction and associated motion information for the current block based on MVInfoj, where j=1 or j=2.

[0950] G7. A method of video processing, comprising: partitioning a current block of a video into a plurality of partitions; and as part of a conversion between the current block and a bitstream representation of the video, performing a motion compensation process on a first sub-region of the current block based on first motion information that is different from second motion information associated with the first sub-region to be stored.

[0951] G8. A method according to scheme G7, wherein the first sub-region includes a set of M×N sample points on a common boundary of at least two partitions among the multiple partitions, wherein the motion compensation process includes bidirectional prediction, and wherein only one motion information set from list 0 or list 1 of the bidirectional prediction is stored.

[0952] G9. A method according to scheme G8, wherein the one motion information set is used to derive spatial motion candidates for neighboring blocks encoded using Merge mode or Advanced Motion Vector Prediction (AMVP) mode.

[0953] G10. A method according to scheme G8, wherein the one motion information set is used to derive temporal motion candidates for subsequent blocks in a picture different from a current picture including the current block.

[0954] G11. A method according to scheme G8, wherein the one motion information set is used for in-loop processing of the current block.

[0955] G12. A method according to scheme G11, wherein the in-loop processing includes deblocking or adaptive loop filtering.

[0956] G13. A method according to scheme G7, wherein a motion compensation process is performed on a second sub-region of the current block based on third motion information associated with the second sub-region, and wherein the first sub-region and the second sub-region are in the same coding unit (CU) including multiple partitions.

[0957] G14. A video processing method, comprising: using a geometric partitioning mode to partition a current block of a video into multiple partitions; based on a partitioning direction or a decoded Merge index or a Merge candidate list associated with the multiple partitions, storing motion information of a first sample point set and a second sample point set, wherein the first sample point set is located on a common boundary of at least two of the multiple partitions, and the second sample point set is located inside one of at least two of the multiple partitions; and based on the stored motion information, performing conversion between the current block and a bitstream representation of the video or conversion between a subsequent block of the video and a bitstream representation.

[0958] G15. A method according to scheme G14, wherein the stored motion information corresponds to motion information associated with one of multiple partitions associated with the selected decoded Merge index.

[0959] G16. A method according to scheme G14, wherein the stored motion information corresponds to the motion candidate in the motion candidate list having the selected Merge index set to the variable k.

[0960] G17. A method according to scheme G16, wherein k=0.

[0961] G18. A method according to scheme G16, wherein the stored motion information corresponds to the motion information of the kth Merge candidate in one of the Merge candidate lists.

[0962] G19. A method according to scheme G15, wherein the selected decoding Merge index is based on the partitioning direction.

[0963] G20. A method according to scheme G15, wherein, when it is determined that the Merge candidate with an index equal to the selected decoded Merge index is based on bidirectional prediction, the stored motion information includes bidirectional prediction motion information.

[0964] G21. A method according to scheme G15, wherein, when it is determined that the Merge candidate with an index equal to the selected decoded Merge index is based on uni-directional prediction, the stored motion information includes uni-directional prediction motion information.

[0965] G22. A method according to scheme G6, wherein the first sub-region includes M×N sample points on a common boundary of at least two partitions among multiple partitions, wherein the motion compensation process includes bidirectional prediction, and wherein only one motion information set from list 0 or list 1 of the bidirectional prediction is stored.

[0966] G23. A method according to scheme G6, wherein the first sub-region includes M×N sample points on the common boundary of at least two partitions among the multiple partitions, wherein the motion compensation process includes bidirectional prediction, wherein only unidirectional prediction information from list X is stored, and wherein X=0 and X=1.

[0967] G24. A method according to scheme G6, wherein the first sub-region includes M×N sample points on a common boundary of at least two partitions among the multiple partitions, wherein the motion compensation process includes unidirectional prediction, wherein the second motion information includes stored unidirectional prediction information from list X, and wherein X=0 and X=1.

[0968] G25. A video processing method, comprising: dividing a current block of a video into multiple partitions; storing virtual bidirectional prediction motion information based on motion information of at least two partitions among the multiple partitions; and performing conversion between the bitstream representation of the current block and the video based on the virtual bidirectional prediction motion information.

[0969] G26. A method according to scheme G25, wherein MvInfo1 and MvInfo2 are motion information of the first partition and the second partition respectively, and wherein the virtual bidirectional prediction motion information is based on modifying the prediction direction of MvInfoA from list X to list Y, where A=1 or A=2, where X=0 or X=1, and where Y=(1-X).

[0970] G27. A method according to scheme G26, wherein the motion vector and reference index of MvInfoA remain unchanged.

[0971] G28. A method according to scheme G26, wherein the reference index of MvInfoA remains unchanged and the motion vector of MvInfoA is set to an opposite value.

[0972] G29. A method according to any one of schemes G26 to G28, wherein MvInfo1 and MvInfo2 are from the same prediction direction.

[0973] G30. A video processing method, comprising: dividing a current block of a video into multiple partitions; based on a low-latency check flag, storing motion information (MvInfo1) of a first partition among the multiple partitions and motion information (MvInfo2) of a second partition among the multiple partitions, wherein the low-latency check flag indicates all reference pictures whose picture order count (POC) values ​​are not greater than the POC values ​​of the current picture including the current block; and based on the stored motion information, performing conversion between the bitstream representation of the current block and the video.

[0974] G31. A method according to scheme G30, wherein MvInfo1 and MvInfo2 are from different prediction directions, and wherein storing the motion information includes storing bidirectional prediction motion information generated by combining MvInfo1 and MvInfo2.

[0975] G32. A method according to scheme G30, wherein MvInfo1 and MvInfo2 come from the same prediction direction (list X), where X=0 or X=1, and wherein the method further comprises: modifying MvInfo2 by setting the prediction direction list of MvInfo2 from list X to list Y, where Y=(1-X), and wherein storing the motion information comprises storing bidirectional prediction motion information generated by combining MvInfo1 and the modified MvInfo2.

[0976] G33. A method according to any one of schemes G25 to G32, wherein storing motion information includes storing motion information of a sub-block of the current block.

[0977] G34. A method according to scheme G33, wherein the sub-block includes samples on a common boundary of at least two partitions among the multiple partitions.

[0978] G35. A method according to scheme G33, wherein the sub-block includes a diagonal line or a counter-diagonal line in the current block.

[0979] G36. A method according to scheme G33, wherein the sub-block is in the lower right corner of the current block.

[0980] G37. A method according to scheme G33, wherein the sub-block is in the right column or bottom row of the current block.

[0981] G38. A method according to any one of schemes G1 to G37, wherein at least one of the multiple partitions is non-square and non-rectangular.

[0982] G39. A method according to any one of schemes G1 to G38, wherein the conversion generates the current block from a bitstream representation.

[0983] G40. A method according to any one of schemes G1 to G38, wherein the conversion generates a bitstream representation from the current block.

[0984] G41. An apparatus in a video system, comprising a processor and a non-transitory memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to implement any one of the methods of implementation G1 to G40.

[0985] G42. A computer program product stored on a non-transitory computer-readable medium, the computer program product comprising a program code for executing the method of any one of schemes G1 to G40.

[0986] Fig.49 4900. The apparatus 4900 may be used to implement one or more methods described herein. The apparatus 4900 may be included in a smartphone, a tablet, a computer, an Internet of Things (IoT) receiver, etc. The apparatus 4900 may include one or more processors 4902, one or more memories 4904, and video processing hardware 4906. The processor(s) 4902 may be configured to implement one or more methods described in this document. The memory(s) 4904 may be used to store data and code for implementing the methods and techniques described herein. The video processing hardware 4906 may be used to implement some of the techniques described in this document in hardware circuits.

[0987] In some embodiments, the video encoding method may use the Fig.49 The method is implemented by the device implemented on the hardware platform.

[0988] Some embodiments of the disclosed technology include making a decision or determination to enable a video processing tool or mode. In one example, when a video processing tool or mode is enabled, the encoder will use or implement the tool or mode in the processing of a video block, but will not necessarily modify the resulting bitstream based on the use of the tool or mode. That is, when enabling a video processing tool or mode based on a decision or determination, the conversion from a block of video to a bitstream representation of a video will use the video processing tool or mode. In another example, when enabling a video processing tool or mode, the decoder will process the bitstream knowing that the bitstream has been modified based on the video processing tool or mode. That is, the conversion from the bitstream representation of a video to a block of video will be performed using a video processing tool or mode enabled based on a decision or determination.

[0989] Some embodiments of the disclosed technology include making a decision or determination to disable a video processing tool or mode. In one example, when a video processing tool or mode is disabled, the encoder will not use the tool or mode to convert blocks of video into a bitstream representation of the video. In another example, when a video processing tool or mode is disabled, the decoder will process the bitstream knowing that the bitstream has not been modified using the video processing tool or mode enabled based on the decision or determination.

[0990] Fig.50 5000 is a block diagram illustrating an example video processing system 5000 in which various techniques disclosed herein may be implemented. Various implementations may include some or all of the components of system 5000. System 5000 may include an input 5002 for receiving video content. The video content may be received in a raw or uncompressed format, for example, 8 or 10 bit multi-component pixel values, or may be in a compressed or encoded format. Input 5002 may represent a network interface, a peripheral bus interface, or a storage interface. Examples of network interfaces include wired interfaces (such as Ethernet, passive optical networks (PONs), etc.) and wireless interfaces (such as Wi-Fi or cellular interfaces).

[0991] System 5000 may include coding component 5004, which may implement various coding methods described in this document. Coding component 5004 may reduce the average bit rate of the video from input 5002 of coding component 5004 to output to generate a coded representation of the video. Therefore, coding techniques are sometimes referred to as video compression or video transcoding techniques. The output of coding component 5004 may be stored, or transmitted via a connected communication, as shown in component 5006. Component 5008 may use the bitstream (or coding) representation of the stored or communicated video received at input 5002 to generate pixel values ​​or displayable video sent to display interface 5010. The process of generating user-visible video from bitstream representation is sometimes referred to as video decompression. In addition, although some video processing operations are referred to as "coding" operations or tools, it should be understood that coding tools or operations are used at the encoder, and the corresponding decoding tools or operations of the inverse coding results will be performed by the decoder.

[0992] Examples of peripheral bus interfaces or display interfaces may include universal serial bus (USB) or high definition multimedia interface (HDMI) or display port, etc. Examples of storage interfaces include SATA (serial advanced technology attachments), PCI, IDE interfaces, etc. The technology described in this document may be embodied in various electronic devices, such as mobile phones, laptops, smart phones, or other devices capable of performing digital data processing and / or video display.

[0993] With respect to the description in this document, the bitstream representation of the current block of video is not necessarily in the form of continuous bits, but may be based on bits appearing in different locations in the bitstream syntax (e.g., header fields and parameter sets).

[0994] In addition, in some disclosed embodiments, for sub-block coding modes, including ATMVP and affine Merge, a Merge list construction process is shared. Here, ATMVP and affine Merge candidates can be added in sequence.

[0995] In some embodiments, for triangle prediction mode, a Merge list construction process for two partitions is shared, and even the two partitions can select their own Merge candidate indexes. When building the Merge list, the spatial neighbors of the block and the two temporal blocks are checked. The motion information derived from the spatial neighbors and the temporal blocks is called regular motion candidates in our IDF. These regular motion candidates are further used to derive multiple TPM candidates. The transformation can be performed at the level of the entire block, and even the two partitions can use different motion vectors to generate their own prediction blocks.

[0996] For the remaining coding blocks, a Merge list construction process is shared. Here, spatial / temporal / HMVP, paired combined bidirectional prediction Merge candidates and zero motion candidates can be inserted in sequence.

[0997] Those skilled in the art will appreciate that a video encoding or decoding method (compression / decompression) is disclosed. Geometric (or geometrical) partitioning of video blocks can be used to accurately track motion information and improve the compression performance of video encoding. It should also be understood that the use of the disclosed method can enable embodiments to achieve further coding gains and / or reduce implementation complexity compared to current video encoding / decoding techniques.

[0998] The disclosed and other schemes, examples, embodiments, modules and functional operations described in this document may be implemented in digital electronic circuits, or in computer software, firmware or hardware, including the structures disclosed in this document and their structural equivalents, or in a combination of one or more of them. The disclosed and other embodiments may be implemented as one or more computer program products, i.e., one or more modules of computer program instructions encoded on a computer-readable medium, for execution by a data processing device or for controlling the operation of the data processing device. The computer-readable medium may be a machine-readable storage device, a machine-readable storage substrate, a memory device, a material combination that implements a machine-readable propagation signal, or a combination of one or more of them. The term "data processing device" covers all devices, equipment and machines for processing data, including, for example, a programmable processor, a computer or multiple processors or computers. In addition to hardware, the device may also include code that creates an operating environment for the computer program in question, for example, code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them. A propagation signal is an artificially generated signal, for example, a machine-generated electrical signal, an optical signal, or an electromagnetic signal, which is generated to encode information for transmission to a suitable receiver device.

[0999] A computer program (also referred to as a program, software, software application, script, or code) may be written in any form of programming language, including compiled or interpreted languages, and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. A program may be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files storing one or more modules, subroutines, or portions of code). A computer program may be deployed to execute on one or more computers located at one site or distributed across multiple sites and interconnected by a communications network.

[1000] The processes and logic flows described herein may be performed by one or more programmable processors running one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows may also be performed by, and apparatus may be implemented as, special purpose logic circuitry, e.g., an FPGA (field programmable gate array) or an ASIC (application specific integrated circuit).

[1001] For example, processors suitable for executing computer programs include general and special purpose microprocessors, and any one or more processors of any type of digital computer. Typically, the processor will receive instructions and data from a read-only memory or a random access memory or both. The basic elements of a computer are a processor for executing instructions and one or more memory devices for storing instructions and data. Typically, a computer will also include one or more mass storage devices for storing data, such as magnetic disks, magneto-optical disks, or optical disks, or be operably coupled to receive data from one or more mass storage devices or to transfer data to them or both. However, a computer does not require such a device. Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, for example, including semiconductor memory devices, such as EPROM, EEPROM, and flash memory devices; magnetic disks, such as internal hard disks or removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks. The processor and memory can be supplemented by or incorporated into a dedicated logic circuit.

[1002] Although this patent document contains many details, these should not be interpreted as limitations on any subject matter or the scope of the claimed protection, but rather as descriptions of features unique to specific embodiments of specific technologies. Certain features described in this patent document in the context of separate embodiments may also be implemented in combination in a single embodiment. Conversely, the various features described in the context of a single embodiment may also be implemented in multiple embodiments individually or in any suitable sub-combination. In addition, although the above-mentioned features may be described as working in certain combinations and even initially claimed to be protected in this way, in some cases, one or more features in the combination may be deleted from the claimed combination, and the claimed combination may point to a sub-combination or a variant of the sub-combination.

[1003] Similarly, although operations are described in a particular order in the drawings, this should not be understood as requiring that the operations be performed in the particular order shown or in sequence, or that all of the operations shown be performed, in order to achieve the desired results. Furthermore, the separation of various system components in the embodiments described in this patent document should not be understood as requiring such separation in all embodiments.

[1004] Only a few implementations and examples are described, and other implementations, enhancements, and variations can be made based on what is described and illustrated in this patent document.

Claims

1. A method for video processing, comprising: making a decision, based on a priority rule, about an order in which to insert motion candidates into a motion candidate list for conversion between a current block of video and a bitstream representation of the video, wherein the current block is encoded using a geometric partitioning mode; as well as Based on the determination and the motion candidate list, performing the conversion, wherein the priority rule is based on a prediction direction of at least one motion candidate, or the priority rule is based on coding mode information associated with a regular motion candidate, The method further comprises: One or more average uni-directional prediction motion candidates from list 0 and list 1 derived from a regular motion candidate are inserted into the motion candidate list for conversion between the current block of video and the bitstream representation of the video.

2. The method according to claim 1, wherein: The order of insertion includes inserting normal motion candidates with uni-directional prediction from list X before inserting any normal motion candidates with uni-directional prediction from list Y.

3. The method according to claim 1, wherein: The order of insertion includes inserting motion candidates with bi-directional prediction before inserting any motion candidates with uni-directional prediction.

4. The method according to claim 1, wherein: The order of insertion is defined as: motion candidates of list X derived from normal motion candidates with bi-directional prediction, followed by motion candidates of list X derived from normal motion candidates with bi-directional prediction, followed by normal motion candidates with uni-directional prediction, where X=0 or X=1.

5. The method according to claim 1, wherein: The order of insertion includes inserting a motion candidate derived from a first regular motion candidate with bidirectional prediction before inserting a motion candidate derived from a second regular motion candidate with bidirectional prediction, and wherein the order of insertion of the current block is the same as the order of insertion of video blocks encoded using a non-geometric partitioning mode.

6. The method according to claim 1, wherein: The order of inserting includes inserting the motion candidates derived from the first normal motion candidate with bi-directional prediction and the motion candidates derived from the second normal motion candidate with bi-directional prediction in an interleaved manner.

7. The method according to claim 6, wherein: The interleaving method includes inserting A The derived truncated list 0 prediction candidates, from C B The derived truncated list 1 prediction candidate, from C A The truncated list derived from 1 prediction candidate and from C B The derived truncated list 0 prediction candidates, where C A and C B are two regular sports candidates.

8. The method according to claim 6, wherein: The interleaving method includes inserting A The derived truncated list 1 prediction candidate, from C B The derived truncated list 0 prediction candidates, from C A The truncated list of derived 0 prediction candidates and from C B The derived truncated list 1 prediction candidates, where C A and C B are two regular sports candidates.

9. The method according to any one of claims 1 to 8, wherein: The conventional motion candidates are motion candidates derived for video blocks encoded using a non-geometric partitioning mode.

10. The method according to claim 1, wherein: The encoding mode information includes encoding performed using a Merge mode or an Advanced Motion Vector Prediction (AMVP) mode.

11. The method according to claim 10, wherein: The order of insertion includes inserting the motion candidate derived from the block encoded in the AMVP mode before inserting the motion candidate derived from the block encoded in the Merge mode.

12. The method according to claim 10, wherein: The order of insertion includes inserting the motion candidate derived from the block encoded in the Merge mode before inserting the motion candidate derived from the block encoded in the AMVP mode.

13. The method according to claim 1, wherein: The coding mode information includes a reference index or a picture order count (POC) difference.

14. The method according to claim 13, wherein: The order of insertion includes inserting motion candidates associated with the first reference index before inserting motion candidates associated with a second reference index greater than the first reference index.

15. The method according to claim 13, wherein: The order of inserting includes inserting motion candidates associated with the first POC difference before inserting motion candidates associated with a second POC difference greater than the first POC difference.

16. The method according to claim 1, wherein: The encoding mode information includes a quantization parameter (QP) or a temporal layer index of a reference picture associated with a picture, a slice, or a slice group including the current block.

17. The method according to claim 16, wherein: The order of insertion includes inserting motion candidates associated with the first QP before inserting motion candidates associated with a second QP greater than the first QP.

18. The method according to claim 1, wherein: The order of insertion includes inserting all truncated list 1 prediction candidates before any truncated list 0 prediction candidates.

19. The method according to claim 1, wherein: The order of inserting includes inserting one or more truncation list 0 prediction candidates and one or more truncation list 1 prediction candidates based on available geometric motion candidates.

20. The method according to claim 1, wherein: The order of inserting includes inserting a first motion candidate with bi-directional prediction before inserting a second motion candidate with bi-directional prediction based on available geometric motion candidates.

21. The method according to claim 1, further comprising: A single average uni-directional prediction motion candidate is selected from the one or more average uni-directional prediction motion candidates for insertion into the motion candidate list.

22. The method according to claim 21, wherein: The selection is based on available geometric motion candidates in the motion candidate list.

23. The method according to claim 21, wherein: The selection is based on the reference picture indices of list 0 and list 1.

24. The method according to claim 21, wherein: The selection is based on the picture order count (POC) distance between the reference pictures of list 0 and list 1 and the current picture.

25. The method of claim 1, wherein: All average uni-directional prediction motion candidates from list 0 are inserted before inserting any average uni-directional prediction motion candidates from list 1.

26. The method of claim 1, wherein: All average uni-directional prediction motion candidates from list 1 are inserted before inserting any average uni-directional prediction motion candidates from list 0.

27. The method of claim 1, further comprising: One or more average motion candidates with uni-directional prediction derived from regular motion candidates with uni-directional prediction are inserted into a motion candidate list for conversion between a current block of video and a bitstream representation of the video, wherein the current block is encoded using a geometric partitioning mode.

28. The method of claim 27, wherein: The average motion candidate with unidirectional prediction from list LX is derived from the normal regular motion candidate with unidirectional prediction from list LX, where X=0 or X=1.

29. The method of claim 27, wherein: An average motion candidate with unidirectional prediction from list LY is derived from the scaled normal motion candidate, wherein the normal motion candidate from list LX is scaled to list LY to generate the scaled normal motion candidate, where X=0 or X=1, and where Y=1-X.

30. The method of claim 1, further comprising: One or more virtual motion candidates derived from the available motion candidates are inserted into a motion candidate list for conversion between a current block of video and a bitstream representation of the video, wherein the current block is encoded using a geometric partitioning mode.

31. The method according to claim 30, wherein: The one or more virtual motion candidates are derived by first generating a truncated list of motion candidates in the motion candidate list.

32. The method of claim 30, wherein: The one or more virtual motion candidates are derived based only on motion candidates with unidirectional prediction.

33. The method of claim 30, wherein: The one or more virtual motion candidates are derived based only on the truncated list of motion candidates with unidirectional prediction and motion candidates predicted from list 1.

34. The method of claim 30, wherein: The one or more virtual motion candidates are derived based on motion candidates having a predetermined reference picture index or having a picture order count (POC) distance within a specific range.

35. The method of claim 30, wherein: The one or more virtual motion candidates are derived without applying a trimming operation.

36. The method according to any one of claims 1 to 8, wherein: The current block is divided into a plurality of partitions based on a partitioning mode.

37. The method of claim 36, wherein: At least one partition of the plurality of partitions is non-square and non-rectangular.

38. The method according to any one of claims 1 to 8, wherein: The geometric partitioning mode includes a triangle partitioning mode.

39. The method according to any one of claims 1 to 8, wherein: The conversion generates the current block from the bitstream representation.

40. The method according to any one of claims 1 to 8, wherein: The conversion generates the bitstream representation from the current block.

41. An apparatus in a video system, comprising a processor and a non-transitory memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to implement the method of any one of claims 1 to 40.

42. A non-transitory computer readable medium comprising program code for executing the method of any one of claims 1 to 40.

Citation Information

Cited By

  • Multiple merge lists and orders for inter prediction with geometric partitioning

    US12689726B2