Motion information derivation for inter prediction

Abstract

Devices, systems, and methods for digital video coding including geometric partitioning are described. An example method for video processing includes making a decision, based on a priority rule, regarding an order in which to insert motion candidates into a motion candidate list for a conversion between a current block of a video and a bitstream representation of the video, wherein the current block is coded using a geometric partitioning mode, and performing the conversion based on the decision and the motion candidate list.

Description

[0001] Cross-references to related applications

[0002] In accordance with applicable patent law and / or the rules of the Paris Convention, this application aims to promptly claim priority and interest in International Patent Application PCT / CN2019 / 090127, filed June 5, 2019, and International Patent Application PCT / CN2019 / 092151, filed June 20, 2019. For all purposes under U.S. law, the entire disclosure of the foregoing applications is incorporated by reference as part of the disclosure of this patent document. Technical Field

[0003] This document covers video encoding and decoding technologies, systems, and devices. Background Technology

[0004] Despite advancements in video compression, digital video still accounts for the largest share of bandwidth usage on the internet and other digital communication networks. As the number of networked user devices capable of receiving and displaying video increases, bandwidth demand for digital video applications is expected to continue to grow. Summary of the Invention

[0005] This paper describes devices, systems, and methods related to digital video coding, including geometric segmentation. The described methods can be applied to existing video coding standards (e.g., High Efficiency Video Coding (HEVC) and / or Versatile Video Coding (VVC)) as well as future video coding standards or codecs.

[0006] In one representative aspect, the disclosed technology can be used to provide a method for video processing. The method includes: making a decision, based on priority rules, regarding the order in which motion candidates are inserted into a list of motion candidates for a transformation between a current block of video and a bitstream representation of the video, wherein the current block is encoded and decoded using a geometric segmentation mode; and performing the transformation based on the decision and the list of motion candidates.

[0007] In another representative aspect, the disclosed technology can be used to provide a method for video processing. This method includes: inserting one or more average unidirectional predicted motion candidates derived from a regular motion candidate list from lists 0 and 1 into a motion candidate list for a transformation between a current block of video and a bitstream representation of the video, wherein the current block is encoded and decoded using a geometric segmentation pattern; and performing the transformation based on the motion candidate list.

[0008] In another representative aspect, the disclosed technology can be used to provide a method for video processing. The method includes: inserting one or more average motion candidates with unidirectional prediction, derived from regular motion candidates with unidirectional prediction, into a motion candidate list for a transformation between a current block of video and a bitstream representation of the video, wherein the current block is encoded and decoded using a geometric segmentation pattern; and performing the transformation based on the motion candidate list.

[0009] In another representative aspect, the disclosed technology can be used to provide a method for video processing. This method includes: inserting one or more virtual motion candidates, derived from available motion candidates, into a motion candidate list for a transformation between a current block of video and a bitstream representation of the video, wherein the current block is encoded and decoded using a geometric segmentation pattern; and performing the transformation based on the motion candidate list.

[0010] In another representative aspect, the disclosed technology can be used to provide a method for video processing. The method includes: making a decision, based on a priority rule, regarding the order in which motion candidates are inserted into a list of motion candidates for a transformation between a current block of video encoded using a geometric segmentation pattern and a bitstream representation of the video, wherein the priority rule is based on the position of the motion candidates derived therefrom or a list of one or more reference images associated with the motion candidates; and performing the transformation based on the decision and the list of motion candidates.

[0011] In another representative aspect, the disclosed technology can be used to provide a method for video processing. The method includes: making a decision, based on priority rules, regarding the order in which motion candidates are inserted into a list of motion candidates for a transformation between a current block of video and a bitstream representation of the video, wherein the current block is encoded and decoded using a geometric segmentation mode, and wherein the insertion order is variable between sequence-to-sequence, picture-to-picture, strip-to-strip, slice-to-slice, or from the current block to a subsequent block of video; and performing a transformation based on the decision and the list of motion candidates.

[0012] In another representative aspect, the disclosed technology can be used to provide a method for video processing. This method includes: dividing a current block of a video encoded using a geometric segmentation pattern into multiple partitions; constructing multiple motion candidate lists, each corresponding to each of the multiple partitions; and performing a conversion between the current block and a bitstream representation of the video based on the multiple motion candidate lists.

[0013] In another representative aspect, the disclosed technology can be used to provide a method for video processing. This method includes: segmenting a current block of a video encoded using a geometric segmentation pattern into multiple segments; constructing multiple motion candidate lists, wherein each of the multiple segments is configured to select the same motion candidate list from the multiple motion candidate lists; and performing a conversion between the current block and a bitstream representation of the video based on the selected same motion candidate list.

[0014] In another representative aspect, the disclosed technology can be used to provide a method for video processing. The method includes performing a conversion between a current block of video in a video unit and a bitstream representation of the video, wherein the bitstream representation includes a field indicating the maximum number of motion candidates allowed in a list of motion candidates for a geometric segmentation mode enabled in a video region.

[0015] In another representative aspect, the disclosed technology can be used to provide a method for video processing. The method includes: for a current block of a video segmented into multiple partitions using a geometric segmentation pattern, selecting a set of weighting factors from a plurality of weighting factor sets, wherein the weighting factor sets are selected based at least on the width or height of the current block; and as part of a conversion between the current block and a bitstream representation of the video, applying the weighting factor sets to samples along the common boundary of at least two of the multiple partitions.

[0016] In another representative aspect, the disclosed technology can be used to provide a method for video processing. The method includes: for samples within a current block of a video segmented into multiple partitions using a geometric segmentation pattern, determining at least one weighting factor based on the angle of the common boundary of at least two of the partitions; and performing a conversion between the current block and a bitstream representation of the video based on the at least one weighting factor.

[0017] In another representative aspect, the disclosed technology can be used to provide a method for video processing. This method includes: for a current block of a video segmented into multiple partitions using a geometric segmentation pattern, performing a motion compensation process on samples along the common boundary of at least two of the multiple partitions, where the sample size differs from a 4×4 sample size; and based on the motion compensation process, performing a conversion between the current block and a bitstream representation of the video.

[0018] In another representative aspect, the disclosed technology can be used to provide a method for video processing. The method includes: dividing a current block of video into multiple partitions using a geometric segmentation pattern; storing a single set of motion information for a K×L region within the current block, derived from motion information associated with the multiple partitions; and performing a conversion between the current block and a bitstream representation of the video using at least one set of motion information associated with at least one of the multiple partitions.

[0019] In another representative aspect, the disclosed technology can be used to provide a method for video processing. The method includes: dividing a current block of video into multiple partitions; and performing a conversion between the current block encoded and decoded using a geometric segmentation pattern and a bitstream representation of the video, wherein a single set of motion information is stored for each M×N block unit of the current block, wherein the single set of motion information is derived from motion information associated with each of the multiple partitions, wherein M or N is not equal to 4, and wherein each sample in the M×N block unit shares the same motion information.

[0020] In another representative aspect, the disclosed technology can be used to provide a method for video processing. This method includes: dividing a current block of video into multiple partitions; and, as part of a conversion between the current block and a bitstream representation of the video, performing a motion compensation process on a first sub-region of the current block based on first motion information, which is different from second motion information associated with the first sub-region to be stored.

[0021] In another representative aspect, the disclosed technology can be used to provide a method for video processing. The method includes: dividing a current block of video into multiple partitions using a geometric segmentation pattern; storing motion information of a first sample set and a second sample set based on a segmentation direction or a decoding merge index or merge candidate list associated with the multiple partitions, wherein the first sample set lies on the common boundary of at least two of the multiple partitions, and the second sample set lies inside one of at least two of the multiple partitions; and performing a conversion between the current block and a bitstream representation of the video, or a conversion between subsequent blocks of the video and a bitstream representation, based on the stored motion information.

[0022] In another representative aspect, the disclosed technology can be used to provide a method for video processing. The method includes: dividing a current block of video into multiple partitions; storing virtual bidirectional predictive motion information based on motion information from at least two of the multiple partitions; and performing a conversion between the current block and a bitstream representation of the video based on the virtual bidirectional predictive motion information.

[0023] In another representative aspect, the disclosed technology can be used to provide a method for video processing. The method includes: dividing a current block of video into multiple partitions; storing motion information (MvInfo1) of a first partition and motion information (MvInfo2) of a second partition among the multiple partitions, based on a low-latency check flag, wherein the low-latency check flag indicates all reference images whose Picture Order Count (POC) value is not greater than the POC value of the current image including the current block; and performing a conversion between the current block and a bitstream representation of the video based on the stored motion information.

[0024] In another representative aspect, the disclosed technology can be used to provide a method for video processing. The method includes: segmenting a current block of video into multiple partitions using a geometric segmentation pattern; and performing a conversion between the current block and a bitstream representation of the video using at least two sets of motion information associated with at least two of the multiple partitions, wherein a single set of motion information is stored for a K×L region located on the common boundary of the at least two partitions, and wherein the single set of motion information includes unidirectional predicted motion information derived from motion information associated with each of the at least two partitions.

[0025] In another representative aspect, the disclosed technology can be used to provide a method for video processing. This method includes: maintaining one or more history-based motion vector prediction (HMVP) tables; performing a conversion between a current block of video and a bitstream representation of the video, wherein the current block is segmented into multiple partitions using a geometric segmentation pattern; and conditionally and selectively updating one or more HMVP tables after performing the conversion.

[0026] In another representative aspect, the disclosed technology can be used to provide a method for video processing. This method includes: for a current block that has been segmented into multiple partitions using a geometric segmentation pattern, performing a conversion between the current block and a bitstream representation of the video, wherein the conversion is based on first motion information of a first partition among the multiple partitions, and wherein the first motion information is derived based on second motion information of a second partition among the multiple partitions.

[0027] In another representative aspect, the disclosed technology can be used to provide a method for video processing. This method includes: for a current block that has been segmented into multiple partitions using a geometric segmentation pattern, performing a conversion between the current block and a bitstream representation of the video, wherein motion information for more than one of the multiple partitions is not derived from the same list of reference images.

[0028] In another representative aspect, the disclosed technology can be used to provide a method for video processing. This method includes: selectively enabling the derivation of motion information from the same reference image for a first and second partition based on a current block that has been segmented into multiple partitions using a geometric segmentation pattern; and performing a conversion between the current block and a bitstream representation of the video based on this selective enabling.

[0029] In another representative aspect, the above methods are embodied in processor-executable code and stored in a computer-readable program medium.

[0030] In another representative aspect, a device configured or operable to perform the above-described methods is disclosed. This device may include a processor programmed to implement the methods.

[0031] In another representative aspect, a video decoder device can implement the methods described herein.

[0032] The above and other aspects and features of the disclosed technology are described in more detail in the accompanying drawings, description and claims. Attached Figure Description

[0033] Figure 1 This is an example of the derivation process used for constructing the Merge candidate list.

[0034] Figure 2 Example locations of spatial merge candidates are shown.

[0035] Figure 3 An example of candidate pairs is shown that take into account for redundancy checks for spatial merge candidates.

[0036] Figure 4A and Figure 4B Example locations of the second prediction unit (PU) are shown in N×2N and 2N×N segmentation.

[0037] Figure 5 This is an example illustration of motion vector scaling for temporal Merge candidates.

[0038] Figure 6 Example candidate positions for time-domain Merge candidates C0 and C1 are shown.

[0039] Figure 7 An example of combined bidirectional prediction of Merge candidates is shown.

[0040] Figure 8 An example derivation of motion vector prediction candidates is shown.

[0041] Figure 9 This is an example illustration of motion vector scaling for spatial motion vector candidates.

[0042] Figure 10 An example of Alternative Temporal Motion Vector Prediction (ATMVP) motion prediction for CU (Coding Unit) is shown.

[0043] Figure 11 An example of a CU with four sub-blocks (AD) and its neighboring blocks (ad) is shown.

[0044] Figure 12This is a flowchart illustrating examples of encoding using different MV (Motion Vector) precision.

[0045] Figure 13A and Figure 13B The diagram illustrates a 135-degree segmentation type (divided from the top left to the bottom right) and a 45-degree segmentation pattern. It also shows an illustration of dividing the CU into two triangular prediction units (two segmentation patterns).

[0046] Figure 14 An example of the location of a neighboring block is shown.

[0047] Figure 15A and Figure 15B An example of motion vector storage is shown.

[0048] Figure 16A and Figure 16B An example of signaling for Triangular Prediction Mode (TPM) is shown.

[0049] Figure 17 An example of adjacent blocks (A and L) used for context selection in TPM flag encoding and decoding is shown.

[0050] Figure 18A and Figure 18B This is an example illustration of a sub-block that applies Overlapped Block Motion Compensation (OBMC).

[0051] Figure 18C An example of unidirectional prediction MV selection for the triangular prediction pattern is shown.

[0052] Figure 19 An example of neighboring samples used to derive IC (Illumination Compensation) parameters is shown.

[0053] Figure 20A and Figure 20B Simplified affine motion models, including a 4-parameter affine model and a 6-parameter affine model, are shown.

[0054] Figure 21 An example of the affine motion vector field (MVF) for each sub-block is shown.

[0055] Figure 22A and Figure 22B Examples of 4-parameter affine models and 6-parameter affine models are shown.

[0056] Figure 23An example of the AF_INTER Motion Vector Predictor (MVP) is shown.

[0057] Figure 24A and Figure 24B Example candidates for AF_MERGE are shown.

[0058] Figure 25 Examples of candidate positions for the affine Merge pattern are shown.

[0059] Figure 26 An example of an optical flow trajectory is shown.

[0060] Figure 27A and Figure 27B Examples of BIO (Bi-directional Optical flow) without block expansion are shown: a) access locations outside the block; b) padding is used to avoid additional memory accesses and computations.

[0061] Figure 28 An example of decoder-side motion vector refinement (DMVR) based on bilateral template matching is shown.

[0062] Figure 29 An example of a block is shown for the process of building a list of motion candidates for different segments.

[0063] Figures 30A-30C Three examples of MV storage regions with Dir equal to 0 are shown.

[0064] Figures 31-53 This is a flowchart of an example of a video processing method.

[0065] Figure 54 This is a block diagram of an example hardware platform used to implement the visual media decoding or visual media encoding techniques described in this document.

[0066] Figure 55 This is a block diagram of an example video processing system in which the disclosed techniques can be implemented. Detailed Implementation

[0067] This document provides a variety of techniques that video bitstream decoders can use to improve the quality of decompressed or decoded digital video. Furthermore, video encoders can implement these techniques during the encoding process to reconstruct decoded frames for further encoding.

[0068] For ease of understanding, chapter headings are used in this document, and the embodiments and techniques are not limited to the corresponding chapters. In this way, embodiments from one chapter can be combined with embodiments from other chapters.

[0069] 1. Overview

[0070] This patent document relates to video codec technology. Specifically, it relates to motion vector encoding and decoding and signaling under geometric segmentation in video codec. It can be applied to existing video codec standards, such as HEVC, or upcoming standards (e.g., Multi-Functional Video Codec (VVC)). It can also be applied to future video codec standards or video codecs.

[0071] 2. Introductory Notes

[0072] Video codec standards have primarily evolved through the development of well-known ITU-T and ISO / IEC standards. ITU-T developed H.261 and H.263, while ISO / IEC developed MPEG-1 and MPEG-4. These two organizations jointly developed the H.262 / MPEG-2 video and H.264 / MPEG-4 Advanced Video Coding (AVC) standards, as well as the H.265 / HEVC standard. Since H.262, video codec standards have been based on a hybrid video codec architecture, using temporal prediction plus transform coding. To explore future video codec technologies beyond HEVC, VCEG and MPEG jointly established the Joint Video Exploration Team (JVET) in 2015. Since then, JVET has adopted many new methods and incorporated them into reference software called the Joint Exploration Model (JEM). In April 2018, the Joint Video Experts Group (JVET) between VCEG (Q6 / 16) and ISO / IEC JTC1 SC29 / WG11 (MPEG) was established to work on the Multi-Functional Video Coding (VVC) standard, with the goal of reducing the bit rate by 50% compared to HEVC.

[0073] The latest version of the VVC draft, namely Multi-Functional Video Codec (Draft 2), can be found at the following URL:

[0074] http: / / phenix.it-sudparis.eu / jvet / doc_end_user / documents / 11_Ljubljana / wg11 / JVET-K1001-v7.zip

[0075] The latest reference software for VVC is called VTM, which can be found at the following website:

[0076] https: / / vcgit.hhi.fraunhofer.de / jvet / VVCSoftware_VTM / tags / VTM-2.1.

[0077] 2.1 Inter-frame prediction in HEVC / H.265

[0078] Each inter-frame predicted PU has motion parameters for one or two lists of reference images. The motion parameters include motion vectors and reference image indices. The use of one of the two reference image lists can also be notified using inter_pred_idc signaling. The motion vectors can be explicitly encoded as increments relative to the predicted values.

[0079] When encoding and decoding a CU using skip mode, a PU is associated with that CU, and there are no significant residual coefficients, no encoded motion vector increments, or reference picture indices. A merge mode is defined, thereby obtaining the motion parameters of the current PU from neighboring PUs, including spatial and temporal candidates. The merge mode can be applied to any PU for inter-frame prediction, not just skip mode. An alternative to the merge mode is explicit transmission of motion parameters, where each PU explicitly signals the motion vector (more precisely, the motion vector difference compared to the predicted motion vector), the corresponding reference picture index for each reference picture list, and the use of the reference picture list. In this disclosure, this mode is referred to as Advanced Motion Vector Prediction (AMVP).

[0080] When signaling indicates that one of two lists of reference images should be used, a PU is generated from a sample block. This is called "one-way prediction". One-way prediction can be used for both P-strips and B-strips.

[0081] When signaling indicates that two reference image lists should be used, PUs are generated from both sample blocks. This is called "bidirectional prediction". Bidirectional prediction can only be used for B-strips.

[0082] The following text provides detailed information about the inter-frame prediction modes specified in HEVC. The description will begin with the Merge mode.

[0083] 2.1.1 List of Reference Images

[0084] In HEVC, the term inter-frame prediction is used to describe predictions derived from data elements (e.g., sample values ​​or motion vectors) of reference images other than the currently decoded image. As in H.264 / AVC, images can be predicted from multiple reference images. The reference images used for inter-frame prediction are organized into one or more reference image lists. A reference index identifies which reference images in the list should be used to create the predicted signal.

[0085] A single list of reference images (list 0) is used for the P-strip, and two lists of reference images (list 0 and list 1) are used for the B-strip. It should be noted that, in terms of capture / display order, the reference images included in lists 0 and 1 can be images from the past and future.

[0086] 2.1.2 Merge Mode

[0087] 2.1.2.1 Derivation of Merge Pattern Candidates

[0088] When predicting a PU using the Merge mode, indices pointing to entries in the Merge candidate list are parsed from the bitstream and used to retrieve motion information. The construction of this list is specified in the HEVC standard and can be outlined in the sequence of the following steps:

[0089] Step 1: Initial Candidate Derivation

[0090] Step 1.1: Spatial Candidate Derivation

[0091] Step 1.2: Redundancy check of spatial candidates

[0092] Step 1.3: Temporal Candidate Derivation

[0093] Step 2: Adding candidate insertions

[0094] Step 2.1: Create bidirectional prediction candidates

[0095] Step 2.2: Insert zero-motion candidates

[0096] These steps are also schematically depicted in Figure 1In the spatial domain merge candidate derivation, up to four merge candidates are selected from five distinct candidates. In the temporal domain merge candidate derivation, up to one merge candidate is selected from two candidates. Since a constant number of candidates is assumed for each PU at the decoder, additional candidates are generated when the number of candidates obtained from step 1 does not reach the maximum number of merge candidates (MaxNumMergeCand) signaled in the stripe header. Because the number of candidates is constant, the index of the best merge candidate is encoded using truncated unary binarization (TU). If the CU size is equal to 8, all PUs of the current CU share a single merge candidate list, which is the same as the merge candidate list of a 2N×2N prediction unit.

[0097] The operations associated with the above steps will be described in detail below.

[0098] 2.1.2.2 Derivation of Airspace Candidates

[0099] In the derivation of the spatial Merge candidate, located in Figure 2 At most four merged candidates are selected from the candidates at the positions depicted. The derivation order is A1, B1, B0, A0, and B2. Position B2 is considered only if any PU at positions A1, B1, B0, or A0 is unavailable (e.g., because it belongs to another strip or slice) or if it is intra-frame encoding / decoding. After adding the candidate at position A1, a redundancy check is performed on the addition of the remaining candidates, which ensures that candidates with the same motion information are excluded from the list, thereby improving encoding / decoding efficiency. To reduce computational complexity, not all possible candidate pairs are considered in the aforementioned redundancy check. Instead, only those with the same motion information are considered. Figure 3 The arrows in the list link pairs, and if the corresponding candidate used for redundancy checking does not have the same motion information, the candidate is only added to the list. Another source of duplicate motion information is a "second PU" associated with a segmentation different from 2N×2N. As an example, Figure 4A and Figure 4B The second prediction unit (PU) is depicted in the N×2N and 2N×N cases, respectively. When the current PU is segmented into N×2N, the candidate at position A1 is not considered for list construction. In fact, adding this candidate would result in two prediction units having the same motion information, which is redundant for an encoding / decoding unit with only one PU. Similarly, when the current PU is segmented into 2N×N, position B1 is not considered.

[0100] 2.1.2.3 Time-domain candidate derivation

[0101] In this step, only one candidate is added to the list. Specifically, in the derivation of this temporal merge candidate, the scaling motion vector is derived based on the co-located PU of the image that has the smallest POC (Picture Order Count) difference with the current image in the given list of reference images. The list of reference images to be used for deriving the co-located PU is explicitly signaled in the strip header. Figure 5 As shown by the dashed lines, the scaled motion vector for the temporal merge candidate is obtained. This scaled motion vector is scaled from the motion vector of the juxtaposed PU using the POC distances tb and td, where tb is defined as the POC difference between the reference image of the current image and the current image, and td is defined as the POC difference between the reference image of the juxtaposed image and the juxtaposed image. The reference image index of the temporal merge candidate is set to zero. The actual implementation of the scaling process is described in the HEVC specification. For the B-strip, two motion vectors (one for reference image list 0 and the other for reference image list 1) are obtained and combined to generate bidirectional prediction merge candidates.

[0102] In the juxtaposed PU(Y) belonging to the reference frame, the position of the temporal candidate is selected between candidate C0 and C1, such as... Figure 6 As described. If the PU at position C0 is unavailable, is intra-frame encoded, or is outside the current CTU row, then position C1 is used. Otherwise, position C0 is used in the derivation of the temporal merge candidate.

[0103] 2.1.2.4 Additional Candidate Insertion

[0104] In addition to spatial and temporal merge candidates, two additional types of merge candidates exist: combined bidirectional prediction merge candidates and zero merge candidates. Combined bidirectional prediction merge candidates are generated by utilizing spatial and temporal merge candidates. These combined bidirectional prediction merge candidates are only used for B-strips. Combined bidirectional prediction candidates are generated by combining the motion parameters of the first reference image list of the initial candidate with the motion parameters of the second reference image list of another initial candidate. If these two tuples provide different motion hypotheses, they will form a new bidirectional prediction candidate. As an example, Figure 7 This illustrates bidirectional prediction merge candidates when two candidates from the original list (on the left) (which have mvL0 and refIdxL0 or mvL1 and refIdxL1) are used to create combinations that are added to the final list (on the right). Numerous rules exist regarding combinations, which are considered to generate these additional merge candidates.

[0105] Zero-motion candidates are inserted to populate the remaining entries in the Merge candidate list, thus reaching the MaxNumMergeCand capacity. These candidates have zero spatial displacements and reference image indices that start at zero and increase each time a new zero-motion candidate is added to the list. The number of reference frames used for these candidates is 1 for unidirectional prediction and 2 for bidirectional prediction. Finally, no redundancy checks are performed on these candidates.

[0106] 2.1.2.5 Motion estimation region for parallel processing

[0107] To accelerate the encoding process, motion estimation can be performed in parallel, thereby simultaneously deriving the motion vectors of all prediction units within a given region. Deriving merge candidates from spatial neighborhoods can interfere with parallel processing because a prediction unit cannot derive motion parameters from neighboring PUs until its associated motion estimation is complete. To mitigate the trade-off between encoding / decoding efficiency and processing latency, HEVC defines a Motion Estimation Region (MER), whose size is signaled in the image parameter set using the "log2_parallel_merge_level_minus2" syntax element. When defining the MER, merge candidates falling into the same region are marked as unavailable and therefore not considered in list construction.

[0108] 2.1.3 AMVP

[0109] AMVP utilizes the spatiotemporal correlation between motion vectors and neighboring PUs, which is used for the explicit transmission of motion parameters. For each list of reference images, a motion vector candidate list is constructed by first checking the availability of temporally neighboring PU locations to the left and top, removing redundant candidates, and adding zero vectors to make the candidate list a constant length. The encoder can then select the best prediction from the candidate list and send the corresponding index indicating the selected candidate. Similar to the Merge index signaling, the index of the best motion vector candidate is encoded using a truncated unary. In this case, the maximum value to be encoded is 2 (see...). Figure 8 The following sections will provide details on the derivation process of the motion vector prediction candidates.

[0110] 2.1.3.1 Derivation of AMVP Candidates

[0111] Figure 8 The derivation process of motion vector prediction candidates is summarized.

[0112] In motion vector prediction, two types of motion vector candidates are considered: spatial motion vector candidates and temporal motion vector candidates. For the derivation of spatial motion vector candidates, the final result is based on... Figure 2 The motion vectors of each PU at five different locations are shown to derive two motion vector candidates.

[0113] For temporal motion vector candidate derivation, a motion vector candidate is selected from two candidates derived based on two different juxtaposition positions. After generating the first spatiotemporal candidate list, duplicate motion vector candidates in the list are removed. If the number of potential candidates is greater than two, motion vector candidates with a reference image index greater than 1 in the associated reference image list are removed from the list. If the number of spatiotemporal motion vector candidates is less than two, additional zero motion vector candidates are added to the list.

[0114] 2.1.3.2 Candidate Spatial Motion Vectors

[0115] In the derivation of spatial motion vector candidates, from the position such as Figure 2 At most two of the five potential candidates derived from the PU at the depicted location are considered, those locations being the same as the location of the motion merge. The derivation order to the left of the current PU is defined as A0, A1, and scaled A0, scaled A1. The derivation order to the top of the current PU is defined as B0, B1, B2, scaled B0, scaled B1, scaled B2. Therefore, for each side, there are four cases that can be used as motion vector candidates, two of which do not require spatial scaling, and two of which do. The four different cases are summarized below:

[0116] • No spatial scaling

[0117] -(1) Same list of reference images, and same index of reference images (same POC)

[0118] -(2) Different lists of reference images, but the same reference image (same POC)

[0119] • Spatial scaling

[0120] -(3) Same list of reference images, but different reference images (different POCs)

[0121] -(4) Different lists of reference images, and different reference images (different POCs)

[0122] First, non-spatial scaling is checked, followed by spatial scaling. Spatial scaling is considered when the POC is not simultaneous with the reference image of the neighboring PU and the reference image of the current PU, regardless of the reference image list. If all candidate PUs on the left are unavailable or are intra-frame encoded / decoded, scaling for the upper motion vector is allowed to aid in the parallel derivation of the left and upper MV candidates. Otherwise, spatial scaling for the upper motion vector is not allowed.

[0123] like Figure 9 The described process involves scaling the motion vectors of neighboring PUs in a manner similar to temporal scaling during spatial scaling. The main difference is that a list of reference images and the index of the current PU are given as input; the actual scaling process is the same as temporal scaling.

[0124] 2.1.3.3 Candidate Motion Vectors in the Time Domain

[0125] Except for the derivation of the reference image index, all the procedures for deriving the temporal Merge candidate are the same as those for deriving the spatial motion vector candidate (see [link]). Figure 6 The reference image index is signaled to the decoder.

[0126] 2.2 A New Inter-Frame Prediction Method in JEM

[0127] 2.2.1 Motion Vector Prediction Based on Sub-CU

[0128] In a JEM with QTBT (Quadtrees plus Binary Trees), each CU can have at most one set of motion parameters for each prediction direction. Two sub-CU level motion vector prediction methods are considered in the encoder by dividing the large CU into sub-CUs and deriving the motion information of all sub-CUs of the large CU. The Optional Temporal Motion Vector Prediction (ATMVP) method allows each CU to extract multiple sets of motion information from multiple blocks smaller than the current CU in the juxtaposed reference image. In the Spatial-Temporal Motion Vector Prediction (STMVP) method, the motion vectors of sub-CUs are recursively derived using temporal motion vector predictions and spatially neighboring motion vectors.

[0129] To maintain a more accurate motion field for sub-CU motion prediction, motion compression of the reference frame is currently disabled.

[0130] 2.2.1.1 Optional temporal motion vector prediction

[0131] refer to Figure 10 In the optional Temporal Motion Vector Prediction (ATMVP) method, Temporal Motion Vector Prediction (TMVP) is modified by extracting multiple sets of motion information (including motion vectors and reference indices) from blocks smaller than the current CU. In the example, the sub-CU is an N×N square block (N is set to 4 by default).

[0132] ATMVP predicts motion vectors for sub-CUs within a CU in two steps. The first step uses so-called temporal vectors to identify corresponding blocks in a reference image, also known as the motion source image. The second step divides the current CU into sub-CUs and obtains the motion vectors from the blocks corresponding to each sub-CU, along with the reference index for each sub-CU.

[0133] In the first step, the reference image and corresponding block are determined by the motion information of the spatial neighboring blocks of the current CU. To avoid repeated scanning of neighboring blocks, the first merge candidate in the current CU's merge candidate list is used. The first available motion vector and its associated reference index are set to the index of the temporal vector and the motion source image. In this way, in ATMVP, the corresponding block can be identified more accurately than in TMVP, where the corresponding block (sometimes called the juxtaposed block) is always located in the lower right or center position relative to the current CU.

[0134] In the second step, the corresponding blocks of the sub-CUs are identified by adding temporal vectors to the coordinates of the current CU, using the temporal vectors in the motion source images. For each sub-CU, the motion information of its corresponding block (the smallest motion grid covering the center sample point) is used to derive the sub-CU's motion information. After identifying the motion information of the corresponding N×N blocks, it is converted into the motion vectors and reference indices of the current sub-CU in the same manner as the TMVP of HEVC, where motion scaling and other processes apply. For example, the decoder checks whether a low-latency condition is met (i.e., the POC of all reference images of the current image is less than the POC of the current image), and may use motion vectors MV. x (For example, the motion vector corresponding to the reference image list X) to predict the motion vector MV of each sub-CU. y (For example, where X equals 0 or 1, and Y equals 1-X).

[0135] 2.2.1.2 Spatiotemporal Motion Vector Prediction (STMVP)

[0136] In this method, the motion vector of the sub-CU is recursively derived according to the raster scan sequence. Figure 11 This concept is illustrated. Let's consider an 8×8 CU containing four 4×4 sub-CUs: A, B, C, and D. The adjacent 4×4 blocks in the current frame are labeled a, b, c, and d.

[0137] Motion derivation for sub-CU A begins by identifying its two spatial neighbors. The first neighbor is the N×N block above sub-CU A (block c). If block c is unavailable or intra-coded, the other N×N blocks above sub-CU A are checked (from left to right, starting with block c). The second neighbor is the block to the left of sub-CU A (block b). If block b is unavailable or intra-coded, the other blocks to the left of sub-CU A are checked (from top to bottom, starting with block b). Motion information obtained from neighboring blocks in each list is scaled to the first reference frame for the given list. Next, the temporal motion vector prediction (TMVP) of sub-block A is derived by following the same procedure as the TMVP derivation specified in HEVC. Motion information for the juxtaposed block at position D is extracted and scaled accordingly. Finally, after retrieving and scaling the motion information, all available motion vectors (up to 3) are averaged individually for each reference list. The averaged motion vector is assigned as the motion vector for the current sub-CU.

[0138] 2.2.1.3 Sub-CU Motion Prediction Mode Signaling

[0139] Sub-CU modes are enabled as additional Merge candidates, and no additional syntax elements are required to signal these modes. Two additional Merge candidates are added to the Merge candidate list for each CU to represent the ATMVP and STMVP modes. Up to seven Merge candidates can be used if the sequence parameter set indicates that ATMVP and STMVP are enabled. The encoding logic for additional Merge candidates is the same as that for Merge candidates in HM, meaning that for each CU in a P-strip or B-strip, two RD checks may be required for the two additional Merge candidates.

[0140] In JEM, all binary numbers (bins) in the Merge index are context-coded using CABAC. In HEVC, however, only the first binary number is context-coded, and the remaining binary numbers are context-bypass encoded.

[0141] 2.2.2 Adaptive Motion Vector Difference Resolution

[0142] In HEVC, when `use_integer_mv_flag` in the strip header is equal to 0, the Motion Vector Difference (MVD) is signaled in units of quarter-luminance samples (QFS) between the motion vector of the PU and the predicted motion vector. In JEM, Locally Adaptive Motion Vector Resolution (LAMVR) is introduced. In JEM, MVD can be encoded and decoded in units of quarter-luminance samples, integer-luminance samples, or four-luminance samples. MVD resolution is controlled at the codec unit (CU) level, and for each CU with at least one non-zero MVD component, an MVD resolution flag is conditionally signaled.

[0143] For a CU with at least one non-zero MVD component, signaling informs a first flag to indicate whether quarter-luminance sample MV precision is used in the CU. When the first flag (equal to 1) indicates that quarter-luminance sample MV precision is not used, signaling informs another flag to indicate whether integer luminance sample MV precision or four-luminance sample MV precision is used.

[0144] When the first MVD resolution flag of the CU is zero or no encoding / decoding is performed for the CU (meaning all MVDs in the CU are zero), a quarter-lumen sample MV resolution is used for the CU. When the CU uses integer lumen sample MV precision or quad lumen sample MV precision, the MVPs in the CU's AMVP candidate list are rounded to the corresponding precision.

[0145] In the encoder, CU-level RD checks are used to determine which MVD resolution should be used for the CU. That is, for each MVD resolution, three CU-level RD checks are performed. To speed up the encoder, the following encoding scheme is applied in JEM:

[0146] • During the RD check of a CU with a normal quarter-luminance sample MVD resolution, the motion information of the current CU (integer luminance sample accuracy) is stored. The stored motion information (after rounding) is used as the starting point for further small-scale motion vector refinement during the RD check of the same CU with integer luminance sample and 4-luminance sample MVD resolutions, so that the time-consuming motion estimation process is not repeated three times.

[0147] • Conditionally invoke the RD check for a CU with 4-luminance sample MVD resolution. For a CU, skip the RD check for 4-luminance sample MVD resolution if the RD cost for integer luminance sample MVD resolution is much greater than the RD cost for quarter-luminance sample MVD resolution.

[0148] The encoding process is as follows Figure 12 shown. First, the 1 / 4 pixel MV is tested, the RD cost is calculated and denoted as RDCost0. Then the integer MV is tested, and the RD cost is denoted as RDCost1. If RDCost1 < th * RDCost0 (where th is a positive value), the 4 pixel MV is tested; otherwise, the 4 pixel MV is skipped. Basically, when checking the integer or 4 pixel MV, the motion information and RD cost for the 1 / 4 pixel MV are known, and this motion information and RD cost can be reused to accelerate the encoding process of the integer or 4 pixel MV.

[0149] 2.2.3 Triangle Prediction Mode

[0150] The concept of the triangle prediction mode (TPM) is to introduce a new triangle partition for motion compensation prediction. As Figure 13A and Figure 13B shown, it divides the CU into two triangle prediction units along the diagonal or the opposite diagonal direction. Each triangle prediction unit in the CU is inter - frame predicted using its own unidirectional prediction motion vector and reference frame index, which are derived from a single unidirectional prediction candidate list. After predicting the triangle prediction units, an adaptive weighting process is performed on the diagonal edges. Then, the transform and quantization processes are applied to the entire CU. Note that this mode is only applicable to the Merge mode (including the skip mode, where the skip mode is regarded as a special Merge mode).

[0151] In this document, the triangle prediction mode (TPM) may alternatively be referred to as the triangle partition mode, triangular prediction mode, triangle segmentation mode, geometric partition mode or geometric segmentation mode.

[0152] In some embodiments, for non - TPM (also referred to as non - geometric) Merge modes, the Merge index for the Merge list is signaled. The Merge list including multiple regular Merge candidates is constructed as: spatial Merge candidates (derived from up to five spatial neighbors), temporal Merge candidates (TMVP), HMVP, pairwise average candidates and default Merge candidates (with zero MV for some assigned reference pictures). In some embodiments, the regular Merge candidates are motion candidates derived for video blocks encoded and decoded using non - geometric (non - TPM) segmentation modes.

[0153] In contrast, for TPM - encoded and decoded blocks, two Merge indices for the Merge list (with predictive encoding and decoding) are signaled.

[0154] In one case, the Merge list can be constructed in a manner similar to the non-TPM segmentation mode described above, which results in the generation of TPM motion candidates.

[0155] In another case, parity can be used to select the unidirectional predicted motion information for two partitions. The two selected Merge indices can be represented as idx0 and idx1. For each Merge candidate, if its reference image list X = (idx0 & idx1) is true, the motion information of the partition is set to the Merge candidate list X information. Otherwise, the motion information of the partition is set to the Merge candidate list Y (Y = 1 - X) information. This results in the final motion information with unidirectional prediction as TPM candidates, and the candidates in the first case are regular Merge candidates (which are generated using the Merge list construction process described for non-TPM mode codec blocks).

[0156] 2.2.3.1 One-way prediction candidate list for TPM

[0157] The unidirectional prediction candidate list, called the TPM motion candidate list, includes five unidirectional prediction motion vector candidates. For example... Figure 14 As shown, it is derived from seven neighboring blocks of the CU, including five spatial neighboring blocks (1 to 5) and two temporal juxtaposition blocks (6 to 7). Motion vectors from the seven neighboring blocks are collected and placed into a unidirectional prediction candidate list in the order of unidirectional predicted motion vector, L0 motion vector of bidirectional predicted motion vector, L1 motion vector of bidirectional predicted motion vector, and the average motion vector of L0 and L1 motion vectors of bidirectional predicted motion vector. If the number of candidates is less than five, a zero motion vector is added to the list. The motion candidates added to this TPM list are called TPM candidates, and the motion information derived from the spatial / temporal blocks is called a regular motion candidate.

[0158] More specifically, the following steps are involved:

[0159] 1) From A1, B1, B0, A0, B2, Col and Col2 (corresponding to Figure 14 Candidates for regular motion are obtained from blocks 1-7. none No trimming required .

[0160] 2) Set the variable numCurrMergeCand = 0.

[0161] 3) For each regular motion candidate derived from A1, B1, B0, A0, B2, Col, and Col2, and numCurrMergeCand less than 5, if the regular motion candidate is a uni-predicted candidate (from list 0 or list 1), it is directly added to the Merge list as a TPM candidate, and numCurrMergeCand is incremented by 1. This TPM candidate is called an "originally uni-predicted candidate".

[0162] application Fully trimmed (full pruning).

[0163] 4) For each motion candidate derived from A1, B1, B0, A0, B2, Col, and Col2, and numCurrMergeCand less than 5, if the regular motion candidate is a bidirectional prediction, the motion information from list 0 is added to the TPM Merge list as a new TPM candidate (i.e., modified to a unidirectional prediction from list 0), and numCurrMergeCand is incremented by 1. This type of TPM candidate is called a "truncated list0-predicted candidate".

[0164] application Fully trimmed .

[0165] 5) For each motion candidate derived from A1, B1, B0, A0, B2, Col, and Col2, and numCurrMergeCand less than 5, if the regular motion candidate is a bidirectional prediction, the motion information from List1 is added to the TPM Merge list (i.e., modified to a unidirectional prediction from List1), and numCurrMergeCand is incremented by 1. This type of TPM candidate is called a "Truncated List1-predicted candidate".

[0166] application Fully trimmed .

[0167] 6) For each motion candidate derived from A1, B1, B0, A0, B2, Col, and Col2, and numCurrMergeCand less than 5, if the regular motion candidate is a bidirectional prediction,

[0168] – If the strip QP of the reference image in List 0 is smaller than the strip QP of the reference image in List 1, then the motion information of List 1 is first scaled to the reference image in List 0, and the average of the two motion videos (one from the original List 0 and the other is the scaled motion video from List 1) is added to the TPM Merge list. Such a candidate is called the averaged uni-prediction from List 0 motion candidate, and numCurrMergeCand is incremented by 1.

[0169] - Otherwise, first scale the motion information of List 0 to the reference image of List 1, and add the average of the two MVs (one from the original List 1 and the other a scaled MV from List 0) to the TPM Merge list. Such TPM candidates are called averaged uni-prediction from List 1 motion candidates, and numCurrMergeCand is increased by 1.

[0170] application Fully trimmed .

[0171] 7) If numCurrMergeCand is less than 5, add a zero motion vector candidate.

[0172] 2.2.3.2 Adaptive Weighting Process

[0173] After predicting each triangular prediction unit, an adaptive weighting process is applied to the diagonal edges between two triangular prediction units to derive the final prediction for the entire CU. The two sets of weighting factors are defined as follows:

[0174] • The first weighting factor group: {7 / 8, 6 / 8, 4 / 8, 2 / 8, 1 / 8} and {7 / 8, 4 / 8, 1 / 8} are used for luminance and chrominance samples, respectively;

[0175] • The second weighting factor group: {7 / 8, 6 / 8, 5 / 8, 4 / 8, 3 / 8, 2 / 8, 1 / 8} and {6 / 8, 4 / 8, 2 / 8} are used for luminance and chrominance samples, respectively.

[0176] The weighting factor group is selected based on a comparison of the motion vectors of two triangular prediction units. The second weighting factor group is used when the reference images of the two triangular prediction units are different from each other or when the difference in their motion vectors is greater than 16 pixels. Otherwise, the first weighting factor group is used. Example: Figures 15A-15B As shown.

[0177] 2.2.3.3 Motion Vector Storage

[0178] The motion vector of the triangular prediction unit ( Figure 16A and Figure 16B Mv1 and Mv2 in the CU are stored in a 4×4 grid. For each 4×4 grid, whether to store a unidirectional or bidirectional predicted motion vector depends on the location of the 4×4 grid within the CU. Figures 16A-16B As shown, the unidirectional predicted motion vector Mv1 or Mv2 is stored in a 4×4 grid located in the unweighted region (i.e., not on the diagonal edge). On the other hand, the bidirectional predicted motion vector is stored in a 4×4 grid located in the weighted region. The bidirectional predicted motion vector is derived from Mv1 and Mv2 according to the following rules:

[0179] 1) When Mv1 and Mv2 have motion vectors from different directions (L0 or L1), Mv1 and Mv2 are simply combined to form a bidirectional predicted motion vector.

[0180] 2) When Mv1 and Mv2 both originate from the same L0 (or L1) direction,

[0181] – If the reference image for Mv2 is the same as an image in the L1 (or L0) reference image list, then Mv2 is scaled to that image. Mv1 and the scaled Mv2 are combined to form a bidirectional predicted motion vector.

[0182] – If the reference image for Mv1 is the same as an image in the L1 (or L0) reference image list, then Mv1 is scaled to that image. The scaled Mv1 and Mv2 are combined to form a bidirectional predicted motion vector.

[0183] Otherwise, only Mv1 is stored for the weighted region.

[0184] 2.2.3.4 Signaling for Triangle Prediction Mode (TPM)

[0185] First, a signaling notification can be sent indicating whether a TPM is being used, using a single bit flag. Subsequently, further signaling notification can be sent indicating two partitioning modes (e.g., ...). Figure 13A and Figure 13B The instructions (described) and the Merge index selected for each of the two partitions.

[0186] 2.2.3.4.1 Signaling of TPM Flags

[0187] Let W and H represent the width and height of a brightness block, respectively. If W*H < 64, then the triangle prediction mode is disabled.

[0188] When a block is encoded or decoded using affine mode, triangle prediction mode is also disabled.

[0189] When a block is encoded or decoded in Merge mode, a signaling bit flag can be used to indicate whether triangle prediction mode is enabled or disabled for that block.

[0190] Based on the following equation, the flag is encoded and decoded using 3 contexts (see...). Figure 17 ).

[0191] Ctx index = ((The left-hand block L is available &&L is encoded / decoded using TPM?) 1:0)

[0192] +((Is block A above encoded / decoded using TPM?) 1:0);

[0193] 2.2.3.4.2 Two partitioning modes (e.g.) Figures 13A-13B The instructions (described) and the signaling for selecting the Merge index for each of the two partitions.

[0194] Note that the partitioning pattern and the merge indexes of the two partitions are jointly encoded and decoded. In the example, the two partitions are restricted from using the same reference index. Therefore, there are 2(partitioning pattern)*N(maximum number of merge candidates)*(N-1) possibilities, where N is set to 5. An indicator is encoded and decoded, and the mapping between the partitioning pattern, the two merge indices, and the encoding / decoding indicator is derived from the array defined below:

[0195] const uint8_t g_TriangleCombination[TRIANGLE_MAX_NUM_CANDS][3]={{0,1,0},{1,0,1},{1,0,2},{0,0,1},{0,2,0},{1 ,0,3},{1,0,4},{1,1,0},{0,3,0},{0,4,0},{0,0,2},{0,1,2},{1,1,2},{0,0,4},{0,0,3},{0,1,3},{0,1,4} ,{1,1,4},{1,1,3},{1,2,1},{1,2,0},{0,2,1},{0,4,3},{1,3,0},{1,3,2},{1,3,4},{1,4,0},{1,3,1},{1,2,3},{1,4,1},{0,4,1},{0,2,3},{1,4,2},{0,3,2},{1,4,3},{0,3,1},{0,2,4},{1,2,4},{0,4,2},{0,3,4}};

[0196] Triangle mode (45 degrees or 135 degrees) = g_TriangleCombination[signaling notification instruction][0];

[0197] Merge index of candidate A = g_TriangleCombination[signaling notification indication];

[0198] Merge index of candidate B = g_TriangleCombination[signaling notification indication];

[0199] Once two motion candidates A and B are derived, motion information for two partitions (PU1 and PU2) can be set based on either A or B. Whether PU1 uses the motion information of merged candidate A or B depends on the prediction directions of the two motion candidates. Table 1 shows the relationship between the two derived motion candidates A and B and the two partitions.

[0200] Table 1: Motion information of partitions derived from the two Merge candidates (A, B)

[0201]

[0202]

[0203] 2.2.3.4.3 indicates entropy encoding / decoding (represented by merge_triangle_idx).

[0204] merge_triangle_idx The range is [0, 39]. K-order Exponential Golomb (EG) codes are used for binarization of merge_triangle_idx, where K is set to 1.

[0205] K-order EG

[0206] To encode larger numbers with fewer bits (at the cost of encoding smaller numbers with more bits), this can be achieved using... non-negative integers Let the parameter k summarize this. To encode a non-negative integer x using k-order exponential Golomb codes:

[0207] 1. Using the above-mentioned zero-order exponent Golomb codes Encode, then

[0208] 2. Using binary representation of x mod 2 k Encode

[0209] Table 2: Examples of Exponential Columbus-k Encoding / Decoding

[0210]

[0211] 2.2.3.5 Latest Developments in TPM

[0212] The regular Merge candidate list is reused for triangle segmentation Merge prediction without additional motion vector pruning. For each Merge candidate in the regular Merge candidate list, one and only one of its L0 or L1 motion vectors is used for triangle prediction. Furthermore, the order of selection of the L0 pair (vs.) L1 motion vectors is based on the parity of their Merge index. Using this scheme, the regular Merge list can be used directly.

[0213] Once the rule merge list is constructed, for each merge candidate in the rule merge candidate list, one and only one of its L0 or L1 motion vectors is used for triangle prediction. Furthermore, the order in which the L0 and L1 motion vectors are selected is based on the parity of their merge index. More specifically, as... Figure 18C As shown, for candidates with even-valued indices, their L0 motion vector is first selected for triangle prediction. If unavailable, their L1 motion vector is used instead. For candidates with odd-valued indices, their L1 motion vector is first selected for triangle prediction. If unavailable, their L0 motion vector is used instead. Figure 18C In the process, for each Merge index, the motion vector marked with "x" is first selected for triangle prediction.

[0214] 7.3.7.7 Merge Data Syntax

[0215]

[0216]

[0217] 2.2.4 Overlapping Block Motion Compensation

[0218] Overlapping Block Motion Compensation (OBMC) was previously used in H.263. In JEM, unlike H.263, OBMC can be enabled and disabled using CU-level syntax. When using OBMC in JEM, it is performed for all Motion Compensation (MC) block boundaries except for the right and bottom boundaries of the CU. Furthermore, it is applied to the luma and chroma components. In JEM, MC blocks correspond to codec blocks. When a CU is encoded or decoded using subCU modes (including subCU Merge, affine, and FRUC (Frame Rate Up Conversion) modes), each sub-block of the CU is an MC block. To handle CU boundaries uniformly, OBMC is performed at the sub-block level for all MC block boundaries, where the sub-block size is set to equal to 4×4, such as... Figures 18A-18B As shown.

[0219] When OBMC is applied to the current sub-block, in addition to the current motion vector, the motion vectors of four adjacent sub-blocks (if available and different from the current motion vector) are used to derive the prediction block for the current sub-block. These multiple prediction blocks based on multiple motion vectors are combined to generate the final prediction signal for the current sub-block.

[0220] The predicted block based on the motion vectors of neighboring sub-blocks is represented as P. N , where N represents the index of the adjacent top, bottom, left, and right sub-blocks, and the predicted block based on the motion vector of the current sub-block is represented as P. C When P N When based on motion information that includes motion information of neighboring sub-blocks that is identical to that of the current sub-block, OBMC does not originate from P. N Executed. Otherwise, P will be executed. N Each sample point is added to P C Among the similar points in the middle, namely P N Four rows / columns were added to P C In the middle. Weighting factors {1 / 4, 1 / 8, 1 / 16, 1 / 32} are used for P. N And the weighting factors {3 / 4, 7 / 8, 15 / 16, 31 / 32} are used for P. C An exception is small MC blocks (i.e., when the height or width of the codec block is equal to 4 or the CU is encoded / decoded using sub-CU mode). For such blocks, only P... N Two rows / columns are added to P C In this case, the weighting factors {1 / 4, 1 / 8} are used for P. N And the weighting factors {3 / 4, 7 / 8} are used for P. C For P generated based on the motion vectors of vertical (horizontal) neighboring sub-blocks N , will P N Samples in the same row (column) are added to P with the same weighting factor. C .

[0221] In JEM, for CUs with a size of 256 lumen samples or less, a signaling flag is sent to the CU level to indicate whether OBMC has been applied to the current CU. For CUs with a size greater than 256 lumen samples or not encoded / decoded using AMVP mode, OBMC is applied by default. At the encoder, when OBMC is applied to the CU, its effects are taken into account during the motion estimation phase. The predicted signal formed by OBMC using motion information from the upper and left neighboring blocks is used to compensate for the upper and left boundaries of the original signal of the current CU, and then the normal motion estimation process is applied.

[0222] 2.2.5 Local lighting compensation

[0223] Local Illumination Compensation (LIC) is based on a linear model of illumination variation, using a scaling factor 'a' and an offset 'b'. It is adaptively enabled or disabled for each inter-frame mode codec's codec unit (CU).

[0224] When LIC is applied to CU, the least squares error method is used to derive parameters a and b by using the nearest samples of the current CU and their corresponding reference samples. More specifically, as Figure 19 As shown, neighboring and corresponding samples (identified by motion information of the current CU or sub-CU) of the CU in the reference image are used for subsampling (2:1 subsampling). IC parameters are derived individually and applied to each prediction direction.

[0225] When encoding and decoding the CU in Merge mode, the LIC flag is copied from the neighboring block in a manner similar to motion information copying in Merge mode; otherwise, the CU signaling notifies the LIC flag, indicating whether the LIC is applicable.

[0226] When LIC is enabled for an image, an additional CU-level RD check is required to determine whether LIC is applied to the CU. When LIC is enabled for a CU, the Mean-Removed Sum of Absolute Difference (MR-SAD) and the Mean-Removed Sum of Absolute Hadamard-Transformed Difference (MR-SATD) (instead of SAD and SATD) are used for integer pixel motion search and fractional pixel motion search, respectively.

[0227] To reduce coding complexity, the following coding scheme is applied in JEM.

[0228] • When there is no significant lighting change between the current image and its reference images, LIC is disabled for the entire image. To identify this situation, histograms of the current image and each reference image of the current image are calculated at the encoder. If the histogram difference between the current image and each reference image of the current image is less than a given threshold, LIC is disabled for the current image; otherwise, LIC is enabled for the current image.

[0229] 2.2.6 Affine Motion Compensation Prediction

[0230] In HEVC, only the translational motion model is applied to motion compensation prediction (MCP). However, in the real world, many types of motion exist, such as zooming in / out, rotation, perspective motion, and other irregular motions. In JEM, a simplified affine transformation motion compensation prediction is applied. For example... Figures 20A-20B As shown, the affine motion field of the block is described by two control point motion vectors.

[0231] The motion vector field (MVF) of a block is described by the following equation:

[0232]

[0233] For 6-parameter affine

[0234]

[0235] Where (v 0x ,v 0y ) is the motion vector of the top left control point, and (v 1x ,v 1y ) is the motion vector of the upper right control point, and (v 2x ,v 2y (x, y) is the motion vector of the bottom left control point, and (x, y) represents the coordinates of the representative point within the current block relative to the top left sample point. In VTM, the representative point is defined as the center position of the sub-block. For example, when the coordinates of the top left corner of the sub-block relative to the top left sample point within the current block are (xs, ys), the coordinates of the representative point are defined as (xs+2, ys+2).

[0236] To further simplify motion compensation prediction, a sub-block-based affine transformation prediction was applied. The sub-block size M×N is derived from Equation 2, where MvPre is the fractional precision of the motion vector (1 / 16 in JEM), (v 2x ,v 2y ) is the motion vector of the lower left control point calculated according to Equation 1.

[0237]

[0238] After deriving from Equation 2, M and N should be adjusted downwards if necessary, so that they are divisors of w and h, respectively.

[0239] To derive the motion vector for each M×N sub-block, such as Figures 20A-20BAs shown, the motion vector of the center sample point of each sub-block is calculated according to Equation 1 and rounded to a fractional precision of 1 / 16. Then, the motion-compensated interpolation filter mentioned in Section 2.2.3 is applied to generate the prediction for each sub-block using the derived motion vector.

[0240] Figure 21 An example of the affine motion vector field MVF for each sub-block is shown.

[0241] After MCP, the high-precision motion vector of each sub-block is rounded and saved with the same precision as the normal motion vector.

[0242] 2.2.6.1 AF_INTER mode

[0243] In JEM, there are two affine motion modes: AF_INTER mode and AF_MERGE mode. AF_INTER mode can be applied to CUs with both width and height greater than 8. Signaling in the bitstream informs the CU-level affine flag to indicate whether AF_INTER mode is used. In this mode, neighboring blocks are used to construct motion vector pairs {(v0, v1) | v0 = {v1, v2, v3, v4, v5, v6, v7, v8, v9, v1, v1, v1, v1, v2 ... A v B v c}, v1={v D v E The candidate list. For example... Figures 22A-22B As shown, v0 is selected from the motion vectors of block A, block B, or block C. The motion vectors from neighboring blocks are scaled based on the reference list and the relationship between the reference POC of neighboring blocks, the reference POC of the current block, and the POC of the current CU. The method for selecting v1 from neighboring blocks D and E is similar. If the number of candidates in the candidate list is less than two, the list is populated by motion vector pairs formed by copying each AMVP candidate. When the candidate list is greater than two, the candidates are first sorted according to the consistency of neighboring motion vectors (the similarity between the two motion vectors in a candidate pair), and only the top two candidates are retained. An RD cost check is used to determine which motion vector pair candidate is selected as the Control Point Motion Vector Prediction (CPMVP) for the current CU. The index of the CPMVP position in the candidate list is indicated by signaling in the bitstream. After determining the CPMVP of the current affine CU, affine motion estimation is applied to find the Control Point Motion Vector (CPMV). Then, the difference between the CPMV and CPMVP is indicated by signaling in the bitstream.

[0244] Figure 23 An example of the motion vector prediction MVP for AF_INTER is shown.

[0245] In AF_INTER mode, when using the 4 / 6 parameter affine mode, 2 / 3 control points are required, and therefore 2 / 3 MVDs need to be encoded and decoded for these control points, such as... Figures 22A-22B As shown. In existing implementations, it is proposed to derive MV in the following manner: predict mvd1 and mvd2 from mvd0.

[0246]

[0247]

[0248]

[0249] in, mvd i mv1 and mv1 are the predicted motion vector, motion vector difference, and motion vector of the top-left pixel (i=0), top-right pixel (i=1), or bottom-left pixel (i=2), respectively. Figure 22B As shown. Note that the sum of two motion vectors (e.g., mvA(xA,yA) and mvB(xB,yB)) is equal to the sum of the two components individually, i.e., newMV = mvA + mvB, where the two components of newMV are set as (xA + xB) and (yA + yB).

[0250] 2.2.6.2 AF_MERGE Mode

[0251] When CU is applied in AF_MERGE mode, it obtains the first block encoded / decoded in affine mode from the valid nearest reconstructed blocks. The selection order of candidate blocks is from left, top, top right, bottom left to top left, as follows: Figure 24A As shown. If the adjacent lower left block A is encoded and decoded in affine mode, as... Figure 24B As shown, the motion vectors v2, v3, and v4 of the upper left, upper right, and lower left corners of the CU containing block A are derived. Then, the motion vector v0 of the upper left corner of the current CU is calculated based on v2, v3, and v4. Next, the motion vector v1 of the upper right corner of the current CU is calculated.

[0252] After deriving the CPMV v0 and v1 of the current CU, the MVF of the current CU is generated according to Equation 1 of the simplified affine motion model. To identify whether the current CU is encoded or decoded in AF_MERGE mode, an affine flag is signaled in the bitstream when at least one neighboring block is encoded or decoded in affine mode.

[0253] In some implementations, the affine Merge candidate list is constructed by the following steps:

[0254] Affine candidate for insertion inheritance

[0255] Inherited affine candidates refer to those derived from the affine motion models of their effective neighboring affine codec blocks. In the common foundation, such as... Figures 24A-24B As shown, the scanning order of the candidate positions is: A1, B1, B0, A0, and B2.

[0256] After deriving candidates, a full pruning process is performed to check if the same candidate has already been inserted into the list. If the same candidate exists, the derived candidate is discarded.

[0257] Affine candidates constructed by insertion

[0258] If the number of candidates in the affine Merge candidate list is less than MaxNumAffineCand (set to 5 in this draft), the constructed affine candidate is inserted into the candidate list. The constructed affine candidate is one that is built by combining the nearest motion information of each control point.

[0259] The motion information of the control points first comes from Figures 24A-24B The derivation is based on the specified spatial and temporal neighbors shown. CPk (k = 1, 2, 3, 4) represents the k-th control point. A0, A1, A2, B0, B1, B2, and B3 are the predicted spatial locations of CPk (k = 1, 2, 3); T is the predicted temporal location of CP4.

[0260] The coordinates of CP1, CP2, CP3 and CP4 are (0,0), (W,0), (H,0) and (W,H) respectively, where W and H are the width and height of the current block.

[0261] Figure 25 An example of candidate positions for the affine Merge pattern is shown.

[0262] Motion information for each control point is obtained according to the following priority order:

[0263] For CP1, the check priority is B2->B3->A2. If B2 is available, then B2 is used. Otherwise, if B2 is available, then B3 is used. If neither B2 nor B3 is available, then A2 is used. If all three candidates are unavailable, motion information for CP1 cannot be obtained.

[0264] For CP2, the inspection priority is B1->B0.

[0265] For CP3, the inspection priority is A1->A0.

[0266] For CP4, use T.

[0267] Secondly, combinations of control points are used to construct affine Merge candidates.

[0268] Constructing a 6-parameter affine candidate requires motion information from three control points. These three control points can be selected from four combinations: {CP1, CP2, CP4}, {CP1, CP2, CP3}, {CP2, CP3, CP4}, and {CP1, CP3, CP4}. The combinations {CP1, CP2, CP3}, {CP2, CP3, CP4}, and {CP1, CP3, CP4} will be transformed into a 6-parameter motion model represented by the top-left, top-right, and bottom-left control points.

[0269] Constructing a 4-parameter affine candidate requires motion information from two control points. These two control points can be selected from the following six combinations ({CP1,CP4}, {CP2,CP3}, {CP1,CP2}, {CP2,CP4}, {CP1,CP3}, {CP3,CP4}). The combination {CP1,CP4}, {CP2,CP3}, {CP2,CP4}, {CP1,CP3}, and {CP3,CP4} will be converted into a 4-parameter motion model represented by the top-left and top-right control points.

[0270] The combinations of constructed affine candidates are inserted into the candidate list in the following order:

[0271] {CP1,CP2,CP3}, {CP1,CP2,CP4}, {CP1,CP3,CP4}, {CP2,CP3,CP4}, {CP1,CP2}, {CP1,CP3}, {CP2,CP3}, {CP1,CP4}, {CP2,CP4}, {CP3,CP4}.

[0272] For a combined reference list X (X is 0 or 1), the reference index with the highest usage rate among the control points is selected as the reference index of list X, and the motion vector pointing to the difference reference image is scaled.

[0273] After candidates are derived, a full pruning process is performed to check if the same candidate has already been inserted into the list. If the same candidate exists, the derived candidate is discarded.

[0274] Fill with zero motion vector

[0275] If the number of candidates in the affine Merge candidate list is less than 5, a zero motion vector with a zero reference index is inserted into the candidate list until the list is full.

[0276] 2.2.7 Bidirectional optical flow

[0277] Bidirectional optical flow (BIO) is a sample-level motion refinement that builds upon block-level motion compensation used for bidirectional prediction. Sample-level motion refinement does not use signaling.

[0278] Figure 26 An example of an optical flow trajectory is shown.

[0279] Assume I (k) It is the brightness value from reference k (k=0,1) after block motion compensation, and They are I (k) The horizontal and vertical components of the gradient. Assuming optical flow is effective, the motion vector field (v...) x ,v y The equation gives the following:

[0280]

[0281] By combining this optical flow equation with Hermite interpolation, the motion trajectory of each sample point is obtained, ultimately yielding the result with respect to the function value I. (k) and derivative The only matching third-order polynomial. The value of this polynomial at t=0 is the BIO prediction:

[0282]

[0283] Here, τ0 and τ1 represent the distances to the reference frame, such as... Figure 28 As shown. Distances τ0 and τ1 are calculated based on the POC of Ref0 and Ref1: τ0 = POC(current) - POC(Ref0), τ1 = POC(Ref1) - POC(current). If two predictions come from the same time direction (either both from the past or both from the future), the signals are different (i.e., τ0·τ1 < 0). In this case, BIO is applied only when the predictions do not come from the same time (i.e., τ0 ≠ τ1), both reference regions have non-zero motion (MVx0, MVy0, MVx1, MVy1 ≠ 0), and the block motion vector is proportional to the time distance (MVx0 / MVx1 = MVy0 / MVy1 = -τ0 / τ1).

[0284] By minimizing points A and B ( Figure 26 The difference Δ between the values ​​of the points where the upper motion trajectory intersects the reference frame plane and the upper motion trajectory plane is used to determine the motion vector field (v). x ,v y The model uses only the first linear term of the local Taylor expansion for Δ:

[0285]

[0286] All values ​​in Equation 5 depend on the sample point location (i′, j′), which has been ignored from the notation so far. Assuming the motion is consistent in the local surrounding region, we minimize Δ within a (2M+1)×(2M+1) square window Ω centered at the current predicted point (i, j), where M equals 2:

[0287]

[0288] For this optimization problem, JEM uses a simplified approach, first minimizing in the vertical direction and then minimizing in the horizontal direction. This yields:

[0289]

[0290]

[0291] in,

[0292]

[0293] To avoid division by zero or very small values, regularization parameters r and m are introduced into equations 7 and 8.

[0294] r = 500·4 d-8 (10)

[0295] m = 700·4 d-8 (11)

[0296] Here, d is the bit depth of the video sample.

[0297] To maintain consistency between BIO memory access and rule-based bidirectional prediction motion compensation, all prediction and gradient values... All calculations are performed only within the current block. In Equation 9, the (2M+1)×(2M+1) square window Ω centered on the current prediction point on the boundary of the prediction block needs to access positions outside the block (e.g., ...). Figure 27A (As shown). In JEM, outside the block The value is set to be equal to the nearest available value within the block. For example, this can be implemented as padding, such as... Figure 27B As shown.

[0298] Using BIO, it is possible to refine the motion field for each sample point. To reduce computational complexity, a block-based BIO design is used in JEM. Motion refinement is calculated based on 4×4 blocks. In block-based BIO, s in Equation 9 aggregates all samples in the 4×4 block. n value, then s nThe aggregated values ​​are used to derive the BIO motion vector offset for a 4×4 block. More specifically, the following formula is used for block-based BIO derivation:

[0299]

[0300] Where b k Let represent the sample set belonging to the k-th 4×4 block of the prediction block. s in equations 7 and 8 n by ((s) n,bk )>>4) instead, to derive the associated motion vector offset.

[0301] In some cases, MV refinement in BIO may be unreliable due to noise or irregular motion. Therefore, in BIO, the magnitude of MV refinement is limited to a threshold thBIO. The threshold is determined based on whether all reference images of the current image come from the same direction. If all reference images of the current image come from the same direction, the threshold is set to 12×2. 14-d Otherwise, it is set to 12×2 13-d .

[0302] The gradient of the BIO is computed simultaneously with motion compensation interpolation using the same operation as the HEVC motion compensation process (2D separable FIR). The input to this 2D separable FIR is a reference frame sample point that is identical to the motion compensation process and the fractional position (fracX, fracY) based on the fractional part of the block motion vector. In the horizontal gradient... In this case, the signal is first vertically interpolated using BIOfilterS corresponding to the fractional position fracY with a de-scaling offset of d-8, and then a gradient filter BIOfilterG corresponding to the fractional position fracX with a de-scaling offset of 18-d is applied in the horizontal direction. In the vertical gradient... In this case, the gradient filter is first applied vertically using BIOfilterG, corresponding to the fractional position fracY with a descaling offset of d-8, and then the signal shift is performed horizontally using BIOfilterS, corresponding to the fractional position fracX with a descaling offset of 18-d. The interpolation filter BIOfilterG and the signal shifting filter BIOfilterF used for gradient calculation are relatively short (6 taps) to maintain reasonable complexity. Table 3 shows the filters used for gradient calculation at different fractional positions of the block motion vector in BIO. Table 4 shows the interpolation filters used for predictive signal generation in BIO.

[0303] Table 3: Filters used for gradient calculation in BIO

[0304] Fractional pixel position Gradient interpolation filter (BIOfilterG) 0 {8,-39,-3,46,-17,5} 1 / 16 {8,-32,-13,50,-18,5} 1 / 8 {7,-27,-20,54,-19,5} 3 / 16 {6,-21,-29,57,-18,5} 1 / 4 {4,-17,-36,60,-15,4} 5 / 16 {3,-9,-44,61,-15,4} 3 / 8 {1,-4,-48,61,-13,3} 7 / 16 {0,1,-54,60,-9,2} 1 / 2 {-1,4,-57,57,-4,1}

[0305] Table 4: Interpolation filters used for predictive signal generation in BIO

[0306] Fractional pixel position Interpolation filters for predicted signals (BIOfilters) 0 {0,0,64,0,0,0} 1 / 16 {1,-3,64,4,-2,0} 1 / 8 {1,-6,62,9,-3,1} 3 / 16 {2,-8,60,14,-5,1} 1 / 4 {2,-9,57,19,-7,2} 5 / 16 {3,-10,53,24,-8,2} 3 / 8 {3,-11,50,29,-9,2} 7 / 16 {3,-11,44,35,-10,3} 1 / 2 {3,-10,35,44,-11,3}

[0307] In JEM, BIO is applied to all bidirectional prediction blocks when two predictions come from different reference images. BIO is disabled when LIC is enabled for CU.

[0308] In JEM, OBMC is applied to blocks following the normal MC process. To reduce computational complexity, BIO is not applied during the OBMC process. This means that when using its own MV, BIO is only applied to the MC process of the block, while when using the MV of an adjacent block during the OBMC process, BIO is not applied to the MC process.

[0309] 2.2.8 Decoder-side motion vector refinement

[0310] In bidirectional prediction, for the prediction of a block region, two prediction blocks formed using motion vectors (MV) from list 0 and MV from list 1 are combined to form a single prediction signal. In the decoder-side motion vector refinement (DMVR) method, the two bidirectional prediction motion vectors are further refined through a bilateral template matching process. Bilateral template matching is applied in the decoder to perform a distortion-based search between the bilateral templates and reconstructed samples in the reference image to obtain the refined MV without sending additional motion information.

[0311] In DMVR, two-sided templates are generated from the initial MV0 of list 0 and MV1 of list 1 as a weighted combination (i.e., average) of the two prediction blocks, such as... Figure 28 As shown. The template matching operation involves calculating a cost metric between the generated template and the sample region (around the initial prediction block) in the reference image. For each of the two reference images, the MV that produces the minimum template cost is considered the updated MV in the list to replace the original MV. In JEM, nine MV candidates are searched for each list. The nine candidate MVs include the original MV and eight surrounding MVs, one of which is a brightness sample offset relative to the original MV in the horizontal or vertical direction, or both. Finally, two new MVs (i.e., as shown) are selected. Figure 28 The MV0′ and MV1′ shown are used to generate the final bidirectional prediction result. The sum of absolute differences (SAD) is used as the cost metric. Note that when calculating the cost of a prediction block generated from a surrounding MV, the rounded MV (rounded to the integer pixel) is actually used to obtain the prediction block, not the actual MV.

[0312] DMVR is applied to bidirectional prediction merge patterns, where one MV comes from a past reference picture and the other from a future reference picture, without transmitting additional syntax elements. In JEM, DMVR is not applied when LIC, affine motion, FRUC, or subCU merge candidates are enabled for a CU.

[0313] 2.2.9 Sub-block Merge Candidate List

[0314] It is recommended to put all sub-block-related motion candidates, except for the rule-based merge list used for non-sub-block merge candidates, into a separate merge list.

[0315] Motion candidates related to sub-blocks are placed in a separate Merge list, which is called the "sub-block Merge Candidate List".

[0316] In one example, the sub-block Merge candidate list includes affine Merge candidates, ATMVP candidates, and / or sub-block-based STMVP candidates.

[0317] 2.2.9.1 Another example of constructing an affine Merge list

[0318] In this version, the ATMVP Merge candidates in the normal Merge list are moved to the first position in the affine Merge list. This ensures that all Merge candidates in the new list (i.e., the sub-block-based Merge candidate list) are based on the sub-block encoding / decoding tool.

[0319] 3. Examples of problems overcome by the implementation plan

[0320] In a triangular partitioning design, a block can be divided into two partitions. To save storage bandwidth due to motion compensation, unidirectional prediction is required for both partitions. During the motion compensation process, bidirectional prediction is used for the diagonal edges, and unidirectional prediction is used for all the remaining parts. If bidirectional prediction is allowed for each partition, samples located at the diagonal edges will have four MVs, two from one partition and two from the other. This design has the following problems:

[0321] 1) The design of the one-way prediction merge list depends on the reference image list, meaning that MVs associated with reference images from reference image list 0 have higher priority. That is, TPM candidates derived from all one-way prediction regular motion candidates are inserted before TPM candidates derived from all two-way prediction regular motion candidates. However, typically, the location of spatially neighboring blocks has a stronger correlation in terms of the similarity of MV information.

[0322] 2) Allowing two partitions to select different TPM candidate indexes, however, using the same Merge candidate list for both is not optimal, as the two partitions may have different relevance to the current block's neighboring blocks.

[0323] 3) The maximum number of TPM candidates is fixed at 5, which is at least unfriendly to software design.

[0324] 4) The weighting factor group is predefined for all types of blocks.

[0325] 5) Motion information is stored in a 4x4 format, which increases memory requirements. How to compress it is a problem that needs to be studied.

[0326] 6) TPM motion information depends on the parity of the two decoded Merge indices. For candidates with even-valued indices, their L0 motion vector is selected first for triangle prediction. If unavailable, their L1 motion vector is used instead. For candidates with odd-valued indices, their L1 motion vector is selected first for triangle prediction. If unavailable, their L0 motion vector is used instead. Therefore, two candidates may come from the same list of reference images.

[0327] 4. Example Implementation

[0328] The proposed method can be applied to any non-square / non-rectangular segmentation, such as geometric segmentation. In the following description, we use "triangle segmentation pattern" to represent an example of non-square / non-rectangular segmentation pattern (TPM), and motion vector prediction candidates inserted into the TPM candidate list are referred to as "TPM candidates." Motion information associated with any previously encoded / decoded block is referred to as "regular motion candidates." It should be noted that other types of segmentation are also applicable.

[0329] The detailed list of techniques below should be considered as examples to illustrate general concepts. These techniques should not be interpreted narrowly. Furthermore, these techniques can be combined in any way.

[0330] Construction process of TPM candidate lists (AMVP / Merge / Others) with different orders

[0331] 1. Instead of always inserting TPM candidates derived from all unidirectional prediction rule motion candidates before those derived from all bidirectional prediction rule motion candidates, a method is proposed that uses prediction direction as a priority criterion to determine which TPM candidates should be added first.

[0332] a. In one example, all regular motion candidates with unidirectional predictions from list X are inserted as TPM candidates into the candidate list before candidates with unidirectional predictions from list Y.

[0333] b. In one example, all TPM candidates derived from bidirectional predictive rule motion candidates (e.g., truncated list 0 predictive candidates, truncated list 1 predictive candidates) are given higher priority, meaning they can be inserted before TPM candidates derived from unidirectional predictive rule motion candidates.

[0334] c. In one example, the order is defined as: all TPM candidates derived from the bidirectional prediction rule motion candidate list X (e.g., truncated list X prediction candidates), all TPM candidates derived from the bidirectional prediction rule motion candidate list Y (e.g., truncated list Y prediction candidates), and all TPM candidates derived from the unidirectional prediction rule motion candidates (e.g., original unidirectional prediction candidates).

[0335] d. In one example, a TPM candidate derived from a bidirectional predictive rule motion candidate can be added to the list before a TPM candidate derived from another bidirectional predictive rule motion candidate.

[0336] e. In one example, a TPM candidate derived from a bidirectional predictive rule motion candidate can be added together with a TPM candidate derived from another bidirectional predictive rule motion candidate in an interleaved manner.

[0337] i. When there are two regular motion candidates C A and C B And when there are four TPM candidates, including those from C A and C B The derived truncation list of 0 / 1 prediction candidates is used. The order in which the list is added can be defined as: from C... A The derived truncation list predicts 0 candidates from C B The derived truncation list 1 predicts candidates from C A The derived truncation list 1 predicts candidates from C B The derived truncation list predicts 0 candidates.

[0338] ii. Alternatively, when there are two regular motion candidates C A and C B And when there are four TPM candidates, including those from C A and C B The derived truncation list 0 / 1 predicts candidates.

[0339] The order in which lists are added can be defined as: starting from C A The derived truncation list 1 predicts candidates from C B The derived truncation list predicts 0 candidates from C A The derived truncation list predicts 0 candidates from C B The derived truncation list 1 predicts candidates.

[0340] 2. Instead of always inserting TPM candidates derived from all unidirectional predictive regular motion candidates before those derived from all bidirectional predictive regular motion candidates, a method is proposed that uses encoding / decoding mode information associated with the regular motion candidates as a priority criterion to determine which TPM candidates should be added first.

[0341] a. Encoding and decoding information may include AMVP or Merge mode.

[0342] i. In one example, if a regular motion candidate C A It is derived from block A, and another block C. B It is derived from block B, and block A is encoded and decoded using AMVP mode, while block B is encoded and decoded using Merge mode, so it is derived from C. A The derived TPM candidates can be found in C B The derived TPM candidates are added to the list beforehand. Alternatively, from C... B The derived TPM candidates can be found in C A The derived TPM candidates are added to the list beforehand.

[0343] b. Encoding and decoding information may include a reference index and / or a point of interest (POC) difference.

[0344] i. In one example, with another TPM candidate C' B In contrast, if a TPM candidate C' A Associated with a smaller reference index, then C' A Can be found in C' B It was previously added to the list.

[0345] ii. In one example, with another TPM candidate C' B In contrast, if a TPM candidate C' A Associated with the POC distance between a smaller reference image and the current image, C' A Can be found in C' B It was previously added to the list.

[0346] c. Encoding and decoding information may include the image / strip / piece group quantization parameter (QP) of the reference image and / or the temporal layer index of the reference image.

[0347] i. In one example, with another TPM candidate C' B In contrast, if a TPM candidate C' A When compared with a reference image having a smaller QP, then C' A Can be found in C' B It was previously added to the list.

[0348] 3. Instead of always inserting TPM candidates derived from all unidirectional prediction rule motion candidates before those derived from all bidirectional prediction rule motion candidates, a method is proposed that uses the position where the motion candidate is derived as a priority criterion to determine which TPM candidates should be added first.

[0349] a. In one example, if a regular motion candidate C A It is derived from block A, and another C. B It is derived from block B, and B is checked after A, so it starts from C. A Derivation (e.g., if C) A Is it a one-way prediction, or each of the two TPM candidates is derived from C? A Those TPM motion candidates that are copied from a related prediction direction (and directly inherited) can be found in block C. B The candidates for derivation are added to the list beforehand.

[0350] b. Alternatively, location and a list of reference images (or predicted orientation) can also be used as prioritization criteria. For example, if a regular motion candidate C A It is derived from block A, and another C. B It is derived from block B, and B is checked after A, then C is derived from list X. A The derived TPM motion candidates can be found in list X, C. B The derived TPM motion candidates were previously added to the list. From list Y, C A The derived TPM motion candidates can be found in list Y of C. B The derived TPM motion candidates were previously added to the list. X equals 0 or 1, and Y equals (1–X).

[0351] 4. It is proposed to add all truncated list 1 prediction candidates before truncated list 0 prediction candidates.

[0352] a. Alternatively, the order of the truncated list 0 prediction candidates and the truncated list 1 prediction candidates can be adaptively changed, for example, based on the available TPM motion candidates.

[0353] i. In one example, if the first TPM motion candidate is the original unidirectional prediction candidate with unidirectional predictions from list 1, then the truncated list 1 prediction candidate can be added before the truncated list 0 prediction candidate.

[0354] ii. Alternatively, if the original one-way prediction candidates come more from list 1, then truncating list 1 prediction candidates can be added before truncating list 0 prediction candidates.

[0355] b. Alternatively, all TPM candidates derived from a bidirectional predictive motion candidate (e.g., a truncated list 0 predictive candidate and a truncated list 0 predictive candidate) can be added to the list before all TPM candidates derived from another bidirectional predictive motion candidate.

[0356] i. Alternatively, for those TPM candidates derived from a motion candidate, the order in which they are added may further depend on the information available for the TPM motion candidates in the list, e.g., 4.a.

[0357] Build process for a TPM candidate list with more TPM candidates (AMVP / Merge / Others)

[0358] 5. Multiple average unidirectional prediction motion candidates derived from list 0 and list 1 from a single regular motion candidate can all be added to the TPM motion candidate list.

[0359] a. Alternatively, only one of them may be added, and which one to add depends on the information available for TPM candidates in the list, e.g., 4.a.

[0360] b. Alternatively, only one of them can be added, and which one is added depends on the reference image index of both prediction lists. The motion vector associated with the smaller reference image index will be retained, and the final prediction direction will be associated with such a reference image.

[0361] c. Alternatively, only one of them can be added, and which one is added depends on the POC distance between the reference image and the current image in both prediction lists. The motion vector associated with the smaller POC distance will be preserved, and the final predicted direction will be associated with such a reference image.

[0362] d. In one example, all motion candidates from the average one-way predictions of list 0 can be inserted before all motion candidates from the average one-way predictions of list 1. Alternatively, all motion candidates from the average one-way predictions of list 1 can be inserted before all motion candidates from the average one-way predictions of list 0.

[0363] 6. It is proposed that the average one-way prediction TPM candidate can be derived using the one-way prediction rule motion candidate.

[0364] a. One-way prediction rule motion candidates from list LX can be used to generate average one-way prediction candidates for list LX, for example, together with two-way prediction rule motion candidates or together with other one-way prediction rule motion candidates from list X to generate average one-way prediction candidates for list LX.

[0365] b. One-way predictive regular motion candidates from list LX can first be scaled to LY (Y = 1-X), and the scaled motion vector can be used to generate average one-way predictive candidates for list LY, for example, together with two-way predictive regular motion candidates or together with other one-way predictive regular motion candidates from list X.

[0366] 7. Virtual TPM motion candidates derived from available TPM motion candidates can also be added to the TPM candidate list.

[0367] a. In one example, after a certain step (e.g., after generating the truncated list 1 predicted candidates), dummy candidates can be derived from those candidates that have already been added to the Merge list.

[0368] b. In one example, a virtual candidate can be derived solely from the original one-way predictive candidate.

[0369] c. In one example, a dummy candidate can be derived from only the original one-way predicted candidate and the truncated list 1 predicted candidate.

[0370] d. In one example, a virtual candidate can be derived from either a candidate with a specific reference image index (i.e., equal to 0) and / or a POC distance within a specific range between the reference image and the current image, or from a list of reference images.

[0371] e. In one example, a dummy candidate can be derived only from the first K TPM motion candidates, for example, K is set to 1.

[0372] i. For example, the MV of two TPM motion candidates can be averaged to obtain virtual candidates.

[0373] f. For its motion vector is determined by C mv The representation and reference index are provided by C. refidx The TPM exercise schedule is indicated

[0374] The virtual can be derived using the following methods:

[0375] i. To C mv Add offsets to the horizontal or vertical components, and use the same C refidx .

[0376] ii. To C mv Add offsets to the horizontal and vertical components, and use the same C... refidx .

[0377] iii. Place C mv Scale to C refidx Another reference image not pointed to. Using scaled motion vectors and a reference image (where C... mvThe index of the scaled-up index is used as a new TPM motion candidate.

[0378] g. When adding a virtual TPM candidate, pruning may not be applied.

[0379] i. Alternatively, pruning can be applied to candidates, excluding those from which virtual TPM candidates are generated.

[0380] Construction process of TPM candidate list (AMVP / Merge / Others) with adaptive insertion order

[0381] 8. The insertion order (priority criterion) of TPM candidates can be changed from sequence to sequence, picture to picture, strip to strip, slice group to slice group, block to block, etc.

[0382] a. In one example, it can depend on the block size / block shape / partitioning pattern.

[0383] b. Alternatively, the sequence can be signaled from the encoder to the decoder in VPS / SPS / PPS / strip head / piece group head / piece / CTU / CU.

[0384] Multiple TPM lists of partitions in a TPM codec block

[0385] 9. Instead of using a single list for all partitions, a separate list of motion candidates can be built for each geometric partition.

[0386] a. In one example, for one partition, its associated candidate list contains only motion candidates predicted from list X, while for another partition, its associated candidate list contains only motion candidates predicted from list Y, where Y is not equal to X.

[0387] b. In one example, different spatial and / or temporal blocks can be accessed to derive motion candidates to be added to the Merge list.

[0388] c. In one example, the location of the spatial and / or temporal blocks of a geometric partition can depend on the location of the partition.

[0389] d. In one example, the location of the spatial and / or temporal blocks of a geometric partition can depend on the partitioning method (from top left to bottom right (45 degrees) or from top right to bottom left (135 degrees)).

[0390] e. In one example, for a 135-degree division pattern (such as...) Figures 13A-13B (As depicted), you can check more of the blocks above to build the Merge list for PU1. Figure 29 The image depicts some examples of the blocks shown above (marked in gray).

[0391] f. In one example, for a 135-degree division pattern (such as...) Figures 13A-13B (As depicted), you can examine more blocks on the left to build the Merge list for PU2. Figure 29 The image depicts some examples of the blocks on the left (marked in gray).

[0392] g. In one example, for a 45-degree division pattern (such as...) Figures 13A-13B (As depicted), you can examine more blocks on the left and above to build the Merge list for PU1. Figure 29 The image depicts some examples of the blocks on the left and the blocks above (marked in gray).

[0393] h. In one example, for a 45-degree division pattern (such as...) Figures 13A-13B (As depicted), more temporal blocks can be examined to build the Merge list for PU2.

[0394] i. In one example, the index of the TPM list for each partition signaling notification.

[0395] 10. Multiple TPM lists can be built and shared for all partitions, and each partition can select one of the TPM lists (e.g., each partition can select the same TPM list from multiple TPM lists).

[0396] a. In one example, the index of the TPM list can be signaled first. Then, the index of the TPM candidate can be signaled further.

[0397] b. In another example, the index of the TPM list and the index of the TPM candidates can be jointly encoded and decoded.

[0398] Maximum allowed signaling notifications for TPM candidates

[0399] 11. Maximum allowed TPM candidates for Merge / AMVP in signaling notifications within SPS / VPS / PPS / Image Header / Strip Header / Piece Group Header / LCU Line / LCU Group.

[0400] a. Alternatively, the maximum allowed TPM motion candidates can be set for non-TPM codec blocks, such as the non-TPMMerge candidate list size or the sub-block Merge candidate list size.

[0401] b. The maximum quantity is binary-coded using unary encoding or truncated unary encoding during signaling notification.

[0402] c. Instead of the maximum number of direct signaling notifications, you can use signaling notification (M - maximum number), for example, M = 5 or 6.

[0403] d. A signaling flag can be used to indicate whether the maximum number of TPM candidates is the same as the maximum number of regular motion candidates / sub-block motion candidates.

[0404] Weighted factor group

[0405] 12. The choice of weighting factor group can depend on the width and / or height of the current block.

[0406] a. In one example, if the ratio between width and height is greater than a threshold, a specific weighting factor group (e.g., group 1) can be selected.

[0407] b. In one example, if the ratio between height and width is greater than a threshold, a specific weighting factor group (e.g., group 1) can be selected.

[0408] c. Multiple sets of weighting factor groups can be predefined, and for different block widths and / or heights, one or two weighting factor groups can be selected.

[0409] 13. The weighting factor can be changed in real time (on-the-fly) based on the location of the sample points.

[0410] a. In one example, it can depend on the angle of the edge that separates the two partitions.

[0411] b. In one example, the weighting factor can be defined as a function of the sample location.

[0412] c. In one example, the final prediction block of a TPM codec block is set to (f(x,y)*P1(x,y)+(2 M –f(x,y))*P2(x,y)+offset)>>M, where P1 and P2 are two predicted values ​​of the sample point located at (x,y), M is an integer value, the offset can be set to (1<<(M-1)), and f(x,y) is a function that returns the weights applied to the first predicted value.

[0413] 14. For blocks located in weighted regions, motion compensation is performed at the 8×8 level instead of the 4×4 level to reduce storage bandwidth.

[0414] a. Alternatively, for blocks located in weighted regions, motion compensation is performed at an 8×4 or 4×8 level instead of a 4×4 level to reduce storage bandwidth.

[0415] b. Alternatively, if motion compensation based on 8×4 or 4×8 is permitted, the choice between 8×4 and 4×8 may depend on...

[0416] i. Block width and / or height.

[0417] ii. Division pattern (e.g., 45 degrees or 135 degrees).

[0418] Storage of motion information

[0419] 15. After decoding the TPM codec block, one or more HMVP tables can be updated.

[0420] a. In one example, one or more HMVP tables can be updated based on the motion information of one partition (excluding the motion information of other partitions).

[0421] b. In one example, one or more HMVP tables can be updated based on stored motion information for weighted regions that do not include unweighted regions or unweighted regions that do not include weighted regions.

[0422] c. In one example, one or more HMVP tables can be updated based on a Merge candidate from a Merge candidate list used to derive motion information for TPM codec blocks.

[0423] i. In one example, a Merge candidate is associated with the Merge index that decodes the Merge index (e.g., merge_triangle_idx0 or merge_triangle_idx1).

[0424] ii. In one example, a Merge candidate is associated with a Merge index that is equal to k (e.g., k = 0).

[0425] 16. The derivation of triangular motion information for one partition can depend on decoded motion information from another partition, for example, based on the predicted direction of another partition.

[0426] d. In one example, suppose a first list of reference images (denoted as LX) with an index equal to merge_triangle_idx0 is selected as a decoding candidate (e.g., based on the parity of merge_triangle_idx0 and the availability of LX). Regardless of the parity of the second decoding candidate index, the motion information associated with the second list of reference images (denoted as LY) (Y = 1-X) of the second candidate (if available) is selected as the motion information for the triangular partition. In the discussion below, the motion information associated with the reference image lists LX or LY may also be referred to as LX motion information or LY motion information.

[0427] i. Alternatively, if the second candidate LY (Y = 1 - X) motion information with merge_triangle_idx1 is not available, the second candidate LX motion information may be used instead.

[0428] ii. Alternatively, if the second candidate LY (Y = 1 - X) motion information with merge_triangle_idx1 is unavailable, the second candidate LX motion information can be used to derive the LY motion information. The derived LY motion information can then be used for TPM encoding / decoding.

[0429] 1) In one example, the derivation process may include scaling the motion vector of the reference image in reference LX to the reference image in LY.

[0430] e. In one example, suppose the first reference image list (denoted as LX) of the decoding candidate with an index equal to merge_triangle_idx1 is selected (e.g., based on the parity of merge_triangle_idx1 and the availability of LX). Regardless of the parity of the second decoding candidate index, the motion information LY (Y = 1 - X) is selected as the motion information for the triangle partition (if available).

[0431] i. Alternatively, if candidate LY (Y = 1 - X) motion information with merge_triangle_idx0 is not available, LX motion information can be used instead.

[0432] ii. Alternatively, if candidate LY (Y = 1-X) motion information with merge_triangle_idx0 is unavailable, candidate LX motion information with merge_triangle_idx0 can be used to derive the LY motion information. The derived LY motion information can then be used for TPM encoding / decoding.

[0433] 1) In one example, the derivation process may include scaling the motion vector of the reference image in reference LX to the reference image in LY.

[0434] 17. For multiple triangular partitions, motion information from the same list of reference images is not allowed.

[0435] f. In one example, the conformance bitstream should satisfy the prediction of two triangular partitions from different lists of reference images.

[0436] g. Alternatively, the derivation process of motion information based on the parity of the Merge index can always output two partitions predicted from two different lists, for example, using the method disclosed in bullet point 15.

[0437] 18. How to store motion information and / or how to derive the motion information to be stored for weighted and / or unweighted regions may depend on the division direction of the triangle partition and / or the decoding Merge index, and / or the Merge candidate list used to derive the motion information of the triangle partition, and / or the reference images in the reference image list (e.g., all reference images have smaller or no larger POC values ​​compared to the current image).

[0438] a. In one example, motion information of Merge candidates associated with a selected decoded Merge index of a partition can be stored.

[0439] i. In one example, the selected decode Merge index can be set to one of the decode Merge candidate indices, such as merge_triangle_idx0 or merge_triangle_idx1.

[0440] ii. In one example, the selected decoding Merge index is set to the variable k.

[0441] 1) In one example, k is set to 0.

[0442] 2) In one example, all information of the k-th Merge candidate can be inherited and stored.

[0443] 3) In one example, partial information of the kth Merge candidate can be inherited, and the remaining information can be modified before storage.

[0444] (a) In one example, the indication of the weighting factor used in the generalized-biprediction (GBi) of the k-th Merge candidate may not be inherited. Alternatively, a specific GBi weighting index may be assigned (e.g., 0) to the motion information stored in the TPM.

[0445] (b) In one example, the indication of motion vector accuracy of the kth Merge candidate may not be inherited.

[0446] iii. Alternatively, the selected decoding merge index may also depend on the partitioning direction.

[0447] iv. In one example, if the Merge candidate whose index is equal to the selected decode Merge index is a bidirectional prediction candidate, then bidirectional prediction motion information can be stored.

[0448] v. In one example, if the Merge candidate whose index is equal to the selected decoded Merge index is a one-way prediction, then the one-way prediction motion information can be stored.

[0449] 1) Alternatively, bidirectional predictive motion information can be stored, and the stored motion information can be derived from the Merge candidates.

[0450] vi. Alternatively, a function can be used to store motion information about the two partitions.

[0451] 1) For example, the average value of the motion vectors of two partitions can be stored.

[0452] b. In one example, assuming MvInfo1 and MvInfo2 indicate motion information for two partitions, virtual bidirectional predicted motion information can be stored by modifying a prediction direction from LX in MvInfoA (A is 1 or 2) to LY (Y = 1-X).

[0453] i. In one example, the motion vector and reference index of MvInfoA remain unchanged.

[0454] ii. In one example, in addition, the reference index of MvInfoA remains unchanged, while the motion vector of MvInfoA is set to the opposite value.

[0455] iii. Virtual bidirectional predictive motion information may include modified MvInfoA and unmodified MvInfoB (where B = 3 - A).

[0456] iv. In one example, this modification can be applied only if MvInfo1 and MvInfo2 both come from the same prediction direction.

[0457] c. In one example, assuming MvInfo1 and MvInfo2 indicate motion information for two partitions, how the motion information is stored may depend on a low-latency check flag, which indicates whether all reference images have smaller or no larger POC values ​​compared to the current image.

[0458] i. In one example, the following procedures are applied sequentially:

[0459] If MvInfo1 and MvInfo2 come from different lists of reference images, then MvInfo0 and MvInfo1 can be combined and stored as bidirectional predictive motion information.

[0460] ii. If MvInfo1 and MvInfo2 come from the same LX (X = 0 or 1) direction, the following procedure can be applied:

[0461] If the low-latency check flag is true (e.g., all reference images have smaller or no larger POC values ​​compared to the current image), MvInfoA can be modified (e.g., A=2) by setting the predicted direction list from LX to LY (Y=1-X), and the modified MvInfoA and the unmodified MVInfoC (C=3-A) can be combined to form virtual bidirectional predicted motion information and stored.

[0462] Otherwise, store MvInfoB (e.g., B=2). That is, store unidirectional predicted motion information.

[0463] iii. In one example, MvInfo1 and Figure 13A PU1 and Figure 13B PU1 is associated with it; MvInfo2 is associated with it. Figure 13A PU2 and Figure 13B It is associated with PU2.

[0464] d. In one example, the above method can be applied to store motion information of certain sub-blocks within a block.

[0465] i. In one example, some sub-blocks could be those sub-blocks within a weighted region.

[0466] ii. In one example, certain sub-blocks can be those sub-blocks within a block that contain diagonals or anti-diagonals.

[0467] iii. In one example, some sub-blocks can be those sub-blocks in the bottom right corner of the block.

[0468] iv. In one example, some sub-blocks can be those sub-blocks in the right column or bottom row of the block.

[0469] v. Alternatively, the above method can be applied to store motion information of all sub-blocks within a block.

[0470] 19. For the TPM model, it is proposed that the motion information of sub-regions within a block can be different from the motion information used in the motion compensation process during the reconstruction of that sub-region.

[0471] a. In one example, for an M×N sub-region located in a weighted region (e.g., 4×4 in the current codec unit using the triangle prediction mode), motion compensation can be accomplished as bidirectional prediction, but only one set of motion information from list 0 or list 1 of the bidirectional prediction can be stored.

[0472] i. Alternatively, for an M×N sub-region located in the weighted region (e.g., 4×4 in the current codec unit using the triangular prediction mode), motion compensation can be performed as bidirectional prediction, but only unidirectional prediction information from list X can be stored, and the stored information is different from the motion information from list X used in the motion compensation process.

[0473] ii. Alternatively, for an M×N sub-region located in the weighted region (e.g., 4×4 in the current codec unit using the triangle prediction mode), motion compensation can be performed as bidirectional prediction, but bidirectional prediction information that is different from the information used in the motion compensation process (e.g., different MVs and / or different reference pictures) can be stored.

[0474] b. In one example, for an M×N sub-region located in an unweighted region (e.g., 4×4 in the current codec unit using the triangle prediction mode), motion compensation can be done as a one-way prediction, but two-way prediction motion information can be stored.

[0475] i. Alternatively, for an M×N sub-region located in an unweighted region (e.g., 4×4 in the current codec unit using a triangular prediction mode), motion compensation can be performed as a one-way prediction, but one-way prediction motion information can be stored, and this one-way prediction motion information can be different from the information used in the motion compensation process.

[0476] c. The stored motion information can be used to encode and decode other blocks (e.g., as a spatial motion candidate in the Merge / AMVP mode of neighboring blocks).

[0477] d. The stored motion information can be used to encode and decode future blocks in different images (e.g., to derive temporal motion vector candidates).

[0478] e. The stored motion information can be used for in-loop processing, such as de-blocking or ALF (adaptive loop filtering).

[0479] f. In one example, the motion information of the first sub-region within the block may be different from the motion information used in the motion compensation process for reconstructing the first sub-region, while the motion information of the second sub-region within the block may be the same as the motion information used in the motion compensation process for reconstructing the second sub-region, and the first and second sub-regions may be located in the same codec unit utilizing the triangular prediction mode.

[0480] 20. A method is proposed to store the same set of motion information for the entire block, regardless of whether the block is located in a weighted region. Assume that MvInfo1 and MvInfo2 indicate the motion information of two partitions (e.g., based on two Merge candidate indices derived from decoding).

[0481] a. In one example, one-way predicted motion information can be stored (e.g., motion information inherited from one of the two partitions or derived from the motion information of both partitions).

[0482] i. In one example, MvInfo1 can be stored for the entire block.

[0483] 1. In one example, MvInfo1 can be stored for all sub-regions within a block.

[0484] ii. In one example, MvInfo2 can be stored for the entire block.

[0485] 1. In one example, MvInfo1 can be stored for all sub-regions within a block.

[0486] iii. In one example, which partition's motion information to store may depend on the POC distance relative to the current image.

[0487] 1. For example, if the absolute value of the POC distance between the current image and the reference image referenced by MvInfo1 is less than the POC distance between the current image and the reference image referenced by MvInfo2, then MvInfo1 can be stored.

[0488] iv. In one example, which partition's motion information to store may depend on the QP of the reference image.

[0489] 1. For example, if the QP of the reference image referenced by MvInfo1 is less than the value of the reference image referenced by MvInfo2, then MvInfo1 can be stored.

[0490] v. In one example, which partition's motion information to store can depend on the reference index of the reference image.

[0491] 1. For example, if the QP of the reference image referenced by MvInfo1 is less than the value of the reference image referenced by MvInfo2, then MvInfo1 can be stored.

[0492] 2. For example, if the reference index of the reference image referenced by MvInfo1 is less than the reference index value of the reference image referenced by MvInfo2, then MvInfo1 can be stored.

[0493] vi. In one example, which partition's motion information is stored can depend on the Merge index associated with a partition.

[0494] 1. For example, if the associated Merge index of the deduced MvInfo1 is less than the associated Merge index of the deduced MvInfo2, then MvInfo1 can be stored.

[0495] 2. For example, if merge_triangle_idx0 is less than merge_triangle_idx1, motion information derived from merge_triangle_idx0 can be stored.

[0496] vii. In one example, a third set of motion information (denoted as MvInfo3) derived from MvInfo1 and MvInfo2 can be stored.

[0497] 1. In one example, the MV of MvInfo3 can be derived as the average of the two MVs of MvInof1 and MvInfo2, or the MV of MvInfo3 can be generated by averaging the motion vectors of one MV and the other scaled or mapped from MvInof1 and MvInfo2.

[0498] 2. In one example, the reference image for MvInfo3 can be one of the two reference images for MvInfo1 and MvInfo2.

[0499] viii. In one example, a motion vector in MvInfoX (where X is 0 or 1) can be scaled to a reference image in MvInfoY (where Y is 1-X), and then it can be used to derive the motion information to be stored.

[0500] 1. In one example, scaled MvInfo1 and MvInfo2 can be used to deduce motion information to be stored, such as the motion information specified in bullet point vii.

[0501] ix. The above method may only be applicable to sub-regions located within the weighted region.

[0502] 1. Alternatively, the above method may be applied only to sub-regions located in unweighted regions.

[0503] 2. Alternatively, the above method may be applied only to certain sub-regions located within the weighted region.

[0504] a. For example, they can be applied to sub-regions located in the upper right and / or lower left of the weighted region.

[0505] b. For example, they can be applied to the upper left and / or lower right sub-regions of the weighted region.

[0506] c. Which sub-region to apply to can depend on the division direction.

[0507] b. In one example, bidirectional predicted motion vectors are derived from and stored from MvInfo1 and MvInfo2.

[0508] i. When MvInfo1 and MvInfo2 have motion vectors from different directions (L0 or L1), MvInfo1 and MvInfo2 are simply combined to form a bidirectional predicted motion vector.

[0509] ii. When MvInfo1 and MvInfo2 both originate from the same LX (X = 0 or 1) direction,

[0510] 1. The stored MV for LX can be derived from one motion vector of MvInfo1 and MvInfo2 or from both of their motion vectors (e.g., by averaging).

[0511] 2. If a reference image of MvInfo1 or MvInfo2 is included in LY (Y = 1 - X), then MvInfo1 and MvInfo2 are simply combined to form a bidirectional predicted motion vector, and one of the predicted directions is set to LY.

[0512] 3. A motion vector can be scaled to a reference image in LY (Y = 1 - X), and the scaled MV and another MV are combined to form a bidirectional predicted motion vector. In one example, the target reference image in LY referenced by the scaled MV can be predefined (e.g., reference image index equal to 0) or can be signaled.

[0513] 4. A motion vector can be mapped to a reference image in LY (Y = 1 - X), and the mapped MV, along with another MV, is combined to form a bidirectional predicted motion vector. In one example, the target reference image in the LY referenced by the mapped MV can be predefined (e.g., reference image index equal to 0) or can be signaled. In one example, the mapping process can be performed without scaling. For example, the mapped MV can be equal to or opposite to the original MV.

[0514] i. The above method only applies to sub-regions located within the weighted region.

[0515] 1) Alternatively, the above method may be applicable only to sub-regions located in unweighted regions.

[0516] 2) Alternatively, the above method may be applicable only to certain sub-regions located within the weighted region.

[0517] (a) For example, they can be applied to subregions located in the upper right and / or lower left of the weighted region.

[0518] (b) For example, they can be applied to sub-regions located in the upper left and / or lower right of the weighted region.

[0519] (c) Which subregion to apply to can depend on the division direction.

[0520] c. Whether to store unidirectional or bidirectional predictive motion information depends on the decoded motion information of the two partitions.

[0521] i. In one example, if MvInfo1 and MvInfo2 both come from the same LX reference list, unidirectional predicted motion information can be stored.

[0522] ii. In one example, if MvInfo1 and MvInfo2 come from different reference lists (one from L0 and the other from L1), bidirectional predictive motion information can be stored.

[0523] iii. In one example, if MvInfo1 and MvInfo2 both come from the same LX reference list, but at least one of the reference images referenced by MvInfo1 or MvInfo2 is also in another reference list LY (Y = 1-X), then bidirectional predictive motion information can be stored.

[0524] d. Alternatively, motion information is still stored based on whether a sub-block is located within a weighted region. For those sub-blocks located within a weighted region, the stored motion information is derived according to the following rules:

[0525] i. In one example, if MvInfo1 and MvInfo2 come from the same LX (X = 0 or 1) direction,

[0526] 1. If a reference image of MvInfo1 or MvInfo2 is included in LY (Y = 1-X), then MvInfo1 and MvInfo2 are simply combined to form a bidirectional predicted motion vector, and one of the predicted directions is set to LY.

[0527] 2. A motion vector can be scaled to a reference image in LY (Y = 1 - X), and the scaled MV and another MV are combined to form a bidirectional predicted motion vector. In one example, the target reference image in LY referenced by the scaled MV can be predefined (e.g., reference image index equal to 0) or can be signaled.

[0528] 3. A motion vector can be mapped to a reference image in LY (Y = 1 - X), and the mapped MV, along with another MV, is combined to form a bidirectional predicted motion vector. In one example, the target reference image in the LY referenced by the mapped MV can be predefined (e.g., reference image index equal to 0) or can be signaled. In one example, the mapping process can be performed without scaling. For example, the mapped MV can be equal to or the reverse of the original MV.

[0529] ii. In one example, if MvInfo1 or MvInfo2 comes from the same LX (X = 0 or 1) direction, then MvInfo1 or MvInfo2 is stored. That is, unidirectional predicted motion vectors are stored.

[0530] iii. In one example, if MvInfo1 and MvInfo2 come from the same LX (X = 0 or 1) direction, one motion vector can be scaled to a reference image of the other motion vector, and the average or weighted average of the scaled motion vector and the other motion vector can be stored.

[0531] iv. In one example, if MvInfo1 and MvInfo2 come from the same LX (X = 0 or 1) direction, and the average or weighted average of the two motion vectors can be stored, then the reference image can be one of the two reference images of MvInfo1 and MvInfo2.

[0532] v. In one example, the stored motion information for the entire block can be derived from the motion information in each sub-block. For example, a weighted sum of the motion values ​​(MVs) in all or some sub-blocks can be used to derive the stored MV for the entire block.

[0533] vi. In the example above, different sub-regions can store different motion information.

[0534] vii. For the example above, sub-regions within the weighted region can store unidirectional or bidirectional prediction motion information.

[0535] e. Alternatively, motion information can still be stored based on the location of sub-blocks. However, for each sub-block, only unidirectional predicted motion information can be stored.

[0536] i. In one example, motion information of subregions in a weighted region can be inherited or derived from MvInfo1 and / or MvInfo2, and can be stored.

[0537] f. In one example, the stored motion information is only used in some modules.

[0538] i. In one example, the stored motion information is used for temporal motion prediction.

[0539] ii. Alternatively, the stored motion information is also used for spatial motion prediction.

[0540] iii. Alternatively, the stored motion information can also be used in a filtering (e.g., deblocking) process.

[0541] g. In one example, the stored motion information can be used in the block's motion compensation process.

[0542] h. In one example, one or more HMVP tables can be updated based on the stored motion information.

[0543] 21. A method for storing motion information based on M×N blocks (where M and N cannot both be equal to 4) is proposed. Each M×N block shares the same motion information.

[0544] a. In one example, M and N are set to 8.

[0545] b. In one example, for an M×N block, if part of it belongs to a weighted region and part of it belongs to an unweighted region, then such a block follows the rules of weighted regions for motion vector storage.

[0546] c. Alternatively, for an M×N block, if its first part belongs to a weighted region but its second part belongs to an unweighted region, then such a block can store motion information according to the rules of the unweighted region.

[0547] 22. Whether TPM is enabled or disabled for two partitions to be predicted from the same reference image can be done under conditions such as the motion vectors of the two partitions should be sufficiently different.

[0548] a. In one example, the reference samples of the two partitions should not overlap.

[0549] b. In one example, abs(MV0[0]–MV1[0]) should be less than TH, where MV0 and MV1 are the motion vectors of the two partitions, and MVX[0] and MVX[1] are the horizontal and vertical components of MVX, respectively. The function abs(x) returns the absolute value of x.

[0550] c. In one example, abs(MV0[1]–MV1[1]) should be less than TH.

[0551] d. In one example, abs(MV0[0]–MV1[0])+abs(MV0[1]–MV1[1]) should be less than TH.

[0552] e. In one example, Max(abs(MV0[0]–MV1[0]), abs(MV0[1]–MV1[1]) should be less than TH, where the function Max(x,y) returns the larger of x and y.

[0553] f. A consistent bitstream should satisfy one or more of the above bullet points being true when TPM is enabled for a block.

[0554] 23. In the disclosed method, the stored motion information can be used to predict motion information of blocks to be subsequently decoded in the current image or other images to be decoded.

[0555] a. The stored motion information may include:

[0556] vii. Motion Vector

[0557] vii. Reference Index

[0558] ix. Indications for unidirectional or bidirectional forecasting

[0559] x. Indicator of inter-frame prediction direction

[0560] xi. Indicators of Generalized Bidirectional Forecasting (GBi)

[0561] xii. Motion Vector Resolution

[0562] xiii. Indicators of affine prediction

[0563] 5. Exemplary embodiments of the disclosed technology

[0564] 5.1 Example #1

[0565] Motion vector storage procedure for triangle Merge pattern

[0566] The variables numSbX and numSbY specify that the number of 4×4 blocks in the horizontal and vertical directions in the current codec block is set to equal numSbX = cbWidth >> 2 and numSbY = cbHeight >> 2.

[0567] cbWidth and cbHeight specify the width and height of the current codec block in the luminance sample, and the variable minSb is set to min(numSbX,numSbY)-1.

[0568] The derivation of the variable cbRatio is as follows:

[0569] cbRatio=(cbWidth>cbHeight)? (cbWidth / cbHeight):(cbHeight / cbWidth)

[0570] For each 4×4 sub-block at sub-block index (xSbIdx, ySbIdx), where xSbIdx = 0…numSbx-1 and ySbIdx = 0…numSbY-1, the following case applies:

[0571] The derivation of variables xidX and yIdx is as follows:

[0572] xIdx=(cbWidth>cbHeight)? (xSbIdx / cbRatio):xSbIdx

[0573] yIdx=(cbWidth>cbHeight)? ySbIdx:(ySbIdx / cbRatio)

[0574] – The derivation of the variable sType is as follows:

[0575] – If triangleDir equals 0, then the following applies:

[0576] sType=(xIdx==yIdx)? 2((xIdx>yIdx)?0:1)

[0577] – Otherwise (triangleDir equals 1), the following applies:

[0578] sType=(xIdx+yIdx==minSb)? 2:((xIdx+yIdx <minSb)?0:1)

[0579] The triangleDir parameter specifies the segmentation direction.

[0580] like Figures 30A-30C As shown, sType equals 0, which corresponds to region P1; sType equals 1, which corresponds to region P2; and sType equals 2, which corresponds to the weighted region.

[0581] The motion information of region P1 is represented as (Mv1, refIdx1); the motion information of region P2 is represented as (Mv2, refIdx2).

[0582] – Depending on the value of sType, the following assignments are made:

[0583] – If sType equals 0, then the motion information of the 4×4 sub-block is (Mv1, refIdx1).

[0584] Otherwise, if sType equals 1 or sType equals 2, and if Mv1 and Mv2 both come from the same reference list, and the current block is in a strip with a backward reference picture (slice.getCheckLDC() is false), then the motion information of the 4×4 sub-block is (Mv2, refIdx2).

[0585] Otherwise (sType equals 2), the following applies:

[0586] refIdxL0=(predListFlagA==0)? refIdx1:refIdx2

[0587] refIdxL1=(predListFlagA==0)? refIdx2:refIdx1

[0588] mvL0=(predListFlagA==0)? Mv1:Mv2

[0589] mvL1=(predListFlagA==0)? Mv2:Mv1

[0590] predListFlagA is the prediction list flag for region P1.

[0591] 5.2 Example #2

[0592] For each 4×4 sub-block at sub-block index (xSbIdx, ySbIdx), where xSbIdx = 0…numSbx-1 and ySbIdx = 0…numSbY-1, the following applies:

[0593] – Depending on the value of sType, the following assignments are made:

[0594] – If sType equals 0, then the motion information of the 4×4 sub-block is (Mv1, refIdx1).

[0595] Otherwise, if sType equals 1, the motion information of the 4×4 sub-block is (Mv2, refIdx2).

[0596] Otherwise (sType equals 2), the following applies:

[0597] refIdxL0=(predListFlagA==0)? refIdx1:refIdx2

[0598] refIdxL1=(predListFlagA==0)? refIdx2:refIdx1

[0599] mvL0=(predListFlagA==0)? Mv1:Mv2

[0600] mvL1=(predListFlagA==0)? Mv2:Mv1

[0601] predListFlagA is the prediction list flag for region P1.

[0602] 5.3 Example #3

[0603] For each 4×4 sub-block at sub-block index (xSbIdx, ySbIdx), where xSbIdx = 0…numSbx-1 and ySbIdx = 0…numSbY-1, the following applies:

[0604] – Depending on the value of sType, the following assignments are made:

[0605] – If sType equals 0, then the motion information of the 4×4 sub-block is (Mv1, refIdx1).

[0606] Otherwise, if sType equals 1 or sType equals 2, and if Mv1 and Mv2 both come from the same reference list, then the motion information for the 4×4 sub-block is (Mv2, refIdx2).

[0607] Otherwise (sType equals 2), the following applies:

[0608] –If Mv1 and Mv2 come from different reference lists, then

[0609] refIdxL0=(predListFlagA==0)? refIdx1:refIdx2

[0610] refIdxL1=(predListFlagA==0)? refIdx2:refIdx1

[0611] mvL0=(predListFlagA==0)? Mv1:Mv2

[0612] mvL1=(predListFlagA==0)? Mv2:Mv1

[0613] predListFlagA is the prediction list flag for region P1.

[0614] 5.4 Example #4

[0615] For each 4×4 sub-block at sub-block index (xSbIdx, ySbIdx), where xSbIdx = 0…numSbx-1 and ySbIdx = 0…numSbY-1, the following applies:

[0616] – Depending on the value of sType, the following assignments are made:

[0617] – If sType equals 0 or sType equals 2, and if Mv1 and Mv2 both come from the same reference list, and the current block is in a strip with a backward reference picture (slice.getCheckLDC() is false), then the motion information of the 4×4 sub-block is (Mv1, refIdx1).

[0618] Otherwise, if sType equals 1, the motion information of the 4×4 sub-block is (Mv2, refIdx2).

[0619] Otherwise (sType equals 2), the following applies:

[0620] refIdxL0=(predListFlagA==0)? refIdx1:refIdx2

[0621] refIdxL1=(predListFlagA==0)? refIdx2:refIdx1

[0622] mvL0=(predListFlagA==0)? Mv1:Mv2

[0623] mvL1=(predListFlagA==0)? Mv2:Mv1

[0624] predListFlagA is the prediction list flag for region P1.

[0625] 5.5 Example #5

[0626] For each 4×4 sub-block at sub-block index (xSbIdx, ySbIdx), where xSbIdx = 0…numSbx-1 and ySbIdx = 0…numSbY-1, the following applies:

[0627] – Depending on the value of sType, the following assignments are made:

[0628] – If sType equals 0 or sType equals 2, and if Mv1 and Mv2 both come from the same reference list, then the motion information for the 4×4 subblock is (Mv1, refIdx1).

[0629] Otherwise, if sType equals 1, the motion information of the 4×4 sub-block is (Mv2, refIdx2).

[0630] Otherwise (sType equals 2), the following applies:

[0631] –If Mv1 and Mv2 come from different reference lists, then

[0632] refIdxL0=(predListFlagA==0)? refIdx1:refIdx2

[0633] refIdxL1=(predListFlagA==0)? refIdx2:refIdx1

[0634] mvL0=(predListFlagA==0)? Mv1:Mv2

[0635] mvL1=(predListFlagA==0)? Mv2:Mv1

[0636] predListFlagA is the prediction list flag for region P1.

[0637] 5.6 Example #6

[0638] For each 4×4 sub-block at sub-block index (xSbIdx, ySbIdx), where xSbIdx = 0…numSbx-1 and ySbIdx = 0…numSbY-1, the following applies:

[0639] – Depending on the value of sType, the following assignments are made:

[0640] – If sType equals 0, then the motion information of the 4×4 sub-block is (Mv1, refIdx1).

[0641] Otherwise, if sType equals 1, the motion information of the 4×4 sub-block is (Mv2, refIdx2).

[0642] Otherwise (sType equals 2), the following applies:

[0643] –If Mv1 and Mv2 come from different reference lists, then

[0644] refIdxL0=(predListFlagA==0)? refIdx1:refIdx2

[0645] refIdxL1=(predListFlagA==0)? refIdx2:refIdx1

[0646] mvL0 = (predListFlagA == 0)? Mv1 : Mv2

[0647] mvL1 = (predListFlagA == 0)? Mv2 : Mv1

[0648] predListFlagA is the prediction list flag for region P1.

[0649] – Otherwise, the motion information for the 4x4 block is ((Mv1 + Mv2) / 2, refIdx1)

[0650] 5.7 Example 7

[0651] For each 4x4 block at sub-block index (xSbIdx, ySbIdx) where xSbIdx = 0…numSbx - 1 and ySbIdx = 0…numSbY - 1, the following applies:

[0652] – Depending on the value of sType, the following assignments are made:

[0653] – If sType equals 0, the motion information for the 4x4 block is (Mv1, refIdx1).

[0654] – Otherwise, if sType equals 1, the motion information for the 4x4 block is (Mv2, refIdx2).

[0655] – Otherwise (sType equals 2), the motion information for the 4x4 block is the motion information of the original Merge candidate corresponding to merge_triangle_idx0.

[0656] 5.8 Example #8

[0657] For each 4x4 block at sub-block index (xSbIdx, ySbIdx) where xSbIdx = 0…numSbx - 1, ySbIdx = 0…numSbY - 1, the following applies:

[0658] – Depending on the value of sType, the following assignments are made:

[0659] – If sType equals 0 or sType equals 2, and if both Mv1 and Mv2 are from the same reference list, and triangleDir equals 0 and ySbIdx < numSbY - 1, the motion information for the 4x4 block is (Mv1, refIdx1).

[0660] Otherwise, if sType equals 1 or sType equals 2, and if Mv1 and Mv2 both come from the same reference list, and triangleDir equals 0 and ySbIdx = numSbY – 1 or triangleDir equals 1, then the motion information of the 4×4 sub-block is (Mv2, refIdx2).

[0661] Otherwise (sType equals 2), the following applies:

[0662] –If Mv1 and Mv2 come from different reference lists

[0663] refIdxL0=(predListFlagA==0)? refIdx1:refIdx2

[0664] refIdxL1=(predListFlagA==0)? refIdx2:refIdx1

[0665] mvL0=(predListFlagA==0)? Mv1:Mv2

[0666] mvL1=(predListFlagA==0)? Mv2:Mv1

[0667] predListFlagA is the prediction list flag for region P1.

[0668] 5.9 Example #9

[0669] For each 4×4 sub-block at sub-block index (xSbIdx, ySbIdx), where xSbIdx = 0…numSbx-1 and ySbIdx = 0…numSbY-1, the following applies:

[0670] – If Mv1 and Mv2 both come from the same reference list, and the current block is in a strip with a backward reference picture (slice.getCheckLDC() is false), then the motion information of the 4×4 sub-block is (Mv2, refIdx2).

[0671] Otherwise, the following applies:

[0672] refIdxL0=(predListFlagA==0)? refIdx1:refIdx2

[0673] refIdxL1=(predListFlagA==0)? refIdx2:refIdx1

[0674] mvL0=(predListFlagA==0)? Mv1:Mv2

[0675] mvL1=(predListFlagA==0)? Mv2:Mv1

[0676] predListFlagA is the prediction list flag for region P1.

[0677] 5.10 Example #10

[0678] For each 4×4 sub-block at sub-block index (xSbIdx, ySbIdx), where xSbIdx = 0…numSbx-1 and ySbIdx = 0…numSbY-1, the following applies:

[0679] refIdxL0=(predListFlagA==0)? refIdx1:refIdx2

[0680] refIdxL1=(predListFlagA==0)? refIdx2:refIdx1

[0681] mvL0=(predListFlagA==0)? Mv1:Mv2

[0682] mvL1=(predListFlagA==0)? Mv2:Mv1

[0683] predListFlagA is the prediction list flag for region P1.

[0684] 5.11 Example #11

[0685] For each 4×4 sub-block at sub-block index (xSbIdx, ySbIdx), where xSbIdx = 0…numSbx-1 and ySbIdx = 0…numSbY-1, the following applies:

[0686] – If Mv1 and Mv2 both come from the same reference list, then the motion information for the 4×4 subblock is (Mv2, refIdx2).

[0687] Otherwise, the following applies:

[0688] refIdxL0=(predListFlagA==0)? refIdx1:refIdx2

[0689] refIdxL1=(predListFlagA==0)? refIdx2:refIdx1

[0690] mvL0=(predListFlagA==0)? Mv1:Mv2

[0691] mvL1=(predListFlagA==0)? Mv2:Mv1

[0692] predListFlagA is the prediction list flag for region P1.

[0693] 5.12 Example #12

[0694] For each 4×4 sub-block at sub-block index (xSbIdx, ySbIdx), where xSbIdx = 0…numSbx-1 and ySbIdx = 0…numSbY-1, the following applies:

[0695] – If Mv1 and Mv2 both come from the same reference list, and the current block is in a strip with a backward reference picture (slice.getCheckLDC() is false), then the motion information of the 4×4 sub-block is (Mv1, refIdx1).

[0696] Otherwise, the following applies:

[0697] refIdxL0=(predListFlagA==0)? refIdx1:refIdx2

[0698] refIdxL1=(predListFlagA==0)? refIdx2:refIdx1

[0699] mvL0=(predListFlagA==0)? Mv1:Mv2

[0700] mvL1=(predListFlagA==0)? Mv2:Mv1

[0701] predListFlagA is the prediction list flag for region P1.

[0702] 5.13 Example #13

[0703] For each 4×4 sub-block at sub-block index (xSbIdx, ySbIdx), where xSbIdx = 0…numSbx-1 and ySbIdx = 0…numSbY-1, the following applies:

[0704] – If Mv1 and Mv2 both come from the same reference list, then the motion information for the 4×4 subblock is (Mv1, refIdx1).

[0705] Otherwise, the following applies:

[0706] –If Mv1 and Mv2 come from different reference lists, then

[0707] refIdxL0=(predListFlagA==0)? refIdx1:refIdx2

[0708] refIdxL1=(predListFlagA==0)? refIdx2:refIdx1

[0709] mvL0=(predListFlagA==0)? Mv1:Mv2

[0710] mvL1=(predListFlagA==0)? Mv2:Mv1

[0711] predListFlagA is the prediction list flag for region P1.

[0712] 5.14 Example #14

[0713] For each 4×4 sub-block at sub-block index (xSbIdx, ySbIdx), where xSbIdx = 0…numSbx-1 and ySbIdx = 0…numSbY-1, the following applies:

[0714] –If Mv1 and Mv2 come from different reference lists, then

[0715] refIdxL0=(predListFlagA==0)? refIdx1:refIdx2

[0716] refIdxL1=(predListFlagA==0)? refIdx2:refIdx1

[0717] mvL0=(predListFlagA==0)? Mv1:Mv2

[0718] mvL1=(predListFlagA==0)? Mv2:Mv1

[0719] predListFlagA is the prediction list flag for region P1.

[0720] Otherwise, the motion information for the 4×4 sub-block is ((Mv1+Mv2) / 2,refIdx1).

[0721] 5.15 Example #15

[0722] For each 4×4 sub-block at sub-block index (xSbIdx, ySbIdx), where xSbIdx = 0…numSbx-1 and ySbIdx = 0…numSbY-1, the motion information of the 4×4 sub-block is the motion information of the original Merge candidate corresponding to merge_triangle_idx0.

[0723] 5.16 Example #16

[0724] For each 4×4 sub-block at sub-block index (xSbIdx, ySbIdx), where xSbIdx = 0…numSbx-1 and ySbIdx = 0…numSbY-1, the following applies:

[0725] – If both Mv1 and Mv2 are from the same reference list, and triangleDir is equal to 0 and ySbIdx < numSbY-1, then the motion information of the 4×4 sub-block is (Mv1, refIdx1).

[0726] – Otherwise, if both Mv1 and Mv2 are from the same reference list, and triangleDir is equal to 0 and ySbIdx = numSbY-1 or triangleDir is equal to 1, then the motion information of the 4×4 sub-block is (Mv2, refIdx2).

[0727] – Otherwise, the following applies:

[0728] refIdxL0 = (predListFlagA == 0)? refIdx1:refIdx2

[0729] refIdxL1 = (predListFlagA == 0)? refIdx2:refIdx1

[0730] mvL0 = (predListFlagA == 0)? Mv1:Mv2

[0731] mvL1 = (predListFlagA == 0)? Mv2:Mv1

[0732] predListFlagA is the prediction list flag of the P1 region.

[0733] Figure 31 Is a flowchart of method 3100 for video processing. Method 3100 includes, at operation 3102, making a decision regarding the order of inserting motion candidates into a motion candidate list for conversion between a current block for video and a bitstream representation of the video, based on priority rules, where the current block is encoded and decoded using a geometric partitioning mode.

[0734] Method 3100 includes, in operation 3104, performing a transformation based on the decision and the motion candidate list.

[0735] Figure 32 This is a flowchart of method 3200 for video processing. Method 3200 includes, in operation 3202, inserting one or more average unidirectional predicted motion candidates derived from a regular motion candidate from list 0 and list 1 into a motion candidate list for the transformation between the current block of video and the bitstream representation of the video, wherein the current block is encoded and decoded using a geometric segmentation pattern.

[0736] Method 3200 includes, in operation 3204, performing a transformation based on the motion candidate list.

[0737] Figure 33 This is a flowchart of method 3300 for video processing. Method 3300 includes, in operation 3302, inserting one or more average motion candidates with unidirectional prediction derived from regular motion candidates with unidirectional prediction into a motion candidate list for the transformation between a current block of video and a bitstream representation of the video, wherein the current block is encoded and decoded using a geometric segmentation pattern.

[0738] Method 3300 includes, in operation 3304, performing a transformation based on the motion candidate list.

[0739] Figure 34 This is a flowchart of method 3400 for video processing. Method 3400 includes, in operation 3402, inserting one or more virtual motion candidates derived from available motion candidates into a list of motion candidates for the transformation between a current block of video and a bitstream representation of the video, wherein the current block is encoded and decoded using a geometric segmentation mode.

[0740] Method 3400 includes performing a transformation based on the motion candidate list in operation 3404.

[0741] Figure 35 This is a flowchart of method 3500 for video processing. Method 3500 includes, in operation 3502, making a decision on the order in which motion candidates are inserted into a list of motion candidates for the transformation between the current block of the video to be encoded and decoded using a geometric segmentation mode and the bitstream representation of the video, based on a priority rule, wherein the priority rule is based on the position of the motion candidate in the motion candidate derived therefrom or a list of one or more reference pictures associated with the motion candidate.

[0742] Method 3500 includes, in operation 3504, performing a transformation based on the decision and the motion candidate list.

[0743] Figure 36This is a flowchart of method 3600 for video processing. Method 3600 includes, in operation 3602, making a decision on the order of inserting motion candidates into a list of motion candidates for the transformation between the current block of the video and the bitstream representation of the video, based on priority rules, wherein the current block is encoded and decoded using a geometric segmentation mode, and wherein the insertion order is variable between sequence-to-sequence, picture-to-picture, strip-to-strip, slice-to-slice, or from the current block to subsequent blocks of the video.

[0744] Method 3600 includes, in operation 3604, performing a transformation based on the decision and the motion candidate list.

[0745] Figure 37 This is a flowchart of method 3700 for video processing. Method 3700 includes, in operation 3702, dividing the current block of a video encoded and decoded using a geometric segmentation mode into multiple partitions.

[0746] Method 3700 includes, in operation 3704, constructing multiple motion candidate lists, each motion candidate list corresponding to each of the multiple partitions.

[0747] Method 3700 includes, in operation 3706, performing a conversion between the current block and the bitstream representation of the video based on a multiple list of motion candidates.

[0748] Figure 38 This is a flowchart of method 3800 for video processing. Method 3800 includes, in operation 3802, dividing the current block of a video encoded and decoded using a geometric segmentation mode into multiple partitions.

[0749] Method 3800 includes, in operation 3804, constructing multiple motion candidate lists, wherein each of the multiple partitions is configured to select the same motion candidate list from the multiple motion candidate lists.

[0750] Method 3800 includes, in operation 3806, performing a conversion between the current block and the bitstream representation of the video based on the selected same list of motion candidates.

[0751] Figure 39 This is a flowchart of method 3900 for video processing. Method 3900 includes, in operation 3902, performing a conversion between a current block of video in a video unit and a bitstream representation of the video, wherein the bitstream representation includes a field indicating the maximum number of motion candidates allowed in a list of motion candidates for a geometric segmentation mode enabled in a video region.

[0752] Figure 40This is a flowchart of a method 4000 for video processing. Method 4000 includes, in operation 4002, selecting a weighting factor group from a plurality of weighting factor groups for a current block of a video segmented into multiple partitions using a geometric segmentation pattern, wherein the weighting factor group is selected based at least on the width or height of the current block.

[0753] Method 4000 includes, in operation 4004, as part of the conversion between the current block and the bitstream representation of the video, applying a weighting factor group to samples along the common boundary of at least two of the plurality of partitions.

[0754] Figure 41 This is a flowchart of a method 4100 for video processing. Method 4100 includes, in operation 4102, determining at least one weighting factor based on the angle of the common boundary of at least two of the multiple partitions for samples within the current block of a video segmented into multiple partitions using a geometric segmentation pattern.

[0755] Method 4100 includes, in operation 4104, performing a conversion between the current block and the bitstream representation of the video based on at least one weighting factor.

[0756] Figure 42 This is a flowchart of method 4200 for video processing. Method 4200 includes, in operation 4202, performing a motion compensation process on samples along the common boundary of at least two of the multiple partitions, for a current block of video segmented into multiple partitions using a geometric segmentation pattern, sample sizes different from 4×4 sample sizes.

[0757] Method 4200 includes, in operation 4204, performing a conversion between the current block and the bitstream representation of the video based on a motion compensation process.

[0758] Figure 43 This is a flowchart of method 4300 for video processing. Method 4300 includes, in operation 4302, dividing the current block of the video into multiple partitions using a geometric segmentation pattern.

[0759] Method 4300 includes, in operation 4304, storing a single set of motion information for a K×L region within the current block, derived from motion information associated with multiple partitions.

[0760] Method 4300 includes, in operation 4306, performing a conversion between the current block and a bitstream representation of the video using at least one set of motion information associated with at least one of the plurality of partitions.

[0761] Figure 44 This is a flowchart of method 4400 for video processing. Method 4400 includes, in operation 4402, dividing the current block of the video into multiple partitions.

[0762] Method 4400 includes, in operation 4404, performing a conversion between the current block encoded and decoded using a geometric segmentation pattern and a bitstream representation of the video, wherein a single set of motion information is stored for each M×N block cell of the current block, wherein the single set of motion information is derived from motion information associated with each of the plurality of partitions, wherein M or N is not equal to 4, and wherein each sample in the M×N block cell shares the same motion information.

[0763] Figure 45 This is a flowchart of method 4500 for video processing. Method 4500 includes, in operation 4502, dividing the current block of the video into multiple partitions.

[0764] Method 4500 includes, in operation 4504, as part of a conversion between the current block and the bitstream representation of the video, performing a motion compensation process on the first sub-region of the current block based on first motion information that is different from the second motion information associated with the first sub-region to be stored.

[0765] Figure 46 This is a flowchart of method 4600 for video processing. Method 4600 includes, in operation 4602, dividing the current block of video into multiple partitions using a geometric segmentation pattern.

[0766] Method 4600 includes, in operation 4604, storing motion information of a first sample set and a second sample set based on a partitioning direction or a decoded Merge index or Merge candidate list associated with a plurality of partitions, wherein the first sample set is located on the common boundary of at least two of the plurality of partitions, and the second sample set is located inside one of the plurality of partitions.

[0767] Method 4600 includes, in operation 4606, performing a conversion between the current block and a bitstream representation of the video, or a conversion between subsequent blocks of the video and a bitstream representation, based on stored motion information.

[0768] Figure 47 This is a flowchart of method 4700 for video processing. Method 4700 includes, in operation 4702, dividing the current block of the video into multiple partitions.

[0769] Method 4700 includes, in operation 4704, storing virtual bidirectional predictive motion information based on motion information of at least two of a plurality of partitions.

[0770] Method 4700 includes, in operation 4706, performing a conversion between the current block and the bitstream representation of the video based on virtual bidirectional predictive motion information.

[0771] Figure 48This is a flowchart of method 4800 for video processing. Method 4800 includes, in operation 4802, dividing the current block of the video into multiple partitions.

[0772] Method 4800 includes, in operation 4804, storing motion information (MvInfo1) of a first partition and motion information (MvInfo2) of a second partition among a plurality of partitions based on a low-latency check flag, wherein the low-latency check flag indicates that the picture sequence count (POC) value is not greater than the POC value of the current picture including the current block of all reference pictures.

[0773] Method 4800 includes, in operation 4806, performing a conversion between the current block and the bitstream representation of the video based on stored motion information.

[0774] Figure 49 This is a flowchart of method 4900 for video processing. Method 4900 includes, in operation 4902, dividing the current block of the video into multiple partitions using a geometric segmentation pattern.

[0775] Method 4900 includes, in operation 4904, performing a conversion between a current block and a bitstream representation of video using at least two sets of motion information associated with at least two of the plurality of partitions, wherein a single set of motion information is stored for a K×L region located on the common boundary of the at least two partitions, wherein the single set of motion information includes unidirectional predicted motion information derived from motion information associated with each of the at least two partitions.

[0776] Figure 50 This is a flowchart of method 5000 for video processing. Method 5000 includes, in operation 5002, maintaining one or more history-based motion vector prediction (HMVP) tables.

[0777] Method 5000 includes, in operation 5004, performing a conversion between the current block of the video and the bitstream representation of the video, wherein the current block is divided into multiple partitions using a geometric segmentation pattern.

[0778] Method 5000 includes, in operation 5006, selectively updating one or more HMVP tables based on conditions and after performing the transformation.

[0779] Figure 51 This is a flowchart of a method 5100 for video processing. Method 5100 includes, in operation 5102, performing a conversion between the current block and a bitstream representation of the video for a current block that has been divided into multiple partitions using a geometric segmentation pattern, the conversion being based on first motion information of a first partition among the multiple partitions, and the first motion information being derived based on second motion information of a second partition among the multiple partitions.

[0780] Figure 52 This is a flowchart of method 5200 for video processing. Method 5200 includes, in operation 5202, performing a conversion between the current block and a bitstream representation of the video for a current block that has been divided into multiple partitions using a geometric segmentation pattern, wherein motion information for more than one of the multiple partitions is not derived from the same list of reference images.

[0781] Figure 53 This is a flowchart of method 5300 for video processing. Method 5300 includes, in operation 5302, selectively enabling the derivation of motion information from the same reference image for a first and a second partition based on conditions of a current block that has been segmented into multiple partitions using a geometric segmentation pattern.

[0782] Method 5300 includes, in operation 5304, performing a conversion between the current block and the bitstream representation of the video based on the selective enable.

[0783] In some embodiments, the following technical solutions may be implemented:

[0784] A1. A method for video processing, comprising: making a decision on the order in which motion candidates are inserted into a list of motion candidates for a transformation between a current block of video and a bitstream representation of the video, based on a priority rule, wherein the current block is encoded and decoded using a geometric segmentation mode; and performing the transformation based on the decision and the list of motion candidates.

[0785] A2. According to the method of scheme A1, the priority rule is based on the predicted direction of at least one motion candidate.

[0786] A3. According to the method of scheme A2, the insertion order includes inserting a regular motion candidate with a unidirectional prediction from list X before inserting any regular motion candidate with a unidirectional prediction from list Y.

[0787] A4. According to the method of scheme A2, the insertion order includes inserting a motion candidate with bidirectional prediction before inserting any motion candidate with unidirectional prediction.

[0788] A5. According to the method of scheme A2, the insertion order is defined as: motion candidates derived from list X of regular motion candidates with bidirectional prediction, followed by motion candidates derived from list X of regular motion candidates with bidirectional prediction, followed by regular motion candidates with unidirectional prediction, where X = 0 or X = 1.

[0789] A6. According to the method of scheme A2, wherein the insertion order includes inserting a motion candidate derived from a first regular motion candidate with bidirectional prediction before inserting a motion candidate derived from a second regular motion candidate with bidirectional prediction, and wherein the insertion order of the current block is the same as the insertion order of the video blocks encoded and decoded using a non-geometric segmentation mode.

[0790] A7. According to the method of scheme A2, wherein the insertion order includes inserting motion candidates derived from a first rule motion candidate with bidirectional prediction and motion candidates derived from a second rule motion candidate with bidirectional prediction in an interleaved manner.

[0791] A8. According to the method of scheme A7, the interleaving method includes inserting truncated list 0 prediction candidates derived from CA, truncated list 1 prediction candidates derived from CB, truncated list 1 prediction candidates derived from CA, and truncated list 0 prediction candidates derived from CB, where CA and CB are two regular motion candidates.

[0792] A9. According to the method of scheme A7, the interleaving method includes inserting truncated list 1 prediction candidates derived from CA, truncated list 0 prediction candidates derived from CB, truncated list 0 prediction candidates derived from CA, and truncated list 1 prediction candidates derived from CB, where CA and CB are two regular motion candidates.

[0793] A10. The method according to any one of schemes A1 to A9, wherein the regular motion candidate is a motion candidate derived for video blocks encoded and decoded using a non-geometric segmentation pattern.

[0794] A11. According to the method of scheme A1, the priority rule is based on encoding / decoding mode information associated with the rule motion candidate.

[0795] A12. According to the method of scheme A11, wherein the encoding / decoding mode information includes encoding / decoding performed using Merge mode or Advanced Motion Vector Prediction (AMVP) mode.

[0796] A13. According to the method of scheme A12, wherein the insertion order includes inserting motion candidates derived from blocks encoded and decoded in AMVP mode before inserting motion candidates derived from blocks encoded and decoded in Merge mode.

[0797] A14. According to the method of scheme A12, wherein the insertion order includes inserting motion candidates derived from blocks encoded and decoded in AMVP mode before inserting motion candidates derived from blocks encoded and decoded in Merge mode.

[0798] A15. According to the method of scheme A11, the encoding / decoding mode information includes the reference index or picture sequence count (POC) difference.

[0799] A16. According to the method of scheme A15, wherein the insertion order includes inserting a motion candidate associated with the first reference index before inserting a motion candidate associated with a second reference index that is greater than the first reference index.

[0800] A17. The method according to scheme A15, wherein the insertion order includes inserting a motion candidate associated with the first POC difference before inserting a motion candidate associated with a reference index that is greater than the first POC difference.

[0801] A18. According to the method of scheme A11, wherein the encoding / decoding mode information includes quantization parameters (QP) or temporal layer indexes of reference pictures associated with the picture, strip, or slice group including the current block.

[0802] A19. The method according to scheme A18, wherein the insertion order includes inserting a motion candidate associated with the first QP before inserting a motion candidate associated with a second QP that is greater than the first QP.

[0803] A20. According to the method of scheme A1, the order of insertion includes inserting all truncated list 1 predicted candidates before any truncated list 0 predicted candidates.

[0804] A21. According to the method of scheme A1, wherein the insertion order includes inserting one or more truncated list 0 prediction candidates and one or more truncated list 1 prediction candidates based on the available geometric motion candidates.

[0805] A22. According to the method of scheme A1, wherein the insertion order includes inserting a first motion candidate with bidirectional prediction based on available geometric motion candidates before inserting a second motion candidate with bidirectional prediction.

[0806] A23. A method for video processing, comprising: inserting one or more average unidirectional predicted motion candidates derived from a regular motion candidate list from lists 0 and 1 into a motion candidate list for a transformation between a current block of video and a bitstream representation of the video, wherein the current block is encoded and decoded using a geometric segmentation pattern; and performing the transformation based on the motion candidate list.

[0807] A24. The method according to scheme A23 further includes: selecting a single average unidirectional predicted motion candidate from one or more average unidirectional predicted motion candidates to insert into the motion candidate list.

[0808] A25. The method according to scheme A24, wherein the selection is based on available geometric motion candidates from the list of motion candidates.

[0809] A26. According to the method of scheme A24, where the selection is based on the reference image indices of list 0 and list 1.

[0810] A27. According to the method of scheme A24, the selection is based on the picture order count (POC) distance between the reference pictures in list 0 and list 1 and the current picture.

[0811] A28. According to the method of scheme A23, all average one-way predicted motion candidates from list 0 are inserted before any average one-way predicted motion candidate from list 1 is inserted.

[0812] A29. According to the method of scheme A23, all average one-way predicted motion candidates from list 1 are inserted before any average one-way predicted motion candidate from list 0 is inserted.

[0813] A30. A method for video processing, comprising: inserting one or more average motion candidates with unidirectional prediction, derived from regular motion candidates with unidirectional prediction, into a motion candidate list for a transformation between a current block of video and a bitstream representation of the video, wherein the one or more average motion candidates with unidirectional prediction are derived from regular motion candidates with unidirectional prediction, wherein the current block is encoded and decoded using a geometric segmentation pattern; and performing a transformation based on the motion candidate list.

[0814] A31. The method according to scheme A30, wherein an average motion candidate with unidirectional predictions from list LX is derived from a regular motion candidate with unidirectional predictions from list LX, where X = 0 or X = 1.

[0815] A32. The method according to scheme A30, wherein an average motion candidate with unidirectional predictions from list LY is derived from a scaled regular motion candidate, wherein regular motion candidates from list LX are scaled to list LY to generate scaled regular motion candidates, wherein X = 0 or X = 1, and wherein Y = 1-X.

[0816] A33. A method for video processing, comprising: inserting one or more virtual motion candidates derived from available motion candidates into a motion candidate list for a transformation between a current block of video and a bitstream representation of the video, wherein the current block is encoded and decoded using a geometric segmentation pattern; and performing the transformation based on the motion candidate list.

[0817] A34. The method according to scheme A33, wherein one or more virtual motion candidates are derived by first generating a truncated list of motion candidates in the motion candidate list.

[0818] A35. The method according to scheme A33, wherein one or more virtual motion candidates are derived based solely on motion candidates with unidirectional prediction.

[0819] A36. The method according to scheme A33, wherein one or more virtual motion candidates are derived solely based on motion candidates with unidirectional prediction and a truncated list of motion candidates predicted from list 1.

[0820] A37. The method according to scheme A33, wherein one or more virtual motion candidates are derived based on motion candidates having a predetermined reference image index or having a picture sequence count (POC) distance within a specific range.

[0821] A38. According to the method of scheme A33, one or more virtual motion candidates are derived without applying pruning operations.

[0822] A39. According to any one of schemes A1 to A38, the current block is divided into multiple partitions based on the partitioning pattern.

[0823] A40. The method according to any one of schemes A1 to A39, wherein at least one of the plurality of partitions is neither square nor rectangular.

[0824] A41. The method according to any one of schemes A1 to A39, wherein the geometric segmentation pattern includes the triangle segmentation pattern.

[0825] A42. The method according to any one of schemes A1 to A41, wherein the transformation generates the current block from the bitstream representation.

[0826] A43. The method according to any one of schemes A1 to A41, wherein the transformation generates a bitstream representation from the current block.

[0827] A44. An apparatus in a video system, comprising a processor and a non-transitory memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to implement the method of any one of A1 to A43.

[0828] A45. A computer program product stored on a non-transitory computer-readable medium, the computer program product comprising program code for performing a method of any one of schemes A1 to A43.

[0829] In some embodiments, the following technical solutions may be implemented:

[0830] B1. A video processing method comprising: making a decision, based on a priority rule, regarding the order in which motion candidates are inserted into a list of motion candidates for a transformation between a current block of video encoded and decoded using a geometric segmentation mode and a bitstream representation of the video, wherein the priority rule is based on the position of the motion candidates derived therefrom or a list of one or more reference pictures associated with the motion candidates; and performing the transformation based on the decision and the list of motion candidates.

[0831] B2. According to the method of scheme B1, when determining the first motion candidate CA derived from block A, the second motion candidate CB derived from block B, and checking block B after block A, the insertion order includes inserting the motion candidate derived from CB after inserting the motion candidate derived from CA into the motion candidate list.

[0832] B3. According to the method of scheme B2, it also includes: checking block B after block A during the construction of the rule motion candidate list.

[0833] B4. According to the method of scheme B2, the insertion order also includes inserting a motion candidate derived from CA with predictions from list X, where X = 0 or X = 1, after inserting a motion candidate derived from CA with predictions from list X.

[0834] B5. According to the method of scheme B2, the insertion order further includes inserting a motion candidate derived from CA with predictions from list Y, where X = 0 or X = 1, and where Y = (1-X).

[0835] B6. According to the method of scheme B2, the insertion order further includes inserting a motion candidate derived from CA with predictions from list Y, and then inserting a motion candidate derived from CB with predictions from list X, where X = 0 or X = 1, and where Y = (1-X).

[0836] B7. According to any one of schemes B1 to B6, the current block is divided into multiple partitions based on the partitioning pattern.

[0837] B8. The method according to any one of schemes B1 to B6, wherein the geometric segmentation pattern includes the triangle segmentation pattern.

[0838] B9. According to any one of schemes B1 to B6, wherein the current block is divided into multiple partitions, wherein at least one partition is neither square nor rectangular.

[0839] B10. The method according to any one of schemes B1 to B9, wherein the transformation generates the current block from the bitstream representation.

[0840] B11. The method according to any one of schemes B1 to B9, wherein the transformation generates a bitstream representation from the current block.

[0841] B12. An apparatus in a video system, comprising a processor and a non-transitory memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to implement the method of any one of B1 to B11.

[0842] B13. A computer program product stored on a non-transitory computer-readable medium, the computer program product comprising program code for performing a method of any one of schemes B1 to B11.

[0843] In some embodiments, the following technical solutions may be implemented:

[0844] C1. A video processing method comprising: making a decision, based on priority rules, regarding the order in which motion candidates are inserted into a list of motion candidates for a transformation between a current block of video and a bitstream representation of the video, wherein the current block is encoded and decoded using a geometric segmentation mode, wherein the order of insertion is variable either between sequence-to-sequence, picture-to-picture, strip-to-strip, slice-to-slice, or from the current block to a subsequent block of video; and performing a transformation based on the decision and the list of motion candidates.

[0845] C2. According to the method of scheme C1, the insertion order is based on the current block or on a sequence, picture, strip, or piece group that includes the current block.

[0846] C3. According to the method of scheme C2, wherein the priority rule is based on at least one of the size, shape or partitioning mode of the current block.

[0847] C4. According to any one of schemes C1 to C3, wherein the order of insertion is notified to the decoder from the encoder signaling in the bitstream representation as a sequence parameter set (SPS), video parameter set (VPS), picture parameter set (PPS), strip header, slice group header, slice, codec tree unit (CTU), or codec unit (CU).

[0848] C5. A video processing method, comprising: dividing a current block of a video encoded using a geometric segmentation pattern into multiple partitions; constructing multiple motion candidate lists, each motion candidate list corresponding to each of the multiple partitions; and performing a conversion between the current block and a bitstream representation of the video based on the multiple motion candidate lists.

[0849] C6. According to the method of scheme C5, wherein the first motion candidate list in the multiple motion candidate lists of the first partition in the multiple partitions includes only motion candidates predicted from list X, wherein the second motion candidate list in the multiple motion candidate lists of the second partition in the multiple partitions includes only motion candidates predicted from list Y, wherein X = 0 or X = 1, and wherein Y = (1-X).

[0850] C7. According to the method of scheme C5, one or more motion candidates inserted into multiple motion candidate lists are based on motion information associated with different spatial or temporal blocks.

[0851] C8. According to the method of scheme C7, the positions of different spatial or temporal blocks used to derive one or more motion candidates of the first partition in a plurality of partitions are based on the positions of the first or second partition in a plurality of partitions in the current block.

[0852] C9. According to the method of scheme C7, the positions of different spatial or temporal blocks used to derive one or more motion candidates of the first or second partition in a plurality of partitions are based on the partitioning pattern of the current block.

[0853] C10. According to the method of scheme C9, the division pattern includes division from the upper right corner to the lower left corner or from the upper left corner to the lower right corner, represented by a 45-degree division pattern or a 135-degree division pattern, respectively.

[0854] C11. According to the method of scheme C10, wherein the partitioning mode includes a 135-degree partitioning mode, and wherein the method further includes: checking whether more upper blocks have a partition including the upper right corner of the current block.

[0855] C12. According to the method of scheme C10, wherein the partitioning mode includes a 135-degree partitioning mode, and wherein the method further includes: checking whether more left blocks have a partition including the lower left corner of the current block.

[0856] C13. According to the method of scheme C5, the index of each motion candidate list in the multiple motion candidate lists corresponding to multiple partitions is signaled.

[0857] C14. According to the method of scheme C5, a bitstream representation including the index of each of the multiple motion candidate lists is parsed for the transformation.

[0858] C15. A video processing method comprising: dividing a current block of a video encoded using a geometric segmentation mode into multiple partitions; constructing multiple motion candidate lists, wherein each of the multiple partitions is configured to select the same motion candidate list from the multiple motion candidate lists; and performing a conversion between the current block and a bitstream representation of the video based on the selected same motion candidate list.

[0859] C16. According to the method of scheme C15, the bit stream represents the index of the first motion candidate list, which includes multiple motion candidate lists.

[0860] C17. According to the method of scheme C16, wherein the bitstream representation further includes indices of motion candidates from a first motion candidate list, and wherein the indices of the motion candidates are signaled after the indices of the first motion candidate list.

[0861] C18. The method according to scheme C16, wherein the bitstream representation further includes an index of a motion candidate from the first motion candidate list following an index of the first motion candidate list, and wherein the bitstream representation is parsed for the conversion.

[0862] C19. The method according to scheme C15, wherein the index of the first motion candidate list is jointly encoded and decoded with the index of the motion candidate from the first motion candidate list in the bitstream representation.

[0863] C20. According to the method of scheme C15, a bitstream representation of the index of the first motion candidate list, including the index of the motion candidate list co-encoded with the index of the motion candidate list from the first motion candidate list, is parsed for the conversion.

[0864] C21. The method of any one of schemes C1 to C20, wherein the current block is divided into multiple partitions based at least on the partitioning pattern.

[0865] C22. The method according to any one of schemes C1 to C21, wherein the geometric segmentation pattern includes the triangle segmentation pattern.

[0866] C23. According to any one of schemes C1 to C21, wherein the current block is divided into multiple partitions, wherein at least one partition is neither square nor rectangular.

[0867] C24. The method according to any one of schemes C1 to C23, wherein the transformation generates the current block from the bitstream representation.

[0868] C25. The method according to any one of schemes C1 to C23, wherein the transformation generates a bitstream representation from the current block.

[0869] C26. An apparatus in a video system, comprising a processor and a non-transitory memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to implement the method of any one of C1 to C25.

[0870] C27. A computer program product stored on a non-transitory computer-readable medium, the computer program product comprising program code for performing a method of any one of schemes C1 to C25.

[0871] In some embodiments, the following technical solutions may be implemented:

[0872] D1. A method for video processing, comprising: performing a conversion between a current block of video in a video unit and a bitstream representation of the video, wherein the bitstream representation includes a field indicating the maximum number of motion candidates allowed in a list of motion candidates for a geometric segmentation mode enabled in the video region.

[0873] D2. According to the method of scheme D1, wherein the field includes explicit signaling indicating the maximum number of motion candidates allowed in the bitstream representation.

[0874] D3. According to the method of scheme D1, the field implicitly indicates that the maximum number of allowed motion candidates is equal to the maximum number of allowed motion candidates in the motion candidate list of another block encoded and decoded using a non-geometric segmentation mode.

[0875] D4. According to any one of schemes D1 to D3, the current block is further encoded and decoded using either Merge mode or Advanced Motion Vector Prediction (AMVP) mode.

[0876] D5. The method according to any one of schemes D1 to D3, wherein the video unit includes a sequence, video, picture, strip, slice group, maximum codec unit (LCU) row or maximum codec unit (LCU) group.

[0877] D6. According to any one of schemes D1 to D3, wherein the maximum number of allowed motion candidates is indicated in the signaling notification field of the Sequence Parameter Set (SPS), Video Parameter Set (VPS), Picture Parameter Set (PPS), Picture Header, Strip Header, Slice Header, Maximum Codec Unit (LCU) Row or Maximum Codec Unit (LCU) Group.

[0878] D7. According to any one of schemes D1 to D6, wherein the indication of the maximum number of allowed motion candidates in the field is binarized using unary encoding or truncated unary encoding.

[0879] D8. According to the method of scheme D2, wherein the indication of the maximum number of allowed motion candidates in the explicit signaling notification field includes the difference between the signaling notification M and the maximum number of allowed motion candidates, where M is an integer.

[0880] D9. According to the method of scheme D8, the maximum number of allowed motion candidates for a block encoded and decoded using the geometric segmentation mode is set to M minus the parsing indicator.

[0881] D10. According to the method of scheme D8 or D9, where M = 5 or M = 6.

[0882] D11. According to the method of any one of schemes D1 to D8, wherein a bitstream representation including an indication of the maximum number of allowed motion candidates in the field is parsed for the conversion.

[0883] D12. According to the method of scheme D1, wherein the bitstream representation includes a bit flag indicating that the maximum number of allowed motion candidates for a video block encoded and decoded using a geometric segmentation mode is the same as the maximum number of allowed regular motion candidates or sub-block motion candidates.

[0884] D13. According to the method of scheme D12, a bitstream representation including a bit flag is parsed for use in the conversion.

[0885] D14. According to any one of schemes D1 to D13, the current block is divided into multiple partitions according to the partitioning pattern.

[0886] D15. The method according to any one of schemes D1 to D14, wherein the geometric segmentation pattern includes the triangle segmentation pattern.

[0887] D16. According to any one of schemes D1 to D14, wherein the current block is divided into multiple partitions, at least one of the multiple partitions is neither square nor rectangular.

[0888] D17. The method of any one of schemes D1 to D16, wherein the transformation generates the current block from the bitstream representation.

[0889] D18. The method according to any one of schemes D1 to D16, wherein the transformation generates a bitstream representation from the current block.

[0890] D19. An apparatus in a video system, comprising a processor and a non-transitory memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to implement the method of any one of D1 to D18.

[0891] D20. A computer program product stored on a non-transitory computer-readable medium, the computer program product comprising program code for performing a method of any one of schemes D1 to D18.

[0892] In some embodiments, the following technical solutions may be implemented:

[0893] E1. A video processing method comprising: for a current block of a video segmented into multiple partitions using a geometric segmentation pattern, selecting a set of weighting factors from a plurality of weighting factor sets, wherein the weighting factor sets are selected based at least on the width or height of the current block; and as part of a conversion between the current block and a bitstream representation of the video, applying the weighting factor sets to samples along a common boundary of at least two of the plurality of partitions.

[0894] E2. According to the method of scheme E1, the selection is based on the ratio between width and height being greater than a threshold.

[0895] E3. According to the method of scheme E1, the selection is based on the ratio between height and width being greater than a threshold.

[0896] E4. According to the method of scheme E1, where multiple weighting factor groups are predefined based on the width or height of the current block.

[0897] E5. According to the method of scheme E4, one or two weighted factor groups are selected from multiple weighted factor groups.

[0898] E6. A video processing method comprising: for samples within a current block of a video segmented into multiple partitions using a geometric segmentation pattern, determining at least one weighting factor based on the angle of a common boundary of at least two of the multiple partitions; and performing a conversion between the current block and a bitstream representation of the video based on the at least one weighting factor.

[0899] E7. According to the method of scheme E6, the determination is also based on the location of the sample points.

[0900] E8. According to any one of schemes E1 to E7, the final predicted value of the sample in the current block is derived as a weighted sum of two predicted values ​​based on at least one weighting factor.

[0901] E9. According to the method of scheme E8, the final predicted block of the current block is (f(x,y)×P1(x,y)+(2M–f(x,y))×P2(x,y)+offset)>>M, where P1(x,y) and P2(x,y) are the first and second predicted values ​​of the sample point with coordinates (x,y), respectively, where M and offset are integers, and where f(x,y) is a function that returns the weights applied to the first predicted value.

[0902] E10. According to the method of scheme E9, where offset = (1<<(M-1)).

[0903] E11. According to any one of schemes E1 to E10, the current block is divided into multiple partitions based on the partitioning pattern.

[0904] E12. The method according to any one of schemes E1 to E11, wherein the geometric segmentation pattern includes the triangle segmentation pattern.

[0905] E13. According to the method of scheme E12, the geometric partitioning pattern includes dividing a block into two partitions.

[0906] E12. According to any one of schemes E1 to E10, wherein the current block is divided into multiple partitions, at least one of the multiple partitions is neither square nor rectangular.

[0907] E13. The method according to any one of schemes E1 to E12, wherein the transformation generates the current block from the bitstream representation.

[0908] E14. The method according to any one of schemes E1 to E12, wherein the transformation generates a bitstream representation from the current block.

[0909] E15. An apparatus in a video system, comprising a processor and a non-transitory memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to implement the method of any one of E1 to E14.

[0910] E16. A computer program product stored on a non-transitory computer-readable medium, the computer program product comprising program code for performing a method of any one of schemes E1 to E14.

[0911] In some embodiments, the following technical solutions may be implemented:

[0912] F1. A video processing method comprising: for a current block of a video segmented into multiple partitions using a geometric segmentation pattern, performing a motion compensation process on samples along the common boundary of at least two of the multiple partitions, the samples having a sample size different from 4×4; and, based on the motion compensation process, performing a conversion between the current block and a bitstream representation of the video.

[0913] F2. According to the method of scheme F1, the motion compensation process is performed with an 8×8 sample size to reduce the storage bandwidth required for conversion.

[0914] F3. According to the method of scheme F1, the motion compensation process is performed with a sample size of 8×4 or 4×8 to reduce the storage bandwidth required for conversion.

[0915] F4. According to the method of scheme F3, the sample size is based on the height or width of the current block.

[0916] F5. According to the method of scheme F3, the sample size is based on the partitioning pattern of multiple partitions of the current block.

[0917] F6. According to the method of scheme F5, the division pattern includes division from the upper right corner to the lower left corner or from the upper left corner to the lower right corner, which are represented by the 45-degree division pattern or the 135-degree division pattern, respectively.

[0918] F7. A video processing method comprising: dividing a current block of a video into multiple partitions using a geometric segmentation pattern; storing a single set of motion information for a K×L region within the current block derived from motion information associated with the multiple partitions; and performing a conversion between the current block and a bitstream representation of the video using at least one set of motion information associated with at least one of the multiple partitions.

[0919] F8. According to the method of scheme F7, where MVInfoi represents the motion information of the i-th partition, where i = 1 and i = 2, and where when it is determined that MVInfo1 comes from the first reference image list LX and MVInfo2 comes from the second reference image list LY, a single motion information set includes bidirectional prediction and associated motion information of the current block based on the combination of MVInfo1 and MVInfo2, where X = 0 or X = 1, and where Y = (1-X).

[0920] F9. According to the method of scheme F7, where MVInfoi represents the motion information of the i-th partition, i = 1 and i = 2, and when it is determined that MVInfo1 and MVInfo2 are both from the reference list LX, a single motion information set includes a unidirectional prediction of the current block based on MVInfoj and associated motion information, j = 1 or j = 2, and X = 0 or X = 1.

[0921] F10. According to the method of scheme F8, the single set of motion information, including bidirectional prediction, is derived by combining the motion vectors of MVInfo1 and MVInfo2 and the reference image index.

[0922] F11. The method according to scheme F9, wherein a single set of motion information, including unidirectional prediction, is based on MVInfo1.

[0923] F12. The method according to scheme F9, wherein a single set of motion information, including unidirectional prediction, is based on MVInfo2.

[0924] F13. The method according to any one of schemes F1 to F12, wherein, when determining that the upper right sample point and the lower left sample point are in two different partitions, MVInfo1 includes a set of motion information associated with the partition covering the upper right sample point, and MVInfo2 includes a set of motion information associated with the partition covering the lower left sample point.

[0925] F14. According to the method of scheme F13, the current block is divided from the top left corner to the bottom right corner.

[0926] F15. The method according to any one of schemes F1 to F12, wherein, when determining that the upper left sample point and the lower right sample point are in two different partitions, MVInfo1 includes a set of motion information associated with the partition covering the upper left sample point, and MVInfo2 includes a set of motion information associated with the partition covering the lower right sample point.

[0927] F16. According to the method of scheme F15, the current block is divided from the upper right corner to the lower left corner.

[0928] F17. According to the method of scheme F7, the stored single set of motion information is used for temporal motion prediction, spatial motion prediction, or one of the filtering processes of the current block of the video.

[0929] F18. According to the method of scheme F7, the stored single set of motion information is used to process other blocks in different pictures.

[0930] F19. The method according to scheme F17, wherein the filtering process includes deblocking or adaptive loop filtering (ALF).

[0931] F20. According to the method of scheme F7, where MVInfoi and MVInfoj represent the motion information of the i-th and j-th partitions, respectively, where MVInfoi and MVInfoj both come from the reference image list LX, where X = 0 or X = 1, and where a single set of motion information is based on the combination of MVInfoi and the scaled motion vector generated by scaling MVInfoj to the reference image list LY, where Y = (1-X).

[0932] F21. According to the method of scheme F20, the value of X is predetermined or signaled in the bitstream representation.

[0933] F22. According to the method of scheme F7, where MVInfoi and MVInfoj represent the motion information of the i-th partition and the j-th partition, respectively, where MVInfoi and MVInfoj both come from the reference image list LX, where X = 0 or X = 1, and where a single motion information set is based on the combination of MVInfoi and the mapped motion vector generated by mapping MVInfoj to the reference image list LY, where Y = (1-X).

[0934] F23. The method according to scheme F22, wherein a single motion information set includes bidirectional predicted motion vectors.

[0935] F24. According to the method of scheme F22, the mapping of MVInfoj to the reference image in list LY does not include scaling operations.

[0936] F25. According to the method of scheme F7, a single set of motion information is stored based on decoded motion information associated with multiple partitions.

[0937] F26. According to the method of scheme F25, MVInfoi and MVInfoj represent the motion information of the i-th partition and the j-th partition, respectively, where MVInfoi and MVInfoj both come from the same reference image list LX, where X = 0 or X = 1, and where a single motion information set includes unidirectional predicted motion information.

[0938] F27. According to the method of scheme F25, MVInfoi and MVInfoj represent the motion information of the i-th partition and the j-th partition, respectively, where MVInfoi comes from the reference image list LX and MVInfoj comes from the reference image list LY, where X = 0 or X = 1, where Y = (1-X), and where a single motion information set includes bidirectional predicted motion information.

[0939] F28. According to the method of scheme F25, MVInfoi and MVInfoj represent the motion information of the i-th partition and the j-th partition, respectively, where MVInfoi and MVInfoj both come from the reference image list LX, where X = 0 or X = 1, where the reference image list LY includes MVInfoi or MVInfoj, where Y = (1-X), and where a single motion information set includes bidirectional predicted motion information.

[0940] F29. According to the method of scheme F7, wherein storing a single set of motion information is based on a sub-region of the current block located along the common boundary of at least two of the multiple partitions.

[0941] F30. According to the method of scheme F29, different sub-regions store different motion information.

[0942] F31. According to the method of scheme F29, the sub-region stores unidirectional or bidirectional predicted motion information.

[0943] F32. According to the method of scheme F7, the storage of a single motion information set is based on the position of the sub-block in the current block, and the sub-block stores only unidirectional predicted motion information.

[0944] F33. The method according to any one of schemes F1 to F32, wherein a single set of motion information is used in the motion compensation process applied to the current block.

[0945] F34. According to the method of scheme F7, where MVInfoi and MVInfoj represent the motion information of the i-th partition and the j-th partition, respectively, where MVInfoi and MVInfoj are both from the reference image list LX, where X = 0 or X = 1, and where a single set of motion information includes bidirectional predicted motion vectors for the reference image list LX derived from (a) MVInfoi, (b) MVInfoj or (c) the average of MVInfoi and MVInfoj.

[0946] F35. According to the method of scheme F7, where MVInfoi and MVInfoj represent the motion information of the i-th partition and the j-th partition, respectively, where MVInfoi and MVInfoj are both from the reference image list LX, where X = 0 or X = 1, and where a single motion information set includes the average or weighted average of MVInfoi and MVInfoj, and where the reference image associated with a single motion information set includes the reference image of MVInfoi or the reference image of MVInfoj.

[0947] F36. According to the method of any one of schemes F1 to F35, wherein the K×L region contains at least one sample point on the common boundary between at least two of the plurality of partitions.

[0948] F37. The method according to any one of schemes F1 to F36, wherein the geometric segmentation pattern includes the triangle segmentation pattern.

[0949] F38. According to any one of schemes F1 to F36, wherein at least one of the plurality of partitions is neither square nor rectangular.

[0950] F39. The method according to any one of schemes F1 to F38, wherein the transformation generates the current block from the bitstream representation.

[0951] F40. The method according to any one of schemes F1 to F38, wherein the transformation generates a bitstream representation from the current block.

[0952] F41. An apparatus in a video system, comprising a processor and a non-transitory memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to implement the method of any one of F1 to F40.

[0953] F42. A computer program product stored on a non-transitory computer-readable medium, the computer program product comprising program code for performing a method of any one of schemes F1 to F40.

[0954] In some embodiments, the following technical solutions may be implemented:

[0955] G1. A method for video processing, comprising: dividing a current block of video into multiple partitions; and performing a conversion between the current block encoded and decoded using a geometric segmentation pattern and a bitstream representation of the video, wherein a single set of motion information is stored for each M×N block unit of the current block, wherein the single set of motion information is derived from motion information associated with each of the multiple partitions, wherein M or N is not equal to 4, and wherein each sample in the M×N block unit shares the same motion information.

[0956] G2. According to the method of scheme G1, where M = 8 and N = 8.

[0957] G3. According to the method of scheme G1, wherein the first part of the M×N block unit includes a first sample set on the common boundary of at least two partitions in the plurality of partitions, and wherein the second part of the M×N block unit includes a second sample set inside one of the plurality of partitions.

[0958] G4. According to the method of scheme G3, where MVInfoi represents the motion information of the i-th partition, i = 1 and i = 2, where when it is determined that MVInfo1 comes from the first reference image list LX and MVInfo2 comes from the second reference image list LY, the same motion information includes bidirectional prediction and associated motion information for the current block based on MVInfoi of multiple partitions, where X = 0 or X = 1, and where Y = (1-X).

[0959] G5. According to the method of scheme G3, where MVInfoi represents the motion information of the i-th partition, i = 1 and i = 2, and when it is determined that MVInfo1 and MVInfo2 both come from the reference image list LX, the same motion information includes the unidirectional prediction and associated motion information for the current block based on MVInfoj, j = 1 or j = 2, and X = 0 or X = 1.

[0960] G6. According to the method of scheme G3, where MVInfoi represents the motion information of the i-th partition, i = 1 and i = 2, and where the same motion information includes the unidirectional prediction and associated motion information for the current block based on MVInfoj, j = 1 or j = 2.

[0961] G7. A video processing method comprising: dividing a current block of video into multiple partitions; and, as part of a conversion between the current block and a bitstream representation of the video, performing a motion compensation process on a first sub-region of the current block based on first motion information, which is different from second motion information associated with the first sub-region to be stored.

[0962] G8. According to the method of scheme G7, wherein the first sub-region includes a set of M×N sample points on the common boundary of at least two of the multiple partitions, wherein the motion compensation process includes bidirectional prediction, and wherein only one set of motion information from list 0 or list 1 of the bidirectional prediction is stored.

[0963] G9. According to the method of scheme G8, the motion information set is used to derive spatial motion candidates of neighboring blocks encoded and decoded using Merge mode or Advanced Motion Vector Prediction (AMVP) mode.

[0964] G10. According to the method of scheme G8, wherein the motion information set is used to derive temporal motion candidates for subsequent blocks in an image that is different from the current image including the current block.

[0965] G11. According to the method of scheme G8, the motion information set is used for in-loop processing of the current block.

[0966] G12. According to the method of scheme G11, the in-loop processing includes deblocking or adaptive loop filtering.

[0967] G13. According to the method of scheme G7, a motion compensation process is performed on the second sub-region of the current block based on third motion information associated with the second sub-region, and the first and second sub-regions are in the same codec unit (CU) comprising multiple partitions.

[0968] G14. A video processing method comprising: dividing a current block of a video into multiple partitions using a geometric segmentation pattern; storing motion information of a first sample set and a second sample set based on a segmentation direction or a decoding Merge index or a Merge candidate list associated with the multiple partitions, wherein the first sample set is located on a common boundary of at least two of the multiple partitions, and the second sample set is located inside one of the multiple partitions; and performing a conversion between the current block and a bitstream representation of the video or a conversion between subsequent blocks of the video and a bitstream representation based on the stored motion information.

[0969] G15. According to the method of scheme G14, the stored motion information corresponds to the motion information associated with one of the multiple partitions associated with the selected decoding Merge index.

[0970] G16. According to the method of scheme G14, wherein the stored motion information corresponds to the motion candidate in the motion candidate list with the selected Merge index set to variable k.

[0971] G17. According to the method of scheme G16, where k = 0.

[0972] G18. According to the method of scheme G16, the stored motion information corresponds to the motion information of the k-th Merge candidate in one of the Merge candidate lists.

[0973] G19. According to the method of scheme G15, the selected decoding merge index is based on the partition direction.

[0974] G20. According to the method of scheme G15, when determining that the Merge candidate whose index is equal to the selected decoding Merge index is based on bidirectional prediction, the stored motion information includes bidirectional prediction motion information.

[0975] G21. According to the method of scheme G15, when determining that the Merge candidate whose index is equal to the selected decoding Merge index is based on unidirectional prediction, the stored motion information includes unidirectional predicted motion information.

[0976] G22. According to the method of scheme G6, wherein the first sub-region includes M×N sample points on the common boundary of at least two of the multiple partitions, wherein the motion compensation process includes bidirectional prediction, and wherein only one set of motion information from list 0 or list 1 of the bidirectional prediction is stored.

[0977] G23. According to the method of scheme G6, wherein the first sub-region includes M×N sample points on the common boundary of at least two partitions in a plurality of partitions, wherein the motion compensation process includes bidirectional prediction, wherein only unidirectional prediction information from list X is stored, and wherein X = 0 and X = 1.

[0978] G24. According to the method of scheme G6, wherein the first sub-region includes M×N sample points on the common boundary of at least two of the multiple partitions, wherein the motion compensation process includes unidirectional prediction, wherein the second motion information includes stored unidirectional prediction information from list X, and wherein X = 0 and X = 1.

[0979] G25. A video processing method, comprising: dividing a current block of a video into multiple partitions; storing virtual bidirectional predictive motion information based on motion information of at least two of the multiple partitions; and performing a conversion between the current block and a bitstream representation of the video based on the virtual bidirectional predictive motion information.

[0980] G26. According to the method of scheme G25, where MvInfo1 and MvInfo2 are motion information of the first and second partitions respectively, and where the virtual bidirectional prediction motion information is based on modifying the prediction direction of MvInfoA from list X to list Y, where A = 1 or A = 2, where X = 0 or X = 1, and where Y = (1-X).

[0981] G27. According to the method of scheme G26, the motion vector and reference index of MvInfoA remain unchanged.

[0982] G28. According to the method of scheme G26, the reference index of MvInfoA remains unchanged, while the motion vector of MvInfoA is set to the opposite value.

[0983] G29. The method according to any one of schemes G26 to G28, wherein MvInfo1 and MvInfo2 come from the same prediction direction.

[0984] G30. A video processing method, comprising: dividing a current block of a video into multiple partitions; storing motion information (MvInfo1) of a first partition and motion information (MvInfo2) of a second partition among the multiple partitions based on a low-latency check flag, wherein the low-latency check flag indicates all reference images whose picture order count (POC) value is not greater than the POC value of the current image including the current block; and performing a conversion between the current block and a bitstream representation of the video based on the stored motion information.

[0985] G31. According to the method of scheme G30, MvInfo1 and MvInfo2 come from different prediction directions, and the storage of motion information includes storing bidirectional predicted motion information generated by combining MvInfo1 and MvInfo2.

[0986] G32. The method according to scheme G30, wherein MvInfo1 and MvInfo2 come from the same prediction direction (list X), wherein X = 0 or X = 1, and wherein the method further comprises: modifying MvInfo2 by changing the prediction direction list of MvInfo2 from list X to list Y, wherein Y = (1-X), and wherein storing motion information includes storing bidirectional predicted motion information generated by combining MvInfo1 and the modified MvInfo2.

[0987] G33. The method according to any one of schemes G25 to G32, wherein storing motion information includes storing motion information of the sub-blocks of the current block.

[0988] G34. According to the method of scheme G33, the sub-block includes sample points on the common boundary of at least two of the multiple partitions.

[0989] G35. According to the method of scheme G33, the sub-block includes the diagonal or anti-diagonal of the current block.

[0990] G36. According to the method of scheme G33, the child block is in the bottom right corner of the current block.

[0991] G37. According to the method of scheme G33, the sub-block is in the right column or bottom row of the current block.

[0992] G38. According to any one of schemes G1 to G37, wherein at least one of the plurality of partitions is neither square nor rectangular.

[0993] G39. The method according to any one of schemes G1 to G38, wherein the transformation generates the current block from the bitstream representation.

[0994] G40. The method according to any one of schemes G1 to G38, wherein the transformation generates a bitstream representation from the current block.

[0995] G41. An apparatus in a video system, comprising a processor and a non-transitory memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to implement the method of any one of G1 to G40.

[0996] G42. A computer program product stored on a non-transitory computer-readable medium, the computer program product comprising program code for performing a method of any one of schemes G1 to G40.

[0997] H1. A method for video processing, comprising: segmenting a current block of video into a plurality of partitions using a geometric segmentation pattern; and performing a conversion between the current block and a bitstream representation of the video using at least two sets of motion information associated with at least two of the plurality of partitions, wherein a single set of motion information is stored for a K×L region located on a common boundary of the at least two partitions, wherein the single set of motion information includes unidirectional predicted motion information derived from motion information associated with each of the at least two partitions.

[0998] H2. The method described in scheme H1, wherein MVInfoi represents the motion information of the i-th partition, where i = 1 and i = 2.

[0999] H3. The method according to scheme H2, wherein storing a single motion information set includes storing MVInfo1 of the K×L region in the current block.

[1000] H4. The method according to scheme H2, wherein storing a single motion information set includes storing MVInfo1 of all sub-regions within the current block.

[1001] H5. The method according to scheme H2, wherein storing a single motion information set includes storing MVInfo2 of the K×L region in the current block.

[1002] H6. The method according to scheme H2, wherein storing a single motion information set includes storing MVInfo2 of all sub-regions within the current block.

[1003] H7. The method according to scheme H1 or H2, wherein a single motion information set includes motion information from a selected partition of a plurality of partitions.

[1004] H8. The method described in scheme H7, wherein a selected partition is selected based on the picture order count (POC) distance relative to the current picture including the current block.

[1005] H9. The method described in scheme H7, wherein a selected partition is selected based on the quantization parameters (QP) of the reference image associated with the current block.

[1006] H10. The method described in scheme H7, wherein a selected partition is selected based on a reference index of a reference image associated with the current block.

[1007] H11. The method according to scheme H2, wherein a single motion information set includes the average value of MVInfoA and MVInfoB, where (A=1, B=2) or (A=2, B=1).

[1008] H12. The method according to scheme H2, wherein a single motion information set comprises the average of MVInfoA and a scaled version of MVInfoB, wherein (A=1, B=2) or (A=2, B=1).

[1009] H13. The method according to scheme H2, wherein a single motion information set comprises the average of MVInfoA and the mapped version of MVInfoB, wherein (A=1, B=2) or (A=2, B=1).

[1010] H14. The method according to scheme H2, wherein when it is determined that the reference index of the reference image associated with MVInfo1 is less than the reference index of the reference image associated with MVInfo2, the single motion information set includes MVInfo1.

[1011] H15. The method according to scheme H2, wherein when it is determined that the Merge index associated with MVInfo1 is less than the Merge index associated with MVInfo2, a single motion information set includes MVInfo1.

[1012] H16. The method according to any one of schemes H1 to H15, wherein the K×L region is a sub-region comprising sample points on the common boundary of at least two partitions.

[1013] H17. The method according to any one of schemes H1 to H16, wherein one of the at least two partitions is selected based on the partitioning direction of the current block.

[1014] H18. The method according to any one of schemes H1 to H17, wherein the upper right corner sample and the lower left corner sample are located in different partitions, and wherein MVInfo1 is a motion information set associated with the partition covering the upper right corner sample, and MVInfo2 is a motion information set associated with the partition covering the lower left corner sample.

[1015] H19. The method described in scheme H18, wherein the division direction is from the upper left sample point to the lower right sample point.

[1016] H20. The method according to any one of schemes H1 to H17, wherein the upper left corner sample and the lower right corner sample are located in different partitions, and wherein MVInfo1 is a motion information set associated with the partition covering the upper left corner sample, and MVInfo2 is a motion information set associated with the partition covering the lower right corner sample.

[1017] H21. The method described according to scheme H20, wherein the division direction is from the upper right corner sample point to the lower left corner sample point.

[1018] H22. The method according to any one of schemes H1 to H21, wherein the geometric segmentation pattern includes a triangular segmentation pattern.

[1019] H23. The method according to any one of schemes H1 to H21, wherein at least one of the plurality of partitions is neither square nor rectangular.

[1020] H24. The method according to any one of schemes H1 to H23, wherein the encoding and decoding of one or more other blocks is based on a single set of motion information.

[1021] H25. The method according to scheme H24 further includes: interpreting a single set of motion information as a spatial motion candidate, wherein encoding and decoding of one or more other blocks includes a Merge pattern of neighboring blocks or an Advanced Motion Vector Prediction (AMVP) pattern.

[1022] H26. The method according to any one of schemes H1 to H23, wherein the encoding and decoding of subsequent blocks in different pictures is based on a single set of motion information.

[1023] H27. The method according to scheme H26 further includes: interpreting a single set of motion information as a candidate temporal motion vector.

[1024] H28. The method according to any one of schemes H1 to H23, wherein the in-loop processing of the current block is based on a single set of motion information.

[1025] H29. The method according to scheme H28, wherein the in-loop processing includes deblocking or adaptive loop filtering (ALF).

[1026] H30. The method according to any one of schemes H1 to H29, wherein the transformation generates the current block from the bitstream representation.

[1027] H31. The method according to any one of schemes H1 to H29, wherein the transformation generates a bitstream representation from the current block.

[1028] H32. An apparatus in a video system, comprising a processor and a non-transitory memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to perform the method according to one or more of schemes H1 to H31.

[1029] H33. A computer program product stored on a non-transitory computer-readable medium, the computer program product including program code for performing the method according to one or more of schemes H1 to H31.

[1030] J1. A video processing method comprising: maintaining one or more history-based motion vector prediction (HMVP) tables; performing a conversion between a current block of video and a bitstream representation of video, wherein the current block is segmented into multiple partitions using a geometric segmentation pattern; and selectively updating one or more HMVP tables based on conditions and after performing the conversion.

[1031] J2. The method according to scheme J1, wherein one or more HMVP tables are updated by including motion information of the first partition among multiple partitions and excluding motion information of the second partition among multiple partitions.

[1032] J3. The method according to scheme J1, wherein one or more HMVP tables are updated by including motion information associated with samples on the common boundary between at least two of the plurality of partitions and excluding motion information associated with samples inside one of the plurality of partitions.

[1033] J4. The method according to scheme J1, wherein one or more HMVP tables are updated by including motion information associated with samples inside one of the plurality of partitions and excluding motion information associated with samples on the common boundary between at least two of the plurality of partitions.

[1034] J5. The method according to scheme J1, wherein one or more HMVP tables are updated based on Merge candidates from a Merge candidate list associated with motion information of the current block divided into multiple partitions.

[1035] J6. A video processing method comprising: for a current block that has been divided into multiple partitions using a geometric segmentation pattern, performing a conversion between the current block and a bitstream representation of a video, wherein the conversion is based on first motion information of a first partition among the multiple partitions, and wherein the first motion information is derived based on second motion information of a second partition among the multiple partitions.

[1036] J7. The method according to scheme J6, wherein the second motion information includes the predicted direction of the second partition.

[1037] J8. The method according to scheme J6, wherein the second motion information includes motion information associated with the reference image list LX, where X = 0 or X = 1.

[1038] J9. The method according to scheme J6, wherein when it is determined that motion information associated with the reference image list LX is unavailable, the second motion information includes motion information associated with the reference image list LY, wherein X = 0 or X = 1, and wherein Y = (1-X).

[1039] J10. A video processing method comprising: for a current block that has been segmented into multiple partitions using a geometric segmentation pattern, performing a conversion between the current block and a bitstream representation of a video, wherein motion information of more than one of the multiple partitions is not derived from the same list of reference images.

[1040] J11. The method according to scheme J10, wherein the consistent bitstream includes an indication that motion information for two of the multiple partitions is derived from different lists of reference images.

[1041] J12. The method described in scheme J10, wherein the parity of the Merge index always indicates that the motion information of two partitions out of multiple partitions is derived from different lists of reference images.

[1042] J13. The method according to any one of schemes J6 to J12 further includes: storing motion information including first motion information or second motion information, wherein one or more historical motion vector prediction (HMVP) tables are updated based on the stored motion information.

[1043] J14. A video processing method comprising: selectively enabling the derivation of motion information from the same reference image for a first and a second partition based on a condition of a current block being segmented into multiple partitions using a geometric segmentation pattern; and performing a conversion between the current block and a bitstream representation of the video based on the selective enabling.

[1044] J15. The method described according to scheme J14, wherein the reference samples of the two partitions do not overlap.

[1045] J16. According to the method described in scheme J14, MV0 and MV1 are the motion vectors of the first partition and the second partition, respectively, and MVX[0] and MVX[1] are the horizontal and vertical components of MVX, respectively.

[1046] J17. The method according to scheme J16, wherein the condition includes abs(MV0[0]–MV1[0]) being less than a threshold, and wherein abs(x) returns the absolute value of x.

[1047] J18. The method according to scheme J16, wherein the condition includes abs(MV0[1]–MV1[1]) being less than a threshold, and wherein abs(x) returns the absolute value of x.

[1048] J19. The method according to scheme J16, wherein the condition includes abs(MV0[0]–MV1[0])+abs(MV0[1]–MV1[1]) being less than a threshold, and wherein abs(x) returns the absolute value of x.

[1049] J20. The method according to scheme J16, wherein the condition includes Max(abs(MV0[0]–MV1[0]), abs(MV0[1]–MV1[1])) being less than a threshold, wherein abs(x) returns the absolute value of x, and wherein Max(a,b) returns x if x>y and y if y>x.

[1050] J21. The method according to any one of schemes J1 to J20, wherein the conversion is based on stored motion information, the motion information including at least one of motion vector, reference index, indication of unidirectional or bidirectional prediction, indication of inter-frame prediction direction, indication of generalized bidirectional prediction (GBi), motion vector resolution, or indication of affine prediction.

[1051] J22. The method according to any one of schemes J1 to J21, wherein at least one of the plurality of partitions is neither square nor rectangular.

[1052] J23. The method according to any one of schemes J1 to J22, wherein the transformation generates the current block from the bitstream representation.

[1053] J24. The method according to any one of schemes J1 to J22, wherein the transformation generates a bitstream representation from the current block.

[1054] J25. An apparatus in a video system, comprising a processor and a non-transitory memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to perform the method according to any one of schemes J1 to J24.

[1055] J26. A computer program product stored on a non-transitory computer-readable medium, the computer program product comprising program code for performing the method according to any one of schemes J1 to J24.

[1056] Figure 54This is a block diagram of a video processing apparatus 5400. Apparatus 5400 can be used to implement one or more methods described herein. Apparatus 5400 can be included in a smartphone, tablet, computer, Internet of Things (IoT) receiver, etc. Apparatus 5400 may include one or more processors 5402, one or more memories 5404, and video processing hardware 5406. The processors (multiple) 5402 can be configured to implement one or more methods described in this document. The memories (multiple memories) 5404 can be used to store data and code for implementing the methods and techniques described herein. The video processing hardware 5406 can be used to implement some of the techniques described in this document in hardware circuitry.

[1057] In some embodiments, the video encoding / decoding method may use, as referenced Figure 54 The device described herein is implemented on a hardware platform.

[1058] Some embodiments of the disclosed technology involve making a decision or determination to enable a video processing tool or mode. In one example, when a video processing tool or mode is enabled, the encoder will use or implement the tool or mode in the processing of video blocks, but not necessarily modify the resulting bitstream based on the use of the tool or mode. That is, when a video processing tool or mode is enabled based on a decision or determination, the conversion from video blocks to a bitstream representation of the video will use that video processing tool or mode. In another example, when a video processing tool or mode is enabled, the decoder will process the bitstream knowing that the bitstream has been modified based on the video processing tool or mode. That is, the conversion from the bitstream representation of the video to video blocks will be performed using the video processing tool or mode enabled based on a decision or determination.

[1059] Some embodiments of the disclosed technology include making a decision or determination to disable a video processing tool or mode. In one example, when a video processing tool or mode is disabled, the encoder will not use the tool or mode to convert blocks of video into a bitstream representation of the video. In another example, when a video processing tool or mode is disabled, the decoder will process the bitstream knowing that the bitstream has not yet been modified using the video processing tool or mode enabled based on the decision or determination.

[1060] Figure 55This is a block diagram illustrating an example video processing system 5500 in which various techniques disclosed herein may be implemented. Various implementations may include some or all of the components of system 5500. System 5500 may include an input 5502 for receiving video content. The video content may be received in a raw or uncompressed format, such as 8 or 10-bit multi-component pixel values, or it may be in a compressed or encoded format. Input 5502 may represent a network interface, a peripheral bus interface, or a storage interface. Examples of network interfaces include wired interfaces (such as Ethernet, passive optical network (PON), etc.) and wireless interfaces (such as Wi-Fi or cellular interfaces).

[1061] System 5500 may include codec component 5504, which may implement the various encoding and codec methods described in this document. Codec component 5504 may reduce the average bit rate of the video from input 5502 to output to produce a codec representation of the video. Therefore, codec techniques are sometimes referred to as video compression or video transcoding techniques. The output of codec component 5504 may be stored or transmitted via connected communication, as shown in component 5506. Component 5508 may use the bitstream (or codec) representation of the stored or transmitted video received at input 5502 to generate pixel values ​​or displayable video sent to display interface 5510. The process of generating user-visible video from the bitstream representation is sometimes referred to as video decompression. Furthermore, although some video processing operations are referred to as “codec” operations or tools, it should be understood that codec tools or operations are used at the encoder and will be performed by the decoder as corresponding decoding tools or operations that reverse the encoded result.

[1062] Examples of peripheral bus interfaces or display interfaces may include Universal Serial Bus (USB), High Definition Multimedia Interface (HDMI), or DisplayPort. Examples of storage interfaces include SATA (Serial Advanced Technology Attachment), PCI, IDE, etc. The technologies described in this document can be found in a variety of electronic devices, such as mobile phones, laptops, smartphones, or other devices capable of performing digital data processing and / or video display.

[1063] Regarding the descriptions in this document, the bitstream representation of the current block of video is not necessarily in the form of consecutive bits, but can be based on bits appearing in different positions in the bitstream syntax (e.g., header fields and parameter sets).

[1064] Furthermore, in some publicly available embodiments, a shared Merge list construction process is used for sub-block encoding / decoding modes, including ATMVP and Affine Merge. Here, ATMVP and Affine Merge candidates can be added sequentially.

[1065] In some embodiments, for a triangular prediction pattern, the process of constructing a merge list for the two partitions is shared, or even the two partitions can choose their own merge candidate indices. When constructing this merge list, the spatial neighboring blocks and two temporal blocks of the block are examined. The motion information derived from the spatial neighbors and temporal blocks is referred to as regular motion candidates in our IDF. These regular motion candidates are further used to derive multiple TPM candidates. The transformation can be performed at the block level, and even the two partitions can use different motion vectors to generate their own prediction blocks.

[1066] For the remaining codec blocks, a shared Merge list construction process is used. Here, spatial / temporal / HMVP, paired bidirectional prediction Merge candidates, and zero-motion candidates can be inserted sequentially.

[1067] Those skilled in the art will understand that video encoding or decoding methods (compression / decompression) are disclosed. Geometric (or geometrical) partitioning of video blocks can be used to accurately track motion information and improve the compression performance of video encoding / decoding. It should also be understood that the use of the disclosed methods can enable embodiments to achieve further encoding / decoding gains and / or reduce implementation complexity compared to current video encoding / decoding technologies.

[1068] The disclosed and other schemes, examples, embodiments, modules, and functional operations described in this document can be implemented in digital electronic circuits, or in computer software, firmware, or hardware, including the structures disclosed in this document and their structural equivalents, or in combinations thereof. The disclosed and other embodiments can be implemented as one or more computer program products, i.e., one or more modules of computer program instructions encoded on a computer-readable medium for execution by or control of the operation of a data processing apparatus. The computer-readable medium can be a machine-readable storage device, a machine-readable storage substrate, a memory device, a combination of materials that implement machine-readable propagating signals, or a combination thereof. The term "data processing apparatus" encompasses all means, devices, and machines for processing data, including, for example, a programmable processor, a computer, or multiple processors or computers. In addition to hardware, the apparatus may also include code that creates an operating environment for the computer program in question, for example, code constituting processor firmware, a protocol stack, a database management system, an operating system, or a combination thereof. Propagating signals are artificially generated signals, such as machine-generated electrical, optical, or electromagnetic signals, which are generated to encode information for transmission to a suitable receiver device.

[1069] Computer programs (also known as programs, software, software applications, scripts, or code) can be written in any programming language, including compiled or interpreted languages, and can be deployed in any form, including as standalone programs or as modules, components, subroutines, or other units suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. A program can be stored as a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), as a single file dedicated to the program in question, or as multiple coordinating files (e.g., a file storing one or more modules, subroutines, or portions of code). Computer programs can be deployed to execute on one or more computers located at a single site or distributed across multiple sites interconnected by a communication network.

[1070] The processes and logic flows described herein can be executed by one or more programmable processors that run one or more computer programs to perform functions by manipulating input data and generating output. The processes and logic flows can also be executed by dedicated logic circuitry, and the devices can be implemented as dedicated logic circuitry, such as FPGAs (Field-Programmable Gate Arrays) or ASICs (Application-Specific Integrated Circuits).

[1071] For example, processors suitable for executing computer programs include general-purpose and special-purpose microprocessors, as well as any one or more processors in any type of digital computer. Typically, a processor receives instructions and data from read-only memory or random access memory, or both. The basic components of a computer are a processor for executing instructions and one or more memory devices for storing instructions and data. Typically, a computer will also include one or more mass storage devices for storing data, such as magnetic disks, magneto-optical disks, or optical disks, or operatively coupled to receive data from or transfer data to one or more mass storage devices, or both. However, a computer does not require such devices. Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including, for example, semiconductor memory devices such as EPROM, EEPROM, and flash memory devices; magnetic disks, such as internal hard disks or removable disks; magneto-optical disks; and CD-ROM and DVD-ROM discs. The processor and memory may be supplemented or incorporated into special-purpose logic circuitry.

[1072] While this patent document contains numerous details, these should not be construed as limiting any subject matter or the scope of the claims, but rather as descriptions of features characteristic of specific embodiments of a particular art. Certain features described in the context of individual embodiments in this patent document may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments. Furthermore, although the foregoing features may be described as functioning in certain combinations, or even initially claimed in this manner, in some cases one or more features may be removed from the claimed combination, and the claimed combination may refer to a sub-combination or a variation of a sub-combination.

[1073] Similarly, although operations are described in a specific order in the accompanying drawings, this should not be construed as requiring these operations to be performed in the specific order or sequence shown, or requiring all of the shown operations to obtain the desired result. Furthermore, the separation of various system components in the embodiments described in this patent document should not be construed as requiring such separation in all embodiments.

[1074] Only a few implementation methods and examples are described, and other implementations, enhancements and variations can be made based on the content described and illustrated in this patent document.

Claims

1. A method of video processing, comprising: maintaining one or more history-based motion vector prediction (HMVP) tables; performing a conversion between a current block of a video and a bitstream of the video, wherein the current block is partitioned using a geometric partition mode into a plurality of partitions; and selectively updating the one or more HMVP tables based on a condition and after performing the conversion; wherein the one or more HMVP tables are updated by including motion information of a first partition of the plurality of partitions and excluding motion information of a second partition of the plurality of partitions; or, the one or more HMVP tables are updated by including motion information associated with samples on a common boundary between at least two partitions of the plurality of partitions and excluding motion information associated with samples inside one partition of the plurality of partitions; or, the one or more HMVP tables are updated by including motion information associated with samples inside one partition of the plurality of partitions and excluding motion information associated with samples on a common boundary between at least two partitions of the plurality of partitions.

2. The method of claim 1, wherein, the one or more HMVP tables are updated based on a Merge candidate from a Merge candidate list associated with motion information of the current block partitioned into the plurality of partitions.

3. The method of claim 1, wherein the conversion is based on first motion information of a first partition of the plurality of partitions, and wherein the first motion information is derived based on second motion information of a second partition of the plurality of partitions.

4. The method of claim 3, wherein, the second motion information includes a prediction direction of the second partition.

5. The method of claim 3, wherein, the second motion information includes motion information associated with a reference picture list LX, where X = 0 or X = 1.

6. The method of claim 5, wherein, the second motion information includes motion information associated with a reference picture list LY when it is determined that motion information associated with the reference picture list LX is unavailable, where X = 0 or X = 1, and where Y = (1−X).

7. The method of claim 1, wherein motion information of more than one partition of the plurality of partitions is not derived from a same reference picture list.

8. The method of claim 7, wherein, a conforming bitstream includes an indication that motion information of two partitions of the plurality of partitions is derived from different reference picture lists.

9. The method of claim 7, wherein, a parity of a Merge index always indicates that motion information of two partitions of the plurality of partitions is derived from different reference picture lists.

10. The method of claim 3, further comprising: storing motion information including the first motion information or the second motion information, wherein the one or more history-based motion vector prediction (HMVP) tables are updated based on the stored motion information.

11. The method of claim 1, wherein, the method further comprises: selectively enabling derivation of motion information of a first partition and a second partition from a same reference picture based on a condition of the current block.

12. The method of claim 11, wherein, reference samples of the two partitions do not overlap.

13. The method of claim 11, wherein, MV0 and MV1 are motion vectors of the first partition and the second partition, respectively, and wherein MVX[0] and MVX[1] are horizontal and vertical components of MVX, respectively.

14. The method of claim 13, wherein, The condition includes abs(MV0[0] - MV1[0]) being less than a threshold, and wherein abs(x) returns an absolute value of x.

15. The method of claim 13, wherein, The condition includes abs(MV0[1] - MV1[1]) being less than a threshold, and wherein abs(x) returns an absolute value of x.

16. The method of claim 13, wherein, The condition includes abs(MV0[0] - MV1[0]) + abs(MV0[1] - MV1[1]) being less than a threshold, and wherein abs(x) returns an absolute value of x.

17. The method of claim 13, wherein, The condition includes Max(abs(MV0[0] - MV1[0]), abs(MV0[1] - MV1[1])) being less than a threshold, wherein abs(x) returns an absolute value of x, and wherein Max(a, b) returns x if x > y, and y if y > x.

18. The method of any one of claims 1 to 17, wherein, The conversion is based on stored motion information including at least one of a motion vector, a reference index, an indication of uni-prediction or bi-prediction, an indication of an inter-prediction direction, an indication of generalized bi-prediction (GBi), a motion vector resolution, or an indication of affine prediction.

19. The method of any one of claims 1 to 17, wherein, At least one of the plurality of partitions is non-square and non-rectangular.

20. The method of any one of claims 1 to 17, wherein, The conversion generates the current block from the bitstream.

21. The method of any one of claims 1 to 17, wherein, The conversion generates the bitstream from the current block.

22. An apparatus in a video system comprising a processor and a non-transitory memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to implement a method according to any of claims 1 to 21.

23. A computer program product stored on a non-transitory computer readable medium, the computer program product comprising program code for executing a method according to any of claims 1 to 21.

Images