Method and apparatus for joint indexing of geometric partition modes in video coding and decoding

Abstract

The present disclosure provides a method that implements a list of candidates for different combinations of geometric segmentation patterns (GPMs). A video encoder generates a list of candidates, each candidate specifying (i) a partition mode (ii) a first prediction mode and a second prediction mode. A first candidate in the candidate list specifies motion information for an inter coding partition of the current block. The video encoder transmits or receives a candidate selected from the candidate list. The video encoder divides the current block into a first partition and a second partition according to the division mode of the selected candidate. The video encoder generates a first prediction and a second prediction for the first partition and the second partition according to the first prediction mode and the second prediction mode of the selected candidate. The video encoder encodes or decodes the current block by using the first prediction and the second prediction.

Description

[Technical field]

[0001] In general, the present disclosure relates to video coding. More specifically, the present disclosure relates to methods for encoding pixel blocks via a geometric partitioning pattern (GPM). [Background technology]

[0002] Unless otherwise indicated herein, the approaches described in this section are not prior art to the claims listed below and are not admitted to be prior art by inclusion in this section.

[0003] High-efficiency video coding (HEVC) is an international video coding standard developed by the Joint Collaborative Team on Video Coding (JCT-VC). HEVC is a block-based discrete cosine transform (DCT)-like hybrid motion compensation coding architecture. The basic unit of compression is called a coding unit (CU), which is a 2Nx2N square pixel block, and each CU can be recursively divided into smaller CU units until a predefined minimum size is reached. Each CU includes one or more prediction units (PUs).

[0004] Versatile video coding (VVC) is the latest video coding standard developed by the Joint Video Expert Team (JVET) of ITU-T SG16 WP3 and ISO / IEC JTC1 / SC29 / WG11. The input video signal is predicted by a reconstruction signal derived from the coded picture area. The prediction residual signal is processed by block transformation. The transform coefficients are quantized and entropy coded into a bitstream along with other side information. The reconstruction signal is generated from the prediction signal and the reconstructed residual signal obtained by inverse transforming the de-quantized transform coefficients. The reconstruction signal is further processed by in-loop filtering to remove coding distortion. The decoded picture is stored in a frame buffer for prediction of subsequent pictures of the input video signal.

[0005] In VVC, a coded image is divided into non-overlapping square block regions, which are represented by related coding tree units (CTUs). The leaf nodes on the coding tree correspond to coding units (CUs). A coded image can be represented as a set of slices, each slice containing an integer number of CTUs. Individual CTUs in a slice are processed in raster-scan order. A bi-predictive (B) slice can be decoded using intra prediction or inter prediction and up to two motion vectors and reference indices to predict the sample values ​​of each block. A predicted (P) slice is decoded using intra prediction or inter prediction and up to one motion vector and reference index to predict the sample values ​​of each block. An intra (I) slice is decoded using only intra prediction.

[0006] A CTU can be partitioned into one or more non-overlapping coding units through a quadtree (QT) with a nested multi-type-tree (MTT) structure to adjust according to various local motion and texture characteristics. A CU can be further divided into smaller CUs using the following five methods: quadtree partitioning, vertical binary tree partitioning, horizontal binary tree partitioning, vertical center-side ternary tree partitioning, and horizontal center-side ternary tree partitioning.

[0007] Each CU includes one or more prediction units. The prediction unit and the associated CU syntax serve as the basic unit for sending predictor information. A specified prediction procedure is used to predict the values ​​of the relevant pixel samples within the PU. Each CU may include one or more transform units (TUs) to represent prediction residual blocks. A transform unit (TU) includes a transform block (TB) of luma samples and two corresponding transform blocks of chroma samples, and each TB corresponds to a residual block of samples of a color component. An integer transform is applied to a transform block. The level values ​​of the quantized coefficients are entropy coded into the bitstream along with other side information. The terms coding tree block (CTB), coding block (CB), prediction block (PB), and transform block (TB) are defined to specify a two-dimensional sample array of a single color component associated with a CTU, CU, PU, ​​and TU, respectively. Therefore, a CTU includes a luma CTB, two chroma CTBs, and related syntax elements. Similar relationships also apply to CU, PU, ​​and TU.

[0008] For each inter-predicted CU, motion parameters including motion vectors, reference picture indices and reference picture list usage indices, and other additional information are used to generate inter-predicted samples. The motion parameters can be sent explicitly or implicitly. When a CU is coded in skip mode, the CU is associated with a PU and has no significant residual coefficients, coded motion vector deltas or reference picture indices. By specifying a merge mode, the current CU can obtain motion parameters from neighboring CUs (including temporal and spatial candidates and schedules introduced in VVC). Merge mode can be applied to any inter-predicted CU. An alternative to merge mode is to explicitly transmit motion parameters, where motion vectors, corresponding reference picture indices for each reference picture list and reference picture list usage flags, and other necessary information are explicitly sent to each CU. [Summary of the invention]

[0009] The following summary of the invention is illustrative only and is not intended to be limiting in any way. That is, the following summary of the invention is only intended to introduce the concepts, highlights, benefits and advantages of the novel and non-obvious technologies described herein. Selected and not all embodiments are further described in the following detailed description. Therefore, the following summary of the invention is not intended to indicate the essential features of the claimed subject matter, nor is it intended to be used to determine the claimed subject matter.

[0010] Certain embodiments of the present disclosure provide a method for implementing a candidate list for different combinations of geometric partition modes (GPMs). A video encoder generates a candidate list, each candidate specifying (i) a partition mode and (ii) a first prediction mode and a second prediction mode. At least a first candidate in the candidate list specifies motion information of an inter-frame coded partition of a current block. The video encoder selects a candidate from the candidate list (e.g., an index assigned based on a cost calculated from the candidate in the candidate list) to send or receive. The video encoder partitions the current block into a first partition and a second partition based on the partition mode of the selected candidate. The video encoder generates a first prediction of the first partition and a second prediction of the second partition based on the first prediction mode and the second prediction mode of the selected candidate. The video encoder encodes or decodes the current block by using the first prediction and the second prediction.

[0011] In some embodiments, the candidates in the candidate list are assigned indices in an order determined based on the costs calculated for the candidates. The costs of the candidates are calculated based on reconstructed samples adjacent to the current block and reference samples derived from the partitioning mode and prediction mode of the candidate. In some embodiments, the candidate with the lowest cost among the costs calculated for all candidates in the candidate list is assigned the shortest code among all candidates in the candidate list.

[0012] The first candidate may specify a refinement of the inter prediction based on the motion information of the first candidate. For GPM-TM, the refinement may be based on minimizing the cost of matching between the reconstructed samples adjacent to the current block and the reconstructed samples adjacent to the reference block identified from the motion information of the first candidate. For GPM-MMVD, the refinement may be based on motion vector differences specified by distance and direction.

[0013] If both the first partition and the second partition of the first candidate are inter-coded, the first candidate in the candidate list may specify motion information for both the first partition and the second partition. For GPM-Intra, the first candidate may specify an intra prediction mode (e.g., intra prediction direction) for an intra-coded partition of the current block. The candidate list may also include a second candidate that specifies two intra prediction modes for two intra-coded partitions.

Brief Description of the Drawings

[0014] Figure 1 Describes intra prediction modes for different directions.

[0015] Figure 2 Template matching based on a search area around an initial motion vector (MV) is conceptually described.

[0016] Figure 3 A merge mode with motion vector difference (MMVD) candidates and their corresponding offsets is conceptually described.

[0017] Figure 4 Describes the partitioning of coding units by the geometric partitioning mode (GPM).

[0018] Figure 5 Describing an example uni-directional prediction candidate list for GPM partitioning and GPM's selection of uni-directional prediction MVs.

[0019] Figure 6 An example partitioned edge blending process for a GPM of coding units is described.

[0020] Figures 7A-7C Describes a GPM with inter and intra prediction for the current block.

[0021] Figure 8 Describes a GPM with intra and intra prediction for the current block.

[0022] Fig. 9 Conceptual description of extending the GPM partition edge to the reference template.

[0023] Fig.10 Example processes that a video encoder may perform to encode or decode a current block of a GPM partition are conceptually described.

[0024] Figures 11A-11C Conceptually describes the unified GPM candidate list used to encode the current block.

[0025] Fig.12 An example video encoder that can encode blocks of pixels using GPM is described.

[0026] Fig.13 Describes a portion of a video encoder that implements a unified GPM candidate list.

[0027] Fig.14 Conceptually describes the process by which a video encoder uses a unified candidate list for GPM combination.

[0028] Fig.15 An example video decoder that can decode GPM coded blocks is described.

[0029] Fig.16 Describes a portion of a video decoder that implements a unified GPM candidate list.

[0030] Fig.17 Conceptually describes the process by which a video decoder uses a unified candidate list combined with a GPM.

[0031] Fig.18 An electronic system for implementing certain embodiments of the present disclosure is conceptually described. [Specific implementation method]

[0032] In the following detailed description, many specific details are set forth by way of example in order to provide a thorough understanding of the relevant teachings. Any variations, derivations, and / or extensions based on the teachings described herein are within the scope of protection of the present disclosure. In some cases, well-known methods, procedures, components, and / or circuits associated with one or more implementation examples disclosed herein are described at a relatively high level without details to avoid unnecessarily obscuring the embodiments of the teachings of the present disclosure.

[0033] I. Intra-frame prediction mode

[0034] A. Intra-frame prediction mode

[0035] The intra prediction method uses a reference tier adjacent to the current prediction unit (PU) and one of a plurality of intra prediction modes to generate a predictor for the current PU. The direction of intra prediction can be selected from a mode set including a plurality of prediction directions. For each PU encoded by intra prediction, an index will be used and encoded to select one of a plurality of intra prediction modes, so that the corresponding prediction will be generated, and then the residual will be derived and converted.

[0036] Figure 1Describes the different directions of intra prediction modes. These intra prediction modes are called directional modes and are not included in DC mode or planar mode. As shown in the figure, there are 33 directional modes (V: vertical direction; H: horizontal direction), so they are labeled with H, H+1~H+8, H-1~H-7, V, V+1~V+8, V-1~V-8. General directional modes can be denoted as H+k mode or V+k mode, where k=±1, ±2, ...±8. Each intra prediction mode is also called an intra-prediction angle. In order to capture the random edge directions shown in naturally generated movies, the number of directional intra modes can be expanded from 33 (as used in HEVC) to 65, making k range from ±1 to ±16. These more dense directional intra prediction modes are applied to blocks of all sizes, and also to luma and chroma intra predictions. With the addition of DC mode and planar mode, the number of intra prediction modes increases to 35 (or 67).

[0037] Among the 35 (or 67) intra prediction modes, some modes (e.g., mode 3 or mode 5) are identified as the most probable modes (MPMs) for intra prediction in the current prediction block. The encoder can reduce the bit rate by signaling an index to indicate that one of the 35 (or 67) intra prediction modes is selected from the MPMs. For example, the intra prediction mode for the left prediction block and the intra prediction mode for the above prediction block are selected as MPMs. When the intra prediction modes of two adjacent blocks use the same intra prediction mode, the intra prediction mode can be selected as the MPM. When only one of the two adjacent blocks is available and is encoded in a directional mode, the two adjacent directional modes next to the directional mode can be selected as the MPM. The DC mode and the planar mode are also identified as MPMs to fill in the available fields of the MPM set, especially when the upper or top neighboring block is not available or is not intra-predicted, or when the intra prediction mode in the adjacent block is not a directional mode. If the intra prediction mode of the current prediction block is one of the MPM sets, one or two bits will be used to indicate which mode it is. Otherwise, the intra prediction mode of the current block is different from any mode in the MPM set, and the current block will be encoded as non-MPM mode. There are 32 non-MPM modes in total, and a fixed-length (five-bit) encoding method is used to indicate this mode.

[0038] The MPM list is built based on the intra modes of the left and above neighboring blocks. Assuming that the mode of the left neighboring block is marked as Left and the mode of the above neighboring block is marked as Above, the unified MPM list can be constructed as follows:

[0039] -When a neighboring block is not available, its intra mode will be set to planar mode by default.

[0040] -If Left and Above are both in non-angular mode:

[0041] .MPM list → {plane, DC, V, H, V–4, V+4}

[0042] -If one of Left and Above is in angular mode and the other is in non-angular mode:

[0043] . Set the larger of Left and Above to Max

[0044] .MPM list → {plane, Max, Max–1, Max+1, Max–2, Max+2}

[0045] -If Left and Above are both angular modes and they are different:

[0046] . Set the larger of Left and Above to Max

[0047] . Set the smaller of Left and Above to Min

[0048] If Max minus Min equals 1:

[0049] -MPM list → {Plane,Left,Above,Min–1,Max+1,Min-2}

[0050] Otherwise, if Max minus Min is greater than or equal to 62:

[0051] -MPM list → {Plane,Left,Above,Min+1,Max-1,Min+2}

[0052] Otherwise, if Max minus Min equals 2:

[0053] -MPM list → {Plane,Left,Above,Min+1,Min-1,Max+1}

[0054] .otherwise:

[0055] -MPM list → {Plane,Left,Above,Min–1,Min+1,Max-1}

[0056] -If Left and Above are both angular modes and they are the same:

[0057] .MPM list → {plane,Left,Left–1,Left+1,Left–2,Left+2}

[0058] The traditional angular intra prediction direction is defined as 45 degrees to -135 degrees clockwise. In Versatile Video Coding (VVC), some traditional angular intra prediction modes are adaptively replaced with wide-angle intra prediction modes for non-square blocks. The replaced mode can be indicated by the index of the original mode, which is remapped to the index of the wide-angle mode after parsing.

[0059] B. Template-based Intra Mode Derivation (TIMD)

[0060] For mode selection, template matching methods can be applied by computing the cost between reconstructed samples and predicted samples. One example is Template-based Intra Mode Derivation (TIMD). TIMD is a coding method that implicitly derives the intra prediction mode of a coding unit (CU) by using a neighboring template at the encoder and decoder, rather than the encoder indicating the correct intra prediction mode to the decoder.

[0061] C. Decoder Side Intra Mode Derivation (DIMD)

[0062] Decoder-side Intra Mode Derivation (DIMD) is a technique that derives two intra prediction modes / angles / directions from reconstructed neighboring samples (templates) of a block and combines these two predictors with a planar mode predictor with gradient-derived weights. DIMD mode is used as an alternative prediction mode and is always checked in high-complexity Rate Distortion Optimization (RDO) mode. To implicitly derive the intra prediction mode for a block, a texture gradient analysis is performed on both the encoder and decoder sides. The process starts with a Histogram of Gradients (HoG) with 65 entries, corresponding to the 65 angular / directional intra prediction modes. The size of these entries is determined during texture gradient analysis.

[0063] II. Template Matching (TM)

[0064] A. TM-based motion vector (MV) refinement

[0065] Template matching (TM) is an MV derivation method at the decoder side, which refines the motion information of the current CU by finding the closest match (i.e., the same size as the template) between the template of the current CU in the current image (e.g., the top and / or left adjacent blocks of the current CU) and a set of pixels in the reference image.

[0066] Figure 2 Template matching based on a search area around an initial motion vector (MV) is conceptually described. As shown, for a current CU 205 in a current image 200, the video encoder searches a reference image or frame 201 for a better or refined MV 211 within a [–8, +8] pixel (pel) search range around the initial MV 210. This search is based on minimizing the difference (or cost) between a current template 220 adjacent to the current CU 205 and a reference template 221 identified through the refined MV 211. Template matching can be performed with a search step size determined based on an adaptive motion vector resolution (AMVR). The template matching process can be cascaded after the bilateral matching process in merge mode.

[0067] In the advanced motion vector prediction (AMVP) mode, an MVP candidate is determined based on the error of template matching to select the one that meets the minimum difference between the current block template and the reference block template, and then the TM process is performed only on this specific MVP candidate for MV refinement. The TM process refines the MVP candidate starting from full-pel motion vector difference (MVD) accuracy (or 4 pixels for 4-pixel AMVR mode) within the pixel search range of [–8, +8] using an iterative diamond search. The MVP candidate is further refined by using cross search with full-pel MVD accuracy (or 4 pixels for 4-pixel AMVR mode) and then using cross search with half-pixel and 1 / 4-pixel depending on the search pattern of the AMVR mode in Table 1.

[0068] Table 1: Search mode of AMVR and merge mode with AMVR

[0069]

[0070] This search process ensures that the MVP candidate still maintains the same MV accuracy as indicated by the AMVR mode after the TM process. During the search process, if the difference between the previous minimum cost and the current minimum cost in the iteration is less than a threshold equal to the block area, the search process is stopped.

[0071] In some embodiments, when the merge mode is used, the video encoder may apply a similar TM search method to refine the merge candidates pointed to by the merge index. As shown in Table 1 above, TM may be performed up to 1 / 8 pixel MVD accuracy, or skip those that exceed half-pixel MVD accuracy, depending on whether an alternative interpolation filter (used when AMVR is in half-pixel mode) is used based on the merged motion information. In addition, when TM mode is enabled, template matching can be performed as a standalone process or as an additional MV refinement process between block-based and subblock-based bilateral matching (BM) methods, depending on whether BM can be enabled according to its enabling condition check.

[0072] B. Adaptive Reordering of MergeCandidates with TM (ARMC-TM)

[0073] In some embodiments, merge candidates can be adaptively reordered via template matching (TM). The reordering method is applied to regular merge mode, template matching (TM) merge mode, and affine merge mode (excluding candidates based on subblock-based temporal motion vector prediction (SbTMVP)). For TM merge mode, merge candidates are reordered before refinement.

[0074] After constructing a merge candidate list, the merge candidates are divided into several subgroups. For the normal merge mode and the TM merge mode, the size of the subgroup is set to 5. For the affine merge mode, the size of the subgroup is set to 3. The merge candidates in each subgroup are reordered ascendingly according to the cost value based on template matching. For simplicity, the merge candidates in the last but not the first subgroup will not be reordered.

[0075] The template matching cost of the merge candidate is measured by the sum of absolute differences (SAD) between the samples of the template of the current block and its corresponding reference samples. The template includes a set of reconstructed samples adjacent to the current block. The reference samples of the template are located using the motion information of the merge candidate. When the merge candidate uses bidirectional prediction, the reference samples of the template of the merge candidate are also generated using bidirectional prediction.

[0076] For subblock-based merge candidates with subblock size equal to Wsub×Hsub, the upper template includes several sub-templates of size Wsub×1, and the left template includes several sub-templates of size 1×Hsub. The motion information of the subblocks in the first row and first column of the current block is used to derive the reference samples for each sub-template.

[0077] III. Merge Mode with Motion Vector Difference (MMVD)

[0078] Unlike the normal merge mode, where the implicitly derived motion information is directly used to generate prediction samples for the current CU, in the merge mode with motion vector difference (MMVD), the derived motion information is further refined by the motion vector difference MVD. MMVD also extends the candidate list of the merge mode by adding additional MMVD candidates based on a predefined offset (also called MMVD offset).

[0079] After sending the skip flag and merge flag, the MMVD flag may be sent to specify whether the CU uses MMVD mode. If MMVD mode is used, the selected merge candidate is refined based on the MVD information. The MVD information also includes a merge candidate flag, a distance index for specifying the motion magnitude, and an index for specifying the direction of motion. The merge candidate flag is sent to specify which of the first two merge candidates will be used as a starting MV.

[0080] The distance index is used to specify the information of the motion amount by indicating a predefined offset from the starting MV. The offset can be added to the horizontal component or the vertical component of the starting MV. Table 2 below details an example of the mapping of distance index to predefined offset:

[0081] Table 2: Distance Index

[0082] Distance Index 0 1 2 3 4 5 6 7 Offset (in luma samples) 1 / 4 1 / 2 1 2 4 8 16 32

[0083] The direction index represents the direction of the MVD relative to the starting point. As shown in Table 3, the direction index can represent one of four directions.

[0084] Table 3: Sign of MV offset specified by direction index

[0085] Direction Index 00 01 10 11 x-axis + – N / A N / A y-axis N / A N / A + –

[0086] It should be noted that the meaning of the MVD symbol may change according to the information of the starting MV. When the starting MV is a unidirectional prediction MV or a bidirectional prediction MV and both MVs or lists (list0 and list1) point to the same side of the current image (i.e., the picture order counts (POC) of both reference images are greater than or less than the POC of the current image), the symbol in Table 3 explicitly indicates the sign of the MV offset to be added to the starting MV. When the starting MV is a bidirectional prediction MV but the two MVs point to different sides of the current image (i.e., the POC of one reference image is greater than the POC of the current image, and the POC of the other reference image is less than the POC of the current image), each symbol in Table 3 explicitly indicates the sign of the MV offset to be added to list0 of the starting MV, and the sign of the MV offset to be added to list1 of the starting MV has the opposite value. In some embodiments, a distance value (MmvdDistance) and a direction sign (MmvdSign) can be derived from a predefined offset (MmvdOffset) of an MMVD candidate and used as its representation.

[0087] Figure 3Conceptual illustration of MMVD candidates and their corresponding offsets. The figure illustrates a merge candidate 310 as a starting MV and several MMVD candidates located in the vertical and horizontal directions. Each MMVD candidate is derived by applying an offset to the merge candidate 310 (starting MV). For example, MMVD candidate 322 is derived by adding an offset of 2 to the horizontal component of the merge candidate 310, and MMVD candidate 324 is derived by adding an offset of -1 to the vertical component of the merge candidate 310. MMVD candidates with offsets in the horizontal direction (such as MMVD candidate 322) are called horizontal MMVD candidates. MMVD candidates with offsets in the vertical direction (such as MMVD candidate 324) are called vertical MMVD candidates.

[0088] IV. Intra Block Copy (IBC) Mode

[0089] Intra-block copying (IBC) is also known as current picture referencing (CPR). IBC (or CPR) motion vectors refer to the motion vectors of the reconstructed reference samples in the current picture. For some embodiments, the IBC prediction mode is considered as a third prediction mode for encoding a CU in addition to intra or inter prediction modes.

[0090] Since the IBC mode is implemented as a block-level coding mode, block matching (BM) is performed at the encoder to find the best block vector (or motion vector) for each CU. Here, the block vector (BV) is used to indicate the displacement from the current block to a reference block that has been reconstructed in the current image. The luma block vector of an IBC-coded CU has integer precision. V. Geometric Partitioning Mode (GPM) and its variants

[0091] A. Geometric segmentation

[0092] A merge mode uses a CU level flag to indicate the geometric partitioning mode (GPM). Other merge modes include normal merge mode, MMVD mode, combined inter and intra prediction (CIIP) mode, and subblock merge mode. For each possible CU size w×h=2 m ×2 n, the geometric partitioning mode supports a total of 64 partitioning modes, where m,n∈{3…6} and does not include 8x64 and 64x8.

[0093] Figure 4 Illustrate the partitioning of a CU by a geometric partitioning pattern (GPM). Each GPM partition or GPM split is a partitioning pattern characterized by a distance-angle pairing that defines a bisecting line or a segmenting line. The figure illustrates an example of a GPM partition grouped by the same angle. As shown in the figure, when using GPM, a CU is divided into at least two parts by a geometrically positioned straight line. The location of the partition line is mathematically derived based on the angle and offset parameters of a particular partition.

[0094] Each partition in a CU formed by the GPM partitioning pattern uses its own motion (vector) for inter-frame prediction. In some embodiments, each partition only allows unidirectional prediction, meaning that each part has a motion vector and a reference index. Similar to traditional bidirectional prediction, unidirectional prediction motion restriction is used to ensure that only two motion compensated predictions are performed for each CU.

[0095] If GPM is used for the current CU, a geometric partitioning index and two merge indices (one for each partition) indicating the partitioning mode of the geometric partitioning (angle and offset) are further sent. According to a partitioning mode, each of the at least two partitions established by geometric partitioning can be assigned a merge index to select a candidate from a unidirectional prediction candidate list (also called a GPM candidate list). Therefore, a pair of merge indices for two partitions selects a pair of merge candidates. The maximum number of candidates for the GPM candidate list can be explicitly indicated in the sequence parameter set (SPS) to specify the binarization of the syntax for the GPM merge index. After predicting each of the at least two partitions, a blending processing with adaptive weights is used to adjust the sample values ​​along the edges of the geometric partition. This is the prediction signal for the entire CU. As with other prediction modes, the conversion and quantization process will be applied to the entire CU. The motion field of the CU as predicted by the GPM is then stored.

[0096] The unidirectional prediction candidate list (GPM candidate list) for a GPM partition can be directly derived from the merge candidate list of the current CU. Figure 5Describes a unidirectional prediction candidate list 500 for a GPM partition in an example, and the selection of the unidirectional prediction MV for the GPM. The GPM candidate list 500 is established in an even-odd manner, with only unidirectional prediction candidates that alternate between two MVs (L0 and L1). Let n be the index of the unidirectional prediction motion in the unidirectional prediction candidate list for the GPM. The LX (i.e., L0 or L1, where X equals the parity of n) motion vector of the nth extended merge candidate is taken as the nth unidirectional prediction motion vector of the GPM. (These motion vectors are marked with "x" in the figure). If the corresponding LX motion vector of the nth extended merge candidate does not exist, the L(1-X) motion vector of the same candidate is used instead as the unidirectional prediction motion vector for the GPM.

[0097] As described above, sample values ​​along the edges of geometric partitions are adjusted using a blending process with adaptive weights. Specifically, after predicting each part of a geometric partition using the motion of the geometric partition itself, a blending process is applied to at least two prediction signals to derive samples around the edges of geometric partitions. The blending weights for each position of the CU are derived based on the distance between each position and the partition edge. The distance from a position (x, y) to the partition edge is derived as follows:

[0098]

[0099]

[0100]

[0101]

[0102] where i,j are the indices of the angle and offset of the geometric partition, which depends on the indicated geometric partition index. x,j and ρ y,j The sign of depends on the angle index i. The weights of each part of the geometric partition are calculated as follows:

[0103] wIdxL(x,y)=partIdx? 32+d(x,y):32-d(x,y) (5)

[0104]

[0105] w 1 (x,y)=1-w 0 (x,y) (7)

[0106] The variable partIdx depends on the angle index i. Figure 6The segmentation edge blending process of the GPM of an example CU 600 is described. In the figure, blending weights are generated based on initial blending weights w0.

[0107] As described above, the motion field of a CU predicted using GPM is stored. Specifically, Mv1 from the first part of the geometric partition, Mv2 from the second part of the geometric partition, and the combination of Mv1 and Mv2 are stored in the motion field of the GPM-coded CU. The method for determining the type of stored motion vectors for each individual position in the motion field is as follows:

[0108] sType=abs(motionIdx)<32?

[0109] 2: (motionIdx ≤ 0 ? (1-partIdx):partIdx) (8)

[0110] where motionIdx is equal to d(4x+2,4y+2) recalculated according to equation (1). partIdx depends on the angle index i. If sType is equal to 0 or 1, Mv0 or Mv1 is stored in the corresponding motion field, otherwise if sType is equal to 2, the combined Mv of Mv0 and Mv2 is stored. The combined Mv is generated by the following process: (i) If Mv1 and Mv2 come from different reference image lists (one from L0 and the other from L1), Mv1 and Mv2 are simply combined to form a bidirectional prediction motion vector; (ii) Otherwise, if Mv1 and Mv2 come from the same list, only the unidirectional prediction motion Mv2 is stored.

[0111] B. GPM with Merge Vector Difference;

[0112] GPM-MMVD)

[0113] In some embodiments, GPM is extended by using motion vector refinement for existing GPM unidirectional MVs. First, a flag pic_fpel_mmvd_enabled_flag is sent to the GPMCU to indicate whether this mode is used. If this mode is used, each geometric partition of the GPM CU can further decide whether to send motion vector differences (MVD). If MVD is sent for a geometric partition, the motion of the partition is further refined by sending MVD information after selecting the GPM merge candidate. All other procedures are the same as in GPM.

[0114] MVD is represented as a pair of distance and direction, similar to MMVD. A total of 9 candidate distances (1 / 4 pixel, 1 / 2 pixel, 1 pixel, 2 pixels, 3 pixels, 4 pixels, 6 pixels, 8 pixels, 16 pixels) and 8 candidate directions (four horizontal / vertical directions and four diagonal directions) are included in the GPM with MMVD (GPM-MMVD). In addition, when the flag pic_fpel_mmvd_enabled_flag is equal to 1, MVD is shifted to the left by 2 like MMVD.

[0115] C. GPM with Template Matching (GPM-TM)

[0116] In some embodiments, template matching (TM) can be used to refine the MV of the GPM partition. When GPM mode is enabled for a CU, a CU-level flag is sent to indicate whether TM is to be used for the two geometric partitions. The motion information of each geometric partition is refined using TM. When TM is selected, the template is built using the left, top, or left and top adjacent samples depending on the partition angle. Table 4 below shows the templates for the first and second geometric partitions, where A represents the use of the top sample, L represents the use of the left sample, and L+A represents the use of the left and top samples.

[0117] Table 4:

[0118] Split Angle 0 2 3 4 5 8 11 12 13 14 Partition 1 A A A A L+A L+A L+A L+A A A Partition 2 L+A L+A L+A L L L L L+A L+A L+A Split Angle 16 18 19 20 21 24 27 28 29 30 Partition 1 A A A A L+A L+A L+A L+A A A Partition 2 L+A L+A L+A L L L L L+A L+A L+A

[0119] The motion is then refined by minimizing the difference between the current template and the template of the reference image using the same search mode with a merge mode with a disabled half-pel interpolation filter. The steps of building the GPM candidate list are as follows: (1) The video encoder directly derives interlaced List-0 MV candidates (List-0 MV candidates) and List-1 MV candidates (List-1 MV candidates) from the regular merge candidate list, where List-0 MV candidates have a higher priority than List-1 MV candidates. Redundant MV candidates are removed using a pruning method with an adaptive threshold based on the current CU size; (2) The video encoder further directly derives interlaced List-1 MV candidates and List-0 MV candidates from the regular merge candidate list, where List-1 MV candidates have a higher priority than List-0 MV candidates. A similar pruning method with adaptive threshold is also used to remove redundant MV candidates; (3) The video encoder fills the GPM candidate list with zero MV candidates with value zero until the GPM candidate list is filled.

[0120] In some embodiments, GPM-MMVD and GPM-TM can only be used for a CU that uses GPM. In some embodiments, this is accomplished by first sending the GPM-MMVD syntax. When both GPM-MMVD control flags are set to false (i.e., GPM-MMVD is disabled on both GPM partitions), the GPM-TM flag is sent to indicate whether template matching refinement is to be performed on the GPM partition, as described in the second part (II.) above. Otherwise (at least one GPM-MMVD flag is set to true), the value of the GPM-TM flag is inferred to be false, and MMVD refinement is performed on the GPM partition, as described in the third part (III.) above.

[0121] D. GPM with inter and intra predictions (GPM-Intra-Inter)

[0122] In GPM with inter and intra prediction, one GPM region is encoded by inter prediction through GPM-inter, and the other GPM region is encoded by intra prediction through GPM-intra. The final prediction samples are generated by weighting the inter prediction samples and intra prediction samples of each GPM-separated region. The inter prediction samples are derived from the inter GPM. The intra prediction samples are derived from the intra prediction mode (IPM) candidate list and the index sent by the encoder. The size of the IPM candidate list is predefined to be 3.

[0123] Figures 7A-7C Depicting the GPM with inter and intra prediction for a current block 700. The current block is divided into an inter coded region 710 and an intra coded region 720 by the GPM. Fig. 7A Region 720 is depicted as being intra-coded via parallel angular mode (parallel mode) with respect to GPM block boundaries. Figure 7B Region 720 is depicted as being intra-coded via perpendicular angular mode (vertical mode) for GPM block boundaries. Figure 7C A region 720 is depicted that is intra-coded via planar mode. In some embodiments, parallel mode, vertical mode, and planar mode are available IPM candidates in the IPM candidate list. Figure 8 A GPM with intra and intra prediction for a current block 800 is depicted.

[0124] GPM with intra-frame and inter-frame prediction can be limited to reduce IPM signaling overhead and avoid increasing the size of the intra-frame prediction circuit of the hardware decoder. In addition, direct motion vector and IPM storage are used in the GPM-blending area to further improve the encoding performance.

[0125] In the intra mode derivation (DIMD) at the decoder side, the neighboring mode based IPM derivation Parallel mode is first registered. Therefore, if the same IPM candidate does not exist in the list, at most two IPM candidates derived from the DIMD method and / or neighboring block derivation can be registered. As for the neighboring mode derivation, at most five positions are provided for the available neighboring blocks, but they are limited by the angle of the GPM block boundary, as shown in Table 4 above. Table 4 describes the positions of available neighboring blocks for IPM candidate derivation based on the angle of the GPM block boundary, which has been used for GPM with template matching (GPM-TM).

[0126] GPM-intra can be combined with GPM with Merged Vector Difference (GPM-MMVD). Template-based intra mode derivation (TIMD) can be applied to IPM candidates of GPM-intra to further improve the performance of encoding. Parallel modes can be registered first, and then TIMD, DIMD, and IPM candidates of neighboring blocks can be registered.

[0127] E. GPM segmentation mode based on TM reordering

[0128] In some embodiments, in TM-based reordering of GPM partitioning modes, individual TM cost values ​​of GPM partitioning modes are calculated given the motion information of the current GPM block. Then, all GPM partitioning modes are reordered in ascending powers according to the TM cost values. Instead of sending the GPM partitioning mode, an index using a Golomb-Rice code is sent to indicate where the exact GPM partitioning mode is located in the reordering list.

[0129] The reordering method of the GPM partition pattern is a two-step process performed after the individual reference templates of the two GPM partitions in the coding unit are generated, and the steps are as follows:

[0130] (1) Extend the GPM segmentation edge into the reference templates of the two GPM partitions, generate 64 reference templates, and calculate the individual TM cost of each of the 64 reference templates.

[0131] (2) Re-order the GPM partitioning patterns according to their TM cost values ​​in ascending order, and mark the best 32 as available partitioning patterns. After re-ordering using the TM cost in ascending order, an index is sent.

[0132] Fig. 9Conceptual description of extending a GPM partition edge to a reference template. This figure depicts a current block 900, which is GPM encoded and partitioned into a GPM partition 910 and a GPM partition 920 by a GPM partition edge 905. The adjacent area of ​​the current block is used as a template 930 (including a top portion and a left portion). In order to calculate the TM cost, the GPM partition edge 905 is extended to divide the template 930 into a template section 930a and a template section 930b. (Template section 930a is used for the TM of GPM partition 910, and template section 930b is used for the TM of GPM partition 920). In some embodiments, although the GPM partition edge extends from the GPM partition edge of the current CU to the template 930, the GPM blending process is not used across the edge in this template area.

[0133] F.GPM decoding program

[0134] Fig.10 An example process 1000 that a video encoder may perform to encode or decode a current block of a GPM partition is conceptually described. In some embodiments, one or more processing units (e.g., processors) of a computing device implementing a video encoder (encoder or decoder) perform the process 1000 by executing instructions stored in a computer-readable medium. In some embodiments, an electronic device implementing a video encoder performs the process 1000.

[0135] The video encoder parses (at block 1010) a GPM syntax, which may include a hybrid index, a GPM mode index, a merge index, an intra index. Each of the two or more GPM partitions may be encoded using GPM variations, such as GPM, GPM-MMVD, GPM-TM, and GPM-Intra. Thus, combinations of GPM variations used to encode a GPM coding block may include {GPM, GPM}, {GPM, GPM-MMVD}, {GPM-MMVD, GPM-Intra}, {GPM-TM, GPM-Intra}, {GPM-TM, GPM-TM}, etc.

[0136] The video encoder generates (at block 1020) a GPM merge candidate list. If GPM-MMVD is used, the video encoder may use MMVD for the selected candidate. If GPM-TM is used for two or more candidates, the video encoder may use the left and / or top templates to calculate the TM cost. If GPM-Intra is used, the video encoder may generate an intra prediction mode (IPM) candidate list.

[0137] The video encoder performs (at block 1030) motion compensation and blending on the GPM partitions. The video encoder may reorder the 64 partition indices and select the partition index by the mode index. If GPM-Intra is used, the video encoder may perform intra prediction on the selected candidate. If the other partitions use GPM-Intra, the video encoder may perform overlapped block motion compensation (OBMC) on the inter-predicted partitions. If GPM-Intra is selected, the video encoder may perform GPM blending on rounded samples.

[0138] The video encoder uses (at block 1040) motion information for the GPM partitions for inter prediction. No motion information is provided for the GPM-Intra partitions. The video encoder may also perform OBMC if GPM-Intra is not used.

[0139] G. Unified GPM Candidate List

[0140] In some embodiments, the video encoder may establish a GPM candidate list, at least one item in the candidate list corresponds to a combination of a partition partition mode and two intra prediction modes, and different combinations of one of the 26 partition modes and three intra prediction modes may be formed. The video encoder may send the index of the candidate selected from the GPM candidate list. In some embodiments, the candidate list is reordered based on the TM cost, where the SAD between the prediction and reconstruction of the template is used for sorting.

[0141] In some embodiments, a joint indexing scheme is used for blocks encoded by GPM or any of its variants or extended applications (e.g., GPM for skip, merge, direct, intra mode, inter mode, and / or IBC mode). In some embodiments, the joint index is used to indicate a combination of:

[0142] ● A partitioning mode and one or more prediction modes for multiple prediction hypotheses

[0143] • A subset from the partitioning mode and one or more prediction modes for multiple prediction hypotheses.

[0144] ●More than one forecasting model for multiple forecasting hypotheses.

[0145] In some embodiments, when GPM / GPM-MMVD / GPM-TM is used to encode a block, a joint index may indicate a combination of a partition mode and two motion candidates / information. A joint index may indicate a combination of two motion candidates / information. A joint index may indicate a combination of a partition mode and a motion candidate / information. An index is sent after power-up reordering using TM cost.

[0146] In some embodiments, for a block encoded in GPM-MMVD, the combination indicated by the joint index includes a partition mode and a GPM-MMVD distance; or, the combination may include a partition mode and a GPM-MMVD direction. Or, the combination may include a partition mode, a GPM-MMVD distance, and a GPM-MMVD direction.

[0147] In some embodiments, a list containing different combinations is reordered according to template matching cost. In some embodiments, each combination in the list includes a segmentation mode and two motion candidates / information. In some embodiments, each combination in the list includes two motion candidates / information, and the template matching cost is determined by the candidates in the list of combinations and the segmentation mode sent. In some embodiments, each combination in the list includes two motion candidates / information, and the template matching cost is determined by the candidates in the combination list and the segmentation mode sent.

[0148] In some embodiments, the joint index indicates a combination that comes from a list of combinations that are reordered by a template matching-based method. In some embodiments, the order in the reordered list of combinations means the sending priority of the combination. That is, the combination that is at the first position in the list of combinations is sent / parsed with the shortest codeword among all the combinations in the list. In some embodiments, the syntax used to indicate the combination that is at the first position in the list of combinations is encoded with one or more context information. For example, the selection of context information may depend on block width, height, area, and adjacent pattern information. For another example, one or more context information used will not be reused by the remaining combinations in the list of combinations. For another example, the syntax used to indicate the combinations after the first position in the list of combinations is not encoded with the context information.

[0149] Figures 11A-11C The unified GPM candidate list 1105 for encoding the current block 1100 is conceptually described. The unified GPM candidate list 1105 can be used for different GPM variants. Fig.11AAs shown, the GPM candidate list 1105 includes items corresponding to different combinations of GPM partition modes and prediction modes of two (or more) GPM partitions. Each candidate in the list indicates (identifies) (i) a GPM partition mode that divides the current block into two or more GPM partitions and (ii) two (or more) prediction modes for two or more GPM partitions of the candidate. The prediction mode of a GPM partition can indicate whether the partition is encoded using intra-frame prediction or inter-frame prediction, and can specify whether an intra-frame prediction mode or motion information (MV and reference image) is to be used. For inter-frame coded partitions, the combination of candidates can further specify how the motion information is refined (e.g., using template matching or using MMVD direction + distance).

[0150] For example, the candidate list 1105 includes a candidate 1140 that divides the current block 1100 into an inter-frame prediction partition and an intra-frame prediction partition, and the item corresponding to the candidate 1140 specifies the GPM partition mode, the intra-frame prediction mode of the intra-frame prediction partition, and the motion information of the inter-frame prediction partition. In another example, the candidate list 1105 includes a candidate 1150 that divides the current block 1100 into two inter-frame prediction partitions, and the item corresponding to the candidate 1150 specifies the GPM partition mode and the motion information of each inter-frame prediction partition. Although not described, the item 1140 and the item 1150 may also specify information on motion refinement of individual inter-frame coding partitions.

[0151] The items in the GPM candidate list 1105 are reordered according to the template matching cost of the candidate, where an index is assigned to the corresponding item / candidate in the list. The template matching cost of the candidate in the list is calculated based on comparing (i) the reconstructed samples of the template area 1110 adjacent to the current block 1100 with (ii) the reference samples derived according to the prediction mode of the GPM partition specified by the candidate.

[0152] Fig. 11B 11. The TM cost calculation for candidate 1140 is described. As shown, the GPM partitioning mode of candidate 1140 divides the current block 1100 into two partitions (partition 1141 and partition 1142). Partition 1141 performs intra prediction using the specific intra prediction mode specified by candidate 1140. Partition 1142 performs inter prediction using the motion information provided by candidate 1140 to refer to a reference block 1120 (in the reference image or the current image). The TM cost for candidate 1140 is calculated from reference samples derived from (i) neighboring samples 1115 of the template region 1110 identified by the specified intra prediction mode and (ii) neighboring samples 1130 of the reference block 1120. The reference samples may be derived by blending according to the GPM partitioning mode specified by candidate 1140.

[0153] Fig. 11C TM cost calculation for candidate 1150 is described. As shown, the GPM partitioning mode of candidate 1150 divides the current block 1100 into two inter-frame prediction partitions (partition 1151 and partition 1152). Partition 1151 has motion information (e.g., MV or BV) that can refer to reference block 1121. Partition 1152 has motion information (e.g., MV or BV) that can refer to reference block 1122. The TM cost calculation for candidate 1150 is based on reference samples derived from (i) neighboring samples 1131 of reference block 1121 and (ii) neighboring samples 1132 of reference block 1122. The reference samples can be derived by blending according to the GPM partitioning mode specified by candidate 1150.

[0154] In some embodiments, a unified candidate list is generated, which includes prediction modes for one or more prediction hypotheses, and different GPM modes (different GPM variations and extensions) share the same unified candidate list (different GPM modes refer to and use the same list). Therefore, only one circuit for generating a unified candidate list is used and is reused by multiple GPM modes. In some embodiments, when a particular GPM mode uses a unified candidate list, how to select one or more candidates from the unified candidate list (and / or how to use the selected one or more candidates in the unified candidate list) may depend on a signal belonging to the particular GPM mode (rather than a signal shared / unified with other GPM modes).

[0155] In some embodiments, a candidate list is generated that includes a combination of (e.g., segmentation mode, motion information, etc.), and different GPM modes share the same unified candidate list. Thus, only one circuit for generating a candidate list is used and is reused by multiple GPM modes. In some embodiments, when a GPM mode uses a unified candidate list, how one or more candidates are selected from the unified candidate list and / or how the selected one or more candidates in the unified candidate list are used may depend on signals belonging to the GPM mode (rather than signals shared / unified with other GPM modes).

[0156] The aforementioned proposed method may be implemented in an encoder and / or a decoder. For example, the proposed method may be implemented in an inter-frame prediction module (and / or an intra-frame block copy prediction module) of an encoder and / or an inter-frame prediction module (and / or an intra-frame block copy prediction module) of a decoder.

[0157] VI. Example Video Encoder

[0158] Fig.12An example video encoder 1200 that can use GPM to encode pixel blocks is described. As shown, the video encoder 1200 receives an input video signal from a video source 1205 and encodes the signal into a bitstream 1295. The video encoder 1200 has several components or modules for encoding the signal from the video source 1205, which include at least some components selected from: a transform module 1210, a quantization module 1211, an inverse quantization module 1214, an inverse transform module 1215, an intra-picture estimation module 1220, an intra-frame prediction module 1225, a motion compensation module 1230, a motion estimation module 1235, an in-loop filter 1245, a reconstructed picture buffer 1250, an MV buffer 1265, an MV prediction module 1275, and an entropy encoder 1290. The motion compensation module 1230 and the motion estimation module 1235 are part of the inter-frame prediction module 1240.

[0159] In some embodiments, the conversion module 1210, the quantization module 1211, the inverse quantization module 1214, the inverse conversion module 1215, the intra-image estimation module 1220, the intra-frame prediction module 1225, the motion compensation module 1230, the motion estimation module 1235, the inter-frame prediction module 1240, the intra-loop filter 1245, the reconstructed image buffer 1250, the MV buffer 1265, the MV prediction module 1275, and the entropy encoder 1290 are modules of software instructions executed by one or more processing units (such as: processors) of a computing device or electronic device. In some embodiments, the conversion module 1210, the quantization module 1211, the inverse quantization module 1214, the inverse conversion module 1215, the intra-image estimation module 1220, the intra-frame prediction module 1225, the motion compensation module 1230, the motion estimation module 1235, the inter-frame prediction module 1240, the intra-loop filter 1245, the reconstructed image buffer 1250, the MV buffer 1265, the MV prediction module 1275, and the entropy encoder 1290 are modules of hardware circuits implemented by one or more integrated circuits (ICs) of an electronic device. Although the conversion module 1210, the quantization module 1211, the inverse quantization module 1214, the inverse conversion module 1215, the intra-image estimation module 1220, the intra-frame prediction module 1225, the motion compensation module 1230, the motion estimation module 1235, the inter-frame prediction module 1240, the intra-loop filter 1245, the reconstructed image buffer 1250, the MV buffer 1265, the MV prediction module 1275, and the entropy encoder 1290 are described as separate modules, some modules may be combined into a single module.

[0160] The video source 1205 provides an original video signal, which presents uncompressed pixel data for each video frame. The subtractor 1208 calculates the difference between the original video pixel data of the video source 1205 and the predicted pixel data 1213 from the motion compensation module 1230 or the intra-frame prediction module 1225 as a prediction residual 1209. The transform module 1210 transforms the difference (or residual pixel data or residual signal) into a transform coefficient 1216 (e.g., by performing a discrete cosine transform (DCT)). The quantization module 1211 quantizes the transform coefficient 1216 into quantized data (or quantized coefficient) 1212, which is then encoded into a bitstream 1295 by the entropy encoder 1290.

[0161] The inverse quantization module 1214 inversely quantizes the quantized data (or quantized coefficients) 1212 to obtain transform coefficients 1216, and the inverse transform module 1215 inversely transforms the transform coefficients 1216 to generate a reconstructed residual 1219. The reconstructed residual 1219 is added to the predicted pixel data 1213 to generate reconstructed pixel data 1217. In some embodiments, when performing intra-picture prediction and spatial MV prediction, the reconstructed pixel data 1217 is temporarily stored in a line buffer (not shown). The reconstructed pixels are filtered by the in-loop filter 1245 and stored in the reconstructed picture buffer 1250. In some embodiments, the reconstructed picture buffer 1250 is a storage device external to the video encoder 1200. In some embodiments, the reconstructed picture buffer 1250 is a storage device internal to the video encoder 1200.

[0162] The intra estimation module 1220 performs intra prediction based on the reconstructed pixel data 1217 to generate intra prediction data. The intra prediction data is provided to the entropy encoder 1290 to encode the intra prediction data into a bitstream 1295. The intra prediction data is also used by the intra prediction module 1225 to generate the predicted pixel data 1213.

[0163] The motion estimation module 1235 performs inter-frame prediction by generating MVs to refer to pixel data of decoded frames stored in the reconstructed image buffer 1250. These MVs are provided to the motion compensation module 1230 to generate predicted pixel data.

[0164] The video encoder 1200 does not encode the complete actual MV into the bitstream, but uses the MV prediction module 1275 to generate a predicted MV, and encodes the difference between the MV used for motion compensation and the predicted MV into residual motion data and stores it into the bitstream 1295.

[0165] The MV prediction module 1275 generates a predicted MV based on a reference MV generated for encoding a previous video frame (i.e., a motion compensated MV for performing motion compensation). The MV prediction module 1275 retrieves a reference MV from a previous video frame in the MV buffer 1265. The video encoder 1200 stores the MV generated for the current video frame in the MV buffer 1265 as a reference MV for generating the predicted MV.

[0166] The MV prediction module 1275 uses the reference MV to create a predicted MV. The predicted MV can be calculated by spatial MV prediction or temporal MV prediction. The difference (residual motion data) between the predicted MV and the motion compensation MV (MC MV) of the current frame is encoded into a bitstream 1295 by the entropy encoder 1290.

[0167] The entropy encoder 1290 encodes various parameters and data into a bitstream 1295 by using entropy coding techniques (e.g., context-adaptive binary arithmetic coding (CABAC) or Huffman encoding). The entropy encoder 1290 encodes various header elements, flags, and quantized transform coefficients 1212 and residual motion data as syntax elements into the bitstream 1295. The bitstream 1295 is in turn stored in a storage device or transmitted to a decoder via a communication medium (e.g., a network).

[0168] The in-loop filter 1245 performs a filtering or smoothing operation on the reconstructed pixel data 1217 to reduce encoding artifacts, particularly at the boundaries of pixel blocks. In some embodiments, the filtering or smoothing operation performed by the in-loop filter 1245 includes a deblocking filter (DBF), a sample adaptive offset (SAO), and / or an adaptive loop filter (ALF).

[0169] Fig.13A portion of a video encoder 1200 that implements a unified GPM candidate list is described. The GPM candidate generation module 1310 specifies various candidate GPM combinations. Each candidate GPM combination can specify a GPM partitioning mode to divide the current block into two or more GPM partitions, and two or more prediction modes used in these two or more GPM partitions. For intra-coded GPM partitions, the candidate combination can specify an intra-frame prediction mode. For inter-coded GPM partitions, the candidate combination can specify motion information (such as: motion vector, reference image, or selected merge candidate). The motion information can be obtained by retrieving from the MV buffer. The candidate combination can also specify a refinement of the motion information according to GPM-MMVD or GPM-TM.

[0170] For each candidate GPM combination, the template identification module 1320 extracts samples of the current template and the reference template from the reconstructed image buffer 1250 based on the prediction mode specified by the candidate GPM combination. For a candidate GPM combination that specifies one or more inter-frame coded partitions, the template identification module 1320 can extract neighboring samples of the current block as the current template and use the motion information specified in the combination to extract neighboring samples of the reference block as the reference template. For a candidate GPM combination that specifies one or more intra-frame coded partitions, the template identification module 1320 can extract neighboring samples of the current block as the current template and extract neighboring samples of the current template identified through the specified intra-frame prediction mode as the reference template.

[0171] The template identification module 1320 provides the reference template and the current template of the currently indicated candidate GPM combination to the cost calculator 1330, which performs template matching to generate the cost of the indicated candidate GPM combination. When determining the TM cost, the cost calculator 1330 can combine the reference templates of different partitions (using edge blending) according to the GPM partitioning mode of the indicated GPM combination. The calculated costs of various candidate GPM combinations are provided to the candidate ranking module 1340, which assigns indexes to various candidate GPM combinations according to the calculated TM costs they correspond to.

[0172] The candidate selection module 1345 may select one from the candidate GPM combinations and provide the index of the selected combination to the entropy encoder 1290 to indicate the index. The contents of the selected GPM combination (including the GPM partition mode, the prediction mode of the GPM partition, motion information, the intra prediction mode, etc.) are provided to the GPM prediction generation module 1350. The GPM prediction generation module 1350 generates predictions for each GPM partition based on the selected GPM combination. The intra prediction module 1225 may be used to generate predictions for intra prediction partitions, and the motion compensation module 1230 may be used to generate predictions for inter prediction partitions. The GPM prediction generation module 1350 then combines the predictions of the different GPM partitions into one GPM predictor for the current block (by performing GPM edge blending) as the predicted pixel data 1213.

[0173] Fig.14 The process 1400 of using a unified candidate list of GPM combinations is conceptually described. In some embodiments, one or more processing units (e.g., processors) of a computing device implementing the encoder 1200 perform the process 1400 by executing instructions stored in a computer-readable medium. In some embodiments, the electronic device implementing the encoder 1200 performs the process 1400.

[0174] The encoder receives (at block 1410) data to be encoded into a current block of pixels of a current picture of a video. The encoder generates (at block 1420) a candidate list, each candidate specifying (i) a partitioning mode and (ii) a first prediction mode and a second prediction mode. At least a first candidate of the candidate list specifies motion information of an inter-coded partition of the current block.

[0175] In some embodiments, candidates in a candidate list are assigned indices in an order based on the cost calculated for the candidate. The cost of the candidate is calculated based on reconstructed samples adjacent to the current block and reference samples derived from the candidate's partitioning mode and prediction mode. In some embodiments, the candidate with the lowest cost among all the calculated costs of the candidates in the list is assigned the shortest codeword among all the candidates in the list.

[0176] The first candidate may specify a refinement of the inter prediction based on the motion information of the first candidate. For GPM-TM, the refinement may be based on minimizing the cost of matching between the reconstructed samples adjacent to the current block and the reconstructed samples adjacent to the reference block identified from the motion information of the first candidate. For GPM-MMVD, the refinement may be based on motion vector differences specified by distance and direction.

[0177] If both the first partition and the second partition of the first candidate are inter-coded, the first candidate in the candidate list may specify motion information for both the first partition and the second partition. For GPM-Intra-Inter, the first candidate may specify the intra prediction mode (e.g., intra prediction direction) of the intra-coded partition of the current block. The candidate list may also include a second candidate that specifies two intra prediction modes for two intra-coded partitions (GPM-Intra).

[0178] (at block 1430) the encoder selects a candidate from the candidate list (e.g., an index assigned according to a cost calculated from the candidate in the list) to send. (at block 1440) the encoder segments the current block into a first partition and a second partition according to the segmentation mode of the selected candidate. (at block 1450) the encoder generates a first prediction and a second prediction for the first partition and the second partition according to the first prediction mode and the second prediction mode of the selected candidate.

[0179] The encoder encodes the current block using the first prediction and the second prediction (at block 1460). In some embodiments, the video encoder combines the first prediction and the second prediction via GPM edge blending to generate a combined prediction, and the combined prediction is used to generate a residual to encode the current block.

[0180] VII. Example Video Decoder

[0181] In some embodiments, the encoder may send (or generate) one or more syntax elements in a bitstream, so that the decoder may parse the one or more syntax elements from the bitstream.

[0182] Fig.15 An example video decoder 1500 capable of decoding GPM coded blocks is described. As shown, the video decoder 1500 is an image decoding or video decoding circuit that receives a bitstream 1595 and decodes the contents of the bitstream into pixel data of a video frame for display. The video decoder 1500 has several components or modules for decoding the bitstream 1595, including some components selected from an inverse quantization module 1511, an inverse transform module 1510, an intra-frame prediction module 1525, a motion compensation module 1530, an inter-frame prediction module 1540, an in-loop filter 1545, a decoded picture buffer 1550, an MV buffer 1565, an MV prediction module 1575, and a parser 1590. The motion compensation module 1530 is a part of the inter-frame prediction module 1540.

[0183] In some embodiments, the inverse quantization module 1511, the inverse transform module 1510, the intra-frame prediction module 1525, the motion compensation module 1530, the inter-frame prediction module 1540, the in-loop filter 1545, the decoded picture buffer 1550, the MV buffer 1565, the MV prediction module 1575, and the parser 1590 are modules of software instructions executed by one or more processing units (e.g., processors) of a computing device. In some embodiments, the inverse quantization module 1511, the inverse transform module 1510, the intra-frame prediction module 1525, the motion compensation module 1530, the inter-frame prediction module 1540, the in-loop filter 1545, the decoded picture buffer 1550, the MV buffer 1565, the MV prediction module 1575, and the parser 1590 are modules of hardware circuits implemented by one or more ICs of an electronic device. Although the inverse quantization module 1511, the inverse transformation module 1510, the intra-frame prediction module 1525, the motion compensation module 1530, the inter-frame prediction module 1540, the intra-loop filter 1545, the decoded image buffer 1550, the MV buffer 1565, the MV prediction module 1575 and the parser 1590 are described as different modules, some modules may be merged into separate modules.

[0184] The parser 1590 (or entropy decoder) receives the bitstream 1595 and performs initial parsing according to the syntax defined by the video coding or image coding standard. The parsed syntax elements include various header elements, flags, and quantization data (or quantization coefficients) 1512. The parser 1590 parses the various syntax elements by using entropy coding techniques such as context adaptive binary arithmetic coding (CABAC) or Huffman coding.

[0185] The inverse quantization module 1511 inversely quantizes the quantized data (or quantized coefficients) 1512 to obtain transform coefficients 1516, and the inverse transform module 1510 performs inverse transform on the transform coefficients 1516 to generate a reconstructed residual signal 1519. The reconstructed residual signal 1519 is added to the predicted pixel data 1513 from the intra prediction module 1525 or the motion compensation module 1530 to generate decoded pixel data 1517. The decoded pixel data 1517 is filtered by the in-loop filter 1545 and stored in a decoded picture buffer 1550. In some embodiments, the decoded picture buffer 1550 is a storage device external to the video decoder 1500. In some embodiments, the decoded picture buffer 1550 is a storage device internal to the video decoder 1500.

[0186] The intra prediction module 1525 receives intra prediction data from the bitstream 1595 and generates predicted pixel data 1513 from the decoded pixel data 1517 stored in the decoded picture buffer 1550 based on the intra prediction data. In some embodiments, the decoded pixel data 1517 is also stored in a line buffer (not shown) for intra prediction and spatial MV prediction.

[0187] In some embodiments, the contents of the decoded image buffer 1550 are used for display. The display device 1555 either retrieves and directly displays the contents of the decoded image buffer 1550 or retrieves the contents of the decoded image buffer to a display buffer. In some embodiments, the display device receives pixel values ​​from the decoded image buffer 1550 via a pixel transmitter.

[0188] The motion compensation module 1530 generates predicted pixel data 1513 from the decoded pixel data 1517 stored in the decoded picture buffer 1550 according to motion compensated MVs (MC MVs). These motion compensated MVs are decoded by adding the residual motion data received from the bitstream 1595 to the predicted MVs received from the MV prediction module 1575.

[0189] The MV prediction module 1575 generates a prediction MV based on a reference MV generated for decoding a previous video frame (e.g., a motion compensated MV for performing motion compensation). The MV prediction module 1575 retrieves the reference MV of the previous video frame from the MV buffer 1565. The video decoder 1500 stores the motion compensated MV generated for decoding a current video frame in the MV buffer 1565 as a reference MV for generating the prediction MV.

[0190] The in-loop filter 1545 performs a filtering or smoothing operation on the decoded pixel data 1517 to reduce encoding distortion, particularly at the boundaries of pixel blocks. In some embodiments, the filtering or smoothing operation performed by the in-loop filter 1545 includes a deblocking filter (DBF), a sample adaptive offset (SAO), and / or an adaptive loop filter (ALF).

[0191] Fig.16A portion of a video decoder 1500 that implements a unified GPM candidate list is described. The GPM candidate generation module 1610 specifies various candidate GPM combinations. Each candidate GPM combination can specify a GPM partitioning mode that divides the current block into two or more GPM partitions, and two or more prediction modes for the two or more GPM partitions. For intra-coded GPM partitions, the candidate combination can specify an intra-frame prediction mode. For inter-coded GPM partitions, the candidate combination can specify motion information (such as: motion vector, reference image, or selected merge candidate). The motion information can be retrieved from the MV buffer 1565. The candidate combination can also specify a refinement of the motion information according to GPM-MMVD or GPM-TM.

[0192] For each candidate GPM combination, the template identification module 1620 retrieves samples of the current template and the reference template from the decoded picture buffer 1550 based on the prediction mode specified in the candidate GPM combination. For a candidate GPM combination that specifies one or more inter-frame coded partitions, the template identification module 1620 may retrieve neighboring samples of the current block as the current template and use the motion information specified in the combination to retrieve neighboring samples of the reference block. For a candidate GPM combination that specifies one or more intra-frame coded partitions, the template identification module 1620 may retrieve neighboring samples of the current block as the current template and retrieve neighboring samples of the current template identified through the specified intra-frame prediction mode as the reference template.

[0193] The template identification module 1620 provides the reference template and the current template of the currently indicated candidate GPM combination to the cost calculator 1630, which performs template matching to generate the cost of the indicated candidate GPM combination. When determining the TM cost, the cost calculator 1630 can combine the reference templates of different partitions (using edge blending) according to the GPM partitioning mode of the indicated GPM combination. The calculated costs of various candidate GPM combinations are provided to the candidate ranking module 1640, which assigns indexes to various candidate GPM combinations according to their corresponding calculated TM costs.

[0194] The candidate selection module 1645 may select one from the candidate GPM combinations based on the index provided by the entropy decoder 1590. The contents of the selected GPM combination (including the GPM partitioning mode, the prediction mode of the GPM partition, the motion information, the intra prediction mode, etc.) are provided to the GPM prediction generation module 1650. The GPM prediction generation module 1650 generates a prediction for each GPM partition according to the selected GPM combination. The intra prediction module 1525 may be used to generate a prediction for the intra prediction partition, and the motion compensation module 1530 may be used to generate a prediction for the inter prediction partition. The GPM prediction generation module 1650 then merges the predictions of the different GPM partitions (by performing GPM edge blending) into one GPM predictor for the current block as the predicted pixel data 1513.

[0195] Fig.17 A process 1700 for using a unified candidate list of GPM combinations is conceptually described. In some embodiments, one or more processing units (e.g., processors) of a computing device implementing the decoder 1500 perform the process 1700 by executing instructions stored in a computer-readable medium. In some embodiments, the electronic device implementing the decoder 1500 performs the process 1700.

[0196] The decoder receives (at block 1710) data to be decoded into a current pixel block of a current picture of a video. The decoder generates (at block 1720) a candidate list, each candidate specifying (i) a partitioning mode and (ii) a first prediction mode and a second prediction mode. At least a first candidate in the candidate list specifies motion information of an inter-coded partition of the current block.

[0197] In some embodiments, candidates in a candidate list are assigned indices in an order based on the cost calculated for the candidate. The cost of the candidate is calculated based on reconstructed samples adjacent to the current block and reference samples derived from the candidate's partitioning mode and prediction mode. In some embodiments, the candidate with the lowest cost among all the calculated costs of the candidates in the list is assigned the shortest codeword among all the candidates in the list.

[0198] The first candidate may specify a refinement of the inter prediction based on the motion information of the first candidate. For GPM-TM, the refinement may be based on minimizing the cost of matching between the reconstructed samples adjacent to the current block and the reconstructed samples adjacent to the reference block identified from the motion information of the first candidate. For GPM-MMVD, the refinement may be based on motion vector differences specified by distance and direction.

[0199] If both the first partition and the second partition of the first candidate are inter-coded, the first candidate in the candidate list may specify motion information for both the first partition and the second partition. For GPM-Intra-Inter, the first candidate may specify the intra prediction mode (e.g., intra prediction direction) of the intra-coded partition of the current block. The candidate list may also include a second candidate that specifies two intra prediction modes for two intra-coded partitions (GPM-Intra).

[0200] (at block 1730) the decoder receives a candidate selected from the candidate list (e.g., by receiving an index assigned a cost calculated from the candidate in the list). (at block 1740) the decoder segments the current block into a first partition and a second partition according to the segmentation mode of the selected candidate. (at block 1750) the decoder generates a first prediction and a second prediction for the first partition and the second partition according to the first prediction mode and the second prediction mode of the selected candidate.

[0201] The decoder reconstructs the current block using the first prediction and the second prediction (at block 1760). In some embodiments, the video encoder combines the first prediction and the second prediction via GPM edge blending to produce a combined prediction, and the combined prediction is used as a prediction block to reconstruct the current block. The decoder may then provide the reconstructed current block for display as part of the reconstructed current image.

[0202] VIII. Example Electronic System

[0203] Many of the above features and applications are implemented as software programs, which are specifically described as a set of instructions recorded on a computer-readable storage medium (also called computer-readable media). When these instructions are executed by one or more computing or processing units (such as one or more processors, processor cores or other processing units), they cause the processing units to perform the actions corresponding to the instructions. Examples of computer-readable media include, but are not limited to, compact disc read only memory (CD-ROM), flash drives, random-access memory (RAM) chips, hard disks, erasable programmable read only memory (EPROM), electronically erasable programmable read only memory (EEPROM), etc. Computer-readable media do not include carrier waves and electronic signals transmitted wirelessly or through wired connections.

[0204] In this specification, the term "software" is intended to include firmware residing in a read-only memory or an application program stored in a magnetic memory that can be read into the memory to be processed by a processor. In addition, in some embodiments, multiple software inventions can be implemented as sub-parts of a larger program while remaining different software inventions. In some embodiments, multiple software inventions can also be implemented as different programs. Finally, any combination of different programs that implement one of the software inventions described herein is within the scope of this disclosure. In some embodiments, when the software program is installed on one or more electronic systems for operation, the software program defines one or more specific machine implementations to perform the operations of the software program.

[0205] Fig.18 An electronic system 1800 for implementing some embodiments of the present disclosure is conceptually described. The electronic system 1800 may be a computer (e.g., a desktop computer, a personal computer, a tablet computer, etc.), a phone, a PDA, or any other type of electronic device. Such an electronic system includes various types of computer-readable media and interfaces for various other types of computer-readable media. The electronic system 1800 includes a bus 1805, a processing unit 1810, a graphics-processing unit (GPU) 1815, a system memory 1820, a network 1825, a read-only memory 1830, a permanent storage device 1835, an input device 1840, and an output device 1845.

[0206] The bus 1805 collectively represents all system, peripheral, and chip buses that communicatively connect the numerous internal devices of the electronic system 1800. For example, the bus 1805 communicatively connects the processing unit 1810 with the GPU 1815, the read-only memory 1830, the system memory 1820, and the permanent storage device 1835.

[0207] From these various memory units, the processing unit 1810 retrieves instructions to be executed and data to be processed to execute the process disclosed herein. In different embodiments, the processing unit can be a single core processor or a multi-core processor. Some instructions are sent to the GPU 1815 for execution. The GPU 1815 can offload various computational burdens or complement the image processing provided by the processing unit 1810.

[0208] The read-only-memory (ROM) 1830 stores static data and instructions used by the processing unit 1810 and other modules of the electronic system. On the other hand, the permanent storage device 1835 is a read-write memory device. This device is a non-volatile memory unit that stores instructions and data even when the electronic system 1800 is turned off. Some embodiments of the present disclosure use a large-capacity storage device (such as a magnetic disk or optical disk and its corresponding magnetic disk drive) as the permanent storage device 1835.

[0209] Other embodiments use removable storage devices (such as floppy disks, USB flash drives, etc. and their corresponding disk drives) as permanent storage devices. Similar to the permanent storage device 1835, the system memory 1820 is a read-write storage device. However, unlike the storage device 1835, the system memory 1820 is a volatile read-write memory, such as a random access memory. The system memory 1820 stores some instructions and data used by the processor during operation. In certain embodiments, the program disclosed herein is stored in the system memory 1820, the permanent storage device 1835 and / or the read-only memory 1830. For example, in certain embodiments, various memory units contain instructions for processing multimedia clips. The processing unit 1810 retrieves the instructions to be executed and the data to be processed from these various memory units to execute the programs of certain embodiments.

[0210] The bus 1805 is also connected to input devices 1840 and output devices 1845. The input devices 1840 allow a user to send information and select commands to the electronic system. The input devices 1840 include an alphanumeric keyboard and a pointing device (also called a "cursor control device"), a camera (such as a webcam), a microphone or a similar device for receiving voice commands, etc. The output devices 1845 display images generated by the electronic system or other output data. The output devices 1845 include printers and display devices, such as cathode ray tubes (CRT) or liquid crystal displays (LCD), and speakers or similar audio output devices. Some embodiments include devices, such as touch screens, that can function as both input devices and output devices.

[0211] Finally, if Fig.18As shown, bus 1805 also couples electronic system 1800 to network 1825 via a network adapter (not shown). In this approach, the computer can be part of a computer network (e.g., a local area network (LAN), a wide area network (WAN), or an intranet), or a network of networks (e.g., a network of networks). Any or all components of electronic system 1800 may be used in conjunction with the present disclosure.

[0212] Certain embodiments include electronic components (e.g., microprocessors, storage devices, and memory) that store computer program instructions on machine-readable or computer-readable media (or computer-readable storage media, machine-readable media, or machine-readable storage media). Some examples of such computer-readable media include RAM, ROM, CD-ROM, recordable compact disc (CD-R), rewritable compact disc (CD-RW), read-only digital versatile discs (e.g., digital versatile disc read-only memory (DVD-ROM), double-layer DVD-ROM), various recordable / rewritable DVDs (e.g., digital versatile disc random access memory (DVD-RAM), rewritable digital versatile disc (digital versatile disc-read / write; DVD-RW), rewritable digital versatile disc (DVD+RW), etc.), flash memory (e.g., Secure Digital Memory Card (SD card), mini SD card, micro SD card, etc.), magnetic and / or solid-state hard drives, read-only and recordable Blu-ray discs Ultra-density optical disks, any other optical or magnetic media, and floppy disks. Computer-readable media can store computer programs that are executable by at least one processing unit and contain sets of instructions for performing various operations. Examples of computer programs or computer program code include machine code generated by a compiler and files of high-level program code that are executed by a computer, electronic component, or microprocessor using an interpreter.

[0213] Although the above discussion focuses on microprocessors or multi-core processors that execute software, many of the above features and applications are implemented on one or more integrated circuits (such as application specific integrated circuits (ASICs) or field programmable gate arrays (FPGAs)). In some embodiments, such integrated circuits execute instructions stored in their own circuits. In addition, some embodiments execute software stored in programmable logic devices (PLDs), ROM, or RAM devices.

[0214] As used in this specification and in the claims to this invention, the terms "computer," "server," "processor," and "memory" refer to electronic or other technological devices. These terms exclude persons or groups of persons. For the purposes of this specification, the term "display" means displayed on an electronic device. As used in this specification and in the claims to this invention, the terms "computer-readable medium" and "machine-readable medium" are entirely limited to tangible physical objects that store information in a form that can be read by a computer. These terms exclude any wireless signals, wired download signals, and any other ephemeral signals.

[0215] Although the present disclosure is described with many references to specific details, it will be appreciated by those skilled in the art that the present disclosure may be implemented in other specific forms without departing from the spirit of the present disclosure. Fig.14 and Fig.17 ) conceptually describe the programs. The specific operations of these programs may not be performed in the exact order shown and described. The specific operations may not be performed in a series of continuous operations, and different specific operations may be performed in different embodiments. In addition, these programs may be implemented using multiple subroutines or as part of a larger macro program. Therefore, it will be understood by those skilled in the art that the present disclosure is not limited to the foregoing illustrative details, but is defined by the appended claims.

[0216] [Additional notes]

[0217] The subject matter described herein sometimes illustrates different components included in or connected to different other components. It should be understood that the architecture depicted in this way is only an example, and in fact many other architectures can achieve the same function. Conceptually, any arrangement of components that achieve the same function is effectively "associated", thereby achieving the desired function. Therefore, regardless of the architecture or intermediate components, any two components that achieve a specific function through combination herein can be regarded as "associated" with each other to achieve the desired function. Similarly, any two components so associated can also be regarded as "operably connected" or "operably coupled" to each other to achieve the desired function, and any two components that can be so associated can also be regarded as "operably couplable" to each other to achieve the desired function. For example, "operably couplable" includes but is not limited to physically compatible and / or physically interactive components and / or wirelessly interactive and / or wirelessly interactive components and / or logically interactive and / or logically interactive components.

[0218] In addition, with respect to the use of substantially any plural and / or singular terms herein, those skilled in the art may translate from the plural to the singular and / or from the singular to the plural as appropriate, depending on the context and / or application. For clarity, various singular / plural arrangements are explicitly set forth herein.

[0219] Furthermore, those skilled in the art will appreciate that, in general, the terms used herein, especially the terms used in the appended claims (e.g., the main body of the appended claims), are generally intended to be “open”.

[0220] Terms such as the word “including” should be interpreted as “including but not limited to,” the word “having” should be interpreted as “at least having,” and the word “including” should be interpreted as “including but not limited to,” etc. Those skilled in the art will further understand that if a specific quantity is intended to be introduced in a claim statement, such intent will be explicitly stated in the claim, and in the absence of such intent, the absence of such intent is absent. For example, to aid understanding, the following appended claims may include the use of the introductory phrases “at least one” and “one or more” to introduce the recitation of the claim statements. However, even if a claim includes the introductory phrases “one or more” or “at least one” and an indefinite article, such as “a” (e.g., “a” and / or “an” should be interpreted as “at least one”).

[0221] or "one or more"), the use of the above phrases should not be interpreted as implying that the above claims including the indefinite article "a" will be limited to only including one embodiment of the above claim; the above also applies to claim statements introduced using definite articles. In addition, even if it is clear that a specific number is recorded in the claim, those skilled in the art will recognize that such a record should be interpreted as at least reaching the above specific number recorded, for example, the record of "two", without other modifiers, means at least two, or two or more. In addition, in those cases where limitations similar to "at least one of A, B and C, etc." are used, generally speaking, such a structure is based on the understanding of the above limitations by those skilled in the art, for example, "a system having at least one of A, B and C" will include but is not limited to systems having A alone, B alone, C alone, A and B at the same time, A and C at the same time, B and C at the same time, and / or A, B and C at the same time. In those cases where a definition similar to "at least one of A, B, and C, etc." is used, generally, such a construction is based on the understanding of the above definition by those skilled in the art, for example, "a system having at least one of A, B, and C" will include but is not limited to systems having A alone, B alone, C alone, A and B together, A and C together, B and C together, and / or A, B, and C together, etc. Those skilled in the art will further understand that in fact, whether in the specification, claims or drawings, any disjunctive words and / or phrases that appear with two or more alternative terms should be understood to include the possibility of one of the terms, any one of the terms, or both of the terms. For example, the phrase "A or B" will be understood to include the possibility of "A" or "B" or "A and B".

[0222] As can be understood from the above, various embodiments of the present disclosure are described herein for illustrative purposes, and various modifications can be made without departing from the scope and spirit of the present disclosure. Therefore, the various embodiments disclosed herein are not intended to be limiting, and the true scope and spirit are represented by the appended application rights.

Claims

1. A video encoding and decoding method, comprising: Receiving data to be encoded or decoded as a current pixel block of a current image of a video; generating a candidate list comprising a plurality of candidates, each candidate in the candidate list specifying (i) a partition mode and (ii) a first prediction mode and a second prediction mode, wherein a first candidate in the candidate list specifies motion information of an inter-frame coded partition of the current block; sending or receiving a candidate selected from the candidate list; According to the segmentation mode of the selected candidate, the current block is divided into a first partition and a second partition; Generate a first prediction of the first partition and a second prediction of the second partition according to the selected first prediction mode and the second prediction mode of the candidate; as well as The current block is encoded or decoded using the first prediction and the second prediction.

2. The video coding method of claim 1, wherein the candidates in the candidate list are assigned indexes according to an order determined based on costs calculated for the candidates.

3. The video coding method of claim 2, wherein the cost of each candidate in the candidate list is calculated based on reconstructed samples adjacent to the current block and reference samples derived according to the partition mode and the prediction modes of the candidate.

4. The video coding method of claim 2, wherein a candidate having a lowest cost among the costs calculated for all candidates in the candidate list is assigned a shortest code among all candidates in the candidate list. 5 . The video coding method of claim 1 , wherein the first candidate specifies a refinement method of an inter-frame prediction based on the motion information of the first candidate. 6 . The video coding method of claim 5 , wherein the refinement method is based on minimizing a matching cost between reconstructed samples adjacent to the current block and reconstructed samples adjacent to a reference block identified by the motion information of the first candidate.

7. The video coding method of claim 5, wherein the refinement method is based on a motion vector difference specified by a distance and a direction. 8 . The video coding method of claim 1 , wherein the first candidate further specifies an intra prediction mode of an intra-coded partition of the current block. 9 . The video coding method of claim 1 , wherein the first candidate in the candidate list specifies motion information of the first partition and the second partition.

10. An electronic device comprising: A video encoder circuit is configured to: Receiving data of a pixel block, the data of the pixel block is to be encoded or decoded into a current block of a current image of a video; generating a candidate list comprising a plurality of candidates, each candidate in the candidate list specifying (i) a partition mode and (ii) a first prediction mode and a second prediction mode, wherein a first candidate in the candidate list specifies motion information of an inter-frame coded partition of the current block; sending or receiving a candidate selected from the candidate list; According to the partition mode of the selected candidate, the current block is divided into a first partition and a second partition; generating a first prediction of the first partition and a second prediction of the second partition according to the first prediction mode and the second prediction mode of the selected candidate; as well as The current block is encoded or decoded using the first prediction and the second prediction.

11. A video decoding method, comprising: Receiving data of a pixel block, the data of the pixel block is to be encoded or decoded into a current block of a current image of a video; generating a candidate list comprising a plurality of candidates, each candidate in the candidate list specifying (i) a partition mode and (ii) a first prediction mode and a second prediction mode, wherein a first candidate in the candidate list specifies motion information of an inter-frame coded partition of the current block; receiving a candidate selected from the candidate list; According to the partition mode of the selected candidate, the current block is divided into a first partition and a second partition; generating a first prediction of the first partition and a second prediction of the second partition according to the first prediction mode and the second prediction mode of the selected candidate; as well as The current block is reconstructed using the first prediction and the second prediction.

12. A video encoding method, comprising: Receiving data of a pixel block, the data of the pixel block is to be encoded or decoded into a current block of a current image of a video; generating a candidate list comprising a plurality of candidates, each candidate in the candidate list specifying (i) a partition mode and (ii) a first prediction mode and a second prediction mode, wherein a first candidate in the candidate list specifies motion information of an inter-frame coded partition of the current block; sending a candidate selected from the candidate list; According to the partition mode of the selected candidate, the current block is divided into a first partition and a second partition; generating a first prediction of the first partition and a second prediction of the second partition according to the first prediction mode and the second prediction mode of the selected candidate; as well as The current block is encoded using the first prediction and the second prediction.

Images