Geometric Partitioning Pattern (GPM) Extension

Abstract

A method using a geometric partitioning pattern (GPM) with regression-based mixing is provided. The current block is divided into at least two GPM partitions. The video encoder derives mixing weights for this partition by regression using template region samples from neighboring current blocks. The video encoder selects a set of candidate pairs from a candidate pair list. When signaling indicates that the mixing weights are derived via regression, a signaling notification can be used to merge the index to select a set of candidate pairs from that list. The video encoder applies the selected candidate pairs to the two GPM partitions and uses the derived mixing weights to generate a predictor for the current block. The video encoder uses the generated predictor to encode or decode the current block.

Description

[0001] Cross-references This disclosure is part of a non-provisional application that claims priority to U.S. Provisional Patent Application No. 63 / 621,649, filed January 17, 2024. The contents of the above application are incorporated herein by reference. Technical Field

[0002] This disclosure generally relates to video coding. Specifically, this disclosure relates to a method for encoding pixel blocks using a Geometric Partitioning Mode (GPM). Background Technology

[0003] Unless otherwise stated herein, the methods described in this section are not prior art as claimed in the following claims, and are not considered prior art by way of inclusion in this section. High-Efficiency Video Coding (HEVC) is an international video coding standard developed by the Joint Collaborative Team Video Coding (JCT-VC). HEVC is based on a hybrid block-based motion-compensated discrete cosine transform (DCT) coding architecture. The basic unit of compression is called a coding unit (CU), which is a 2Nx2N square pixel block. Each CU can be recursively divided into four smaller CUs until a preset minimum size is reached. Each CU contains one or more prediction units (PUs). Versatile Video Coding (VVC) is the latest international video coding standard developed by the Joint Video Experts Team (JVET) of ITU-TSG16WP3 and ISO / IEC JTC1 / SC29 / WG11. The input video signal is predicted from a reconstructed signal derived from the coded image region. The prediction residual signal is processed through block transform. The transform coefficients, along with other auxiliary information, are quantized and entropy-coded in the bitstream. The reconstructed signal is generated from the predicted signal and the inverse-transformed reconstructed residual signal, with the inverse transform applied to the dequantized transform coefficients. The reconstructed signal is further filtered within a loop to remove coding artifacts. The decoded image is stored in a frame buffer and used to predict subsequent images in the input video signal. In VVC, the encoded image is divided into non-overlapping square block regions represented by relevant coding tree units (CTUs). The leaf nodes of the coding tree correspond to coding units (CUs). The encoded image can consist of multiple slices, each containing an integer number of CTUs. The CTUs within a slice are processed in raster scan order. Double-predictive (B) slices are decoded via intra-frame or inter-frame prediction, using at most two motion vectors (MVs) and a reference index to predict the sample values ​​for each block. Predictive (P) slices are decoded via intra-frame or inter-frame prediction, using at most one motion vector and a reference index to predict the sample values ​​for each block. Intra-frame (I) slices are decoded only via intra-frame prediction. The Coding Unit (CTU) can be divided into one or more non-overlapping Coding Units (CUs) using a quadtree (QT) or nested multi-type-tree (MTT) structure to accommodate different local motion and texture characteristics. CUs can be further partitioned into smaller CUs using one of five partitioning types: quadtree partitioning, vertical binary tree partitioning, horizontal binary tree partitioning, vertical center-side ternary tree partitioning, and horizontal center-side ternary tree partitioning. Each CU contains one or more prediction units (PUs). The prediction unit, along with the associated CU syntax, serves as the basic unit for signaling predictor information. The specified prediction process is used to predict the values ​​of associated pixel samples within the PU. Each CU may contain one or more transform units (TUs) to represent prediction residual blocks. A transform unit (TU) consists of one luma transform block (TB) and two corresponding chroma transform blocks, each TB corresponding to a residual sample block for one color component. Integer transforms are applied to the transform blocks. The level values ​​of the quantization coefficients, along with other auxiliary information, are entropy-encoded in the bitstream. Coding tree blocks (CTBs), coding blocks (CBs), prediction blocks (PBs), and transform blocks (TBs) are used to specify the two-dimensional sample arrays of monochromatic components associated with the CTU, CU, PU, ​​and TU, respectively. Therefore, a CTU consists of one luma CTB, two chroma CTBs, and associated syntax elements. CUs, PUs, and TUs have similar relationships. For each inter-frame predicted CU, motion parameters include motion vectors, reference image indices, reference image list usage indices, and other additional information used for inter-frame predicted sample generation. Motion parameters can be signaled explicitly or implicitly. When a CU is coded in skip mode, it is associated with a PU and has no significant residual coefficients, and no coded motion vector increments or reference image indices. Merging mode specifies that the motion parameters of the current CU can be obtained from neighboring CUs, including spatial and temporal candidates, as well as additional scheduling introduced in VVC. Merging mode can be applied to any inter-frame predicted CU. An alternative to merging mode is explicit signaling notification of motion parameters, i.e., each CU explicitly signals the motion vectors, the corresponding reference image index for each reference image list, the reference image list usage flag, and other necessary information. Intrablockcopy (IBC) or currentpicturereferencing (CPR) refers to encoding pixel blocks by referencing the same pixel locations in the current image as the current block using block vectors. In Advanced Motion Vector Prediction (AMVP) mode, motion vector predictor (MVP) candidates are determined based on template matching (TM) error. The candidate with the smallest difference between the current block template and the reference block template is selected, and then TM is performed only on that specific MVP candidate to refine the MV. The TM process can refine the MVP candidate through iterative search according to the adaptive motion vector resolution (AMVR) mode search pattern. Summary of the Invention

[0004] The following abstract is for illustrative purposes only and is not intended to be limiting in any way. That is, the abstract aims to introduce the concepts, key points, benefits, and advantages of the novel and non-obvious techniques described herein. Some implementations will be further elaborated in the detailed description below, not all implementations. Therefore, the following abstract is not intended to identify the essential features of the claimed subject matter, nor to determine the scope of the claimed subject matter.

[0005] Certain embodiments of this disclosure provide a method using a geometric segmentation pattern (GPM) with regression-based mixing. The current block is divided into at least two GPM partitions. A video encoder derives mixing weights for this partition by regression, utilizing template region samples adjacent to the current block. The video encoder selects a set of candidate pairs from a candidate pair list. A signaling notification can be used to merge the index to select a set of candidate pairs from this list when the mixing weights are indicated by regression derivation. The video encoder applies the selected candidate pairs to the two GPM partitions and utilizes the derivation of the mixing weights to generate a predictor for the current block. The video encoder uses the generated predictor to encode or decode the current block.

[0006] In some embodiments, the video encoder selects a set of candidate pairs by choosing from multiple lists of candidate pairs, wherein each set of candidate pairs in a particular list contains at least one candidate for a particular encoding tool. In some embodiments, the video encoder selects a set of candidate pairs by choosing from only one list of candidate pairs, which contains pairs of any two candidates for two or more encoding tools. For example, the candidate pair list may include different candidate pairs from inter-frame merge candidates and affine merge candidates. As another example, the candidate pair list may include different candidate pairs from inter-frame merge candidates, affine merge candidates, intra-frame merge candidates, and intra-block copy (IBC) merge candidates.

[0007] In some embodiments, the candidate pairing list is reordered according to the template cost, and a corresponding merge index is assigned to each candidate pairing in the list, so that the video encoder can use signaling to notify the merge index to identify the selected candidate pairing in the list. In some embodiments, the merge index of the candidate pairing list is signaled when a flag indicating that the blending weights are derived through regression (e.g., gpm_implicit_flag) is indicated.

[0008] In some embodiments, when only the first GPM partition among at least two GPM partitions of the current block has the LIC enable flag and an inherited LIC parameter set, the video encoder refines the inherited LIC parameter set and applies the refined LIC parameter set to the second GPM partition among the at least two GPM partitions. The inherited LIC parameters of the second GPM partition can be refined using the neighboring templates and corresponding reference blocks of the current block. Attached Figure Description

[0009] The accompanying drawings are included in this specification to further understand this disclosure and form part of this disclosure. The drawings illustrate embodiments of this disclosure and, in conjunction with the specification, explain the principles of this disclosure. It is understood that the drawings are not necessarily drawn to scale, as some components may differ in size from their actual dimensions for the purpose of clearly illustrating the concepts of this disclosure. Figure 1 Spatial and temporal candidates for merging patterns are shown. Figure 2 This demonstrates the control point motion vector (CPMV) of the current block encoded by affine motion field. Figure 3 The affine motion field for each sub-block is displayed. Figure 4 This demonstrates the concept of CPMV inheritance. Figure 5 This demonstrates how coding units (CUs) are partitioned using the geometric partitioning pattern (GPM). Figure 6 An example of a single prediction candidate list for GPM partitioning and the selection of a single prediction motion vector (MV) for GPM are shown. Figure 7 An example of the GPM partition edge blending process for CU is shown. Figure 8 The ramp function for GPM mixing weights is shown based on the displacement from the predicted sample location to the GPM partition boundary and the mixing region. Figure 9 This shows the GPM partition edge extending from the current CU, used for partition template regions. Figure 10 Conceptually, it presents a list of candidate pairings that includes all possible candidate pairings to be merged. Figure 11 This conceptually presents a list of multiple candidate pairings corresponding to different encoding tools. Figure 12 An example video encoder that can implement GPM is shown. Figure 13 The video encoder portion of GPM, which implements regression mixing, is shown. Figure 14 This concept demonstrates the process of using a candidate pairing list to predict GPM by regressing mixed weights. Figure 15 An example of a video decoder that can implement GPM is shown. Figure 16 The video decoder portion of GPM, which implements regression mixing, is shown. Figure 17This concept demonstrates the process of using a candidate pairing list to predict GPM by regressing mixed weights. Figure 18 This invention conceptually demonstrates an electronic system that can implement some embodiments of the present disclosure. Detailed Implementation

[0010] In the following detailed description, numerous specific details are illustrated by way of example to provide a comprehensive understanding of the related techniques. Any variations, derivatives, and / or extensions based on the techniques described herein are within the scope of this disclosure. In some cases, to avoid unnecessarily obscuring certain aspects of the techniques disclosed, well-known methods, processes, components, and / or circuits involved in one or more example embodiments may be described at a higher level without going into detail.

[0011] I. Inter-frame merging mode Skip and Merge modes acquire motion information from spatially neighboring blocks (spatial candidates) or temporally co-located blocks (temporal candidates). When the prediction unit (PU) is in Skip or Merge mode, motion information is not encoded; only the index of the selected candidate is encoded. For Skip mode, the residual signal is forced to zero and is not encoded. If a particular block is encoded as Skip or Merge, signaling is used to notify the candidate index to identify the candidate used for merging in the candidate set. Each merged PU reuses the motion vector (MV), prediction direction, and reference image index of the selected candidate.

[0012] Figure 1 The spatial and temporal candidates for merging patterns are shown. As illustrated, up to four spatial MV candidates can be derived from spatial neighbors A0, A1, B0, and B1, and from T... BR or TCT R Derive a time MV candidate (preferably using T) BR If T BR If unavailable, use TCT. R If any of the four spatial MV candidates is unavailable, position B2 is used to derive an MV candidate as a replacement. After deriving the four spatial MV candidates and one temporal MV candidate, redundancy removal (pruning) is applied to remove redundant MV candidates. If, after redundancy removal (pruning), the number of available MV candidates is less than five, three additional candidates are derived and added to the candidate set (candidate list). The encoder selects a final candidate from the candidate set for the Skip or Merge mode based on the Rate-Distortion Optimization (RDO) decision and transmits the index to the decoder.

[0013] II. Affine Prediction A. Affine Stadium Objects in a video may exhibit different types of motion, including translation, scaling, rotation, perspective motion, and other irregular motions. In some implementations, block-based affine transformation motion compensation prediction is employed to accommodate these different types of motion. Specifically, the affine motion field of the current block at position (x, y) is... mv x , mv y It takes the form of a linear model: mv x = a*x + b*y + c ; mv y = d*x + e*y + f (0) The coefficients {a, b, c, d, e, f} are the parameters of the linear model. In some implementations, the affine motion field at position (x, y) can be described by motion information from two control points (CPs) (e.g., the top right and top left corners of the block) (a four-parameter model) or by motion information from three control points (e.g., the top right, top left, and bottom left corners of the block) (a six-parameter model).

[0014] For a four-parameter affine motion model, Equation 0 (the motion vector of the sample position (x, y) in the block) can be written as: (1) For the six-parameter affine motion model, Equation 0 can be written as: (2) Where (mv 0x MV 0y ) is the motion vector of the top-left control point (top-left CPMV, or mv0), (mv 1x MV 1y ) is the motion vector of the upper right control point (upper right CPMV, or mv1), (mv 2x MV 2y ) is the motion vector of the lower left control point (lower left CPMV, or mv2).

[0015] In some implementations, a two-parameter model can be used to refine the inter-frame translation prediction candidates (e.g., regular merged candidates): MVx regress = MV x orig + biasX MV y regress = MV y orig + biasY (3) Figure 2 This demonstrates the control point motion vector (CPMV) of the current block encoded by an affine motion field. The current block has CPMVs at its top left (mv0), top right (mv1), and bottom left (mv2). The affine motion field mv' of the current block #6200 at position (x, y) can be derived using affine motion models such as Equation 1 (four-parameter affine model) or Equation 2 (six-parameter affine model).

[0016] To simplify motion compensation prediction, a block-based affine transformation prediction method is used. Figure 3 The affine motion field for each sub-block is shown. This figure illustrates the motion vector motion field of a block containing 16 4x4 sub-blocks. To derive the motion vector for each 4×4 luma sub-block, the motion vector of the center sample of each sub-block is calculated according to Equation 1 or Equation 2 and rounded to 1 / 16 fractional precision. Motion-compensated interpolation filters can be applied to generate predictions for each sub-block using the derived motion vectors. The sub-block size for the chroma component is also set to 4×4. The motion vector (MV) of the 4×4 chroma sub-block is calculated as the average of the motion vectors of the upper-left and lower-right luma sub-blocks within the corresponding 8x8 luma region.

[0017] B. Affine Merging Pattern In affine merging mode, the control point motion vector (CPMV) of the current coding unit (CU) is generated based on the motion information of spatially neighboring CUs. There can be up to five control point motion vector prediction (CPMVP) candidates, and the candidate index for the current CU is notified by signaling. The following three types of CPMV candidates are used to form the affine merging candidate list: (1) inherited affine merging candidates extrapolated from the CPMVs of neighboring CUs; ​​(2) affine merging candidate CPMVPs derived using the translational motion vectors of neighboring CUs; ​​and (3) zero motion vector.

[0018] Affine candidates are inherited by directly obtaining the CPMV from neighboring blocks, inheriting the affine model from the neighboring blocks (one from the left neighboring CU and one from the top neighboring CU). The neighboring blocks used for inheriting affine candidates are shown above. Figure 1As shown. For the left predictor, the candidate block scanning order is A1 (A0), and for the upper predictor, the candidate block scanning order is B1 (B0) (B2). When a neighboring affine CU is identified, its control point motion vector is used to derive CPMVP candidates in the affine merging list of the current CU.

[0019] Figure 4 This conceptually demonstrates the inheritance of control point motion vectors. As shown in the figure, for the current block 410, if the lower-left neighboring block A is encoded in affine mode, the motion vectors mv2, mv3, and mv4 of the upper-left, upper-right, and lower-left corners of CU420 containing block A can be inherited by the current block 410. When block A is encoded using a four-parameter affine model, the two CPMVs of the current CU410 can be calculated based on mv2 and mv3. If block A is encoded using a six-parameter affine model, the three CPMVs of the current CU410 can be calculated based on mv2, mv3, and mv4.

[0020] Constructing affine candidates involves building candidates by combining the translational motion information of each control point's neighbors. The motion information of the control points is generated by... Figure 1 The derivation of specified spatial and temporal neighbors is shown. (CPMV) k (k=1, 2, 3, 4) represents the k-th control point. For CPMV1, check blocks B2 (B3) (A2) and use the MV of the first available block. For CPMV2, check block B1 (B0), and for CPMV3, check block A1 (A0). If the Temporal Motion Vector Prediction (TMVP) is available, it is used as CPMV4. After obtaining the MVs of the four control points, affine merging candidates are constructed based on this motion information. The following combinations of control point MVs are used in sequence for construction: {CPMV1, CPMV2, CPMV3}, {CPMV1, CPMV2, CPMV4}, {CPMV1, CPMV3, CPMV4}, {CPMV2, CPMV3, CPMV4}, {CPMV1, CPMV2}, {CPMV1, CPMV3}. Combinations of three CPMVs construct a six-parameter affine merging candidate, and combinations of two CPMVs construct a four-parameter affine merging candidate. To avoid issues during motion scaling, if the reference indices of the control points are different, the relevant control point MV combinations are discarded. After checking the inherited affine merge candidates and constructing the affine merge candidates, if the list is still not full, a zero motion vector is inserted at the end of the list.

[0021] C. Affine AMVP Prediction The affine AMVP mode can be applied to CUs with a width and height greater than or equal to 16. Whether the affine AMVP mode is used is signaled in the CU-level bitstream, and then indicated by another signal as either a four-parameter affine or a six-parameter affine. In this mode, the difference between the current CU's CPMV and its predictor CPMVP is signaled in the bitstream. The affine AMVP candidate list is of size 2 and is generated in the following four CPMV candidate orders: (1) inherited affine AMVP candidates extrapolated from the CPMV of neighboring CUs, (2) constructed affine AMVP candidate CPMVP derived using the translational MV of neighboring CUs, (3) translational MV from neighboring CUs, and (4) zero MV.

[0022] The checking order for inherited affine AMVP candidates is the same as that for inherited affine merge candidates. The only difference is that for AVMP candidates, only affine CUs with the same reference image as the current block are considered. No pruning is performed when inserting inherited affine motion predictors into the candidate list.

[0023] The construction of AMVP candidates is based on the above. Figure 1 The specified spatial neighbor derivation is shown. The same checking order is used when constructing affine merge candidates. Additionally, the reference image indices of neighboring blocks are checked. The first block in the checking order is inter-coded and uses the same reference image as the current CU. When the current CU is coded in a four-parameter affine mode and both mv0 and mv1 are available, they are added as a candidate to the affine AMVP list. When the current CU is coded in a six-parameter affine mode and all three CPMVs are available, they are added as a candidate to the affine AMVP list. Otherwise, the AMVP candidate construction is set to unavailable.

[0024] If the affine AMVP candidate list still has fewer than two candidates after inserting valid inherited affine AMVP candidates and constructing AMVP candidates, then add mv0, mv1, and mv2 as translation MVs in sequence to predict the MVs of all control points in the current CU (if available). Finally, if the list is still not full, fill the affine AMVP candidate list with zero MVs.

[0025] III. Geometric Partitioning (GPM) and Extensions A. GPM Geometric Partitioning (GPM) signals a merging mode using a CU-level flag. Other merging modes include regular merging, Merge Mode with Motion Vector Difference (MMVD), Combined Inter-Intra Prediction (CIIP), and sub-block merging. For each possible CU size, GPM supports a total of 64 merging modes. ,in Excluding 8x64 and 64x8.

[0026] Figure 5 This illustrates how the CU is partitioned using Geometric Partitioning Patterns (GPM). Each GPM partition or segmentation is a partitioning pattern based on distance-angle pairing, used to define a partition or segment line. The figure shows an example of a GPM segmentation grouped by the same angle. As shown, when using GPM, the CU is partitioned into at least two parts by a geometrically positioned straight line. The position of the partition line is mathematically derived based on the angle and offset parameters of the specific partition.

[0027] Each partition in a CU formed by the partitioning pattern of GPM performs inter-frame prediction using its own motion (vector). In some implementations, each partition only allows unidirectional prediction, meaning each partition has one motion vector and one reference index. The unidirectional prediction motion constraint is used to ensure that, similar to traditional bidirectional prediction, each CU performs motion-compensated prediction only twice.

[0028] If the current CU uses GPM, a geometric partition index is also signaled, indicating the partitioning pattern (angle and offset) of the geometric partition, along with two merge indices (one for each partition). Each of the at least two partitions created from the geometric partitions according to the partitioning pattern can be assigned a merge index to select a candidate from the unidirectional prediction candidate list (also known as the GPM candidate list). The merge indices of the two partitions are paired, thus selecting a set of merge candidates. The maximum number of candidates in the GPM candidate list can be explicitly signaled in the Sequence Parameter Set (SPS) to specify the syntax binarization of the GPM merge indexes. After predicting each of the at least two partitions, sample values ​​along the edges of the geometric partitions are adjusted through a mixing process with adaptive weights. This is the prediction signal for the entire CU, and the transformation and quantization processes are applied to the entire CU as with other prediction patterns. The motion field of the CU predicted by GPM is then stored.

[0029] The one-way prediction candidate list for GPM partitions (GPM candidate list) can be directly derived from the merge candidate list of the current CU. Figure 6 This diagram illustrates a unidirectional prediction candidate list 600 for the GPM partition and an example of unidirectional prediction MV selection for the GPM. The GPM candidate list 600 is constructed in an even-odd manner, containing only unidirectional prediction candidates for alternating L0MV and L1MV. Let n be the index of the unidirectional prediction motion in the GPM unidirectional prediction candidate list. The LX (i.e., L0 or L1) motion vector of the nth extended merge candidate, where X equals the parity of n, is used as the nth unidirectional prediction motion vector for the GPM (these motion vectors are labeled "x" in the diagram). If the corresponding LX motion vector of the nth extended merge candidate does not exist, then the L(1) of the same candidate... The X-axis motion vector will be used as the unidirectional predicted motion vector for GPM.

[0030] As previously mentioned, sample values ​​along the edges of the geometric partitions are adjusted through a mixing process with adaptive weights. Specifically, after using each portion of the respective motion prediction geometric partition, at least two predicted signals are mixed to derive samples around the edges of the geometric partition. The mixing weights at each location of the CU are derived based on the distance of that location from the partition edge. The distance to the partition edge is derived as follows: (2) (3) ； (4) ； (5) in It is an index of the angle and offset of the geometric partition, which depends on the geometric partition index of the signaling notification. and The sign depends on the angle index. The weights for each part of the geometric partition are derived as follows: ； ( 6) ； ( 7) ； ( 8) The variable partIdx depends on the angle index. . Figure 7An example of the GPM partition edge blending process for the CU700 is shown. In this figure, the blending weights are generated based on the initial blending weight w0.

[0031] As mentioned earlier, the motion field of the CU predicted using GPM is stored. Specifically, Mv1 from the first geometric partition, Mv2 from the second geometric partition, and Mv formed by the combination of Mv1 and Mv2 are stored in the motion field of the GPM-encoded CU. The type of motion vector stored at each location in each motion field is determined as follows: sType = abs(motionIdx)<32 ? 2: (motionIdx ≤ 0 ? (1-partIdx):partIdx). (9) Where motionIdx equals d(4x+2, 4y+2), this value is recalculated from formula (2). partIdx depends on the angle index. If sType equals 0 or 1, then Mv0 or Mv1 is stored in the corresponding motion field; otherwise, if sType equals 2, then Mv, which is a combination of Mv0 and Mv2, is stored. The combined Mv is generated through the following process: (i) if Mv1 and Mv2 come from different reference image lists (one from L0 and the other from L1), then Mv1 and Mv2 are directly combined to form a dual predicted motion vector; (ii) otherwise, if Mv1 and Mv2 come from the same list, then only a single predicted motion Mv2 is stored.

[0032] B. GPM with merged vector difference (GPM-MMVD) In some embodiments, GPM is extended by applying motion vector refinement to existing GPM unidirectional motion vectors. First, a flag is signaled to the GPMCU to indicate whether this mode is to be used. If this mode is used, each geometric partition of the GPMCU can further decide whether to signal the Motion Vector Difference (MVD). If MVD is signaled for a geometric partition, the motion of that partition is further refined using the signaled MVD information after selecting a GPM merging candidate. All other procedures remain consistent with GPM.

[0033] MVD signals using a pairing of distance and direction, similar to MMVD. GPM with MMVD (GPM-MMVD) involves nine candidate distances (¼-pel, ½-pel, 1-pel, 2-pel, 3-pel, 4-pel, 6-pel, 8-pel, 16-pel) and eight candidate directions (four horizontal / vertical directions and four diagonal directions). Furthermore, when pic_fpel_mmvd_enabled_flag equals 1, MVD is shifted left by 2 bits, just like MMVD.

[0034] C. Template-matching based GPM (GPM-TM) In some embodiments, motion vectors of GPM partitions can be refined using template matching (TM). When the CU enables GPM mode, a CU-level flag is signaled to indicate whether TM is applied to both geometric partitions. Motion information for each geometric partition is refined using TM. When TM is selected, a template is constructed using neighboring samples from the left and top, or from both the left and top, based on the partition angle. Table 1 below shows the templates for the first and second geometric partitions, where A indicates the use of the top sample, L indicates the use of the left sample, and L+A indicates the use of both the left and top samples.

[0035] Table 1:

[0036] Then, by minimizing the difference between the current template and the template in the reference image, the motion is refined using the same search mode of the merging mode and disabling the half-pixel interpolation filter. The GPM candidate list is constructed as follows: (1) The video encoder directly derives interleaved List-0 motion vector candidates and List-1 motion vector candidates from the regular merging candidate list, where List-0 motion vector candidates have a higher priority than List-1 motion vector candidates. An adaptive threshold pruning method based on the current CU size is used to remove redundant motion vector candidates; (2) The video encoder further derives interleaved List-1 motion vector candidates and List-0 motion vector candidates directly from the regular merging candidate list, where List-1 motion vector candidates have a higher priority than List-0 motion vector candidates. The same adaptive threshold pruning method is also used to remove redundant motion vector candidates; (3) The video encoder fills the GPM candidate list with zero motion vector candidates until the GPM candidate list is full.

[0037] In some embodiments, GPM-MMVD and GPM-TM are mutually exclusive and enabled only for one CU using GPM. In some embodiments, the GPM-MMVD syntax is first signaled. When both GPM-MMVD control flags are false (i.e., neither GPM partition has GPM-MMVD enabled), the GPM-TM flag is signaled to indicate whether template matching refinement should be applied to the GPM partition. Otherwise (if at least one GPM-MMVD flag is true), the value of the GPM-TM flag is inferred to be false.

[0038] D. Adaptive Hybrid GPM For GPM prediction, the final predicted samples are generated by weighted averaging of the predictions from the two predicted signals. Two integer mixing matrices (W0 and W1) are used. In some embodiments, the weights in the GPM mixing matrix can be derived using a ramp function based on the displacement from the predicted sample location to the GPM partition boundary. The mixing region size is fixed at 2 (2 samples on each side of the GPM partition boundary). Figure 8 This demonstrates the displacement (d) from the predicted sample location to the GPM partition boundary and the mixed region. The figure shows the GPM blending weight ramp function. It also illustrates four additional blending region sizes defined by the blending process (one-quarter, one-half, double, and four times the existing region size). A CU-level flag can be encoded to signal the selected blending region size.

[0039] E. Template-match-based GPM partition pattern reordering In some embodiments, for the motion information of the current GPM block, the respective TM cost of each GPM partition pattern is calculated. Then, all GPM partition patterns are reordered in ascending order according to their TM cost. Instead of transmitting the GPM partition patterns, an index is transmitted via Golomb-Rice code to indicate the exact position of the GPM partition pattern in the reordering list. The GPM partition pattern reordering method is a two-step process performed after generating the respective reference templates of the two GPM partitions in the coding unit, as follows: (1) extending the edges of the GPM partitions into the reference templates of the two GPM partitions to obtain 64 reference templates; (2) calculating the respective TM cost of each of the 64 reference templates; (3) sorting the GPM partition patterns in ascending order according to their TM cost and marking the best 32 partition patterns as available partition patterns. Figure 9 This demonstrates the GPM partition edge extending from the current CU to partition the stencil region. The GPM blending process is not applied to the stencil region around the edge.

[0040] F. GPM with affine prediction In some embodiments, GPM is extended to support affine motion compensation (AMC). Therefore, GPM partitions can be predicted using AMC inter-frame prediction, non-AMC inter-frame prediction, or intra-frame prediction. Furthermore, GPM partitions predicted by AMC can be combined with another GPM partition predicted by AMC, non-AMC, or intra-frame prediction.

[0041] When AMC is applied, the single-predict affine merge candidate list is constructed from the sub-block-based merge candidate list after discarding sub-TMVP candidates, similar to the construction of the single-predict merge candidate list for GPM in VVC. AMC uses the control point motion vector (CPMV) of a merge candidate from the single-predict affine merge candidate list for the GPM partition.

[0042] For each GPM partition, a gpm_affine_flag is transmitted to indicate whether AMC is applied to that GPM partition. Merge candidate indexes for GPM partitions are transmitted using different arithmetic context models depending on whether AMC is applied or not.

[0043] G. Regression-based GPM mixture For some embodiments, a regression-based GPM mixing pattern is provided as an additional implicit GPM pattern, where two integer mixing matrices (W0 and W1) are derived from the template region (the row above and the column to the left of the current block). The mixing matrix is ​​modeled as an affine linear function of the sample position (x, y) in the current CU: W0(x,y) = ax + by + c; and W1(x,y) = 1 - W0(x,y).

[0044] The parameters (a, b, c) are derived from a reference template, employing a method that minimizes the mean square error (MSE) between the corresponding input and output samples in the neighboring template regions of the current block. MSE minimization is achieved by calculating the autocorrelation matrix of the input samples and the cross-correlation vector between the input and output samples (including parameters a, b, c). The autocorrelation matrix is ​​decomposed using LDL, and the final filter coefficients are calculated through back-substitution. The autocorrelation matrix is ​​calculated using the reconstructed values ​​of the luminance and / or chrominance samples.

[0045] The candidate pair list is constructed from regular GPM candidates and reordered according to template cost. The GPM implicit pattern is transmitted via a CU-level flag (gpm_implicit_flag). If gpm_implicit_flag is true, merge-idx is encoded to transmit the used GPM candidate pairs. If gpm_implicit_flag is false, regular GPM syntax elements are transmitted.

[0046] In some embodiments, an additional implicit GPM pattern (i.e., the same two integer mixing matrices as W0 and W1 mentioned above) can be derived and selected from the template region when GPM with affine prediction is enabled. In some embodiments, a flag is passed after gpm_affine_flag for both GPM partitions to indicate whether regression-based GPM mixing is enabled. When this flag and gpm_affine_flag for at least one GPM partition are enabled, reference templates for the two GPM partitions are used to derive W0 and W1 by minimizing sample differences in the template region, and the GPM partitions predicted by AMC and non-AMC are mixed in the CU according to W0 and W1. For one or more GPM partitions predicted by AMC, the reference template is predicted in a sub-block-based manner, i.e., reference template samples are obtained from the top or left reconstructed region of each affine prediction sub-block. When this flag is enabled and gpm_affine_flag for both GPM partitions are disabled, the two GPM partitions predicted by non-AMC are mixed in the CU according to W0 and W1.

[0047] In some embodiments, when regression-based GPM blending is enabled, a GPM with an affine prediction mode can be selected.

[0048] H.GPM Candidate Pairing List In some embodiments, a merge candidate pairing list is constructed, with each pair containing two candidates used to encode or decode two GPM partitions, respectively. In some embodiments, the candidate pairing list is constructed from inter-frame merge candidates and affine merge candidates. Pairings in the list may include pairs of two inter-frame merge candidates, pairs of two affine merge candidates, and pairs of one inter-frame merge candidate and one affine merge candidate, used for deriving the regression blending weights. The list may be reordered based on template costs. If `gpm_implicit_flag` is true, `merge-idx` is encoded to transmit candidate pairings for GPM prediction. If `gpm_implicit_flag` is false, regular GPM syntax elements are transmitted.

[0049] In some embodiments, first and second candidate pairing lists are constructed from inter-frame merge candidates and affine merge candidates, respectively. The first list is constructed from inter-frame merge candidates, and the second list is constructed from inter-frame merge candidates and affine merge candidates, but pairs of two inter-frame merge candidates are excluded. If gpm_implicit_flag is true, an additional flag is transmitted to indicate the list used. The flag is followed by merge-idx encoding to transmit GPM candidate pairs from the selected list for the derivation of the regression blending weights. If gpm_implicit_flag is false, regular GPM syntax elements are transmitted.

[0050] In some embodiments, two candidate pairing lists are constructed, one from inter-frame merge candidates and the other from affine merge candidates. The first list is constructed from inter-frame merge candidates, and the second list from affine merge candidates. If `gpm_implicit_flag` is true, an additional flag is transmitted to indicate the list used. The flag is followed by a encoded `merge-idx` (merge index) to transmit GPM candidate pairings from the selected list for the derivation of the regression blending weights.

[0051] In some embodiments, when regression-based GPM blending is enabled, GPM with inter-frame, intra-frame, IBC, and affine prediction modes can be selected. All candidate pairs for inter-frame merging, intra-frame, IBC, and affine merging are divided into multiple lists. If gpm_implicit_flag is true, several bits are transmitted to indicate the list used, followed by merge-idx to select candidate pairs from the lists for the derivation of regression blending weights.

[0052] In some implementations, the candidate pairing list is constructed from inter-frame merge, intra-frame, intra-block copy (IBC), and affine merge candidates. This list is reordered based on template cost. If `gpm_implicit_flag` is true, a merge index is encoded to indicate GPM candidate pairings, including two sets of inter-frame merge candidate pairings, two sets of intra-frame candidate pairings, two sets of IBC candidate pairings, two sets of affine merge candidate pairings, one set of inter-frame merge candidates paired with intra-frame, IBC, or affine merge candidate pairings, one set of intra-frame candidates paired with inter-frame merge, IBC, or affine merge candidate pairings, one set of IBC candidates paired with inter-frame merge, intra-frame, or affine merge candidate pairings, and one set of affine merge candidates paired with inter-frame merge, intra-frame, or IBC candidate pairings, for regression derivation of the mixed weights. If `gpm_implicit_flag` is false, regular GPM syntax elements are indicated.

[0053] In some implementations, the two candidate pairing lists are constructed from inter-frame merge, intra-frame merge, IBC, and affine merge candidates. The first list is constructed from inter-frame merge candidates, and the second list is constructed from inter-frame merge, intra-frame merge, IBC, and affine merge candidates, but does not include the two sets of inter-frame merge candidate pairings. If gpm_implicit_flag is true, additional flags are indicated to specify which list to use.

[0054] In some implementations, the four candidate pairing lists are constructed from inter-frame merge, intra-frame merge, IBC, and affine merge candidates. The first list is constructed from inter-frame merge candidates, the second from intra-frame candidates, the third from IBC candidates, and the fourth from affine merge candidates. If gpm_implicit_flag is true, two additional bits are used to indicate which list is used.

[0055] Figure 10 Conceptually, this presents a list of candidate pairs containing all possible merge candidate pairings. Each item in list 1000 is a pair of two merge candidates, which can come from any of the three coding tools A, B, and C. (Candidates A:1, A:2, etc., can represent inter-frame merge candidates; B:1, B:2, etc., can represent affine merge candidates; C:1, C:2, etc., can represent IBC merge candidates.) List 1000 contains all or most possible pairings between merge candidates. The candidates in list 1000 can be further reordered and indexed according to template cost.

[0056] Figure 11 Conceptually, this presents multiple candidate pairing lists corresponding to different encoding tools. List 1110 contains all or most candidate pairings to be merged with encoding tool A. List 1120 contains all or most candidate pairings to be merged with encoding tool B. List 1130 contains all or most candidate pairings to be merged with encoding tool C. The candidates in lists 1110, 1120, and 1130 can be further reordered and indexed according to template cost.

[0057] J. GPM with Local Illumination Compensation (LIC) Local illumination compensation (LIC) is an inter-frame prediction technique used to model the local illumination variation between the current block and its predicted block as a function of the variation between the current block template and the reference block template. The parameters of this function can be represented by a scaling factor α and an offset β, forming a linear equation, α*p[x] + β, used to compensate for illumination variations, where p[x] is a reference sample on the reference image pointed to at position x by a motion vector (MV). In some implementations, since the parameters α and β can be derived based on the current block template and the reference block template, no signaling notification is required. The video encoder can signal the LIC flag to enable or disable the use of LIC.

[0058] In some implementations, GPM can be enabled together with LIC. In this case, the motion information for both GPM partitions includes the LIC flag and LIC parameters. If the LIC process is enabled for both GPM partitions, the corresponding LIC parameters can be applied to the two partitions separately without needing to re-derive the LIC parameters during motion compensation (MC). In some implementations, if only one GPM partition contains the LIC flag and corresponding LIC parameters, the LIC can be directly applied to both GPM partitions. In this case, the same LIC parameters will be applied to both GPM partitions.

[0059] In some implementations, if only one GPM partition contains the LIC flag and the corresponding LIC parameters (inherited LIC parameters), the LIC parameters can be applied to that GPM partition. For the other GPM partition, refined LIC parameters will be applied. The refined LIC parameters are derived by refining the inherited LIC parameters. In some implementations, the inherited LIC parameters are refined using the template of the current block (e.g., the upper and left adjacent regions) and the corresponding reference block.

[0060] In some implementations, a signaling notification flag (cu_gpm_lic_flag) is used in the bitstream to indicate whether the GPM can be enabled together with the LIC. If cu_gpm_lic_flag is enabled, the GPM can only be inserted into the merge list if the merge candidate contains the enabled LIC flag and the corresponding LIC parameter in the merge information.

[0061] Any of the methods described above can be implemented in the encoder and / or decoder. For example, any of the methods described above can be implemented in the inter-frame / intra-frame / prediction module of the encoder, and / or in the inter-frame / intra-frame / prediction module of the decoder. Alternatively, any of the methods described above can be implemented as circuitry coupled to the inter-frame / intra-frame / prediction module of the encoder and / or the inter-frame / intra-frame / prediction module of the decoder, in order to provide the required information to the inter-frame / intra-frame / prediction module.

[0062] IV. Example Video Encoder Figure 12 An example video encoder 1200 capable of implementing GPM is shown. As shown, the video encoder 1200 receives an input video signal from a video source 1205 and encodes the signal into a bitstream 1295. The video encoder 1200 has multiple components or modules for encoding the signal from the video source 1205, including at least some selected from the following components: a transform module 1210, a quantization module 1211, an inverse quantization module 1214, an inverse transform module 1215, an intra-frame image estimation module 1224, an intra-frame prediction module 1225, a motion compensation module 1230, a motion estimation module 1235, a loop filter 1245, a reconstructed image buffer 1250, a motion vector (MV) buffer 1265, a motion vector prediction module 1275, and an entropy encoder 1290. The motion compensation module 1230 and the motion estimation module 1235 belong to the inter-frame prediction module 1240. Intra-frame prediction module 1225 and intra-frame prediction estimation module 1224 belong to the current image prediction module 1220, which uses the current image reconstruction sample as the reference sample for predicting the current block.

[0063] In some embodiments, modules 1210–1290 are software instruction modules executed by one or more processing units (e.g., processors) of a computing device or electronic device. In some embodiments, modules 1210–1290 are hardware circuit modules implemented by one or more integrated circuits (ICs) of an electronic device. Although modules 1210–1290 are illustrated as independent modules, some of these modules may be combined into a single module.

[0064] Video source 1205 provides an uncompressed video signal, presenting pixel data for each video frame. Subtractor 1208 calculates the difference between the raw video pixel data of video source 1205 and the predicted pixel data 1213 of motion compensation module 1230 or intra-frame prediction module 1225, as the prediction residual 1209. Transform module 1210 converts this difference (or residual pixel data or residual signal 1208) into transform coefficients (e.g., by performing a Discrete Cosine Transform (DCT)). Quantization module 1211 quantizes the transform coefficients into quantized data (or quantization coefficients) 1212, which are then encoded into bitstream 1295 by entropy encoder 1290.

[0065] The inverse quantization module 1214 inverse-quantizes the quantized data (or quantization coefficients) 1212 to obtain transform coefficients 1218. The inverse transform module 1215 performs an inverse transform on the transform coefficients 1218 to generate a reconstruction residual 1219. The reconstruction residual 1219 is added to the predicted pixel data 1213 to generate reconstructed pixel data 1217. In some embodiments, the reconstructed pixel data 1217 is temporarily stored in the line buffer 1227 (or the intra-frame prediction buffer) for intra-frame image prediction and spatial motion vector (MV) prediction. After being filtered by the loop filter 1245, the reconstructed pixels are stored in the reconstructed image buffer 1250. In some embodiments, the reconstructed image buffer 1250 is external memory to the video encoder 1200. In some embodiments, the reconstructed image buffer 1250 is internal memory to the video encoder 1200.

[0066] The intra-frame image estimation module 1224 performs intra-frame prediction based on the reconstructed pixel data 1217 to generate intra-frame prediction data. The intra-frame prediction data is provided to the entropy encoder 1290 and encoded into the bitstream 1295. The intra-frame prediction data is also used by the intra-frame prediction module 1225 to generate prediction pixel data 1213.

[0067] The motion estimation module 1235 performs inter-frame prediction of reference pixel data from previously decoded frames stored in the reconstructed image buffer 1250 by generating motion vectors (MVs). These motion vectors are provided to the motion compensation module 1230 for generating predicted pixel data.

[0068] The video encoder 1200 does not encode the complete actual motion vector into the bitstream. Instead, it uses motion vector prediction to generate predicted motion vectors and encodes the difference between the motion vector used for motion compensation and the predicted motion vector as residual motion data and stores it in the bitstream 1295.

[0069] The motion vector prediction module 1275 generates a predicted motion vector based on the reference motion vector generated for encoding the previous video frame, i.e., the motion-compensated motion vector used to perform motion compensation. The motion vector prediction module 1275 retrieves the reference motion vector of the previous video frame from the motion vector buffer 1265. The video encoder 1200 stores the motion vector generated for the current video frame in the motion vector buffer 1265 as the reference motion vector for generating the predicted motion vector.

[0070] The motion vector prediction module 1275 generates a predicted motion vector using a reference motion vector. The predicted motion vector can be calculated through spatial motion vector prediction or temporal motion vector prediction. The difference (residual motion data) between the predicted motion vector and the motion compensated motion vector (MCMV) of the current frame is encoded into the bit stream 1295 by the entropy encoder 1290.

[0071] The entropy encoder 1290 uses entropy coding techniques (such as context-adaptive binary arithmetic coding (CABAC) or Huffman coding) to encode various parameters and data into a bitstream 1295. The entropy encoder 1290 encodes various header elements, flags, quantization transform coefficients 1212, and residual motion data as syntax elements into the bitstream 1295. The bitstream 1295 is then stored in a storage device or transmitted to the decoder via a communication medium such as a network.

[0072] Loop filter 1245 filters or smooths the reconstructed pixel data 1217 to reduce coded artifacts, especially at pixel block boundaries. In some embodiments, the filtering or smoothing operations performed by loop filter 1245 include deblocking filter (DBF), sample adaptive offset (SAO), and / or adaptive loop filter (ALF). In some embodiments, lumamapping chromascaling (LMCS) is performed before loop filtering.

[0073] Figure 13 This diagram illustrates a portion of a video encoder 1200 that implements a regression-blending-based GPM. The figure shows the components that generate GPM predictions as predicted pixel data 1213. As shown, at least two predictions 1310 and 1320 are generated for at least two GPM partitions. Each partition prediction can be generated by intra-frame prediction 1220, inter-frame prediction 1240, or current image reference (IBC) prediction 1312. Inter-frame predictions can be further refined by a local illumination compensation (LIC) module 1314. Predictions can be performed at the sub-block level to implement an affine merging mode or an AMVP-like mode for each GPM partition.

[0074] Each partition prediction is generated based on candidates from the encoding tool. For example, the first partition prediction 1310 may be generated from affine merge candidates, and the second partition prediction 1320 may be generated from inter-frame merge candidates. The encoder selects a set of candidate pairs from a candidate pair list (or multiple lists) 1330 by providing a merge index, since the candidate pair list 1330 can be reordered according to template cost. When regression-based GPM blending is enabled, the merge index may be provided by the entropy encoder 1290.

[0075] Two partition predictions, 1310 and 1320, are merged by the GPM mixing module 1350 based on the GPM partition parameters (provided by the entropy encoder 1290) to form a GPM prediction. Samples along the GPM partition boundaries / edges are mixed according to weighting factors (mixing weights) provided by the regression module 1360. The regression module 1360 performs mean squared error (MSE) minimization using the corresponding input and output samples in the template region to determine the weighting factors.

[0076] When the LIC parameter applies only to one of the two GPM partitions, the LIC parameter refinement module 1340 can refine the LIC parameter of the other GPM partition based on samples of neighboring templates and corresponding reference blocks stored in the reconstructed image buffer 1250 or the line buffer 1227.

[0077] Figure 14 A conceptual illustration of process 1400 is provided, which utilizes a candidate pairing list to perform GPM prediction via regression with mixed weights. In some embodiments, one or more processing units (such as processors) of a computing device implementing encoder 1200 execute process 1400 by executing instructions stored in a computer-readable medium. In some embodiments, an electronic device implementing encoder 1200 executes process 1400.

[0078] In step 1410, the encoder receives pixel data of the current block to be encoded into the current image. In step 1420, the encoder divides the current block into at least two geometric partition pattern (GPM) partitions.

[0079] In step 1430, the encoder uses template region samples from neighboring current blocks to derive mixed weights for the partition via regression. In some embodiments, the encoder performs mean squared error (MSE) minimization based on the corresponding input and output samples in the template regions to determine the weighting factors.

[0080] The encoder selects a set of candidate pairs from a candidate pair list (at module 1440). In some implementations, the encoder selects a set of candidate pairs by choosing from multiple candidate pair lists, where each set of candidate pairs in a particular list contains at least one candidate for a particular encoding tool. In some implementations, the encoder selects a set of candidate pairs by choosing from only one candidate pair list, which contains pairs of any two candidates for two or more encoding tools. For example, the candidate pair list may include different candidate pairs from inter-frame merge candidates and affine merge candidates. Another example is that the candidate pair list may include different candidate pairs from inter-frame merge candidates, affine merge candidates, intra-frame merge candidates, and intra-blockcopy (IBC) merge candidates.

[0081] In some implementations, the candidate pairing list is reordered according to the template cost, and a corresponding merge index is assigned to each candidate pairing in the list so that the encoder can identify the position of the candidate pairing in the list by signaling the merge index. In some implementations, the merge index of the candidate pairing list is signaled when a flag indicating that the blending weights are derived through regression (e.g., gpm_implicit_flag) is indicated.

[0082] In some implementations, when only the first GPM partition among at least two GPM partitions of the current block has a LIC enable flag and an inherited LIC parameter set, the encoder refines the inherited LIC parameter set and applies the refined LIC parameter set to the second GPM partition among the at least two GPM partitions. The inherited LIC parameters of the second GPM partition can be refined using the neighboring templates and corresponding reference blocks of the current block.

[0083] The encoder applies the selected candidate pairing (at module 1450) to the two GPM partitions and utilizes the derived mixed weights to generate a predictor for the current block. The encoder uses the generated predictor to encode the current block (at module 1460) to produce a prediction residual.

[0084] V. Example Video Decoder In some implementations, the encoder may signal (or generate) one or more syntax elements in the bitstream so that the decoder can parse the one or more syntax elements from the bitstream.

[0085] Figure 15 An example video decoder 1500 capable of implementing GPM is shown. As shown, the video decoder 1500 is an image decoding or video decoding circuit that receives a bitstream 1595 and decodes the bitstream content into video frame pixel data for display. The video decoder 1500 has several components or modules for decoding the bitstream 1595, including some selected from an inverse quantization module 1514, an inverse transform module 1515, an intra-frame prediction module 1525, a motion compensation module 1530, a loop filter 1545, a decoded image buffer 1550, an MV buffer 1565, an MV prediction module 1575, and a parser 1590. The motion compensation module 1530 is part of the inter-frame prediction module 1540. The intra-frame prediction module 1525 is part of the current image prediction module 1520, which uses current image reconstructed samples as reference samples for predicting the current block.

[0086] In some embodiments, modules 1514–1590 are software instruction modules executed by one or more processing units (e.g., processors) of a computing device. In some embodiments, modules 1514–1590 are hardware circuit modules implemented by one or more integrated circuits (ICs) of an electronic device. Although modules 1514–1590 are illustrated as independent modules, some of these modules may be combined into a single module.

[0087] Parser 1590 (or entropy decoder) receives bitstream 1595 and performs preliminary parsing according to the syntax defined by the video coding or image coding standard. The parsed syntax elements include various header elements, flags, and quantization data (or quantization coefficients) 1512. Parser 1590 then uses entropy coding techniques, such as context-adaptive binary arithmetic coding (CABAC) or Huffman coding, to parse out the various syntax elements.

[0088] The inverse quantization module 1514 inverse-quantizes the quantized data (or quantization coefficients) 1512 to obtain transform coefficients, and the inverse transform module 1515 inverse-transforms the transform coefficients 1518 to generate the reconstructed residual signal 1519. The reconstructed residual signal 1519 is added to the predicted pixel data 1513 from the intra-frame prediction module 1525 or the motion compensation module 1530 to generate the decoded pixel data 1517. The decoded pixel data is filtered by the loop filter 1545 and then stored in the decoded image buffer 1550. In some embodiments, the decoded image buffer 1550 is an external memory of the video decoder 1500. In some embodiments, the decoded image buffer 1550 is an internal memory of the video decoder 1500.

[0089] Intra-prediction module 1525 receives intra-prediction data from bitstream 1595 and generates predicted pixel data 1513 from decoded pixel data 1517 stored in decoded image buffer 1550 accordingly. In some embodiments, decoded pixel data 1517 is also stored in line buffer 1527 (or intra-prediction buffer) for intra-image prediction and spatial MV prediction.

[0090] In some embodiments, the contents of the decoded image buffer 1550 are used for display. The display device 1505 can directly retrieve the contents of the decoded image buffer 1550 for display, or retrieve the contents of the decoded image buffer into a display buffer. In some embodiments, the display device receives pixel values ​​from the decoded image buffer 1550 via pixel transfer.

[0091] The motion compensation module 1530 generates predicted pixel data 1513 from the decoded pixel data 1517 stored in the decoded image buffer 1550 based on the motion compensation MV (MCMV). These motion compensation MVs are decoded by adding the residual motion data received from the bitstream 1595 to the predicted MV received from the MV prediction module 1575.

[0092] The MV prediction module 1575 generates a predicted MV based on a reference MV generated for decoding the previous video frame, such as a motion-compensated MV used for performing motion compensation. The MV prediction module 1575 retrieves the reference MV of the previous video frame from the MV buffer 1565. The video decoder 1500 stores the motion-compensated MV generated for decoding the current video frame in the MV buffer 1565 as a reference MV for generating the predicted MV.

[0093] Loop filter 1545 filters or smooths the decoded pixel data 1517 to reduce coding artifacts, especially at pixel block boundaries. In some embodiments, the filtering or smoothing operations performed by loop filter 1545 include deblocking filter (DBF), sample adaptive offset (SAO), and / or adaptive loop filter (ALF). In some embodiments, lumamapping chromascaling (LMCS) is performed before loop filtering.

[0094] Figure 16 A portion of the video decoder 1500 is shown, implementing a geometric partitioning (GPM) mode with regression-based merging. The figure illustrates the components that generate GPM predictions as predicted pixel data 1513. As shown, predictions 1610 and 1620 are generated for at least two GPM partitions. Each partition prediction can be generated by intra-frame prediction 1520, inter-frame prediction 1540, or current image reference (intra-block copy, IBC) prediction 1612. Inter-frame predictions can also be further refined by a local illumination compensation (LIC) module 1614. Predictions can be performed at the sub-block level to implement an affine merging mode or an affine AMVP mode for each GPM partition.

[0095] Each partition prediction is generated based on candidates from the encoding tool. For example, the first partition prediction 1610 may be generated from affine merge candidates, while the second partition prediction 1620 may be generated from inter-frame merge candidates. The decoder selects a set of candidate pairs from a candidate pair list (or multiple lists) 1630 by providing a merge index, since the candidate pair list 1630 can be reordered according to the template cost. When regression-based GPM blending is enabled, the merge index may be provided by the entropy decoder 1590.

[0096] Two partition predictions, 1610 and 1620, are combined by the GPM mixing module 1650 to form a GPM prediction based on the GPM partition parameters (provided by the entropy decoder 1590). Samples along the GPM partition boundaries / edges are mixed according to weighting factors (mixing weights) provided by the regression module 1660. The regression module 1660 performs mean squared error (MSE) minimization using corresponding input and output samples in the template region to determine the weighting factors.

[0097] When the LIC parameter can only be used in one of the two GPM partitions, the LIC parameter refinement module 1640 can refine the LIC parameter of the other GPM partition based on the samples of the neighboring templates and corresponding reference blocks stored in the reconstructed image buffer 1550 or the line buffer 1527.

[0098] Figure 17 This conceptually illustrates a process 1700 for GPM prediction based on regression mixed weights using a list of candidate pairings. In some embodiments, one or more processing units (such as processors) of a computing device implementing decoder 1500 execute process 1700 by executing instructions stored on a computer-readable medium. In some embodiments, an electronic device implementing decoder 1500 executes process 1700.

[0099] In step 1710, the decoder receives pixel data of the current block to be decoded into the current image of the video. In step 1720, the decoder divides the current block into at least two geometric partitioning pattern (GPM) partitions.

[0100] In step 1730, the decoder uses template region samples from neighboring current blocks to derive mixed weights for the partition via regression. In some embodiments, the decoder performs mean squared error (MSE) minimization based on corresponding input and output samples in the template regions to determine the weighting factors.

[0101] In step 1740, the decoder selects a set of candidate pairs from the candidate pairing list. In some embodiments, the decoder selects a set of candidate pairs by choosing from multiple candidate pairing lists, wherein each set of candidate pairs in a particular list contains at least one candidate for a particular coding tool. In some embodiments, the decoder selects a set of candidate pairs by choosing from only one candidate pairing list, which contains pairs of any two candidates for two or more coding tools. For example, the candidate pairing list may include different candidate pairs from inter-frame merge candidates and affine merge candidates. Another example is that the candidate pairing list may include different candidate pairs where each candidate is selected from inter-frame merge candidates, affine merge candidates, intra-frame merge candidates, and intra-block copy (IBC) merge candidates.

[0102] In some embodiments, the candidate pairing list is reordered according to the template cost, and a corresponding merge index is assigned to each candidate pairing in the list, such that the decoder can signal the merge index to identify the position of the selected candidate pairing in the list. In some embodiments, the merge index of the candidate pairing list is signaled when a flag (such as gpm_implicit_flag) is indicated to indicate that the blending weights are derived by regression.

[0103] In some embodiments, when only the first GPM partition among at least two GPM partitions of the current block has a LIC enable flag and an inherited LIC parameter set, the decoder refines the inherited LIC parameter set and applies the refined LIC parameter set to the second GPM partition among the at least two GPM partitions. The inherited LIC parameters of the second GPM partition can be refined using the neighboring templates and corresponding reference blocks of the current block.

[0104] In step 1750, the decoder applies the selected candidate pairings to the two GPM partitions and utilizes the derived mixing weights to generate a predictor for the current block. In step 1760, the decoder reconstructs the current block by combining the generated predictor with the prediction residual. Subsequently, the decoder can use the reconstructed current block for display as part of the reconstructed current image.

[0105] VI. Example Electronic System Many of the features and applications described above are implemented as software processes, which are specified as a set of instructions recorded on a computer-readable storage medium (also known as a computer-readable medium). When these instructions are executed by one or more computing or processing units (such as one or more processors, processor cores, or other processing units), the processing unit performs the operations indicated in the instructions. Examples of computer-readable media include, but are not limited to, CD-ROMs, flash drives, random access memory (RAM) chips, hard disks, erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), etc. Computer-readable media do not include carrier waves and electronic signals transmitted via wireless or wired connections.

[0106] In this specification, the term "software" includes firmware residing in read-only memory or application programs stored in magnetic storage, which can be read into memory and processed by a processor. Furthermore, in some embodiments, multiple software inventions may be implemented as sub-parts of a larger program while remaining independent software inventions. In some embodiments, multiple software inventions may also be implemented as independent programs. Finally, any combination of independent programs that collectively implement the software inventions described herein falls within the scope of this disclosure. In some embodiments, when a software program is installed and runs on one or more electronic systems, one or more specific machine implementations are defined that execute and perform the operations of the software program.

[0107] Figure 18 An electronic system 1800 implementing certain embodiments of the present disclosure is conceptually illustrated. The electronic system 1800 may be a computer (such as a desktop computer, personal computer, tablet computer, etc.), a telephone, a personal digital assistant (PDA), or any other type of electronic device. Such electronic systems include various types of computer-readable media and their interfaces. The electronic system 1800 includes a bus 1805, a processing unit 1810, a graphics-processing unit (GPU) 1815, system memory 1820, a network 1825, a read-only memory 1830, a permanent storage device 1835, an input device 1840, and an output device 1845.

[0108] Bus 1805 represents all system, peripheral, and chipset buses used to communicate with the numerous internal devices of electronic system 1800. For example, bus 1805 communicates with processing unit 1810, graphics processing unit (GPU) 1815, read-only memory (ROM) 1830, system memory 1820, and permanent storage device 1835.

[0109] Processing unit 1810 retrieves instructions from these different memory units for execution and retrieves data for processing, thereby executing the processes of this disclosure. The processing unit may be a single-core processor or a multi-core processor in different embodiments. Some instructions are passed to GPU 1815 and executed thereon. GPU 1815 is capable of offloading various computations or supplementing image processing provided by processing unit 1810.

[0110] Read-only memory (ROM) 1830 stores static data and instructions used by processing unit 1810 and other modules of the electronic system. Permanent storage device 1835 is a read-write storage device. This device is a non-volatile storage unit that can store instructions and data even when the electronic system 1800 is powered off. Some embodiments of this disclosure use mass storage devices (such as disks or optical discs and their corresponding disk drives) as permanent storage device 1835.

[0111] Other embodiments use removable storage devices (such as floppy disks, flash memory devices, etc., and their corresponding disk drives) as permanent storage devices. Similar to permanent storage device 1835, system memory 1820 is also a read-write storage device. However, unlike storage device 1835, system memory 1820 is a volatile read-write memory, such as random access memory. System memory 1820 stores some instructions and data used by the processor during runtime. In some embodiments, processes conforming to this disclosure are stored in system memory 1820, permanent storage device 1835, and / or read-only memory 1830. For example, various storage units include instructions for processing multimedia segments, conforming to certain embodiments. From these different storage units, processing unit 1810 retrieves instructions for execution and retrieves data for processing, thereby executing the processes of some embodiments.

[0112] Bus 1805 also connects input device 1840 and output device 1845. Input device 1840 enables a user to pass information to the electronic system and select commands. Input device 1840 includes an alphanumeric keypad and pointing device (also known as a "cursor control device"), a camera (such as a webcam), a microphone, or similar devices for receiving voice commands. Output device 1845 displays images generated by the electronic system or otherwise outputs data. Output device 1845 includes printers and display devices such as cathode ray tube (CRT) or liquid crystal display (LCD), as well as speakers or similar audio output devices. Some embodiments include devices such as touchscreens that function as both input and output devices.

[0113] Finally, as Figure 18As shown, bus 1805 also connects electronic system 1800 to network 1825 via a network adapter (not shown). In this way, the computer can become part of a computer network (such as a local area network (LAN), wide area network (WAN), or intranet, or a network of networks, such as the Internet). Any or all components of electronic system 1800 can be used in conjunction with this disclosure.

[0114] Some embodiments include electronic components, such as microprocessors, storage, and memory, that store computer program instructions in a machine-readable or computer-readable medium (also referred to as a computer-readable storage medium, machine-readable medium, or machine-readable storage medium). Examples of such computer-readable media include random access memory (RAM), read-only memory (ROM), read-only optical disc (CD-ROM), recordable optical disc (CD-R), rewritable optical disc (CD-RW), read-only digital versatile optical discs (such as DVD-ROM, dual-layer DVD-ROM), various recordable / rewritable DVDs (such as DVD-RAM, DVD-RW, DVD+RW, etc.), flash memory (such as SD cards, mini-SD cards, micro-SD cards, etc.), magnetic and / or solid-state drives, read-only and recordable Blu-ray® optical discs, ultra-high density optical discs, any other optical or magnetic media, and floppy disks. The computer-readable medium may store a computer program executable by at least one processing unit and contain a set of instructions for performing various operations. Examples of computer programs or computer code include machine code (such as code generated by a compiler) and files containing high-level code that are executed by a computer, electronic component, or microprocessor via an interpreter.

[0115] While the foregoing discussion primarily concerns microprocessors or multi-core processors that execute software, many of the aforementioned features and applications are implemented by one or more integrated circuits, such as application-specific integrated circuits (ASICs) or field-programmable gate arrays (FPGAs). In some embodiments, such integrated circuits execute instructions stored within the circuit itself. Furthermore, some embodiments execute software stored in programmable logic devices (PLDs), ROM, or RAM devices.

[0116] In this specification and any claim of this application, the terms "computer," "server," "processor," and "memory" refer to electronic or other technical devices. These terms do not include individuals or groups. For the purposes of this specification, the term "display" or "show" means display on an electronic device. In this specification and any claim of this application, the terms "computer-readable medium," "computer-readable media," and "machine-readable medium" are fully defined as tangible, physical objects that store information and can be read by a computer. These terms do not include any wireless signals, wired download signals, or other temporary signals.

[0117] Although this disclosure has been described in conjunction with numerous specific details, those skilled in the art will recognize that this disclosure may take other specific forms without departing from its spirit. Furthermore, many figures (including...) Figure 14 and Figure 17 The diagrams provided are conceptual illustrations of the process. The specific operations of these processes may not be performed in the order shown and described. Specific operations may not be performed in a continuous sequence; different specific operations may be performed in different embodiments. Furthermore, the process may be implemented through multiple sub-processes or as part of a larger macro-process. Therefore, those skilled in the art should understand that this disclosure should not be limited to the foregoing exemplary details but should be defined by the appended claims.

[0118] Supplementary Explanation The topics described herein sometimes demonstrate different components contained within or connected to other different components. It should be understood that such architectures are merely examples, and many other architectures can actually be implemented to achieve the same functionality. Conceptually, any arrangement of components that achieves the same function is considered "associated" in order to achieve the desired functionality. Therefore, any two components combined in this document to achieve a specific function can be considered "associated" with each other to achieve the desired functionality, regardless of the architecture or intermediate components. Similarly, any two such associated components can be considered "operationally connected" or "operationally coupled" to achieve the desired functionality, and any two components that can be suchly associated can be considered "operationally coupled" to achieve the desired functionality. Specific examples of operational coupling include, but are not limited to, physically matable and / or physically interactive components and / or wirelessly interactive and / or logically interactive components.

[0119] Furthermore, regarding the use of almost all plural and / or singular terms in this document, those skilled in the art can appropriately convert plural to singular and / or singular to plural depending on the context and / or application. For clarity, various singular / plural permutations may be explicitly listed herein.

[0120] Furthermore, those skilled in the art will understand that the terms used herein, particularly in appended claims, such as the body portion of appended claims, are generally intended as “open-ended” terms. For example, “comprising” should be interpreted as “including but not limited to,” “having” should be interpreted as “having at least,” and “including” should be interpreted as “including but not limited to,” etc. Those skilled in the art will also understand that if a claim intends to introduce a particular quantity, that intention will be explicitly stated in the claim; otherwise, there is no such intention. For example, for ease of understanding, the following appended claims may contain the introductory phrases “at least one” and “one or more” to introduce the content of the claim. However, the use of such phrases should not be construed as limiting any particular claim containing that content to containing only one such content, even if the same claim contains the introductory phrases “one or more” or “at least one” and indefinite articles such as “a,” for example, “a” should be interpreted as “at least one” or “one or more”; the same applies to definite articles used to introduce the content of the claim. Furthermore, even if the claims explicitly specify a particular quantity, those skilled in the art will recognize that such a statement should be interpreted as at least the stated quantity. For example, stating only "two items" without other modifiers indicates at least two items, or two or more items. Additionally, when using conventions such as "at least one of A, B, and C," such structures are generally intended for those skilled in the art to understand the meaning of the convention. For example, "a system having at least one of A, B, and C" includes, but is not limited to, systems with only A, only B, only C, A and B, A and C, B and C, and systems where A, B, and C are present simultaneously. Similarly, when using conventions such as "at least one of A, B, or C," such structures are generally intended for those skilled in the art to understand the meaning of the convention. For example, "a system having at least one of A, B, or C" includes, but is not limited to, systems with only A, only B, only C, A and B, A and C, B and C, and systems where A, B, and C are present simultaneously. Those skilled in the art will also understand that almost all disjunctive terms and / or phrases appearing in the specification, claims, or drawings, when presenting two or more alternative terms, should be understood to include one, any, or both terms. For example, the phrase “A or B” should be understood as including the possibility of “A” or “B” or “A and B”.

[0121] As can be seen from the foregoing, various embodiments of this disclosure have been described herein for illustrative purposes, and various modifications may be made without departing from the scope and spirit of this disclosure. Therefore, the various embodiments disclosed herein are not intended to be limiting, and the true scope and spirit are indicated by the following claims.

Claims

1. A video encoding / decoding method, comprising: Receive data of the pixels of the current block to be encoded or decoded into the current image of the video; Divide the current block into first and second geometric partitioning modes (GPM); Using template region samples adjacent to the current block, the mixed weights are derived for this partition through regression. Select a set of candidate pairs from the candidate pair list; The selected candidate pairings are applied to the first and second GPM partitions, and the derived mixed weights are used to generate a predictor for the current block. as well as Use the generated predictor to encode or decode the current block.

2. The video encoding / decoding method of claim 1, wherein the candidate pairing list includes different candidate pairs from inter-frame merge candidates and affine merge candidates.

3. The video encoding and decoding method as described in claim 1, wherein the candidate pairing list includes different candidate pairs from inter-frame merge candidates, affine merge candidates, intra-frame merge candidates, and intra-block copy (IBC) merge candidates.

4. The video encoding / decoding method as described in claim 1, wherein the candidate pairing list is reordered according to template cost, and a corresponding merge index is assigned to each candidate pairing in the list.

5. The video encoding / decoding method of claim 1, wherein selecting a set of candidate pairs includes signaling a merge index to identify the position of the set of candidate pairs in the candidate pair list.

6. The video encoding / decoding method of claim 5, wherein the merge index is signaled when the signaling notification flag indicates that the blend weights are derived by regression.

7. The video encoding / decoding method of claim 1, wherein selecting a set of candidate pairs includes selecting from a plurality of candidate pair lists, wherein each set of candidate pairs in a particular list contains at least one candidate for a particular encoding / decoding tool.

8. The video encoding / decoding method of claim 1, wherein selecting a set of candidate pairs includes selecting from only one list of candidate pairs containing any two candidate pairs for two or more encoding tools.

9. The video encoding / decoding method as described in claim 1, wherein only the first GPM partition of the current block has a Local Illumination Compensation (LIC) enable flag and an inherited LIC parameter set, the method further comprising refining the inherited LIC parameter set and applying the refined LIC parameter set to the second GPM partition.

10. The video encoding / decoding method of claim 9, wherein the inherited LIC parameters are refined using the neighboring templates and corresponding reference blocks of the current block for use in the second GPM partition.

11. An electronic device, comprising: The video encoding / decoding circuit is configured to perform the following operations: Receive data of the pixels of the current block to be encoded or decoded into the current image of the video; Divide the current block into first and second geometric partitioning modes (GPM); Using template region samples adjacent to the current block, the mixed weights are derived for this partition through regression. Select a set of candidate pairs from the candidate pair list; The selected candidate pairings are applied to the first and second GPM partitions, and the derived mixed weights are used to generate a predictor for the current block. as well as Use the generated predictor to encode or decode the current block.

12. A video decoding method, comprising: Receive the pixel data of the current block to be decoded into the current image of the video; Divide the current block into first and second geometric partitioning modes (GPM); Using template region samples adjacent to the current block, the mixed weights are derived for this partition through regression. Select a set of candidate pairs from the candidate pair list; The selected candidate pairings are applied to the first and second GPM partitions, and the derived mixed weights are used to generate a predictor for the current block. as well as Use the generated predictor to reconstruct the current block.

13. A video coding method, comprising: Receive data of the pixels of the current block to be encoded into the current image of the video; Divide the current block into first and second geometric partitioning modes (GPM); Using template region samples adjacent to the current block, the mixed weights are derived for this partition through regression. Select a set of candidate pairs from the candidate pair list; The selected candidate pairings are applied to the first and second GPM partitions, and the derived mixed weights are used to generate a predictor for the current block. as well as Use the generated predictor to encode the current block.

Images