Method and apparatus for hybrid prediction in video coding system

Abstract

The invention relates to a shared candidate list of GPM and SGPM. According to the method, a shared intra candidate list is determined. If GPM is selected, the current block is divided into two GPM regions according to the first target partition, a first hypothesis for GPM intra prediction is derived from the shared intra candidate list, and a hybrid predictor is determined using first information including the first hypothesis for GPM intra prediction. If SGPM is selected, the current block is divided into two SGPM regions according to a second target partition, one or more second hypotheses of intra prediction are derived from the shared intra candidate list, and a hybrid predictor is determined using second information including the one or more second hypotheses of intra prediction. And encoding or decoding the current block by using prediction data including the hybrid predictor.

Description

[0001] Cross-references

[0002] This application is a non-provisional application of and claims priority to U.S. Provisional Patent Application No. 63 / 369,092, filed on July 22, 2022. The entire contents of this U.S. Provisional Patent Application are incorporated herein by reference. Technical Field

[0003] The present invention relates to a video coding system. In particular, the present invention relates to a hybrid predictor using an efficient shared candidate list for Geometric Partitioning Mode (GPM) intra prediction and SGPM (Spatial GPM). Background Art

[0004] Versatile Video Coding (VVC) is the latest international video coding standard jointly developed by the Joint Video Experts Group (JVET) of the ITU-T Video Coding Experts Group (VCEG) and the ISO / IEC Moving Picture Experts Group (MPEG). The standard has been published as an ISO standard: ISO / IEC 23090-3:2021, Information technology - Coded representation of immersive media - Part 3: Versatile video coding, published in February 2021. VVC is based on its predecessor HEVC (High Efficiency Video Coding), by adding more codec tools to improve codec efficiency, and can also handle various types of video sources, including 3D video signals.

[0005] Figure 1A An exemplary adaptive inter / intra video coding system including a loop process is illustrated. For intra prediction 110, prediction data is derived from previously encoded video data in the current picture. For inter prediction 112, motion estimation (ME) is performed on the encoder side and motion compensation (MC) is performed based on the results of ME to provide prediction data derived from other pictures and motion data. Switch 114 selects intra prediction 110 or inter prediction 112 and the selected prediction data is provided to adder 116 to form a prediction error, also called a residual. The prediction error is then processed by transform (T) 118 and subsequent quantization (Q) 120. The transformed and quantized residual is then encoded by entropy encoder 122 to be included in a video bitstream corresponding to the compressed video data. The bitstream associated with the transform coefficients is then packaged with auxiliary information (e.g., motion and coding modes associated with intra prediction and inter prediction) and other information (e.g., parameters associated with loop filters applied to underlying image areas). Auxiliary information associated with intra prediction 110, inter prediction 112, and in-loop filter 130 is provided to entropy encoder 122, such as Figure 1AAs shown. When using inter-frame prediction mode, one or more reference pictures must also be reconstructed at the encoder end. Therefore, the transformed and quantized residual is processed by inverse quantization (IQ) 124 and inverse transform (IT) 126 to recover the residual. The residual is then added back to the prediction data 136 at reconstruction (REC) 128 to reconstruct the video data. The reconstructed video data can be stored in a reference picture buffer 134 and used to predict other frames.

[0006] like Figure 1A As shown, the input video data undergoes a series of processes in the encoding system. Due to the series of processes, the reconstructed video data from REC128 may be subject to various impairments. Therefore, the loop filter 130 is often applied to the reconstructed video data before it is stored in the reference picture buffer 134 to improve the video quality. For example, a deblocking filter (DF), sample adaptive offset (SAO), and adaptive loop filter (ALF) may be used. It may be necessary to merge the loop filter information into the bitstream so that the decoder can correctly recover the required information. Therefore, the loop filter information is also provided to the entropy encoder 122 to be merged into the bitstream. Figure 1A In , a loop filter 130 is applied to the reconstructed video before the reconstructed samples are stored in a reference picture buffer 134 . Figure 1A The system in is intended to illustrate an exemplary structure of a typical video encoder. It may correspond to the High Efficiency Video Coding (HEVC) system, VP8, VP9, ​​H.264, or VVC.

[0007] like Figure 1B As shown, the decoder can use similar or identical functional blocks as the encoder, except for transform 118 and quantization 120, because the decoder only needs inverse quantization 124 and inverse transform 126. Instead of entropy encoder 122, the decoder uses entropy decoder 140 to decode the video bitstream into quantized transform coefficients and required encoding information (such as ILPF information, intra-frame prediction information, and inter-frame prediction information). Intra-frame prediction 150 on the decoder side does not need to perform mode search. Instead, the decoder only needs to generate intra-frame prediction based on the intra-frame prediction information received from entropy decoder 140. In addition, for inter-frame prediction, the decoder only needs to perform motion compensation (MC 152) based on the inter-frame prediction information received from entropy decoder 140 without motion estimation.

[0008] According to VVC, similar to HEVC, the input picture is divided into non-overlapping square block areas called CTUs (Coding Tree Units). Each CTU can be divided into one or more coding units (CUs) of smaller size. The generated CU partitions can be square or rectangular. In addition, VVC divides CTUs into prediction units (PUs) as units for applying prediction processes such as inter-frame prediction, intra-frame prediction, etc.

[0009] The VVC standard incorporates various new codec tools to further improve the coding efficiency beyond the HEVC standard. Among the various new codec tools, some codec tools related to the present invention are summarized as follows.

[0010] Inter-frame prediction overview

[0011] According to Section 3.4 of JVET-T2002, (Jianle Chen et al., “Algorithmic Description of Generic Video Coding and Test Model 11 (VTM11)”, Joint Video Experts Group (JVET) of ITU-T SG 16WP 3 and ISO / IEC JTC 1 / SC 29, 20th Meeting, Teleconference, October 7–16, 2020, Archives: JVET-T2002), for each inter-predicted CU, the motion parameters contain the motion vector, reference picture index, and reference picture list usage flag, as well as additional information required by the new coding features of VVC for inter-predicted sample generation. The motion parameters can be signaled in an explicit or implicit manner. When a CU is coded in skip mode, the CU is associated with one PU and has no significant residual coefficients, no coded motion vector increments or reference picture indexes. To specify the merge mode, the motion parameters of the current CU are obtained from neighboring CUs, including spatial and temporal candidates, and the additional scheduling introduced in VVC. The merge mode can be applied to any inter-predicted CU, not just skip mode. An alternative to merge mode is explicit transmission of motion parameters, where the motion vectors, the corresponding reference picture index for each reference picture list and the reference picture list usage flag, along with other required information, are explicitly sent per CU.

[0012] In addition to the inter-frame coding capabilities in HEVC, VVC also includes many new and improved inter-frame prediction codec tools, listed below:

[0013] Extended merge prediction

[0014] Merge mode with MVD(MMVD)

[0015] Symmetric MVD (SMVD) signalling

[0016] Affine motion compensated prediction

[0017] Subblock-based temporal motion vector prediction (SbTMVP)

[0018] Adaptive motion vector resolution(AMVR)

[0019] Motion field storage: 1 / 16 luminance sample MV storage and 8x8 motion field compression

[0020] Bi-prediction with CU-level weight(BCW)

[0021] Bi-directional optical flow(BDOF)

[0022] Decoder side motion vector refinement (DMVR)

[0023] Geometric partitioning mode(GPM)

[0024] Combined inter and intra prediction(CIIP)

[0025] The following description provides details of those inter prediction methods specified in VVC.

[0026] Extended Merger Forecast

[0027] In VVC, the merge candidate list is constructed by including the following five categories of candidates in sequence:

[0028] Spatial MVP from spatially neighboring CUs

[0029] Time MVP from collocated CU

[0030] History-based MVP from FIFO table

[0031] Average MVP in pairs

[0032] Zero MV.

[0033] The size of the merge list is signaled in the sequence parameter set (SPS) header and the maximum allowed size of the merge list is 6. For each CU encoded in merge mode, the index of the best merge candidate is encoded using truncated unary binarization (TU). The first binary string of the merge index (hereafter referred to as bin) is context encoded and bypass encoding is used for the remaining bins.

[0034] This section provides the derivation process of merge candidates for each category. As done in HEVC, VVC also supports parallel derivation of merge candidate lists (or merge candidate lists) for all CUs in a certain size region.

[0035] Spatial candidate derivation

[0036] The derivation of spatial merge candidates in VVC is the same as in HEVC, except that the positions of the first two merge candidates are swapped. Figure 2 Select up to four merge candidates (B) of the current CU 210 from the candidates at the positions shown. 0 , A 0 , B 1 and A 1 ). The order of export is B 0 , A 0 , B 1 , A 1 and B 2 . Location B 2 Only in position B 0 , A 0 , B 1 , A 1 One or more neighboring CUs are not available (for example, belong to another slice or tile) or are intra-coded. Add position A 1 After selecting the candidates, the remaining candidates are added for redundancy check to ensure that the candidates with the same motion information are excluded from the list, thereby improving the coding efficiency. In order to reduce the computational complexity, not all possible candidate pairs are considered in the mentioned redundancy check. Instead, only Figure 3 The pairs are linked by arrows in , and candidates are added to the list only if the corresponding candidates used for redundancy checking do not have the same motion information.

[0037] Time candidate derivation

[0038] In this step, only one candidate is added to the list. Specifically, in the derivation of the temporal merge candidate for the current CU 410, based on the candidate belonging to Figure 4The collocated CU 420 of the collocated reference picture shown derives the scaled motion vector. The reference picture list and the reference index of the collocated CU used to derive the collocated CU are explicitly signaled in the slice header. Figure 4 As shown by the dotted line in , a scaled motion vector 430 of the temporal merge candidate is obtained, which is scaled from the motion vector 440 of the collocated CU using the POC (Picture Order Count) distances tb and td, where tb is defined as the POC difference between the reference picture of the current picture and the current picture, and td is defined as the POC difference between the reference picture of the collocated picture and the collocated picture. The reference picture index of the temporal merge candidate is set equal to zero.

[0039] The position of the temporal candidate is in candidate C 0 and C 1 Choose between Figure 5 If position C 0 If the CU is not available, is intra-coded, or is outside the current CTU row, then position C is used. 1 Otherwise, position C 0 Used to derive temporal merge candidates.

[0040] History-based Merge Candidate Derivation

[0041] History-based MVP (HMVP) merge candidates are added to the merge list after spatial MVP and TMVP. In this method, the motion information of the previously coded block is stored in a table and used as the MVP of the current CU. A table with multiple HMVP candidates is maintained during the encoding / decoding process. The table will be reset (cleared) when a new CTU row is encountered. Whenever there is a non-sub-block inter-coded CU, the associated motion information will be added to the last entry of the table as a new HMVP candidate.

[0042] The HMVP table size S is set to 6, which indicates that a maximum of 5 history-based MVP (HMVP) candidates can be added to the table. When inserting a new motion candidate into the table, a constrained first-in, first-out (FIFO) rule is used, where a redundancy check is first applied to find if the same HMVP exists in the table. If found, the same HMVP is removed from the table and all subsequent HMVP candidates are moved forward, and the same HMVP is inserted into the last entry of the table.

[0043] HMVP candidates can be used in the merge candidate list construction process. The latest HMVP candidates in the table are checked in turn and inserted into the candidate list after the TMVP candidate. Redundancy check is applied to HMVP candidates to spatial or temporal merge candidates.

[0044] In order to reduce the number of redundant check operations, the following simplifications are introduced:

[0045] The last two entries in the table are for A 1 and B 1 Spatial candidates are checked for redundancy.

[0046] Once the total number of available merge candidates reaches the maximum number of allowed merge candidates - 1, the merge candidate list construction process of HMVP is terminated.

[0047] Pairwise average merge candidate derivation

[0048] Paired average candidates are generated by averaging predefined candidate pairs in the existing merge candidate list using the first two merge candidates. The first merge candidate is defined as p0Cand, and the second merge candidate can be defined as p1Cand, respectively. The average motion vector is calculated for each reference list separately according to the availability of motion vectors of p0Cand and p1Cand. If two motion vectors are available in one list, they are averaged and their reference pictures are set to one of p0C and p0C even if the two motion vectors point to different reference pictures; if only one motion vector is available, one is used directly; if no motion vector is available, this list is kept invalid. In addition, if the half-pixel interpolation filter index of p0Cand and p1Cand is different, it is set to 0.

[0049] When the merge list is not full after adding pairwise average merge candidates, zero MVPs are inserted at the end until the maximum number of merge candidates is encountered.

[0050] Merge Mode with MVD (MMVD)

[0051] In addition to the merge mode in which the implicitly derived motion information is directly used for prediction sample generation of the current CU, a merge mode with motion vector difference (MMVD) is introduced in VVC. The MMVD flag is sent immediately after the regular merge flag is sent to specify whether the MMVD mode is used for the CU.

[0052] In MMVD, after a merge candidate (referred to as a basic merge candidate in the present invention) is selected, it is further refined by signaling MVD information. Further information includes a merge candidate identifier, an index for specifying the magnitude of motion, and an index for indicating the direction of motion. In MMVD mode, one of the first two candidates in the merge list is selected as the MV basis. An MMVD candidate identifier is issued to specify which one to use between the first and second merge candidates.

[0053] The distance index specifies the motion magnitude information and indicates a predefined offset from the start point (612 and 622) of the L0 reference block 610 and the L1 reference block 620. Figure 6 As shown, the offset is added to the horizontal component or vertical component of the starting MV, where small circles of different styles correspond to different offsets from the center. The relationship between the distance index and the predefined offset is specified in Table 1.

[0054] Table 1 – Relationship between distance index and predefined offsets

[0055]

[0056] The direction index indicates the direction of the MVD relative to the starting point. The direction index can indicate four directions as shown in Table 2. It should be noted that the meaning of the MVD symbol can change according to the information of the starting MV. When the starting MV is a non-predicted MV or a bi-predicted MV, both lists point to the same side of the current picture (that is, the POCs of both references are greater than the POC of the current picture, or are less than the POC of the current picture), and the symbol in Table 2 specifies the sign of the MV offset added to the starting MV. When the starting MV is a bi-predicted MV, the two MVs point to different sides of the current picture (that is, the POC of one reference is greater than the POC of the current picture, and the POC of the other reference is less than the POC of the current picture), and the difference in POC in list 0 (which can be abbreviated as list 0 or L0) is greater than the POC in list 1 (which can be abbreviated as list 1 or L1), the symbol in Table 2 specifies that the sign of the MV offset of the MV component of list 0 added to the starting MV has an opposite value to the sign of the MV in list 1. Otherwise, if the difference in POC in List 1 is greater than List 0, then the sign in Table 2 specifies that the sign of the MV offset added to the List 1 MV component of the starting MV has the opposite value to that of the List 0 MV.

[0057] The MVD is scaled by the difference in the POC in each direction. If the difference in the POC is the same in both lists, no scaling is required. Otherwise, if the difference in the POC in list 0 is greater than that in list 1, the MVD of list 1 is scaled by defining the POC difference of L0 as td and the POC difference of L1 as tb, as Figure 5If the POC difference of L1 is greater than L0, the MVD of list 0 is scaled in the same way. If the starting MV is unidirectionally predicted, the MVD is added to the available MVs.

[0058] Table 2 – MV offset symbols specified by direction index

[0059]

[0060] Affine Motion Compensated Prediction

[0061] In HEVC, only the translational motion model is applied to motion compensation prediction (MCP). In the real world, there are many kinds of motion, such as zooming in / out, rotation, perspective motion, and other irregular motions. In VVC, block-based affine transformation motion compensation prediction is applied. As shown in the figure. In FIG. 7A-B, the affine motion domain of block 710 is composed of Fig. 7A The motion information of the two control points (4 parameters) in Figure 7B The three control point motion vectors (6 parameters) are described in .

[0062] For the 4-parameter affine motion model, the motion vector at the sample position (x, y) in the block is derived as:

[0063]

[0064] For the 6-parameter affine motion model, the motion vector at a sample position (x, y) in the block is derived as:

[0065]

[0066] Where (mv 0x ,mv 0y ) is the motion vector of the upper left control point, (mv 1x ,mv 1y ) is the motion vector of the upper right corner control point, (mv 2x ,mv 2y ) is the motion vector of the bottom-left corner control point.

[0067] To simplify motion compensated prediction, block-based affine transform prediction is applied. To derive the motion vector for each 4×4 luminance sub-block, the motion vector of the center sample of each sub-block is Figure 8 As shown, it is calculated according to the above equation and rounded to 1 / 16 fractional precision. Then, a motion compensated interpolation filter is applied to generate a prediction for each subblock with a derived motion vector. The subblock size of the chroma component is also set to 4×4. The MV of the 4×4 chroma subblock is calculated as the average of the MVs of the upper left and lower right luminance subblocks in the collocated 8x8 luminance area.

[0068] For translational motion inter prediction, there are also two affine motion inter prediction modes: affine merge mode and affine AMVP mode.

[0069] Affine merge prediction (AF_MERGE)

[0070] AF_MERGE mode can be applied to CUs with width and height greater than or equal to 8. In this mode, the CPMV (control point MV) of the current CU is generated based on the motion information of the spatial neighboring CUs. There can be up to five CPMVP (CPMV prediction) candidates, and an index is signaled to indicate which one to use for the current CU. The following three types of CPVM candidates are used to form the affine merge candidate list:

[0071] Inherited affine merge candidates inferred from the CPMV of neighboring CUs

[0072] Construct affine merge candidate CPMVP derived from the translation MV of the adjacent CU

[0073] Zero MV

[0074] In VVC, there are at most two inherited affine candidates, which come from the affine motion models of neighboring blocks, one from the left neighboring CU and one from the upper neighboring CU. Figure 2 For the left predictor, the scanning order is A 0 ->A 1 , for the upper predictor, the scan order is B0->B1->B2. Only the first inherited candidate of each side is selected. No pruning check is performed between two inherited candidates. When the adjacent affine CU is identified, its control point motion vectors are used to derive the CPMVP candidate in the affine merge list of the current CU. Fig. 9 As shown, if the lower left neighboring block A of the current block 910 is encoded in affine mode, the motion vectors v of the upper left corner, the upper right corner and the lower left corner of the containing block A of the CU 920 are obtained. 2 、v 3 and v 4 When block A is encoded using a 4-parameter affine model, according to v 2 、v 3 Calculate the two CPMVs of the current CU (i.e., v 0 and v 1 ). In the case where block A is encoded using a 6-parameter affine model, according to v 2 、v 3 、v 4 Calculate the three CPMVs of the current CU.

[0075] Constructing affine candidates means constructing candidates by combining the adjacent translation motion information of each control point. Fig.10 As shown, the motion information of the control point is derived from the specified spatial neighbors and temporal neighbors of the current block 1010. CPMV k (k=1, 2, 3, 4) represents the kth control point. 1 , check B2->B3->A2 blocks and use the MV of the first available block. For CPMV 2 , check B1->B0 block, for CPMV 3 , check the A1->A0 block. If TMVP is available, use it as CPMV 4 .

[0076] After obtaining the MVs of the four control points, an affine merge candidate is constructed based on the motion information. The following combinations of control point MVs are used to construct in order:

[0077] {CPMV 1 ,CPMV 2 ,CPMV 3},{CPMV 1 ,CPMV 2 ,CPMV 4},{CPMV 1 ,CPMV 3 ,CPMV 4},{CPMV 2 ,CPMV 3 ,CPMV 4},{CPMV 1 ,CPMV 2},{CPMV 1 ,CPMV 3}

[0078] The combination of 3 CPMVs constructs a 6-parameter affine merge candidate and the combination of 2 CPMVs constructs a 4-parameter affine merge candidate. To avoid the motion scaling process, the relevant combination of control point MVs is discarded if the reference indexes of the control points are different.

[0079] After checking inherited affine merge candidates and constructed affine merge candidates, if the list is still not full, a zero MV is inserted at the end of the list.

[0080] Affine AMVP prediction

[0081] Affine AMVP mode can be applied to CUs with width and height both greater than or equal to 16. A CU-level affine flag is signaled in the bitstream to indicate whether to use affine AMVP mode, and then another flag is signaled to indicate whether to use 4-parameter affine or 6-parameter affine. In this mode, the difference between the current CU's CPMV and its predictor CPMVP is signaled in the bitstream. The affine AVMP candidate list size is 2, and is generated from the following four CPVM candidates in sequence:

[0082] Inherited affine AMVP candidates inferred from the CPMV of neighboring CUs

[0083] The constructed affine AMVP candidate CPMVP is derived using the translation MV of the adjacent CU

[0084] Translation MV from neighboring CU

[0085] Zero MV

[0086] The order of checking inherited affine AMVP candidates is the same as that of checking inherited affine merge candidates. The only difference is that for AVMP candidates, only affine CUs with the same reference picture as the current block are considered. When inserting inherited affine motion prediction into the candidate list, the cropping process is not applied.

[0087] The constructed AMVP candidates are Fig.10 The specified spatial neighbors are derived as shown in . The same checking order as in the affine merge candidate construction is used. In addition, the reference picture indexes of neighboring blocks are checked. In the checking order, the first block that uses inter coding and has the same reference picture as the current CU is used. When the current CU is coded using the 4-parameter affine mode and mv 0 and mv 1 When all three CPMVs are available, they are added as a candidate to the affine AMVP list. When the current CU is encoded using the 6-parameter affine mode and all three CPMVs are available, they are added as a candidate to the affine AMVP list. Otherwise, the constructed AMVP candidate is set to unavailable.

[0088] If the number of affine AMVP candidates in the list is still less than 2 after inserting the valid inherited affine AMVP candidates and the constructed AMVP candidates, mv is added. 0 ,mv 1 With mv 2 As the translation MV to predict all control point MVs of the current CU when available. Finally, if the affine AMVP list is still not full, fill it with zero MVs.

[0089] Affine motion information storage

[0090] In VVC, the CPMV of an affine CU is stored in a separate buffer. The stored CPMV is only used to generate inherited CPMVP for the most recently encoded CU in affine merge mode and affine AMVP mode. The sub-block MV derived from the CPMV is used for motion compensation, merging of translation MVs / MV derivation of AMVP lists, and deblocking.

[0091] To avoid an extra picture row buffer for CPMV, the affine motion data inherited from the CU of the above CTU is treated differently to inheritance from a normal neighboring CU. If the candidate CU for affine motion data inheritance is in the above CTU row, the bottom left and bottom right sub-block MVs in the row buffer are used for affine MVP derivation instead of the CPMVs. In this way, the CPMVs are only stored in the local buffer. If the candidate CU is 6-parameter affine coded, the affine model degenerates to a 4-parameter model. Fig.11 As shown, along the top CTU boundary, the bottom left and bottom right sub-block motion vectors of the CU are used for affine inheritance of the CU in the bottom CTU. Fig.11 , lines 1110 and 1112 represent the x and y coordinates of the picture with the origin (0, 0) at the upper left corner. Legend 1120 shows the meaning of various motion vectors, where arrow 1122 represents the CPMV for affine inheritance in the local buffer, arrow 1124 represents the sub-block vector for MC / Merge / Skip MVP / Deblocking / TMVP in the local buffer and affine inheritance in the row buffer, and arrow 1126 represents the sub-block vector for MC / Merge / Skip MVP / Deblocking / TMVP.

[0092] Adaptive Motion Vector Resolution (AMVR)

[0093] In HEVC, when use_integer_mv_flag in the slice header is equal to 0, the motion vector difference (MVD) (between the CU's motion vector and the predicted motion vector) is signaled in units of quarter-luminance samples. In VVC, a CU-level adaptive motion vector resolution (AMVR) scheme is introduced. AMVR allows the MVD of a CU to be encoded with different precisions. Depending on the mode of the current CU (normal AMVP mode or affine AVMP mode), the MVD of the current CU can be adaptively selected as follows:

[0094] Normal AMVP mode: quarter brightness samples, half brightness samples, integer brightness samples, or quad brightness samples.

[0095] Affine AMVP mode: quarter-luminance samples, integer-luminance samples, or 1 / 16-luminance samples.

[0096] If the current CU has at least one non-zero MVD component, the CU-level MVD resolution indication is conditionally signaled.If all MVD components (ie, horizontal and vertical MVDs for reference list L0 and reference list L1) are zero, a quarter luma sample MVD resolution is inferred.

[0097] For CUs with at least one non-zero MVD component, a first flag is signaled to indicate whether quarter luma sample MVD precision is used for the CU. If the first flag is 0, no further signaling is required and quarter luma sample MVD precision is used for the current CU. Otherwise, a second flag is issued to indicate that half luma samples or other MVD precision (integer or four luma samples) are used for normal AMVP CUs. In the case of half luma samples, a 6-tap interpolation filter is used for the half luma sample positions instead of the default 8-tap interpolation filter. Otherwise, a third flag is signaled to indicate whether integer luma samples or four luma sample MVD precision is used for normal AMVP CUs. In the case of affine AMVP CUs, the second flag is used to indicate whether integer luma samples or 1 / 16 luma sample MVD precision is used. To ensure that the reconstructed MV has the expected precision (quarter luma samples, half luma samples, integer luma samples, or four luma samples), the motion vector predictor of the CU is rounded to the same MVD precision before being added to the MVD. Motion vector prediction values ​​are rounded towards zero (i.e., negative motion vector prediction values ​​are rounded towards positive infinity, and positive motion vector prediction values ​​are rounded towards negative infinity).

[0098] The encoder uses RD check to determine the motion vector resolution of the current CU. In order to avoid always performing four CU-level RD checks for each MVD resolution, the RD check for MVD precision other than quarter luma samples is only called conditionally in VTM11. For normal AVMP mode, the RD cost of quarter luma sample MVD precision and integer luma sample MV precision is first calculated. Then, the RD cost of integer luma sample MVD precision is compared with the RD cost of quarter luma sample MVD precision to decide whether it is necessary to further check the RD cost of four luma sample MVD precision. When the RD cost of quarter luma sample MVD precision is much smaller than the RD cost of integer luma sample MVD precision, the RD check of four luma sample MVD precision is skipped. Then, if the RD cost of integer luma sample MVD precision is significantly greater than the best RD cost of the previously tested MVD precision, the check of half luma sample MVD precision is skipped. For the affine AMVP mode, if the affine inter mode, affine AMVP mode is not selected after checking the rate-distortion cost of the affine merge / skip mode, merge / skip mode, quarter luma sample MVD accuracy normal AMVP mode, and quarter luma sample MVD accuracy, the 1 / 16 luma sample MV accuracy and 1 pixel MV accuracy affine inter modes are not checked. In addition, the affine parameters obtained in the quarter luma sample MV accuracy affine inter mode are used as the starting search points for the 1 / 16 luma sample and quarter luma sample MV accuracy affine inter modes.

[0099] Combined Inter and Intra Prediction (CIIP)

[0100] In VVC, when a CU is encoded in merge mode, if the CU contains at least 64 luma samples (i.e., the CU width multiplied by the CU height is equal to or greater than 64), and if both the CU width and the CU height are less than 128 luma samples, an additional flag is sent to indicate whether the combined inter / intra prediction (CIIP) mode is applied to the current CU. As its name suggests, CIIP prediction combines the inter prediction signal with the intra prediction signal. CIIP mode P inter

[0101] The inter-frame prediction signal in is derived using the same inter-frame prediction process applied to the normal merge mode; and the intra-frame prediction signal P intra is derived after the conventional intra prediction process with planar mode. Then, the intra and inter prediction signals are combined using weighted averaging, where the weight value wt is calculated based on the top and left neighboring blocks of the current CU 1210 (e.g. Fig.12 The encoding mode of ) is calculated as follows:

[0102] If the top neighbor is available and intra-coded, set isIntraTop to 1, otherwise set isIntraTop to 0;

[0103] If the left neighbor is available and intra-coded, set isIntraLeft to 1, otherwise set isIntraLeft to 0;

[0104] If (isIntraLeft+isIntraTop) is equal to 2, then wt is set to 3;

[0105] Otherwise, if (isIntraLeft+isIntraTop) is equal to 1, wt is set to 2;

[0106] Otherwise, set wt to 1.

[0107] The CIIP forecast is formed as follows:

[0108] P CIIP =((4-wt)*P inter +wt*P intra +2)>>2 (4)

[0109] Geometric Partitioning Mode (GPM)

[0110] In VVC, geometric partition mode (GPM) is supported for inter prediction as described in JVET-W2002 (Adrian Browne et al., Algorithmic Description of Generic Video Coding and Test Model 14 (VTM 14), ITU-T / ISO / IEC Joint Video Exploration Team (JVET), 23rd Meeting, by Teleconference, July 7-16, 2021, Document: Document JVET-M2002). The geometric partition mode is signaled using a CU-level flag as the merge mode. Other merge modes include normal merge mode, MMVD mode, CIIP mode, and sub-block merge mode. For each possible CU size, the geometric partition mode supports a total of 64 partitions, w×h=2 m ×2 n , where m,n∈{3...6} does not include 8x64 and 64x8. The GPM mode can be applied to skip or merge CUs with sizes within the above limits and with at least two normal merge modes.

[0111] When using this mode, the CU is partitioned into two parts by geometrically positioned straight lines at a certain angle. In VVC, GPM uses a total of 20 angles and 4 offset distances, which is reduced from 24 angles in earlier drafts. The location of the partition line is mathematically derived based on the angle and offset parameters of the specific partition. In VVC, there are a total of 64 partitions, such as Fig.13 As shown in the figure, the partitions are grouped by angle and the dotted lines represent redundant partitions. Each part of the geometric partition in the CU uses its own motion for inter-frame prediction; each partition only allows single prediction, that is, each part has a motion vector and a reference index. Fig.13 , each line corresponds to the boundary of a partition. The partitions are grouped according to their angles. For example, partition group 1310 consists of three vertical GPM partitions (i.e., 90°). Partition group 1320 consists of four inclined GPM partitions with a small angle to the vertical direction. Moreover, partition group 1330 consists of three vertical GPM partitions similar to those of group 1310, but with the opposite direction (i.e., 270°). A uni-prediction motion constraint is applied to ensure that each CU requires only two motion compensated predictions, the same as traditional bi-directional prediction. The uni-prediction motion for each partition is derived using the process described later.

[0112] If geometry partition mode is used for the current CU, a geometry partition index indicating the selected partition mode (angle and offset) of the geometry partition is further signaled as well as two merge indices (one for each partition). The number of maximum GPM candidate sizes is explicitly indicated in the SPS (Sequence Parameter Set) and the syntax for GPM merge indices is binarized. After predicting each part of the geometry partition, the sample values ​​along the edges of the geometry partition are adjusted using a blending process with adaptive weights using the process described later. This is the prediction signal for the entire CU and, as with other prediction modes, the transform and quantization process will be applied to the entire CU. Finally, the motion field of the CU predicted using the geometry partition mode is stored using the procedure described later.

[0113] One-way prediction candidate list construction

[0114] The uni-prediction candidate list is derived directly from the merge candidate list constructed according to the extended merge prediction process. Denote n as the index of the uni-prediction motion in the geometric uni-prediction candidate list. The LX motion vector of the n-th extended merge candidate (X=0 or 1, i.e. LX=L0 or L1) is taken as the n-th uni-prediction motion vector, where X is equal to the parity geometric partition mode of n. These motion vectors are Fig.14 If the corresponding LX motion vector of the n-th extended merge candidate does not exist, the L(1-X) motion vector of the same candidate is used as the unidirectional prediction motion vector of the alternative geometric partition mode.

[0115] Blending along geometric partition edges

[0116] After predicting each part of the geometric partition using its own motion, blending is applied to the two prediction signals to derive samples around the edges of the geometric partition. The blending weight for each position of the CU is derived based on the distance between each position and the partition edge.

[0117] Two integer mixing matrices (W 0 and W 1 ) is used in the GPM mixing process. The weights in the GPM mixing matrix contain values ​​in the range [0,8] and are derived based on the displacement from the sample location to the GPM partition boundary 1540, as Fig.15 shown.

[0118] Specifically, the weights are given by a discrete ramp function with a displacement and two thresholds, as Fig.16 As shown, the two endpoints of the slope (i.e., -τ and τ) correspond to Fig.15 Lines 1542 and 1544.

[0119] Here, the threshold τ defines the width of the GPM mixing area and is chosen as a fixed value in VVC. In other words, as described in JVET-Z0137 (Han Gao et al., "Non-EE2: Adaptive mixing of GPM", ITU-T SG 16WP 3 and ISO / IEC JTC1 / SC 29 Joint Video Experts Group (JVET), 26th Meeting, by Teleconference, April 20-29, 2022, JVET-Z0137), the mixing strength or mixing area width θ is fixed for all different contents.

[0120] The weight values ​​in the blending mask can be adjusted by using a ramp function.

[0121]

[0122] function) gives: (4)

[0124] In the current ECM (VVC) design, with a fixed θ = 2 pixels, the ramp function can be quantized as:

[0125] ω m，n =Clip3(0,8,(d(m,n)+32+4)>>3) (5)

[0126] The distance from the position (x, y) to the partition boundary is:

[0127]

[0128] where i,j are the angle and offset indices of the geometry partition, which depend on the signaled geometry partition index. x,j and ρ y,j The sign of depends on the angle index i.

[0129] Fig.17 An example of GPM mixing according to ECM 4.0 is shown (Muhammed Coban et al., "Algorithmic Description of Enhanced Compression Model 4 (ECM 4)", Joint Video Experts Group (JVET) 3 of ITU-T SG 16WP and ISO / IEC JTC 1 / SC29, 26th Meeting, by Teleconference, April 20-29, 2022, JVET-Y2025). Fig.17 In , the size of the mixed region on each side of the partition boundary is represented by θ. The weights of each part of the geometric partition are calculated as follows:

[0130] wIdxL(x,y)=partIdx? 32+d(x,y):32-d(x,y) (10)

[0132]

[0133] w 1 (x,y)=1-w 0 (x,y) (12)

[0134] partIdx depends on the angle index i. Fig.15 The weight w is shown in 0 An example of where the angle is indicated for the GPM index i 1510 and offset ρ i 1520, and point 1530 corresponds to the center of the block. Line 1540 corresponds to the GPM partition boundary.

[0135] Motion field storage in geometric partitioning mode

[0136] Mv1 from the first part of the geometric partition, Mv2 from the second part of the geometric partition, and the combined MV of Mv1 and Mv2 are stored in the motion field of the CU encoded in the geometric partition mode.

[0137] The type of stored motion vector for each individual position in the motion field is determined as:

[0138] sType=abs(motionIdx)<32?2:(motionIdx≤0?(1-partIdx):partIdx) (13)

[0139] where motionIdx is equal to d(4x+2,4y+2), which is recalculated according to equation (6). partIdx depends on the angle index i.

[0140] If sType is equal to 0 or 1, then Mv0 or Mv1 is stored in the corresponding motion field, otherwise if sType is equal to 2, the combined MV from Mv0 and Mv2 is stored. The combined Mv is generated using the following process:

[0141] If Mv1 and Mv2 are from different reference picture lists (one from L0 and the other from L1), then Mv1 and Mv2 are simply combined to form a bi-directional prediction motion vector.

[0142] Otherwise, if Mv1 and Mv2 are from the same list, only the unidirectional predicted motion Mv2 is stored.

[0143] Intra-mode coding with 67 intra-prediction modes

[0144] To capture arbitrary edge directions present in natural videos, the number of directional intra modes in VVC is extended from the 33 used in HEVC to 65. The new directional modes not present in HEVC are depicted as dashed arrows in Figure 7, while the planar and DC modes remain unchanged. These denser directional intra prediction modes are applicable to all block sizes and for luma and chroma intra prediction.

[0145] In VVC, for non-square blocks, several traditional angular intra prediction modes are adaptively replaced with wide-angle intra prediction modes.

[0146] In HEVC, each intra-coded block has a square shape and the length of each of its sides is a power of 2. Therefore, no division operation is required to generate intra predictors using DC mode. In VVC, blocks can have a rectangular shape, which in general requires the use of division operations for each block. To avoid division operations for DC prediction, only the longer sides are used to calculate the average for non-square blocks.

[0147] In order to keep the complexity of the most probable mode (MPM) list generation low, an intra mode coding method with 6 MPMs is used by considering two available adjacent intra modes. The following three aspects are considered to construct the MPM list:

[0148] Default intra mode

[0149] Adjacent Intra Mode

[0150] Intra mode for export.

[0151] A unified 6-MPM list is used for intra blocks, regardless of whether the MRL and ISP codecs are applied. The MPM list is constructed based on the intra modes of the left and upper neighboring blocks. Assuming that the mode on the left is denoted as Left and the mode of the upper block is denoted as Above, the unified MPM list is constructed as follows:

[0152] When a neighboring block is not available, its intra mode is set to planar by default.

[0153] If both Left and Above modes are non-angle modes:

[0154] MPM list > {plane, DC, V, H, V-4, V+4}

[0155] If one of Left and Above modes is angular and the other is non-angular:

[0156] Set the mode Max to the larger mode between Left and Above

[0157] – MPM list → {plane, Max, DC, Max-1, Max+1, Max-2}

[0158] If Left and Above are both angled and they are different:

[0159] Set the mode Max to the larger mode between Left and Above

[0160] If the difference between the modes Left and Above is in the range 2 to 62, inclusive

[0161] MPM list → {Plane,Left,Above,DC,Max-1,Max+1}

[0162] otherwise

[0163] MPM list → {Plane,Left,Above,DC,Max-2,Max+2}

[0164] If Left and Above are both angled and they are the same:

[0165] –MPM list → {plane,Left,Left-1,Left+1,DC,Left-2}

[0166] In addition, the first bin of the MPM index codeword is CABAC context coded. A total of three contexts are used, corresponding to whether the current intra block is MRL enabled, ISP enabled, or a normal intra block.

[0167] In the 6MPM list generation process, pruning is used to remove repeated patterns so that only unique patterns can be included in the MPM list. For entropy coding of the 61 non-MPM patterns, truncated binary code (TBC) is used.

[0168] Wide-angle intra prediction for non-square blocks

[0169] Conventional angular intra prediction directions are defined as clockwise from 45 degrees to -135 degrees. In VVC, several traditional angular intra prediction modes are adaptively replaced with wide-angle intra prediction modes for non-square blocks. The replaced modes are signaled using the original mode index, which is remapped to the index of the wide-angle mode after parsing. The total number of intra prediction modes remains unchanged, i.e. 67, and the intra mode encoding method remains unchanged.

[0170] To support these predictions, Fig.19A and Fig.19B As shown, a top reference with a length of 2W+1 and a left reference with a length of 2H+1 are defined.

[0171] The number of alternative modes in wide-angle direction mode depends on the aspect ratio of the block. The alternative intra prediction modes are shown in Table 3.

[0172] Table 3 – Intra prediction modes replaced by Wide mode

[0173]

[0174]

[0175] In VVC, 4:2:2, 4:4:4, and 4:2:0 chroma formats are supported. The chroma derived mode (DM) derivation table for the 4:2:2 chroma format was originally ported from HEVC, expanding the number of entries from 35 to 67 to keep pace with the expansion of intra prediction modes. Since the HEVC specification does not support prediction angles below -135° and above 45°, the luma intra prediction modes are mapped from 2 to 5 to 2. Therefore, the chroma DM derivation table for the 4:2:2 chroma format is updated by replacing some values ​​of the mapping table entries to more accurately convert the prediction angles of the chroma blocks.

[0176] Decoder-side Intra Mode Derivation (DIMD)

[0177] When DIMD is applied, two intra modes are derived from the reconstructed neighboring samples and these two predictions are combined with the planar mode prediction with weights derived from the gradients. DIMD mode is used as an alternative prediction mode and is always checked in high complexity RDO mode.

[0178] To implicitly derive the intra prediction mode of a block, texture gradient analysis is performed on both the encoder and decoder side. This process starts with an empty gradient histogram (HoG) with 65 entries, corresponding to 65 angular modes. The magnitudes of these entries are determined during texture gradient analysis.

[0179] In the first step, DIMD selects a template of T=3 columns and rows from the left and top of the current block, respectively. This area is used as a reference for gradient-based intra prediction mode derivation.

[0180] In the second step, horizontal and vertical Sobel filters are applied to all 3×3 window locations, centered on the pixels in the template midline. At each window location, the Sobel filter computes the pure horizontal and vertical intensities G x and G y . The texture angle of the window is then calculated as:

[0181] angle=arctan(G x / G y ), (14)

[0182] can be converted to one of 65 angular intra prediction modes. Once the intra prediction mode index of the current window is derived as idx, the magnitude of its entry in HoG[idx] is updated by adding:

[0183] ampl=|G x |+|G y | (15)

[0184] Figures 20A-C show examples of the HoG computed after applying the above operations to all pixel locations in the template. Fig. 20A An example of a template 2020 selected for a current block 2010 is illustrated. The template 2020 includes T rows above the current block and T columns to the left of the current block. For intra prediction of the current block, an area 2030 above and to the left of the current block corresponds to a reconstructed area, while an area 2040 below and to the right of the block corresponds to an unavailable area. Fig. 20B An example of T=3 is illustrated and the HoG is calculated for a pixel 2060 in the middle row and a pixel 2062 in the middle column. For example, for pixel 2052, a 3x3 window 2050 is used. Fig. 20C An example of the amplitude (Ampl) calculated based on equation (2) for the angular intra prediction mode as determined from equation (1) is illustrated.

[0185] Once the HoG is calculated, the indices with the two highest histogram bins are selected as the two implicitly derived intra prediction modes for the block and further combined with the planar mode as the prediction for the DIMD mode. Prediction fusion is applied as a weighted average of the three predictors mentioned above. For this purpose, the weight of the plane is fixed to 21 / 64 (~1 / 3). The remaining 43 / 64 (~2 / 3) weights are then shared between the two HoG IPMs, proportional to the amplitude of their HoG bins. Fig.21 An example of the mixing process is shown. Fig.21 As shown, two intra modes (M1 2112 and M2 2114) are selected according to the index of the two highest bars of the histogram bar 2110.

[0186] 2140, Pred2 2142 and Pred3 2144) are used to form a hybrid prediction. The three predictors correspond to applying M1, M2 and planar intra modes (respectively 2120, 2122 and 2124) to the reference pixel 2130 to form the corresponding predictor. The three predictors are weighted by the corresponding weighting factors (ω1, ω2 and ω3) 2150. The weighted predictors are summed using adder 2152 to generate a hybrid predictor 2160.

[0187] In addition, two implicitly derived intra modes are included in the MPM list so that the DIMD process is performed before constructing the MPM list. The primary derived intra mode of a DIMD block is stored with the block and used for MPM list construction for neighboring blocks.

[0188] Template-based Intra Mode Derivation (TIMD)

[0189] Template-based intra mode derivation (TIMD) mode implicitly derives the intra prediction mode of a CU using neighboring templates at the encoder and decoder, instead of signaling the intra prediction mode to the decoder. Fig. 22 As shown, the reference samples (2220 and 2222) of the template of each candidate mode are used to generate the prediction samples (2212 and 2214) of the template of the current block 2210. The cost is calculated as the SATD (sum of absolute transform differences) between the prediction samples and the reconstructed samples of the template. The intra prediction mode with the smallest cost is selected as the DIMD mode and used for intra prediction of the CU. The candidate mode can be 67 intra prediction modes as in VVC or extended to 131 intra prediction modes. Typically, MPM can provide clues to indicate the directional information of the CU. Therefore, in order to reduce the intra mode search space and utilize the characteristics of the CU, the intra prediction mode can be implicitly derived from the MPM list.

[0190] For each intra prediction mode in the MPM, the SATD between the predicted and reconstructed samples of the template is calculated. The first two intra prediction modes with the smallest SATD are selected as TIMD modes. These two TIMD modes are fused with weights after applying the PDPC process, and this weighted intra prediction is used to encode the current CU. Positiondependent intra prediction combination (PDPC) is included in the derivation of TIMD modes.

[0191] The costs of the two selection modes are compared to a threshold, and in the test a cost factor of 2 is applied as follows:

[0192] costMode2<2*costMode1.

[0193] Among them, costMode2 is the cost of mode 2, and costMode1 is the cost of mode 1.

[0194] If this condition is true, fusion is applied, otherwise only mode 1 is used. The weight of the mode is calculated based on its SATD cost as follows:

[0195] weight1=costMode2 / (costMode1+costMode2)

[0196] weight2=1-weight1.

[0197] Internal Subpartition (ISP)

[0198] Intra sub-partitioning (ISP) divides the luma intra prediction block into 2 or 4 sub-partitions vertically or horizontally depending on the block size. For example, the minimum block size of ISP is 4x8 (or 8x4). If the block size is larger than 4x8 (or 8x4), the corresponding block will be partitioned into 4 sub-partitions. It is worth noting that M×128 (M≤64) and 128×N (N≤64) ISP blocks may cause potential problems for 64×64 VDPU (virtual decoder pipeline unit). For example, an M×128 CU in a single tree case has an M×128 luma TB and two corresponding M / 2×64 chroma TBs. If the CU uses ISP, the luma TB will be divided into 4 M×32 TBs (only horizontal partitioning is possible), each TB is smaller than 64×64 blocks. However, in the current ISP design, the chroma blocks are not divided. Therefore, the size of both chroma components will be larger than 32×32 blocks. Similarly, a 128×N CU using ISP can also create a similar situation. Therefore, both cases are a problem for a 64×64 decoder pipeline. Therefore, the CU size that can use ISP is limited to a maximum of 64×64. Fig.23A and Fig. 23B Examples of two possibilities are shown. All sub-partitions satisfy the condition of having at least 16 samples.

[0199] In ISP, 1xN and 2xN sub-block predictions are not allowed to rely on reconstructed values ​​of previously decoded 1xN and 2xN sub-blocks of the coding block, making the minimum prediction width of the sub-block four samples. For example, an 8xN (N>4) coding block coded using ISP with vertical partitioning is divided into two prediction regions (each of size 4xN) and four transforms of size 2xN. In addition, a 4xN coding block coded using ISP with vertical partitioning is predicted using the full 4xN block; all four transforms use 1xN. Although 1xN and 2xN transform sizes are allowed, it can be asserted that the transforms of these blocks in the 4xN region can be performed in parallel. For example, when a 4xN prediction region contains four 1xN transforms, there is no transform in the horizontal direction; the transform in the vertical direction can be performed as a single 4xN transform in the vertical direction. Similarly, when a 4xN prediction region contains two 2xN transform blocks, the transform operations of the two 2xN blocks in each direction (horizontally and vertically) can be performed in parallel. Therefore, processing these smaller blocks does not add any latency compared to processing 4x4 regular coded intra blocks.

[0200] Table 4

[0201]

[0202] For each subpartition, the reconstructed samples are obtained by adding the residual signal to the prediction signal. Here, the residual signal is generated through processes such as entropy decoding, inverse quantization, and inverse transformation. Therefore, the reconstructed sample values ​​of each subpartition can be used to generate the prediction of the next subpartition, and each subpartition is processed consecutively. In addition, the first subpartition to be processed is the subpartition containing the samples in the upper left corner of the CU, and then continues downward (horizontal partition) or to the right (vertical partition). As a result, the reference samples used to generate the subpartition prediction signal are only located to the left and above the line. All subpartitions share the same intra-frame mode. The following is a summary of the interaction of ISP with other coding tools.

[0203] Multiple Reference Lines (MRL): If the MRL index of a block is not 0, the ISP coding mode will be inferred to be 0, so the ISP mode information will not be transmitted to the decoder.

[0204] Entropy coding coefficient group size: The size of the entropy coded sub-blocks has been modified so that they have 16 samples in all possible cases, as shown in Table 4. Note that the new size only affects blocks produced by ISP where one of the dimensions is less than 4 samples. In all other cases, the coefficient groups maintain 4×4 dimensions.

[0205] CBF encoding: It is assumed that at least one subpartition has a non-zero CBF. Thus, if n is the number of subpartitions and the first n-1 subpartitions produce a 0 CBF, the CBF of the nth subpartition is inferred to be 1.

[0206] Transform size restrictions: All ISP transforms greater than 16 points in length use DCT-II.

[0207] MTS flag: If the CU uses ISP coding mode, the MTS CU flag will be set to 0 and will not be transmitted to the decoder. Therefore, the encoder will not perform RD tests on the different available transforms for each generated sub-partition. Instead, the transform selection for ISP mode will be fixed and selected based on the intra mode, processing order and block size used. Therefore, no signaling is required. For example, let t G and t V The horizontal and vertical transforms are selected separately for the w×h subpartition, where w is the width and h is the height. The transforms are then selected according to the following rules:

[0208] If w=1 or h=1, there is no horizontal or vertical transformation, respectively.

[0209] If w ≥ 4 and w ≤ 16, then t H =DST-VII, otherwise, t H =DCT-II

[0210] If h≥4 and h≤16, then t V =DST-VII, otherwise, t V =DCT-II

[0211] In ISP mode, all 67 intra modes are allowed. PDPC is also applied if the corresponding width and height are at least 4 samples long. In addition, the conditions for the reference sample filtering process (reference smoothing) and the intra interpolation filter selection no longer exist, and the fractional position interpolation in ISP mode always uses a cubic (DCT-IF) filter.

[0212] Multiple Hypothesis Prediction (MHP)

[0213] In the multi-hypothesis inter-frame prediction mode (JVET-M0425), in addition to the traditional dual prediction signal, one or more additional motion compensation prediction signals are sent. The final overall prediction signal is obtained by superposition of sample weights. bi and the first additional inter-frame prediction signal / hypothesis h 3 The resulting prediction signal p 3 as follows:

[0214] p 3 =(1-α)pbi +αh 3 (16)

[0215] The weight factor α is specified by a new syntax element add_hyp_weight_idx according to the following mapping (Table 5):

[0216] Table 5. Mapping α to add_hyp_weight_idx

[0217]

[0218] Similar to the above, more than one additional prediction signal may be used. The resulting overall prediction signal is iteratively accumulated with each additional prediction signal.

[0219] p n+1 =(1-α n+n ) n +α n+1 h n+1 (17)

[0220] As the last p n (i.e., the p with the largest index n n ) to obtain the resulting overall prediction signal. For example, at most two additional prediction signals can be used (ie, n is limited to 2).

[0221] The motion parameters for each additional prediction hypothesis can be signaled explicitly by specifying the reference index, motion vector predictor index and motion vector difference, or implicitly by specifying the merge index. A separate multi-hypothesis merge flag distinguishes these two signaling modes.

[0222] For inter-AMVP mode, MHP is applied only when non-equal weights in BCW are selected in bi-prediction mode. Details of MHP for VVC can be found in JVET-W2025 (Muhammed Coban et al., "Algorithmic Description of Enhanced Compression Model 2 (ECM 2)", Joint Video Experts Group (JVET) of ITU-T SG 16WP 3 and ISO / IEC JTC 1 / SC29, 23rd Meeting, Teleconference, July 7-16, 2021, Document: JVET-W2025).

[0223] GPM Extensions

[0224] Several variants of the GPM mode (JVET-W0097 (Zhipin Deng et al., "AEE2-related: Combination of EE2-3.3, EE2-3.4 and EE2-3.5", Joint Video Experts Group (JVET) of ITU-T SG 16WP 3 and ISO / IEC JTC 1 / SC29, 23rd meeting, by teleconference, July 7-16, 2021, document: JVET-W0097) and JVET-Y0065 (Yoshitaka Kidani et al., "EE2-3.1: GPM with inter- and intra-frame prediction (JVET-X0166)", ITU-T SG16WP 3 and ISO / IEC JTC 1 / SC 29 Joint Video Experts Team (JVET), 25th Meeting, by Teleconference, January 12-21, 2022, Document: JVET-Y0065)) has been proposed to improve the coding efficiency of the GPM mode in VVC. This method is included in the Exploration Experiment (EE2) for further evaluation, and its main technical aspects are described as follows:

[0225] EE2-3.3 Regarding GPM with MMVD (GPM-MMVD): 1) Additional MVDs are added to existing GPM merge candidates; 2) MVDs are signaled in the same way as MMVDs in VVC, i.e., a distance index plus a direction index; 3) Two flags are signaled to control whether MMVD is applied to each GPM partition respectively.

[0226] EE2-3.4-3.5 Regarding GPM with template matching (GPM-TM): 1) Extend template matching to GPM mode by refining the GPM MV based on the left and top neighboring samples of the current CU; 2) Select template samples according to the GPM partition direction; 3) Signal a single flag to jointly control whether template matching is applied to the MVs of two GPM partitions.

[0227] JVET-W0097 proposes a combination of EE2-3.3, EE2-3.4 and EE2-3.5 to further improve the coding efficiency of GPM mode. Specifically, in the proposed combination, the existing designs in EE2-3.3, EE2-3.4 and EE2-3.5 remain unchanged, while the following modifications are further applied to coordinate the two codec tools:

[0228] GPM-MMVD and GPM-TM are enabled for only one GPM CU. This is done by first signaling the GPM-MMVD syntax. When both GPM-MMVD control flags are equal to false (i.e., GPM-MMVD is disabled for both GPM partitions), the GPM-TM flag is signaled to indicate whether template matching is applied to both GPM partitions. Otherwise (at least one GPM-MMVD flag is equal to true), the value of the GPM-TM flag is inferred to be false.

[0229] The GPM merge candidate list generation method in EE2-3.3 and EE2-3.4-3.5 is directly combined in the form of the MV pruning scheme in EE2-3.4-3.5 (where the MV pruning threshold is adjusted according to the current CU size) to replace the preset MV pruning scheme applied in EE2-3.3; in addition, as in EE2-3.4-3.5, multiple zero MVs are added until the GPM candidate list is completely filled.

[0230] In JVET-Y0065, in GPM with inter and intra prediction (or called GPM intra), the final prediction sample is generated by weighting the inter prediction sample and the intra prediction sample for each GPM separation area. The inter prediction samples are derived through the same scheme as the current GPM in ECM, while the intra prediction samples are derived through the intra prediction mode (IPM) candidate list and the index signaled from the encoder. The IPM candidate list size is predefined as 3. The available IPM candidates are parallel angle mode (parallel mode) relative to the GPM block boundary, vertical angle mode (vertical mode) relative to the GPM block boundary, and planar mode: as shown in the figure. See respectively. Fig.24A -C. In addition, Fig.24D The GPM shown with intra and intra prediction is restricted in the proposed method to reduce the signaling overhead of IPM and avoid increasing the size of intra prediction circuitry on the hardware decoder. In addition, direct motion vector and IPM storage is introduced in GPM mixed regions to further improve coding performance.

[0231] Space GPM

[0232] Similar to inter-GPM, spatial GPM (SGPM) consists of a partition mode and two associated intra prediction modes. If these modes are directly signaled in the bitstream, e.g. Fig.25A As shown, significant overhead will be generated. In order to express the necessary partition and prediction information more efficiently in the bitstream, a candidate list is used and only the candidate index is signaled in the bitstream. Each candidate in the list can derive a combination of a partition mode and two intra prediction modes, such as Fig.25B shown.

[0233] The template is used to generate the candidate list. The shape of the template is as follows Fig.26 For each possible combination of a partition mode and two intra prediction modes, the partition weights extended to the template are used to generate a prediction of the template, as Fig.26 As shown. The combinations are sorted in ascending order of the SATD between their prediction and reconstruction in the template. The length of the candidate list is set equal to 16, and these candidates are considered as the most likely SGPM combinations of the current block. Both the encoder and the decoder build the same candidate list based on the template.

[0234] To reduce the complexity of constructing the candidate list, the number of possible partition modes and the number of possible intra prediction modes are pruned. In the following tests, 26 of the 64 partition modes are used, and only the MPMs of the 67 intra prediction modes are used.

[0235] Recently, more schemes to speed up the encoding time of SGPM and improve the gain of SGPM have been disclosed, and some details can be found in JVET-AA0118 (Fan Wang et al., “EE2-1.4: Spatial GPM”, ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29 Joint Video Experts Team (JVET), 27th Meeting, by teleconference, July 13-22, 2022, document: JVET-AA0118).

[0236] Encoding fast algorithm

[0237] In JVET-Z0124 (Fan Wang et al., “Non-EE2: Spatial GPM”, Joint Video Experts Group (JVET) of ITU-T SG 16WP 3 and ISO / IEC JTC 1 / SC 29, 26th Meeting, by Teleconference, April 20-29, 2022, Document: JVET-Z0124), a full RDO process is performed on each candidate in a candidate list of size 16. In the EE2-1.4 test, the SAD / SATD cost is used to filter the candidates before full RDO. In particular, if the SAD / SATD cost of a candidate is greater than a threshold, the candidate will not enter the full RDO. This threshold is the best SAD / SATD cost of the current block multiplied by a ratio. The maximum number of SGPM full RDOs per block is limited to 8.

[0238] Simplified export of candidate lists

[0239] In JVET-Z0124, when deriving the candidate list, for each possible combination of a partition mode and two intra-frame prediction modes, the partition weights extended to the template are used to generate predictions for the template, and the SATD between the prediction and reconstruction of the template is used as the sorting criterion. In the EE2-1.4 test, the GPM mixing process is not used in the template, but SAD is used as the sorting criterion. The weights in the template are 1 or 0. For each MPM, two SADs of the two parts of each partition mode are calculated and saved. In order to obtain the SAD of a combination, just add the two corresponding SADs.

[0240] Template matching based reordering of GPM partition patterns

[0241] In the template matching based reordering for GPM partition patterns, given the motion information of the current GPM block, the corresponding TM cost value of the GPM partition pattern is calculated. Then, all GPM partition patterns are reordered in ascending order according to the TM cost value. Instead of sending the GPM partition pattern, an index is signaled using the Golomb-Rice code to indicate where the exact GPM partition pattern is located in the reordered list.

[0242] The reordering method of the GPM partition mode is a two-step process performed after generating the reference templates of the two GPM partitions in the coding unit, as follows:

[0243] Extending the GPM partition edge into the reference templates of the two GPM partitions, generating 64 reference templates and calculating the respective TM cost for each of the 64 reference templates;

[0244] The GPM partition patterns are re-sorted in ascending order according to their TM cost values, and the best 32 partition patterns are marked as usable partition patterns.

[0245] The edge 2720 on the template (2730 and 2732) extends from the edge of the current CU 2710, as shown in FIG. Fig. 27 As shown, but no GPM blending is used in the template area that crosses the edge. The index is signaled after ascending reordering using the TM cost.

[0246] GPM and SGPM use similar processes and have shown improvements in encoding and decoding performance, and it is desirable to develop a unified approach to simplify the process and / or further improve encoding performance. Summary of the invention

[0247] A method and apparatus for video encoding and decoding are disclosed. According to the method, pixel data associated with a current block is received at an encoder side, or encoded data associated with a current block to be decoded is received at a decoder side, wherein the current block is encoded using a codec tool including a first codec tool and a second codec tool. A shared intra-frame candidate list is determined. In response to selecting the first codec tool for the current block: the current block is divided into two regions according to a first target partition, a first hypothesis of intra-frame prediction is derived from the shared intra-frame candidate list, and a hybrid predictor is determined using first information including the first hypothesis of intra-frame prediction. In response to selecting the second codec tool for the current block: the current block is divided into two regions according to a second target partition, more than one second hypothesis of intra-frame prediction is derived from the shared intra-frame candidate list, and a hybrid predictor is determined using second information including the more than one second hypothesis of intra-frame prediction. The current block is encoded or decoded by using prediction data including the hybrid predictor.

[0248] In one embodiment, the shared intra candidate list includes an intra MPM (most probable mode) or a subset of an intra MPM. In another embodiment, the shared intra candidate list includes any subset of 67 intra prediction modes. In another embodiment, the shared intra candidate list includes one or more DIMD (decoder-side intra mode derivation) modes derived from any predefined template area. In another embodiment, the shared intra candidate list includes one or more TIMD (template-based intra mode derivation) modes derived from any predefined template area.

[0249] In one embodiment, different shared intra candidate lists are determined for different block sizes.

[0250] In one embodiment, a reordering process is applied to the candidates of the shared intra-frame candidate list. In one embodiment, the candidates of the shared intra-frame candidate list are reordered according to the template matching cost associated with the candidates of the shared intra-frame candidate list. In one embodiment, the shortest codeword is assigned to the candidate with the smallest template matching cost. In one embodiment, unified signaling between the first codec tool and the second codec tool is used, wherein a joint index is signaled or parsed to indicate the candidate selected from the reordered shared intra-frame candidate list. As an example consistent with the SGPM design, after reordering, only the top K candidates with smaller template matching costs are retained in the list for signaling, where K is a predefined positive number less than or equal to all possible candidates.

[0251] In one embodiment, a unified hybrid process is applied to the first codec tool and the second codec tool.

[0252] In one embodiment, each candidate in the shared intra candidate list includes a partition mode and an intra prediction mode. In another embodiment, each candidate in the shared intra candidate list includes a partition mode, motion candidate / information, and an intra prediction mode. In yet another embodiment, each candidate in the shared intra candidate list includes a motion candidate / information and an intra prediction mode. In yet another embodiment, each candidate in the shared intra candidate list includes a partition mode and motion candidate / information. BRIEF DESCRIPTION OF THE DRAWINGS

[0253] Figure 1A An exemplary adaptive inter / intra video coding system incorporating loop processing is shown.

[0254] Figure 1B Shows Figure 1A The corresponding decoder of the encoder in .

[0255] Figure 2 Neighboring blocks used to derive spatial merging candidates for VVC are shown.

[0256] Figure 3 The possible candidate pairs considered for redundancy checking in VVC are described.

[0257] Figure 4 An example of temporal candidate derivation is shown, where scaled motion vectors are derived according to the POC (Picture Order Count) distance.

[0258] Figure 5 The position of the temporal candidate selected between candidates C0 and C1 is shown.

[0259] Figure 6 The distance offset from the starting MV in the horizontal and vertical directions according to the MVD merge mode (MMVD) is marked.

[0260] Fig. 7A An example of an affine motion field of a block described by the motion information (4 parameters) of two control points is shown.

[0261] Figure 7B An example of an affine motion field of a block described by motion information of three control point motion vectors (6 parameters) is shown.

[0262] Figure 8 An example of block-based affine transform prediction is shown, where the motion vector of each 4×4 luma sub-block is derived from the control point MV.

[0263] Fig. 9 An example of deriving inherited affine candidates based on the control point MVs of neighboring blocks is shown.

[0264] Fig.10An example of constructing affine candidates by combining the translational motion information of each control point from its spatial neighbors and time is shown.

[0265] Fig.11 An example of affine motion information storage for motion information inheritance is shown.

[0266] Fig.12 An example of weight value derivation for combined inter and intra prediction (CIIP) according to the coding modes of the top and left neighboring blocks is shown.

[0267] Fig.13 An example of 64 partitions used in the VVC standard is shown, where the partitions are grouped according to their angles and the dashed lines represent redundant partitions.

[0268] Fig.14 An example of unipredictive MV selection for geometric partitioning patterns is shown.

[0269] Fig.15 The bending weight ω using the geometric partitioning pattern is indicated 0 Example.

[0270] Fig.16 An example of the GPM mixing process based on a discrete ramp function of the mixing region around the boundary is shown.

[0271] Fig.17 An example of the GPM mixing process of GPM mixing in ECM 4.0 is shown.

[0272] Fig.18 The intra-frame prediction mode adopted by the VVC video coding standard is marked.

[0273] Figure 19A-B shows a block that is wider than it is tall ( Fig.19A ) and blocks that are taller than they are wide ( Fig.19B ) is an example of wide-angle intra prediction.

[0274] Fig. 20A An example of a template selected for the current block is shown, where the template includes T rows above the current block and T columns to the left of the current block.

[0275] Fig. 20B An example of T=3 is shown, and the HoG (Histogram of Gradients) is calculated for the pixels in the middle row and the pixels in the middle column.

[0276] Fig. 20C Examples of the amplitude (ampl) of the angular intra prediction modes are marked.

[0277] Fig.21An example of the blending process is shown, where two angular intra modes (M1 and M2) are selected based on the index with the two highest bins of the histogram bins.

[0278] Fig. 22 An example of Template-based Intra Mode Derivation (TIMD) mode is shown, where TIMD implicitly derives the intra prediction mode of a CU using neighboring templates at both the encoder and the decoder.

[0279] Fig.23A An example of intra sub-partitioning (ISP) is shown, where a block is divided into two sub-blocks horizontally or vertically.

[0280] Fig. 23B An example of intra sub-partitioning (ISP) is shown, where a block is divided into four sub-blocks horizontally or vertically.

[0281] Fig.24A -C shows an example of available IPM candidates: parallel angle patterns relative to GPM block boundaries (parallel patterns, Fig.24A ), the vertical angle mode relative to the GPM block boundary (vertical mode, Fig. 24B ) and planar mode ( Fig.24C ).

[0282] Fig.24D An example of a GPM with intra and intra prediction is shown, where the intra prediction is restricted to reduce the signaling overhead and hardware decoder cost of the IPM.

[0283] Fig.25A The syntax encoding for the spatial GPM (SGPM) before using the simplification method is shown.

[0284] Fig.25B An example of a simplified syntax encoding of a spatial GPM (SGPM) is shown.

[0285] Fig.26 An example of a template for a spatial GPM (SGPM) is shown.

[0286] Fig. 27 An example is shown where an edge on a template extends from the template of the current CU, but no GPM blending is used in the template region across the edge.

[0287] Fig.28 An example of adaptive blending with separate blending sizes for two blending regions according to one embodiment of the present invention is shown.

[0288] Fig.29 An example of determining costs associated with various blend sizes based on a template and an extended blend region according to one embodiment of the present invention is shown.

[0289] Fig.30 An example of neighboring pattern information for a candidate list is illustrated.

[0290] Fig.31 A flow chart of an exemplary video encoding system utilizing a shared candidate list for GPM intra prediction and SGPM according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0291] It is readily understood that the components of the present invention, as generally described and illustrated in the accompanying drawings herein, may be arranged and designed in a variety of different configurations. Therefore, the following more detailed description of the embodiments of the systems and methods of the present invention as represented in the accompanying drawings is not intended to limit the scope of the claimed invention, but is merely representative of selected embodiments of the present invention. References to "one embodiment," "an embodiment," or similar language throughout the specification mean that a particular feature, structure, or characteristic described in conjunction with the embodiment may be included in at least one embodiment of the present invention. Therefore, the phrases "in one embodiment" or "in an embodiment" appearing in various places throughout the specification do not necessarily all refer to the same embodiment.

[0292] In addition, the described features, structures or characteristics can be combined in any suitable manner in one or more embodiments. However, those skilled in the relevant art will recognize that the present invention can be practiced without one or more specific details, or using other methods, components, etc. In other cases, well-known structures or operations are not shown or shown. Detailed description to avoid blurring aspects of the present invention. The illustrated embodiments of the present invention will be best understood by reference to the accompanying drawings, in which the same parts are represented by the same numbers throughout. The following description is only by way of example, and simply describes certain selected embodiments of the apparatus and method consistent with the present invention as claimed herein.

[0293] In order to improve the efficiency of video coding, one or more prediction hypotheses are mixed with one or more existing prediction hypotheses to achieve better prediction accuracy. In one embodiment, the prediction hypothesis means prediction based on motion with a predefined direction (list 0 or list 1). In another embodiment, the prediction hypothesis refers to a prediction generated from a motion candidate (e.g., a merge candidate or an AMVP candidate). In another embodiment, the prediction hypothesis means prediction based on motion with a predefined direction (list 0 or list 1) or bidirectional prediction. In another embodiment, the prediction hypothesis means prediction based on intra-frame candidates or motion candidates. In another embodiment, the prediction hypothesis means prediction from intra-frame candidates. In another embodiment, the hybrid prediction tool refers to (but is not limited to) any one or more of the tools listed below or any combination of the listed tools.

[0294] - Hybrid prediction tools include bi-predictive motion candidates, which can be merge candidates and / or AMVP candidates.

[0295] - The hybrid prediction tool includes GPM, one or more variants of GPM extension, and / or spatial GPM.

[0296] Since more than one prediction hypothesis is used for the current block, a blending process is required to form the final prediction for the current block.

[0297] In the present invention, an adaptive hybrid process is proposed to improve the weighting scheme used in hybrid prediction. GPM is taken as an example of a hybrid tool. In the following, the proposed adaptive hybrid process can also be applied to one or more of the mentioned hybrid prediction tools and / or any combination of the mentioned hybrid prediction tools.

[0298] In a first aspect, a partition line (eg, a GPM partition boundary) is defined to divide the current block 2810 into two prediction regions ( Fig.28 As shown). The area close to the partition line (e.g., θ1 line 2832 to partition line 2820 and partition line 2820 to -θ2 line 2830) is defined as a mixed area. In the mixed area, multiple (e.g., 2, first_hyp_pred and secondary_hyp_pred) prediction hypotheses are combined with weights (refer to W0[x][y]). Outside the mixed area: for samples located in the first prediction area, the weight of the second prediction hypothesis is zero, and the weight of the first prediction hypothesis is N; for samples located in the second prediction area, the weight of the first prediction hypothesis is zero, and the weight of the second prediction hypothesis is N. Several embodiments are proposed:

[0299] In one embodiment, Final_pred[x][y]=(first_hyp_pred[x][y]*W0[x][y]+second_hyp_pred[x][y]*(N-W0[x][y])+offset1)>>shift1

[0300] (x,y) is the sample position in the current block.

[0301] For the sample at (x,y) in the current block, W0[x][y] is the weight of first_hyp_pred, and (N–W0[x][y]) is the weight of secondary_hyp_pred.

[0302] In another embodiment, N is predefined as a fixed positive integer (eg, 8, 16, 32, or 64) or specified by block-level, SPS-level, PPS-level, APS-level, PH-level, and / or SH-level syntax.

[0303] In another embodiment, offset1 and shift1 are determined according to N and / or BitDepth. Taking N=8 as an example,

[0304] shift1=Max(5,17-BitDepth)

[0305] offset1=1<<(shift1-1)

[0306] In another embodiment, if the sample (x, y) is located in the second prediction region (i.e., the distance (x, y) is less than or equal to -theta2), then W0[x][y] is defined to be 0. (Or W0[x][y] is defined after the derivation of the mixed region, and the result is equal to or close to 0.)

[0307] In another embodiment, if the sample (x, y) is located in the first prediction region (i.e., the distance (x, y) is greater than or equal to theta1), then W0[x][y] is defined to be N. (Or W0[x][y] is defined after the derivation of the mixed region, and the result is equal to N or a value close to N.)

[0308] In another embodiment, a1 equal to 0 means that there is no mixing in the first prediction region. That is, W0[x][y] is defined as N in the first prediction region.

[0309] In another embodiment, a2 equal to 0 means that there is no mixing in the second prediction region. That is, W0[x][y] is defined as 0 in the second prediction region.

[0310] In another embodiment, if the sample (x, y) is located in the mixed region (i.e., the distance (x, y) is greater than -theta2 and less than theta1), W0[x][y] and / or θ2 are defined according to the distance theta1. For example, W0[x][y] is defined according to the existing GPM weight derivation (e.g., VVC method) by setting theta (used in GPM weight derivation) to the suggested theta1 or theta2.

[0311] In another sub-embodiment, if the sample (x, y) is located in the mixed region within the first prediction region (i.e., the distance (x, y) is greater than 0 and less than theta1), then W0[x][y] is defined based on the distance and theta1. For example, W0[x][y] is defined as (N*(distance(x, y)+theta1)) / (2*theta1) or can be simplified through quantization. For example, after quantization, W0[x][y] is defined as ((distance'(x, y)+16*theta1+offset2)>>shift2) and is clipped to [0,N].

[0312] The distance' in the GPM introduction section can be wIdxL

[0313] offset2=theta1>>1

[0314] shift2=log2(theta1)

[0315] For another example, after quantization, W0[x][y] is defined as ((distance'(x,y)+16*theta1+offset3)>>shift3) and is clipped to [0,N].

[0316] The distance' in the GPM introduction section can be wIdxL

[0317] Offset3 can be N shifted right by 1. Shift3 can be log2(N). For example, if N is 8, Offset3 will be 4 and shift3 will be 3.

[0318] In another sub-embodiment, if the sample (x, y) is located in the mixed region within the second prediction region (i.e., the distance (x, y) is less than 0 and greater than -theta2), then W0[x][y] is defined based on the distance and theta2. For example, W0[x][y] is defined as (N*(distance(x, y)+theta2)) / (2*theta2) or can be simplified through quantization. For example, after quantization, W0[x][y] is defined as ((distance'(x, y)+16*theta2+offset2)>>shift2) and is clipped to [0,N].

[0319] The distance' in the GPM introduction section can be wIdxL

[0320] offset2=theta2>>1

[0321] shift2=log2(theta2)

[0322] For another example, after quantization, W0[x][y] is defined as ((distance'(x,y)+16*theta2+offset3)>>shift3) and is clipped to [0,N].

[0323] The distance' in the GPM introduction section can be wIdxL

[0324] Offset3 can be N shifted right by 1. Shift3 can be log2(N). For example, if N is 32, Offset3 will be 16 and shift3 will be 5.

[0325] In another sub-embodiment, if the sample (x, y) is located in the mixed region of the partition line (i.e., distance(x, y) is equal to 0), then W0[x][y] is defined as the case of "the sample (x, y) is in the mixed region of the first prediction region", the case of "the sample (x, y) is in the mixed region of the second prediction region", any of the proposed embodiments, or as equal weights (N>>1).

[0326] In the second aspect, the following embodiments are proposed to determine the values ​​of theta1 and theta2.

[0327] In one embodiment, theta1 is predefined as a fixed value (eg, 0, 1 / 2, 1 / 4, 1, 2, 4, or 8) or specified by block level, SPS level, PPS level, APS level, PH level, and / or SH level syntax. This embodiment is applicable to theta 2.

[0328] In another embodiment, theta1 is selected from a candidate set including at least one candidate value. This embodiment is applicable to theta2.

[0329] In one sub-embodiment, the candidate set includes at least one of {0, 1 / 2, 1 / 4, 1, 2, 4, or 8} or any combination of the above values.

[0330] In another sub-embodiment, the candidate set includes at least one of {a / b, a, b*a} or any combination of the above values, where a and b are set to positive integers, such as a=2 and b=4.

[0331] In another sub-embodiment, the candidate set varies with the block width, block height and / or block area. For example, when the short side of the current block is equal to or less than a predefined threshold, only the smaller value is included in the candidate set; otherwise, only the larger value is included in the candidate set.

[0332] In another embodiment, theta1 may be the same as or different from theta2. The benefit of allowing different theta1 and theta2 values ​​(allowing asymmetric theta1 and theta2) is that the best mixing quality for different video sequences may require different mixing areas for the first prediction area and the second prediction area. For example, if the area of ​​the first prediction area is smaller, theta1 should be smaller than theta2. Or conversely, if the area of ​​the first prediction area is larger, theta1 should be smaller than theta2.

[0333] In one sub-embodiment, theta1 and theta2 each have their own candidate sets (e.g., theta1_set and theta2_set). For example, the number of candidates in theta1_set and theta2_set (i.e., the number of candidates = the number of candidates in the candidate set) may be different. For another example, one candidate set is a subset of the other candidate set. For another example, the number of candidates in theta1_set and theta2_set is the same.

[0334] In another sub-embodiment, theta1 and theta2 share a single candidate set. For example, theta1 and theta2 are the same. For another example, theta1 and theta2 can be the same or different.

[0335] In another embodiment, the number of candidates in the candidate set is defined as a fixed value (eg, 3 or 5) or specified by block level, SPS level, PPS level, APS level, PH level, and / or SH level syntax.

[0336] In another embodiment, the selection of theta1 and theta2 depends on a clear signal.

[0337] In one sub-embodiment, two separate syntaxes are signaled at the block level, SPS level, PPS level, APS level, PH level, and / or SH level syntax to indicate theta1 and theta2, respectively. For example, theta1 and theta2 are selected from a candidate set including {0, 1, 2, 4, 8}, respectively. An index (e.g., index_theta1, ranging from 0 to 4) is issued to select a value from the candidate set, and an index (e.g., index_theta2, ranging from 0 to 4) is issued to select a value from the candidate set.

[0338] In one sub-embodiment, syntax is signaled at the block level, SPS level, PPS level, APS level, PH level, and / or SH level syntax to indicate the combination of theta1 and theta2.

[0339] -theta1 and theta 2 are selected from a candidate set consisting of {0,1,2,4,8}. The candidate combination of theta1 and theta2, denoted as (theta1,theta2), can be

[0340] (0,0),(0,1),(0,2),(0,4),(0,8),(1,0),(1,1),(1,2),(1,4),(1,8),(2,0),(2,1),(2,2),(2,4),(2,8),(4,0),(4,1),(4,2),(4,4),(4,8),(8,0),(8,1),(8,2),(8,4),(8,8). (The number of candidate combinations can be reduced by other methods proposed in the present invention.)

[0341] -Signaling index (ranges from 0 to the number of candidate combinations - 1).

[0342] In one way, the index can be signaled using a truncated unary code.

[0343] In another way, the index can be context encoded.

[0344] In another way, the candidate combinations are sorted in ascending order by their template costs to form a reordered list. (Template cost measurement can refer to the part related to implicit derivation rules in the present invention.) The index indicated by the signal refers to the position of the used combination in the reordered list. Both the encoder and the decoder build the same reordered list based on the template.

[0345] The candidate combination with the minimum template cost uses the shortest codeword among all candidate combinations.

[0346] In another embodiment, the selection of theta1 and theta2 is subject to implicit derivation.

[0347] In one sub-embodiment, template matching is used as an implicit derivation rule:

[0348] -Step 1: Use the template (or the neighboring area of ​​the current block, encoded or decoded before the current block) to measure the cost of each candidate combination of theta1 and theta2. For example, select theta1 and theta2 from the candidate set including {0,1,2,4,8}. The candidate combination of theta1 and theta2, denoted as (theta1,theta2), can be

[0349] (0,0),(0,1),(0,2),(0,4),(0,8),(1,0),(1,1),(1,2),(1,4),(1,8),(2,0),(2,1),(2,2),(2,4),(2,8),(4,0),(4,1),(4,2),(4,4),(4,8),(8,0),(8,1),(8,2),(8,4),(8,8). (The number of candidate combinations can be reduced by other methods proposed in the present invention.)

[0350] -Step 2: For each candidate combination, calculate the template cost based on the distortion between the template’s “prediction” and reconstruction.

[0351] A "prediction" is generated by applying the GPM and blending (i.e. using a combination of candidates) to the template. Fig.29 As shown, the partition lines extend to the template.

[0352] The distortion can be SATD, SAD, MSE, SSE or any distortion measurement equation / metric.

[0353] -Step 3: theta1 and theta2 are implicitly set via the combination with the minimum stencil cost.

[0354] In another embodiment, the GPM change / extension can be any inter-frame or intra-frame mode.

[0355] -Partition the current block into two regions with partition directions (i.e. partition modes)

[0356] - Generate multiple forecast hypotheses through different forecast models

[0357] o Prediction mode refers to motion candidates, motion information derived from one or more motion candidates, intra prediction mode…

[0358] - Combine multiple prediction hypotheses to form a prediction for the current block

[0359] o Combined with sample-based weighting. That is, each sample gets its own weight

[0360] oFor samples close to the partition direction, the combined weight of each prediction hypothesis is non-zero. That is, the prediction samples near the partition direction are a combination of the prediction samples based on one prediction mode and the prediction samples based on another prediction mode.

[0361] In one sub-embodiment, the GPM variation / extension refers to GPM-MMVD, GPM-TM, GPM-Intra, or SGPM.

[0362] In another sub-embodiment, the GPM variation / extension is referred to as GPM-Intra or SGPM.

[0363] In another embodiment, for a block encoded using any of the GPM and / or GPM variants / extensions, a joint index is used to indicate a combination of a partition mode and one or more prediction modes for multiple prediction hypotheses, a combination of a subset of partition modes and one or more prediction modes for multiple prediction hypotheses, or a combination of more than one prediction modes for multiple prediction hypotheses.

[0364] In one sub-embodiment, SGPM is used to encode and decode the block. For example, the combination includes a partition mode and two intra-frame prediction modes. For another example, the combination includes two intra-frame prediction modes.

[0365] In another sub-embodiment, the block is GPM-intra coded. For example, the combination includes a partition mode, motion candidates / information, and an intra prediction mode. For another example, the combination includes a motion candidate / information and an intra prediction mode. For another example, the combination includes a partition mode and an intra prediction mode. For another example, the combination includes a partition mode and a motion candidate / information.

[0366] In another sub-embodiment, the combined list is reordered according to a template matching-based method. For example, the combined list comprising partition mode, motion candidates / information, and intra-frame prediction mode is reordered according to a template matching-based method. For another example, the combined list comprising motion candidates / information and intra-frame prediction mode is reordered according to a template matching-based method, and the template matching cost is determined for the combined list and the partition mode signaled. For another example, the combined list comprising partition mode and intra-frame prediction mode is reordered according to a template matching-based method, and the template matching cost is determined for the combined list and the motion candidates / information signaled. For another example, the combined list comprising partition mode and motion candidates / information is reordered according to a template matching-based method, and the template matching cost is determined for the combined list and the intra-frame prediction mode signaled.

[0367] In another sub-embodiment, the joint index is signaled / parsed in the bitstream. For example, the joint index is encoded using truncated unary codewords. The following example shows an example of the combination of the shortest codewords.

[0368] In one example, when GPM-Intra is used to encode and decode a block, the combination with the shortest codeword comprises a predefined pattern.

[0369] The predefined mode may be a motion candidate / information with a merge index equal to M, where M may be a positive integer, such as 0, 1, ..., or (size of the merge candidate list - 1).

[0370] The predefined mode may be an intra prediction mode, such as one of planar, DC, horizontal, vertical, parallel mode and vertical mode.

[0371] The predefined pattern may be a partition pattern in a vertical direction, a horizontal direction, or a diagonal direction.

[0372] For another example, the predefined rules depend on the block width, height, area, and neighboring pattern information.

[0373] In another sub-embodiment, the joint index indicates a combination from the combination list. For example, the order in the combination list implies the signal priority of the combination. That is, the combination at the first position in the combination list is signaled / parsed using the shortest codeword among all combinations. For another example, the combination at the first position in the combination list is predefined.

[0374] In one manner, when the current block is GPM-intra or SGPM, one of planar, DC, horizontal, vertical, parallel mode, and vertical mode is predefined in the first position in the combination list.

[0375] In other words, the predefined rules depend on the block width, height, area, and neighboring pattern information.

[0376] As another example, the syntax used to indicate the combination at the first position in the combination list is encoded using one or more contexts.

[0377] In one approach, context selection depends on block width, height, area, and neighboring pattern information.

[0378] In another way, one or more of the used contexts are not reused by the remaining combinations in the combination list.

[0379] As another example, the syntax for indicating a combination at a non-first position in a combination list is not encoded using a context.

[0380] In another sub-embodiment, the joint index indicates a combination from a reordered combination list according to a template matching based method. For example, the order in the reordered combination list implies a signal priority of the combination. That is, the combination at the first position in the combination list is signaled / parsed with the shortest codeword among all combinations. For another example, the syntax for indicating the combination at the first position in the combination list is encoded with one or more contexts.

[0381] In one approach, context selection depends on block width, height, area, and neighboring pattern information.

[0382] In another way, one or more of the used contexts are not reused by the remaining combinations in the combination list.

[0383] As another example, the syntax for indicating a combination at a non-first position in a combination list is not encoded using a context.

[0384] In another embodiment, for a block encoded using any of the GPM and / or GPM variants / extensions, an index is used to indicate a reordered list of prediction modes according to a template matching based approach, wherein the template matching cost is calculated via another signaled index to the prediction mode, the signaled partition mode, and a prediction mode in another list of prediction modes to indicate another list of prediction modes.

[0385] In another sub-embodiment, the block is GPM-intra coded. For example, the prediction mode list includes one or more intra prediction modes, and the list is reordered according to a template matching based method, wherein a template matching cost is calculated for the intra prediction modes in the intra prediction mode list, the signaled partition mode, and the inter prediction mode, wherein the inter prediction mode is determined by a signaled index to indicate the list of inter prediction modes.

[0386] In another embodiment, it is proposed to unify the design between GPM modes (eg, GPM and / or different GPM variants / extensions). The benefit is that, through the unified design, the circuit can be reused by GPM and / or different GPM variants / extensions.

[0387] In another sub-embodiment, the unified design refers to a hybrid design (eg, an adaptive hybrid process). For example, the candidate sets used in the adaptive hybrid process may be unified.

[0388] On the one hand, the candidates in the set are uniform.

[0389] For unification, the candidates for the first unified GPM mode are the same subset as the candidates for the second unified GPM mode.

[0390] In other words, the number of candidates in the set is uniform.

[0391] For unification, the number of candidates for the first unified GPM mode is the same as the number of candidates for the second unified GPM mode.

[0392] For another example, a selection rule for selecting a candidate from a candidate set used in the adaptive mixing process may be unified.

[0393] In one approach, the selection rule for the first unified GPM mode is the same as the selection rule for the candidate for the second unified GPM mode.

[0394] In another way, the selection rule depends on the signaling / parsing in the bitstream.

[0395] In other words, the selection rule depends on the block width, block height, block area or neighboring pattern information.

[0396] In another sub-embodiment, the unified design refers to the generation of candidate lists (eg, prediction modes for obtaining prediction hypotheses). For example, the candidate lists are used to generate hypotheses for intra prediction of GPM-Intra and one or more hypotheses for intra prediction of SGPM.

[0397] In one approach, the candidate list is an IPM candidate list.

[0398] In another way, the candidate list is the MPM list or any subset of the MPM list used in normal intra mode (eg, intra mode codec with 67 intra prediction modes or any extension from 67 intra prediction modes, such as 131 intra prediction modes).

[0399] This subset may be the first N candidates in the MPM list used in normal intra mode, where N may be any positive integer, such as 1, 2, 3, 4, 5, 6, ... or (size_of_MPM_list-1).

[0400] In another way, the candidate list includes neighboring mode information (such as neighboring intra prediction modes), where the neighboring blocks can be one or more of the following: Fig.30 shown.

[0401] Left neighboring block (L) or any block adjacent to the left border of the current block

[0402] In the adjacent block (A) or any block adjacent to the above boundary of the current block

[0403] Lower left adjacent block (BL)

[0404] Upper right adjacent block (AR)

[0405] Upper left adjacent block (AL)

[0406] In another way, the candidate list includes one or more DIMD intra prediction modes (i.e., having the two highest histogram bins)

[0407] In another manner, the candidate list includes one or more of DC, HOR, and VER.

[0408] For another example, the order in the candidate list implies the priority order of the signaling or parsing of the candidates. That is, the candidate at the first position in the list will use the shortest codeword among all the candidates for signaling / parsing. For another example, the candidate at the first position in the candidate list is predefined.

[0409] In one manner, when the current block is GPM-intra or SGPM, one of planar, DC, horizontal, vertical, parallel mode, and vertical mode is predefined in the first position in the candidate list.

[0410] In other words, the predefined rules depend on the block width, height, area, and neighboring pattern information.

[0411] As another example, the syntax used to indicate the candidate at the first position in the candidate list is encoded using one or more contexts.

[0412] In one approach, context selection depends on block width, height, area, and neighboring pattern information.

[0413] In another way, one or more of the used contexts are not reused by the remaining candidates in the candidate list.

[0414] For another example, the syntax for indicating a candidate at a non-first position in the candidate list is not encoded or decoded using a context.

[0415] The method proposed by the present invention can be unified with multiple hybrid tools. For example, the method proposed for GPM, GPM extension and / or spatial GPM is unified.

[0416] In one embodiment, the method proposed in the present invention can only be applied to some predefined partition lines among all candidate partition lines.

[0417] The method proposed in the present invention can be based on implicit rules (such as block width, height or area) or based on explicit rules (such as syntax about block, tile, slice, picture, SPS or PPS level). For example, when the block area is greater than a threshold, the proposed method is applied. For another example, when the longer block edge is greater than or equal to a threshold (such as 2) multiplied by the shorter block edge, the proposed method is applied.

[0418] The term “block” in the present invention may refer to a TU / TB, a CU / CB, a PU / PB, a predefined area, or a CTU / CTB.

[0419] The AMVP in the present invention is similar to the "AMVP" in JVET-T2002 (VVC tool description). AMVP motion comes from motion candidates with syntax "merge flag" equal to false (for example, general_merge_flag equal to false in VVC).

[0420] Any combination of the methods proposed in the present invention may be applied.

[0421] Any of the aforementioned proposed hybrid prediction methods with a shared candidate list for GPM intra prediction and SGPM can be implemented in an encoder and / or decoder. For example, any of the proposed hybrid prediction methods with a shared candidate list for GPM intra prediction and SGPM can be implemented in an intra / inter coding module ( Figure 1A Intra-frame prediction 150 and / or inter-frame prediction 112 in the encoder and decoder ( Figure 1B Intra-frame prediction 150) and / or motion compensation module ( Figure 1B Alternatively, any proposed method may be implemented as a circuit coupled to an intra / inter coding module and / or a motion compensation module of an encoder, or a merge candidate derivation module of a decoder.

[0422] Fig.31 A flowchart of an exemplary video encoding system using a shared candidate list for GPM intra prediction and SGPM according to an embodiment of the present invention is shown. The steps shown in the flowchart can be implemented as executable program codes on one or more processors (e.g., one or more CPUs) on the encoder side. The steps shown in the flowchart can also be implemented based on hardware, such as one or more electronic devices or processors arranged to perform the steps in the flowchart. According to the method, in step 3110, pixel data associated with a current block is received on the encoder side or encoded data associated with a current block to be decoded is received on the decoder side, wherein the current block is encoded using one of a plurality of codecs including a first codec and a second codec. In step 3120, a shared intra candidate list is determined. In step 3130, it is checked whether the current block is encoded in the first codec mode or the second codec mode. If the current block is encoded in the first codec mode, steps 3140 to 3144 are performed. If the current block is encoded in the second codec, steps 3150 to 3154 are performed. In step 3140, the current block is divided into two regions according to the first target partition. In step 3142, a first hypothesis for intra prediction is derived from the shared intra candidate list. In step 3144, a hybrid predictor is determined using first information including the first hypothesis for intra prediction. In step 3150, the current block is divided into two regions according to a second target partition. In step 3152, more than one second hypothesis for intra prediction is derived from the shared intra candidate list. In step 3154, a hybrid predictor is determined using second information including the more than one second hypothesis for intra prediction. In step 3160, the current block is encoded or decoded by using prediction data including the hybrid predictor.

[0423] The flow chart shown is intended to illustrate an example of video encoding according to the present invention. Without departing from the spirit of the present invention, those skilled in the art may modify each step, rearrange the steps, split the steps, or combine the steps to implement the present invention. In the present disclosure, specific syntax and semantics have been used to illustrate examples to implement embodiments of the present invention. Without departing from the spirit of the present invention, those skilled in the art may implement the present invention by replacing syntax and semantics with equivalent syntax and semantics.

[0424] The above description is provided to enable those of ordinary skill in the art to practice the present invention provided in the context of a specific application and its requirements. Various modifications to the described embodiments will be apparent to those skilled in the art, and the general principles defined herein may be applied to other embodiments. Therefore, the present invention is not intended to be limited to the specific embodiments shown and described, but to conform to the widest scope consistent with the principles and novel features disclosed herein. In the above detailed description, various specific details are illustrated to provide a thorough understanding of the present invention. However, those skilled in the art will appreciate that the present invention may be implemented.

[0425] The embodiments of the present invention as described above can be implemented in various hardware, software codes or a combination of the two. For example, one embodiment of the present invention can be one or more circuits integrated into a video compression chip or a program code integrated into the video compression software to perform the processing described herein. The embodiments of the present invention can also be a program code to be executed on a digital signal processor (DSP) to perform the processing described herein. The present invention can also be related to many functions performed by a computer processor, a digital signal processor, a microprocessor or a field programmable gate array (FPGA). These processors can be configured to perform specific tasks according to the present invention by executing machine-readable software codes or firmware codes that define the specific methods embodied by the present invention. The software code or firmware code can be developed in different programming languages ​​and different formats or styles. The software code can also be compiled for different target platforms. However, the different code formats, styles and languages ​​of the software code and other ways of configuring the code to perform tasks according to the present invention will not depart from the spirit and scope of the present invention.

[0426] The invention may be embodied in other specific forms without departing from its spirit or essential characteristics. The described examples should be considered in all respects as illustrative and not restrictive. Therefore, the scope of the invention is indicated by the appended claims rather than by the foregoing description. All changes that fall within the meaning and scope of the equivalents of the claims should be included within their scope.

Claims

1. A video encoding and decoding method, the method comprising: Receiving pixel data related to a current block at an encoder side or receiving encoded data to be decoded related to the current block at a decoder side, wherein the current block is encoded and decoded using one of a plurality of codec tools including a first codec tool and a second codec tool; determining a shared intra candidate list; In response to selecting a first codec for the current block: Dividing the current block into two regions according to a first target partition; deriving a first hypothesis for intra prediction from the shared intra candidate list; and determining a hybrid predictor using first information including a first hypothesis for said intra prediction; In response to selecting a second codec for the current block: Dividing the current block into two regions according to a second target partition; deriving more than one second hypothesis for intra prediction from the shared intra candidate list; and determining a hybrid predictor using second information including more than one second hypothesis for said intra prediction; and The current block is encoded or decoded by using prediction data including a hybrid predictor.

2. The method according to claim 1, characterized in that The shared intra candidate list includes an intra most probable mode (MPM) or a subset of the intra MPM.

3. The method according to claim 1, characterized in that The shared intra candidate list includes any subset of the 67 intra prediction modes.

4. The method according to claim 1, characterized in that: The shared intra candidate list includes one or more decoder-side intra mode derivation (DIMD) modes derived from any predefined template region.

5. The method according to claim 1, characterized in that: The shared intra candidate list includes one or more template-based intra mode derivation (TIMD) modes derived from any predefined template region.

6. The method according to claim 1, characterized in that Different shared intra candidate lists are determined for different block sizes.

7. The method according to claim 1, characterized in that A reordering process is applied to the candidates of the shared intra candidate list.

8. The method according to claim 7, characterized in that The candidates of the shared intra candidate list are reordered according to the template matching costs associated with the candidates of the shared intra candidate list.

9. The method according to claim 8, characterized in that The shortest codeword is assigned to the candidate with the smallest template matching cost.

10. The method according to claim 8, characterized in that The first codec and the second codec use unified signaling, where a joint index is signaled or parsed to indicate a candidate selected from a reordered shared intra candidate list.

11. The method according to claim 1, characterized in that A unified hybrid process is applied to the first codec tool and the second codec tool.

12. The method according to claim 1, characterized in that Each candidate in the shared intra candidate list includes a partition mode and an intra prediction mode.

13. The method according to claim 1, characterized in that Each candidate in the shared intra candidate list includes a partition mode, motion candidate / information, and an intra prediction mode.

14. The method according to claim 1, characterized in that Each candidate of the shared intra candidate list includes motion candidate / information and an intra prediction mode.

15. The method according to claim 1, characterized in that Each candidate of the shared intra candidate list includes a partition mode and motion candidate / information.

16. An apparatus for video encoding, the apparatus comprising one or more electronic devices or processors arranged to: Receive pixel data related to the current block at the encoder side or receive encoded data to be decoded related to the current block at the decoder side, wherein: encoding and decoding the current block using one of a plurality of encoding and decoding tools including a first encoding and decoding tool and a second encoding and decoding tool; determining a shared intra candidate list; In response to selecting a first codec for the current block: Dividing the current block into two regions according to a first target partition; deriving a first hypothesis for intra prediction from the shared intra candidate list; and determining a hybrid predictor using first information including a first hypothesis for said intra prediction; In response to selecting a second codec for the current block: Dividing the current block into two regions according to a second target partition; deriving more than one second hypothesis for intra prediction from the shared intra candidate list; as well as determining a hybrid predictor using second information including more than one second hypothesis for said intra prediction; and The current block is encoded or decoded by using prediction data including the hybrid predictor.

Images