Method and apparatus for multiple hypothesis prediction in video coding system

Abstract

A predictive encoding and decoding method and related device, the method comprising: receiving input data associated with a current block, the input data comprising pixel data of the current block to be encoded at an encoder side or encoded data associated with the current block to be decoded at a decoder side; determining prediction members related to the existing prediction, where each prediction member corresponds to a weight sum of the existing prediction using a weight and at least one target prediction candidate from a prediction member combination, where the weight and the at least one target prediction candidate are jointly decided by a joint index; and encoding or decoding the current block using a final prediction determined based on the joint index.

Description

[0001] Cross Reference To

[0002] This application is a divisional application of the application patent application No. 202211702799.X, entitled “Method and apparatus for multiple hypothesis prediction in video coding system”.

[0003] This application claims priority to U.S. Provisional Patent Application No. 63 / 294,424, filed December 29, 2021, which is hereby incorporated by reference in its entirety. TECHNICAL FIELD

[0004] The present invention relates to video coding systems. In particular, the present invention relates to multiple hypothesis prediction coding. BACKGROUND

[0005] Versatile Video Coding (VVC) is the latest international video coding standard developed by the Joint Video Expert Team (JVET) of ITU-T Video Coding Experts Group (VCEG) and ISO / IEC Moving Picture Experts Group (MPEG). The standard has been published as an ISO standard: ISO / IEC 23090-3:2021, Information technology — Coding of immersive media — Part 3: Versatile Video Coding, published in February 2021. VVC is based on its predecessor, HEVC (High Efficiency Video Coding), by adding more coding tools to improve coding efficiency, and can also handle various types of video sources, including 3-dimensional (3D) video signals.

[0006] Figure 1A An exemplary adaptive inter / intra video coding system incorporating in-loop processing is illustrated. For intra prediction 110, prediction data is derived from previously coded video data in the current picture. For inter prediction 112, motion estimation (ME) is performed at the encoder side and motion compensation (MC) is performed based on the results of ME to provide prediction data derived from other pictures and motion data. A switch 114 selects either intra prediction 110 or inter prediction 112 and the selected prediction data is provided to a summer 116 to form prediction error, also referred to as residual. The prediction error is then processed by a transform (T) 118 and subsequent quantization (Q) 120. The transformed and quantized residual is then coded by an entropy coder 122 to be included in a video bitstream corresponding to compressed video data. The bitstream associated with the transform coefficients is then packaged with side information (e.g., motion and coding modes associated with intra and inter prediction) and other information (e.g., parameters associated with in-loop filters applied to regions of the underlying picture). Side information associated with intra prediction 110, inter prediction 112, and in-loop filters 130 is provided to the entropy coder 122, as is Figure 1AAs shown. When using inter-frame prediction mode, one or more reference images must also be reconstructed at the encoder end. Therefore, the transformed and quantized residuals are processed by inverse quantization (IQ) 124 and inverse transform (IT) 126 to recover the residuals. Then, at reconstruction (REC) 128, the residuals are added back to the prediction data 136 to reconstruct the video data. The reconstructed video data can be stored in the reference image buffer 134 and used to predict other frames.

[0007] like Figure 1A As shown, the input video data undergoes a series of processing steps in the encoding system. Due to these processing steps, the reconstructed video data from REC128 may suffer various forms of degradation. Therefore, loop filters 130 are often applied to the reconstructed video data before it is stored in the reference picture buffer 134 to improve video quality. For example, deblocking filters (DF), sample adaptive offset (SAO), and adaptive loop filters (ALF) can be used. It may be necessary to incorporate loop filter information into the bitstream so that the decoder can correctly recover the required information. Therefore, loop filter information is also provided to the entropy encoder 122 for incorporation into the bitstream. Figure 1A In this process, the loop filter 130 is applied to the reconstructed video before the reconstructed samples are stored in the reference image buffer 134. Figure 1A The system described herein is intended to illustrate an exemplary architecture of a typical video encoder. It may correspond to a High Efficiency Video Coding (HEVC) system, VP8, VP9, ​​H.264, or VVC.

[0008] like Figure 1B As shown, the decoder can use similar or identical function blocks to the encoder, except for transform 118 and quantization 120, since the decoder only needs inverse quantization 124 and inverse transform 126. Instead of entropy encoder 122, the decoder uses entropy decoder 140 to decode the video bitstream into quantized transform coefficients and the required coding information (e.g., ILPF information, intra-frame prediction information, and inter-frame prediction information). Intra-frame prediction 150 on the decoder side does not require mode search. Instead, the decoder only needs to generate intra-frame predictions based on the intra-frame prediction information received from entropy decoder 140. Furthermore, for inter-frame prediction, the decoder only needs to perform motion compensation (MC 152) based on the inter-frame prediction information received from entropy decoder 140 without motion estimation.

[0009] According to VVC, similar to HEVC, the input image is divided into non-overlapping square block regions called CTUs (Coding Tree Units). Each CTU can be further divided into one or more smaller coding units (CUs). The resulting CU partitions can be squares or rectangles. Furthermore, VVC divides the CTUs into prediction units (PUs) as units for applying prediction processes, such as inter-frame prediction, intra-frame prediction, etc.

[0010] The VVC standard incorporates various new encoding tools to further improve encoding efficiency beyond the HEVC standard. Among these new encoding tools, some relevant to this invention are summarized below.

[0011] Inter-frame prediction overview

[0012] According to Section 3.4 of JVET-T2002 (Jianle Chen et al., “Algorithm description of Universal Video Coding and Test Model 11 (VTM11)”, Joint Video Experts Group (JVET) of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29, 20th Meeting, Teleconference, October 7–16, 2020, Archive: JVET-T2002), for each inter-frame prediction CU, the motion parameters include motion vectors, reference picture indexes, and a reference picture list using identifiers, as well as additional information required by the new coding features of VVC for inter-frame prediction sample generation. Motion parameters can be signaled explicitly or implicitly. When a CU is coded in skip mode, the CU is associated with a PU and has no significant residual coefficients, no encoded motion vector increments, and no reference picture indexes. A merge mode is specified where the motion parameters of the current CU are obtained from neighboring CUs, including spatial and temporal candidates, as well as additional scheduling introduced in VVC. The merge mode can be applied to any inter-frame prediction CU, not just skip mode. An alternative to the merged mode is the explicit transmission of motion parameters, in which motion vectors, the corresponding reference image index for each reference image list, the reference image list identifier, and other necessary information are explicitly sent per CU.

[0013] In addition to the inter-frame coding features in HEVC, VVC also includes many new and improved inter-frame predictive coding tools, as listed below:

[0014] –Extended merge prediction

[0015] – Merge mode with MVD (MMVD)

[0016] -Symmetric MVD (SMVD) signaling

[0017] – Affine motion compensated prediction

[0018] – Subblock-based temporal motion vector prediction (SbTMVP)

[0019] -Adaptive motion vector resolution (AMVR)

[0020] – Sports field storage: 1 / 16 luminance sample MV storage and 8x8 sports field compression

[0021] –Bi-prediction with CU-level weight (BCW)

[0022] – Bidirectional optical flow (BDOF)

[0023] Decoder-side motion vector refinement (DMVR)

[0024] – Geometric partitioning mode (GPM)

[0025] – Combined inter and intra prediction (CIIP)

[0026] The following description provides details of the inter-frame prediction methods specified in VVC.

[0027] Extended merge forecast

[0028] In VVC, the merge candidate list is constructed by sequentially including the following five categories of candidates:

[0029] 1) Spatial MVP from spatially adjacent CUs

[0030] 2) Time MVP from co-located CU

[0031] 3) Historical MVPs from FIFO tables

[0032] 4) Paired average MVP

[0033] 5) Zero MV.

[0034] The size of the merge list is signaled in the Sequence Parameter Set (SPS) header, and the maximum allowed size of the merge list is 6. For each CU encoded in merge mode, the index of the best merge candidate is encoded (TU) using truncated univariate binarization. The first binary string of the merge index (hereinafter referred to as bin) is context-encoded, and bypass encoding is used for the remaining bins.

[0035] This section provides the derivation process for merge candidates for each category. Similar to HEVC, VVC also supports parallel derivation of the merge candidate list (or merge candidate list) for all CUs within a given region.

[0036] Spatial candidate derivation

[0037] Except for swapping the positions of the first two merge candidates, the derivation of spatial merge candidates in VVC is the same as in HEVC. Figure 2 At most four merge candidates (B0, A0, B1, and A1) of the current CU 210 are selected from the candidates at the indicated positions. The derived order is B0, A0, B1, A1, and B2. Position B2 is considered only if one or more adjacent CUs of positions B0, A0, B1, and A1 are unavailable (e.g., belonging to another slice or tile) or are intra-coded. After adding the candidate for position A1, redundancy checks are performed on the addition of the remaining candidates to ensure that candidates with the same motion information are excluded from the list, thereby improving coding efficiency. To reduce computational complexity, not all possible candidate pairs are considered in the redundancy check mentioned. Instead, only... Figure 3 The pairs are linked by arrows, and a candidate is added to the list only if the corresponding candidate used for redundancy checking does not have the same motion information.

[0038] Time Candidate Derivation

[0039] In this step, only one candidate is added to the list. Specifically, in the derivation of this time-merging candidate for the current CU 410, based on belonging to... Figure 4 The scaled motion vector is derived from the collocated reference picture's collocated CU 420. The reference picture list and reference index used to derive the collocated CU are explicitly sent as signals in the slice header. For example... Figure 4As shown by the dashed line, the scaled motion vector 430 of the temporal merge candidate is obtained, which is scaled from the motion vector 440 of the co-located CU using POC (Picture Order Count) distances tb and td, where tb is defined as the POC difference between the reference image and the current image, and td is defined as the POC difference between the reference image and the co-located image. The reference image index of the temporal merge candidate is set to zero.

[0040] The position of the time candidate is selected between candidate C0 and C1, such as... Figure 5 As shown. If the CU at position C0 is unavailable, intra-coded, or outside the current CTU line, then position C1 is used. Otherwise, position C0 is used to derive time-merging candidates.

[0041] Historical Merger Candidate Derivation

[0042] Historically based MVP (HMVP) merge candidates are added to the merge list following the spatial MVP and TMVP. In this method, motion information from previous coded blocks is stored in a table and used as the MVP for the current CU. A table with multiple HMVP candidates is maintained during encoding / decoding. This table is reset (cleared) when a new CTU row is encountered. Whenever a non-sub-block inter-coded CU exists, the associated motion information is added as the last entry in the table as a new HMVP candidate.

[0043] The HMVP table size S is set to 6, indicating that a maximum of 5 history-based MVP (HMVP) candidates can be added to the table. When a new motion candidate is inserted into the table, a constrained First-In-First-Out (FIFO) rule is used, where a redundancy check is first applied to find if a duplicate HMVP already exists in the table. If found, the duplicate HMVP is removed from the table, and all subsequent HMVP candidates are shifted forward, with the duplicate HMVP inserted as the last entry in the table.

[0044] HMVP candidates can be used in the candidate list construction process. The latest few HMVP candidates in the table are checked sequentially, and inserted into the candidate list after the TMVP candidates. Redundancy checks are applied to HMVP candidates for spatial or temporal merging.

[0045] To reduce the number of redundant check operations, the following simplification is introduced:

[0046] 1. The last two entries in the table perform redundancy checks on candidates in space A1 and B1, respectively.

[0047] 2. Once the total number of available merge candidates reaches the maximum allowed number of merge candidates minus 1, the HMVP merge candidate list construction process terminates.

[0048] Derivation of Pairwise Average Merge Candidates

[0049] Pairwise averaging candidates are generated by averaging predefined candidate pairs in the existing merge candidate list using the first two merge candidates. The first merge candidate is defined as p0Cand, and the second merge candidate can be defined as p1Cand. The average motion vector is calculated for each reference list based on the availability of motion vectors for p0Cand and p1Cand. If both motion vectors are available in a list, they are averaged even if they point to different reference images, and their reference images are set to one of p0C and p1C; if only one motion vector is available, it is used directly; if no motion vector is available, the list remains invalid. Furthermore, if the half-pixel interpolation filter indices of p0Cand and p1Cand are different, they are set to 0.

[0050] If the merge list is not full after adding pairwise average merge candidates, insert zero MVPs at the end until the maximum number of merge candidates is encountered.

[0051] Merging estimation areas

[0052] Merge Estimation Region (MER) allows for the independent derivation of merge candidate lists for Cues (CUs) within the same MER. Candidate blocks within the same MER as the current CU are not included in the generation of the current CU's merge candidate list. Furthermore, merge candidate lists are only generated if (xCb+cbWidth)>>Log2ParMrgLevel is greater than xCb>>Log2ParMrgLevel and (yCb+cbHeight)>>Log2ParMrgLevel is greater than (yCb>>Log2ParMrgLevel), where (xCb, yCb) is the top-left luminance sample position of the current CU in the image, and (cbWidth, cbHeight) is the CU size. The MER size is selected at the encoder end and signaled as log2_parallel_merge_level_minus2 in the Sequence Parameter Set (SPS).

[0053] Merge pattern with MVD (MMVD)

[0054] In addition to the merging pattern in which implicitly derived motion information is directly used in the generation of prediction samples for the current CU, a merging pattern with motion vector difference (MMVD) is introduced in VVC. An MMVD flag is issued immediately after the regular merging flag to specify whether the MMVD pattern is used for the CU.

[0055] In MMVD, after selecting a merging candidate (referred to as a basic merging candidate in this invention), it is further refined by MVD information transmitted via signals. This further information includes a merging candidate identifier, an index specifying the motion amplitude, and an index indicating the motion direction. In MMVD mode, one of the first two candidates in the merging list is selected as the MV basis. An MMVD candidate identifier is issued to specify which one to use between the first and second merging candidates.

[0056] The distance index specifies motion amplitude information and indicates a predefined offset from the starting points (612 and 622) of L0 reference block 610 and L1 reference block 620. For example... Figure 6 As shown, offsets are added to the horizontal or vertical component of the starting MV, where different styles of small circles correspond to different offsets from the center. The relationship between the distance index and the predefined offsets is specified in Table 1.

[0057] Table 1 - Relationship between Distance Index and Predefined Offset

[0058]

[0059] The direction index indicates the direction of the MVD relative to the starting point. The direction index can represent the four directions shown in Table 2. It is important to note that the meaning of the MVD sign can vary depending on the information of the starting MV. When the starting MV is a non-predictive MV or a double-predictive MV, and both lists point to the same side of the current image (i.e., both reference POCs are greater than or less than the current image's POC), the sign in Table 2 specifies the sign of the MV offset added to the starting MV. When the starting MV is a bidirectional predictive MV, and the two MVs point to different sides of the current image (i.e., one reference POC is greater than the current image's POC, and the other reference POC is less than the current image's POC), and the difference in POCs in list 0 (which can be simply referred to as list 0 or L0) is greater than the difference in POCs in list 1 (which can be simply referred to as list 1 or L1), the sign in Table 2 specifies that the sign of the MV offset of the MV component in list 0 added to the starting MV has the opposite value to the sign of the MV in list 1. Otherwise, if the difference of POC in List 1 is greater than that in List 0, then the sign in Table 2 specifies that the sign of the MV offset of the List 1 MV component added to the starting MV has the opposite value to the sign of the List 0 MV.

[0060] MVD is scaled based on the difference in POC in each direction. If the difference in POC is the same in both lists, no scaling is needed. Otherwise, if the difference in POC in list 0 is greater than the difference in list 1, the MVD of list 1 is scaled by defining the difference in POC in L0 as td and the difference in POC in L1 as tb, as shown below. Figure 5As shown. If the POC difference of L1 is greater than that of L0, the MVD of list 0 is scaled in the same way. If the initial MV is unidirectionally predicted, the MVD is added to the available MV.

[0061] Table 2 - MV Offset Symbols Specified by Direction Index

[0062] Direction Index 00 01 10 11 X-axis + - N / A N / A Y-axis N / A N / A + -

[0063] Affine Motion Compensation Prediction

[0064] In HEVC, only the translational motion model is applied to motion compensation prediction (MCP). However, in the real world, there are many types of motion, such as zooming in / out, rotation, perspective motion, and other irregular motions. In VVC, block-based affine transformation motion compensation prediction is applied, as shown in the figure. Figure 7A In -B, the affine motion domain of block 710 is determined by... Figure 7A Motion information of two control points (4 parameters) or Figure 7B The motion vectors (6 parameters) of the three control points are described.

[0065] For the 4-parameter affine motion model, the motion vector at the sample position (x, y) in the block is derived as:

[0066]

[0067] For the 6-parameter affine motion model, the motion vector at the sample position (x, y) in the block is derived as:

[0068]

[0069] Where (mv 0x ,mv 0y (mv) is the motion vector of the top-left control point. 1x ,mv 1y (mv) is the motion vector of the upper right control point. 2x ,mv 2y ) is the motion vector of the bottom-left corner control point.

[0070] To simplify motion compensation prediction, a block-based affine transformation prediction is applied. To derive the motion vector for each 4×4 brightness sub-block, the motion vector of the center sample of each sub-block is as follows: Figure 8 As shown, calculations are performed according to the above equations, and rounded to 1 / 16 fractional precision. Then, a motion-compensated interpolation filter is applied to generate predictions for each sub-block with derived motion vectors. The sub-block size for the chroma components is also set to 4×4. The MV of the 4×4 chroma sub-block is calculated as the average of the MVs of the top-left and bottom-right luminance sub-blocks in the juxtaposed 8x8 luminance region.

[0071] For translational motion inter-frame prediction, there are also two affine motion inter-frame prediction modes: affine merging mode and affine AMVP mode.

[0072] Affine Merge Prediction (AF_MERGE)

[0073] The AF_MERGE mode can be applied to CUs with a width and height greater than or equal to 8. In this mode, the CPMV (Control Point MV) of the current CU is generated based on the motion information of spatially adjacent CUs. There can be up to five CPMV prediction candidates, and an index is signaled to indicate which one should be used for the current CU. The following three types of CPMV candidates are used to construct the affine merging candidate list:

[0074] – Inherited affine merge candidates inferred from the CPMV of adjacent CUs

[0075] – Constructing affine merge candidate CPMVPs using translational MV derivations of adjacent CUs

[0076] -Zero MV

[0077] In VVC, there are at most two inherited affine candidates, which come from the affine motion models of neighboring blocks: one from the left-neighboring CU and one from the top-neighboring CU. Candidate blocks and Figure 2 The same applies. For the left predictor, the scan order is A0->A1, and for the upper predictor, the scan order is B0->B1->B2. Only the first inherited candidate for each side is selected. No pruning check is performed between two inherited candidates. When identifying adjacent affine CUs, their control point motion vectors are used to derive CPMVP candidates from the affine merge list of the current CU. Figure 9 As shown, if the lower-left adjacent block A of the current block 910 is encoded in affine mode, then the motion vectors v2, v3, and v4 of the upper-left corner, upper-right corner, and lower-left corner containing block A of CU 920 are obtained. When block A is encoded using a 4-parameter affine model, the two CPMVs (i.e., v0 and v1) of the current CU are calculated based on v2 and v3. When block A is encoded using a 6-parameter affine model, the three CPMVs of the current CU are calculated based on v2, v3, and v4.

[0078] Constructing affine candidates refers to building candidates by combining the translational motion information of adjacent control points. For example... Figure 10As shown, the motion information of the control point is derived from the specified spatial and temporal neighbors of the current block 1010. CPMVk (k = 1, 2, 3, 4) represents the k-th control point. For CPMV1, check the B2->B3->A2 block and use the MV of the first available block. For CPMV2, check the B1->B0 block, and for CPMV3, check the A1->A0 block. If the TMVP is available, it is used as CPMV4.

[0079] After obtaining the motion signatures (MVs) of the four control points, affine merging candidates are constructed based on motion information. The following combinations of control point MVs are used for sequential construction:

[0080] {CPMV1,CPMV2,CPMV3},{CPMV1,CPMV2,CPMV4},{CPMV1,CPMV3,CPMV4},

[0081] {CPMV2,CPMV3,CPMV4},{CPMV1,CPMV2},{CPMV1,CPMV3}

[0082] Combinations of three CPMVs construct a 6-parameter affine merging candidate, and combinations of two CPMVs construct a 4-parameter affine merging candidate. To avoid motion scaling, combinations of control point MVs are discarded if the reference indices of the control points are different.

[0083] After checking the affine merge candidates of inheritance and construction, if the list is still not full, insert zero MVs at the end of the list.

[0084] Affine AMVP Prediction

[0085] The affine AMVP mode can be applied to CUs with a width and height greater than or equal to 16. A CU-level affine identifier is signaled in the bitstream to indicate whether the affine AMVP mode is used, and another identifier is signaled to indicate whether a 4-parameter affine or a 6-parameter affine is used. In this mode, the difference between the current CU's CPVM and its predictor variable CPMVP is signaled in the bitstream. The affine AMVP candidate list is of size 2 and is generated sequentially from the following four CPVM candidates:

[0086] – Inherited affine AMVP candidates inferred from the CPMV of adjacent CUs

[0087] – The constructed affine AMVP candidate CPMVP is derived using the translation MV of adjacent CUs.

[0088] – Translation MV from adjacent CU

[0089] -Zero MV

[0090] The checking order for inherited affine AMVP candidates is the same as that for inherited affine merge candidates. The only difference is that for AVMP candidates, only affine CUs with the same reference picture as the current block are considered. No pruning process is applied when inserting inherited affine motion predictions into the candidate list.

[0091] The constructed AMVP candidates are from Figure 10 The specified spatial neighbors are shown in the diagram. The reference picture indices of adjacent blocks are also checked using the same checking order as in the affine merge candidate construction. In the checking order, the first block using inter-frame coding and having the same reference picture as in the current CU is selected. If the current CU is coded using a 4-parameter affine mode and both mv0 and mv1 are available, they are added as a candidate to the affine AMVP list. If the current CU is coded using a 6-parameter affine mode and all three CPMVs are available, they are added as a candidate to the affine AMVP list. Otherwise, the constructed AMVP candidate is set to unavailable.

[0092] If the number of affine AMVP candidates is still less than 2 after inserting valid inherited affine AMVP candidates and constructed AMVP candidates, then mv0, mv1, and mv2 are added as translational MVs to predict all control point MVs of the current CU when available. Finally, if the affine AMVP list is still not full, it is filled with zero MVs.

[0093] Affine motion information storage

[0094] In VVC, the CPMV of an affine CU is stored in a separate buffer. The stored CPMV is only used to generate inherited CPMVPs for the most recently coded CU in affine merge mode and affine AMVP mode. Subblock MVs derived from the CPMV are used for motion compensation, merging / AMVP lists of MVs, MV export, and deblocking.

[0095] To avoid additional CPMV image row buffers, affine motion data inherited from the CUs of the aforementioned CTU is treated differently, inheriting from normal adjacent CUs. If the candidate CU for affine motion data inheritance is in the aforementioned CTU row, the bottom left and bottom right sub-blocks of MV in the row buffer, instead of CPMV, are used for affine MVP derivation. Thus, CPMV is only stored in the local buffer. If the candidate CU is a 6-parameter affine code, the affine model degenerates into a 4-parameter model. Figure 11 As shown, along the top CTU boundary, the motion vectors of the lower left and lower right sub-blocks of the CU are used for affine inheritance of the CU in the bottom CTU. Figure 11In the diagram, rows 1110 and 1112 represent the x and y coordinates of the origin (0, 0) in the top-left corner of the image. Legend 1120 illustrates the meaning of various motion vectors, where arrow 1122 represents the CPMV used for affine inheritance in the local buffer, arrow 1124 represents inheritance in the row buffer for MC / merge / skip MVP / deblock / TMVP and affine sub-block vectors, and arrow 1126 represents the sub-block vector of MC / merge / skip MVP / deblock / TMVP.

[0096] Adaptive Motion Vector Resolution (AMVR)

[0097] In HEVC, when `use_integer_mv_flag` in the slice header is equal to 0, the motion vector difference (MVD) (between the CU's motion vector and the predicted motion vector) is transmitted as a signal in quarter-luminance samples. In VVC, a CU-level adaptive motion vector resolution (AMVR) scheme is introduced. AMVR allows the CU's MVD to be encoded with different precisions. Depending on the current CU's mode (normal AMVP mode or affine AVMP mode), the current CU's MVD can be adaptively selected as follows:

[0098] - Normal AMVP mode: quarter brightness sample, half brightness sample, integer brightness sample or four brightness sample.

[0099] – Affine AMVP mode: quarter brightness sample, integer brightness sample, or 1 / 16 brightness sample.

[0100] If the current CU has at least one non-zero MVD component, the CU-level MVD resolution indication is conditionally signaled. If all MVD components (i.e., the horizontal and vertical MVD of reference list L0 and reference list L1) are zero, the quarter-luminance sample MVD resolution is inferred.

[0101] For a CU with at least one non-zero MVD component, a first flag is signaled to indicate whether quarter-luminance sample MVD accuracy is used for the CU. If the first flag is 0, no further signaling is required, and quarter-luminance sample MVD accuracy is used for the current CU. Otherwise, a second flag is signaled to indicate whether half-luminance sample or other MVD accuracy (integer or quad-luminance sample) is used for the normal AMVP CU. In the case of half-luminance samples, a 6-tap interpolation filter is used at the half-luminance sample location instead of the default 8-tap interpolation filter. Otherwise, a third flag is signaled to indicate whether integer or quad-luminance sample MVD accuracy is used for the normal AMVP CU. In the case of affine AMVP CUs, the second flag indicates whether integer or 1 / 16 luminance sample MVD accuracy is used. To ensure that the reconstructed MV has the expected accuracy (quarter-luminance sample, half-luminance sample, integer luminance sample, or quad-luminance sample), the CU's motion vector predictor is rounded to the same MVD accuracy before being added to the MVD. The predicted motion vector values ​​are rounded to zero (i.e., negative motion vector predicted values ​​are rounded to positive infinity, and positive motion vector predicted values ​​are rounded to negative infinity).

[0102] The encoder uses RD checks to determine the current CU's motion vector resolution. To avoid always performing four CU-level RD checks for each MVD resolution, RD checks for MVD accuracy, except for quarter-luminance samples, are conditionally invoked only in VTM11. For normal AVMP mode, the RD costs for quarter-luminance sample MVD accuracy and integer luminance sample MVD accuracy are first calculated. Then, the RD costs for integer luminance sample MVD accuracy are compared with those for quarter-luminance sample MVD accuracy to determine if further checks of the four-luminance sample MVD accuracy's RD cost are necessary. The four-luminance sample MVD accuracy RD check is skipped when the RD cost for quarter-luminance sample MVD accuracy is significantly less than that for integer luminance sample MVD accuracy. Then, if the RD cost for integer luminance sample MVD accuracy is significantly greater than the optimal RD cost of the previously tested MVD accuracy, the half-luminance sample MVD accuracy check is skipped. For affine AMVP mode, if the affine inter-frame mode and affine AMVP mode are not selected after checking the rate-distortion cost of affine merge / skip mode, merge / skip mode, AMVP mode with normal quarter-luminance sample MVD accuracy, and quarter-luminance sample MVD accuracy, then the 1 / 16 luminance sample MV accuracy and 1 pixel MV accuracy affine inter-frame modes are not checked. Furthermore, the affine parameters obtained in the quarter-luminance sample MV accuracy affine inter-frame mode are used as the starting search point for the 1 / 16 luminance sample and quarter-luminance sample MV accuracy affine inter-frame modes.

[0103] Bidirectional prediction with CU-level weights (BCW)

[0104] In HEVC, the dual prediction signal P bi-pred It is generated by averaging two prediction signals P0 and P1 obtained from two different reference images and / or using two different motion vectors. In VVC, the dual prediction mode is extended beyond simple averaging to allow for a weighted average of the two prediction signals.

[0105] P bi-pred =((8-w)*P0+w*P1+4)>>3 (3)

[0106] Weighted average bidirectional prediction allows five weights, w∈{-2,3,4,5,10}. For each bidirectionally predicted CU, the weight w is determined by one of two methods: 1) for non-merged CUs, the weight index is signaled after the motion vector difference; 2) for merged CUs, the weight index is inferred from neighboring blocks based on the merge candidate index. BCW is only applicable to CUs with 256 or more luminance samples (i.e., CU width multiplied by CU height is greater than or equal to 256). For low-latency images, all 5 weights are used. For non-low-latency images, only 3 weights are used (w∈{3,4,5}). At the encoder, a fast search algorithm is applied to find the weight index without significantly increasing the encoder's complexity. These algorithms are summarized below. Detailed information is disclosed in the VTM software and document JVET-L0646 (Yu-Chi Su et al., “CE4 related: Generalized bidirectional prediction improvement combining JVET-L0197 and JVET-L0296”, Joint Video Experts Group (JVET) of ITU-T SG 16WP 3 with ISO / IEC JTC1 / SC 29, 12th meeting: Macau, October 3-12, 2018, document: JVET-L0646).

[0107] When combined with AMVR, if the current image is a low-latency image, unequal weights are conditionally checked only for 1-pixel and 4-pixel motion vector precision.

[0108] When combined with affine, affine ME is performed on unequal weights if and only if the affine mode is selected as the current best mode.

[0109] When the two reference images in bidirectional prediction are the same, unequal weights are only conditionally checked.

[0110] Unequal weights are not searched when certain conditions are met, depending on the POC distance between the current image and its reference image, the encoding QP, and the time level.

[0111] The BCW weight index is encoded using a context-coded bin and a bypass-coded bin. The first context-coded bin indicates whether equal weights are used; if unequal weights are used, the bypass-coded bin signals the additional bin to indicate which unequal weights were used.

[0112] Weight Prediction (WP) is an encoding tool supported by the H.264 / AVC and HEVC standards that efficiently encodes video content with fading. Support for WP has also been added to the VVC standard. WP allows sending weight parameters (weights and offsets) for each reference picture in each of the reference picture lists L0 and L1. The weights and offsets of the corresponding reference pictures are then applied during motion compensation. WP and BCW are designed for different types of video content. To avoid interaction between WP and BCW, which would complicate the VVC decoder design, if a CU uses WP, the BCW weight index is not notified, and the weight w is inferred as 4 (i.e., equal weights are applied). For merged CUs, the weight index is inferred from neighboring blocks based on the merge candidate index. This can be applied to both normal merge mode and inherited affine merge mode. For constructed affine merge mode, the affine motion information is constructed based on the motion information of up to 3 blocks. The BCW index of a CU using a constructed affine merge mode is simply set to be equal to the BCW index of the first control point MV.

[0113] In VVC, CIIP and BCW cannot jointly request a CU. When a CU uses CIIP mode encoding, the BCW index of the current CU is set to 2, which is equivalent to w=4 for equal weights. Equal weights imply the default value of the BCW index.

[0114] Combined Inter-Frame and Intra-Frame Prediction (CIIP)

[0115] In VVC, when a CU is encoded in merged mode, if the CU contains at least 64 luma samples (i.e., CU width multiplied by CU height equal to or greater than 64), and if both the CU width and CU height are less than 128 luma samples, an additional flag is sent to indicate whether Combined Inter-Frame / Intra-Frame Prediction (CIIP) mode is applied to the current CU. As its name suggests, CIIP prediction combines inter-frame prediction signals with intra-frame prediction signals. CIIP mode P inter The inter-frame prediction signal in the model is derived using the same inter-frame prediction process applied to the regular combining mode; and the intra-frame prediction signal P intra This is derived after a regular intra-frame prediction process with a planar pattern. Then, the intra-frame and inter-frame prediction signals are combined using a weighted average, where the weight value wt is based on the top and left adjacent blocks of the current CU 1210 (e.g., ...). Figure 12 The encoding pattern shown is calculated as follows:

[0116] If the top neighbor is available and intra-frame encoding is used, set isIntraTop to 1; otherwise, set isIntraTop to 0.

[0117] If the left neighbor is available and intra-frame coding is used, set isIntraLeft to 1; otherwise, set isIntraLeft to 0.

[0118] If (isIntraLeft+isIntraTop) equals 2, then wt is set to 3;

[0119] Otherwise, if (isIntraLeft+isIntraTop) equals 1, then wt is set to 2;

[0120] Otherwise, set wt to 1.

[0121] The CIIP predictions are as follows:

[0122] P CIIP =((4-wt)*P inter +wt*P intra +2)>>2 (4)

[0123] CCLM (Cross-Component Linear Model)

[0124] The main idea behind CCLM mode (sometimes abbreviated as LM mode) is that there is often some correlation between the color components (e.g., Y / Cb / Cr, YUV, and RGB) of a color image. These colors may be referred to as the first color, the second color, and the third color in this disclosure. CCLM technology utilizes this correlation by predicting the chromaticity components of a block from juxtaposed reconstructed luminance samples using a linear model whose parameters are derived from the reconstructed luminance and chromaticity samples adjacent to the block.

[0125] In VVC, the CCLM mode utilizes inter-channel dependencies by predicting chromaticity samples from reconstructed luminance samples. This prediction is performed using a linear model of the following form.

[0126] P(i,j)=a·rec′ L (i,j)+b. (5)

[0127] Here, P(i,j) represents the predicted chromaticity sample in CU, while rec′ L (i, j) represent reconstructed luminance samples from the same CU, which are downsampled for non-4:4:4 color formats. Model parameters a and b are derived from adjacent luminance and chrominance samples reconstructed at the encoder and decoder ends, without explicit signaling.

[0128] VVC specifies three CCLM modes: CCLM_LT, CCLM_L, and CCLM_T. These three modes differ in the location of the reference samples used for model parameter derivation. CCLM_T mode involves samples only from the top boundary, CCLM_L mode involves samples only from the left boundary, and CCLM_LT mode uses samples from both the top and left boundaries.

[0129] In general, the prediction process of the CCLM model includes three steps:

[0130] 1) Downsampling of the luminance block and its adjacent reconstructed samples to match the size of the corresponding chrominance block.

[0131] 2) Derivation of model parameters based on reconstructed neighboring samples, and

[0132] 3) Apply model equation (1) to generate intra-frame chroma prediction samples.

[0133] Luminance component downsampling: To match the chroma sample positions of 4:2:0 or 4:2:2 color format video sequences, two types of downsampling filters can be applied to the luminance samples, both with a 2:1 downsampling rate in both the horizontal and vertical directions. These two filters correspond to "Type-0" and "Type-2" 4:2:0 chroma format content, respectively, and are defined by...

[0134]

[0135] Based on SPS-level identification information, a two-dimensional 6th-order (f2) or 5th-order (f1) filter is applied to the luminance samples within the current block and their adjacent luminance samples. SPS-level refers to the Sequence Parameter Set level. An anomaly occurs if the row above the current block is a CTU boundary. In this case, a one-dimensional filter [1,2,1] / 4 is applied to the aforementioned adjacent luminance samples to avoid using multiple luminance queues above the CTU boundary.

[0136] Model parameter derivation process: The model parameters a and b from equation (5) are derived based on adjacent luminance and chrominance samples reconstructed at the encoder and decoder ends to avoid requiring any signaling overhead. In the initial CCLM model version, a linear minimum mean square error (LMMSE) estimator was used for parameter derivation. However, in the final design, only four samples are involved to reduce computational complexity. Figure 13 The relative sample positions of the M×N chroma block 1310, the corresponding 2M×2N luminance block 1320, and their adjacent samples (displayed as solid circles and triangles) of "Type 0" content are shown.

[0137] existFigure 13 The example shows four samples used in CCLM_LT mode, labeled with triangles. They are located at the upper boundaries M / 4 and M·3 / 4, and the left boundaries N / 4 and N·3 / 4. In CCLM_T and CCLM_L modes, the upper and left boundaries are extended to the size of (M+N) samples, and the four samples used for model parameter derivation are located at (M+N) / 8, (M+N)·3 / 8, (M+N)·5 / 8, and (M+N)·7 / 8.

[0138] Once four samples are selected, four comparison operations are used to determine the two smallest and two largest brightness sample values. Let X l Let X represent the average of two maximum brightness sample values. s This represents the average of the two minimum brightness sample values. Similarly, let Y... l and Y s This represents the average value of the corresponding chromaticity sample values. Then, the linear model parameters are obtained according to the following equation:

[0139]

[0140] b = Y s-a ·X s (7)

[0141] In this equation, the division operation for parameter 'a' is performed using a lookup table. To reduce the memory required to store this table, the diff value—the difference between the maximum and minimum values—and parameter 'a' are represented using exponential notation. Here, the diff value is approximated with a 4-digit significant part and an exponent. Therefore, the diff table contains only 16 elements. This approach reduces both computational complexity and the memory required to store the table.

[0142] MMLM Overview

[0143] As the name suggests, the original CCLM model uses a linear model to predict the chromaticity samples of the entire CU's luminance samples, while MMLM (Multi-Model CCLM) can have two models. In MMLM, the neighboring luminance samples and neighboring chromaticity samples of the current block are divided into two groups, each group serving as a training set to derive the linear model (i.e., deriving specific α and β for a specific group). Furthermore, the samples of the current luminance block are also classified based on the same rules as the classification of neighboring luminance samples.

[0144] The threshold is the average value of adjacent reconstructed brightness samples. Neighbors with Rec′L[x,y] <= Threshold are classified into group 1; while neighbors with Rec′L[x,y] > Threshold are classified into group 2.

[0145] Accordingly, the chromaticity prediction is obtained using a linear model:

[0146]

[0147] Chroma intra-frame mode coding

[0148] For chroma intra-mode coding, a total of eight intra-modes are allowed. These modes include five traditional intra-modes and three cross-component linear model modes (CCLM, LM_A, and LM_L). The chroma mode signaling and derivation process are shown in Table 3. Chroma mode coding directly depends on the intra-prediction mode of the corresponding luma block. Due to the separate block partitioning structure for luma and chroma components enabled in the I-slice, one chroma block can correspond to multiple luma blocks. Therefore, for chroma-derived (DM) mode, the intra-prediction mode of the corresponding luma block covering the center position of the current chroma block is directly inherited.

[0149] Table 3. Deriving Chroma Prediction Mode from Luminance Mode when CCLM is Enabled

[0150]

[0151] As shown in Table 4, a single binary table is used regardless of the value of sps_cclm_enabled_flag.

[0152] Table 4. Unified Binarization Table for Colorimetric Prediction Mode

[0153]

[0154] The first bin indicates whether it is in normal mode (0) or LM mode (1). If it is in LM mode, the next bin indicates whether it is LM_CHROMA (0) or not (1). If it is not LM_CHROMA, the next bin indicates whether it is LM_L (0) or LM_A (1). In this case, when sps_cclm_enabled_flag is 0, the first bin of the binarization table corresponding to intra_chroma_pred_mode can be ignored before entropy encoding. Or, in other words, the first bin is inferred to be 0 and therefore not encoded. This single binarization table is used for cases where sps_cclm_enabled_flag equals 0 and 1. The first two bins are context-encoded using their own context model, and the remaining bins are side-encoded.

[0155] Multiple Hypothesis Prediction (MHP)

[0156] In the multi-hypothesis inter-frame prediction mode (JVET-M0425), in addition to the traditional dual prediction signal, one or more additional motion-compensated prediction signals are emitted. The final overall prediction signal is obtained by superimposing sample weights. Using the dual prediction signal p... bi The resulting prediction signal p3, obtained by combining the first additional inter-frame prediction signal / hypothesis h3, is as follows:

[0157] p3=(1-α)p bi +αh3 (8)

[0158] According to the following mapping (Table 5), the weight factor α is specified by the new syntax element add_hyp_weight_idx:

[0159] Table 5. Mapping α to add_hyp_weight_idx

[0160] add_hyp_weight_idx α 0 1 / 4 1 -1 / 8

[0161] Similar to the above, more than one additional prediction signal can be used. The resulting overall prediction signal is iteratively accumulated together with each additional prediction signal.

[0162] p n+1 =(1-α) n+1 )p n +α n+1 h n+1 (9)

[0163] As the last p n (That is, p with the largest index n) n The resulting overall prediction signal is obtained. For example, up to two additional prediction signals can be used (i.e., n is limited to 2).

[0164] The motion parameters for each additional prediction hypothesis can be explicitly signaled by specifying a reference index, a motion vector prediction index, and a motion vector difference, or implicitly signaled by specifying a merging index. A single multi-hypothesis merging identifier distinguishes between these two signaling modes.

[0165] For inter-frame AMVP mode, MHP is applied only when non-equal weights are selected in BCW under bidirectional prediction mode. Detailed information on MHP for VVC can be found in JVET-W2025 (Muhammed Coban et al., “Algorithmic description of Enhanced Compression Model 2 (ECM 2)”, Joint Video Experts Group (JVET) of ITU-T SG 16WP 3 and ISO / IEC JTC 1 / SC29, 23rd meeting, teleconference, July 7-16, 2021, document: JVET-W2025).

[0166] Combining MHP and BDOF is possible, but BDOF is only applied to the double-predictive signal portion of the predicted signal (i.e., the ordinary first two assumptions). Summary of the Invention

[0167] In one embodiment of the present invention, a predictive encoding / decoding method is disclosed. The method includes: receiving input data associated with a current block, wherein the input data includes pixel data of the current block to be encoded on the encoder side or encoded data associated with the current block to be decoded on the decoder side; determining prediction members associated with existing predictions, wherein each prediction member corresponds to the weighted sum of the existing predictions using weights and at least one target prediction candidate from the combination of prediction members, wherein the weights and the at least one target prediction candidate are jointly determined by a joint index; and encoding or decoding the current block using a final prediction determined based on the joint index.

[0168] In one embodiment of the present invention, the joint prediction index is disclosed to indicate a combination of a target motion candidate and a target weight, the combination being used to combine the existing prediction and the at least one target prediction candidate, the target prediction candidate being a prediction from the target motion candidate, the target motion candidate being a combination from m motion candidates, and the target weight being a combination from n weights, where m and n are positive integers.

[0169] In one embodiment of the present invention, the joint index is disclosed to be associated with the final predicted target motion candidate and the target weight, and is transmitted as a signal in the bitstream at the encoder end or parsed from the bitstream at the decoder end.

[0170] In one embodiment of the present invention, it is disclosed that the joint index is related to the final predicted target motion candidate and the target weight, and is implicitly determined.

[0171] In one embodiment of the present invention, it is further disclosed to include determining the boundary matching cost associated with the predicted member, wherein each boundary matching cost of each predicted member is determined using the predicted sample of the current block and the neighboring reconstructed samples of the current block, wherein the predicted sample of the current block is generated based on each predicted member, and wherein the final prediction is determined based on the boundary matching cost.

[0172] In one embodiment of the present invention, an apparatus for predictive encoding and decoding is disclosed. The apparatus includes one or more electronic devices or processors arranged to: receive input data associated with a current block, wherein the input data includes pixel data of the current block to be encoded on the encoder side or encoded data associated with the current block to be decoded on the decoder side; determine prediction members associated with existing predictions, wherein each prediction member corresponds to a weighted sum of the existing predictions using weights and at least one target prediction candidate from a combination of prediction members, wherein the weights and the at least one target prediction candidate are jointly determined by a joint index; and encode or decode the current block using a final prediction determined based on the joint index. Attached Figure Description

[0173] Figure 1A An exemplary adaptive inter-frame / intra-frame video coding system incorporating loop processing is illustrated.

[0174] Figure 1B The diagram shows... Figure 1A The corresponding decoder of the encoder in the code.

[0175] Figure 2 The diagram illustrates adjacent blocks used to derive spatial merge candidates for VVC.

[0176] Figure 3 The illustration shows possible candidate pairs for redundancy checks in VVC.

[0177] Figure 4 An example of temporal candidate derivation is illustrated, in which a scaled motion vector is derived based on the POC (picture order count) distance.

[0178] Figure 5 The diagram illustrates the position of the time candidate selected between candidate C0 and C1.

[0179] Figure 6 The diagram illustrates the distance offset from the starting MV in the horizontal and vertical directions according to the merge mode with MVD (MMVD).

[0180] Figure 7A An example of an affine motion field for a block described by motion information from two control points (4 parameters) is shown.

[0181] Figure 7B An example of an affine motion field of a block is illustrated, described by motion information from three control point motion vectors (6 parameters).

[0182] Figure 8 An example of block-based affine transformation prediction is illustrated, where the motion vector of each 4×4 luma subblock is derived from the control point MV.

[0183] Figure 9The illustration shows an example of deriving inheritance affine candidates based on the control point MV derivation of adjacent blocks.

[0184] Figure 10 An example is shown of constructing an affine candidate by combining translational motion information from each control point in space and time.

[0185] Figure 11 The illustration shows an example of affine motion information storage used for motion information inheritance.

[0186] Figure 12 The illustration shows an example of deriving weight values ​​for combining inter-frame and intra-frame prediction (CIIP) based on the coding patterns of the top and left adjacent blocks.

[0187] Figure 13 The illustration shows an example of model parameter derivation for CCLM (Cross Component Linear Model) using adjacent chroma samples and adjacent luminance samples.

[0188] Figure 14 The illustration shows an example of boundary samples and neighboring samples used to derive the boundary matching cost of candidate patterns.

[0189] Figure 15 An exemplary multiple prediction hypothesis is shown, which includes the existing hypothesis as well as two hypotheses predicting h1 and predicting h2.

[0190] Figure 16 The illustration shows a flowchart of an exemplary video coding system utilizing multiple predictive hypotheses according to an embodiment of the present invention.

[0191] Figure 17 The illustration shows a flowchart of another exemplary video coding system utilizing multiple predictive hypotheses according to an embodiment of the present invention. Detailed Implementation

[0192] It is readily understood that the components of the present invention, as generally described and illustrated in the accompanying drawings, can be arranged and designed in a variety of different configurations. Therefore, the following more detailed description of embodiments of the systems and methods of the present invention as illustrated in the drawings is not intended to limit the scope of the claimed invention, but only represents selected embodiments of the invention. References throughout the specification to “one embodiment,” “an embodiment,” or similar language mean that a particular feature, structure, or characteristic described in connection with that embodiment may include in at least one embodiment of the invention. Therefore, the phrases “in one embodiment” or “in an embodiment” appearing in various places throughout the specification do not necessarily refer to the same embodiment.

[0193] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. However, those skilled in the art will recognize that the invention can be practiced without one or more specific details, or using other methods, components, etc. In other instances, well-known structures or operations are not shown or not illustrated. Detailed description is provided to avoid obscuring aspects of the invention. The illustrated embodiments of the invention will be best understood by referring to the accompanying drawings, wherein like parts are represented by like numbers throughout. The following description is by way of example only and simply illustrates certain selected embodiments of devices and methods consistent with the invention as claimed herein.

[0194] With improvements in video encoding and decoding, more encoding and decoding tools have been developed. However, the encoding gain of new encoding and decoding tools is not additive. A possible reason is...

[0195] - Due to syntactic overhead, not all new encoding / decoding modes can be candidate modes for blocks.

[0196] - As the number of candidate modes in a block increases, longer codewords are needed to indicate the encoding / decoding mode from multiple candidate modes.

[0197] Taking merge mode as an example. Compared to HEVC, it is recommended to add new merge candidates (e.g., pairwise average merge candidates, HMVP merge candidates, etc.) to the merge candidate list. As mentioned earlier, the index of the best merge candidate is encoded / decoded to indicate the merge candidate selected for the current block. However,

[0198] - The number of merge candidates in the merge candidate list is limited to a predefined number, so not all merge candidates can be added to the merge candidate list.

[0199] - If the number of merge candidates in the merge candidate list increases, the codeword length of the best merge candidate index becomes longer.

[0200] Therefore, a scheme is proposed to adaptively reorder candidate patterns. The general rules are as follows.

[0201] - Calculate the cost of each candidate pattern, and may (but is not limited to) merge candidate patterns and / or extend to candidate patterns of any other tool.

[0202] Next, the priority of these candidate patterns will be determined based on cost.

[0203] In one embodiment, candidate patterns with lower costs receive higher priority.

[0204] In another embodiment, candidate patterns with lower costs are given lower priority.

[0205] Next, the candidate patterns are reordered according to priority.

[0206] Smaller index values ​​indicate higher priority candidate patterns.

[0207] In other words, an index refers to a reordered candidate pattern.

[0208] Initially, the value of the index refers to the index of the candidate pattern.

[0209] After applying the suggested reordering scheme, the index value refers to the reordered index of the candidate pattern.

[0210] In another embodiment, shorter codewords are used for encoding / decoding candidate patterns with higher priority. In yet another embodiment, the highest priority candidate pattern is implicitly set as the encoding pattern for the current block. In this invention, the scheme for determining priority includes boundary matching. For each candidate pattern, a boundary matching cost is calculated. The priority depends on the boundary matching costs of these candidate patterns.

[0211] Boundary matching cost of candidate patterns

[0212] The boundary matching cost of a candidate pattern refers to the current prediction generated from the candidate pattern. Figure 14 The boundary matching cost of a candidate pattern represents the discontinuity between the current prediction (shown as a thick outline within the current block 1410) and adjacent reconstructions (reconstruction samples within one or more adjacent blocks), including top boundary matching and / or left boundary matching. For example, the boundary matching cost of a candidate pattern represents the discontinuity between the current prediction and adjacent reconstructions. Top boundary matching refers to the comparison between the current top prediction sample and the adjacent top reconstruction sample 1420, and left boundary matching refers to the comparison between the current left prediction sample and the adjacent left reconstruction sample 1430.

[0213] In one embodiment, the boundary matching cost is calculated using a predefined subset of the current predictions. This is achieved using n1 rows of the top boundary within the current block and / or m1 columns of the left boundary within the current block, where n1 and m1 are integers greater than or equal to 0. Additionally, n2 rows of the top adjacent reconstruction and / or m2 columns of the left adjacent reconstruction are used, where n2 and m2 are integers greater than or equal to 0.

[0214] Here is an example of calculating the boundary matching cost for n1=2, m1=2, n2=2, and m2=2.

[0215]

[0216] In the above equation, the weights (a, b, c, d, e, f, g, h, i, j, k, l) can be any positive integers, for example a = 2, b = 1, c = 1, d = 2, e = 1, f = 1, g = 2, h = 1, i = 1, j = 2, k = 1, l = 1.

[0217] This is another example of calculating the boundary matching cost for n1=2, m1=2, n2=1, and m2=1.

[0218]

[0219] In the above equation, the weights (a,b,c,g,h,i) can be any positive integers, such as a=2, b=1, c=1, g=2, h=1, i=1.

[0220] This is another example of calculating the boundary matching cost for n1=2, m1=1, n2=2, and m2=2.

[0221]

[0222] In the above equation, the weights (d,e,f,j,k,l) ​​can be any positive integers, such as d=2, e=1, f=1, j=2, k=1, l=1.

[0223] This is another example of calculating the boundary matching cost for n1=1, m1=1, n2=1, and m2=1.

[0224]

[0225] In the above equation, the weights (a,c,g,i) can be any positive integers, such as a=1, c=1, g=1, i=1.

[0226] This is another example of calculating the boundary matching cost for n1=2, m1=1, n2=2, and m2=1.

[0227]

[0228] In the above equation, the weights (a,b,c,d,e,f,g,i) can be any positive integers, such as a=2, b=1, c=1, d=2, e=1, f=1, g=1, i=1.

[0229] This is another example of calculating the boundary matching cost for n1=1, m1=2, n2=1, and m2=2.

[0230]

[0231] In the above equation, the weights (a,c,g,h,i,j,k,l) ​​can be any positive integers, such as a=1, c=1, g=2, h=1, i=1, j=2, k=1, l=1.

[0232] The examples below for n1 and m1 can also be applied to n2 and m2.

[0233] For example, nl can be any positive integer, such as 1, 2, 3, 4, etc.

[0234] For example, m1 can be any positive integer, such as 1, 2, 3, 4, etc.

[0235] For example, n1 and / or m1 vary with the block width, height, or area.

[0236] As an example, for larger blocks (e.g., area > threshold), a larger m1 is used.

[0237] Threshold = 64, 128 or 256.

[0238] When the area exceeds the threshold, m1 increases to 2. (Initially, m1 is 1.)

[0239] When the area exceeds the threshold, m1 increases to 4. (Initially, m1 is 1 or 2.)

[0240] According to another example, for taller blocks (e.g., height > threshold * width), use a larger m1 and / or a smaller n1.

[0241] Threshold = 1, 2 or 4.

[0242] When the height exceeds the threshold multiplied by the width, m1 increases to 2. (Initially, m1 was 1.)

[0243] When the height exceeds the threshold multiplied by the width, m1 increases to 4. (Originally, m1 was 1 or 2.)

[0244] According to another example, for larger blocks (e.g., area > threshold), use a larger n1.

[0245] Threshold = 64, 128 or 256.

[0246] When the area exceeds the threshold, n1 increases to 2. (Initially, n1 is 1.)

[0247] When the area exceeds the threshold, n1 increases to 4. (Initially, n1 is 1 or 2.)

[0248] According to another example, for wider blocks (e.g., width > threshold * height), use a larger n1 and / or a smaller m1.

[0249] Threshold = 1, 2 or 4.

[0250] When the width exceeds the threshold multiplied by the height, n1 increases to 2. (Originally, n1 was 1.)

[0251] When the width exceeds the threshold multiplied by the height, n1 increases to 4. (Originally, n1 was 1 or 2.)

[0252] For another example, nl and / or ml can be defined in the standard or depend on the signaling / parsing of the syntax from CU / CB, PU / PB, TU / TB, CTU / CTB, slice level, picture level, SPS (sequence parameter set) level and / or PPS (picture parameter set) level.

[0253] In another embodiment, when the current block is located at the top boundary within a CTU row, top boundary matching is not used and / or only left boundary matching is used. (Adjacent reconstructed samples across CTU rows are not used.)

[0254] In another embodiment, when the current block is located at the left boundary within the CTU, left boundary matching is not used and / or only upper boundary matching is used.

[0255] In another embodiment, when the current block is high (e.g., height > threshold * width), only the left boundary is used for matching.

[0256] In another embodiment, when the current block is wider (e.g., width > threshold * height), only the top boundary is used for matching.

[0257] In another embodiment, the top-left adjacent reconstructed sample can be used for boundary matching. For example, the boundary matching cost is added to the following:

[0258] |reco -1,1 -pred 0,0 |

[0259] Merging candidates as candidate patterns

[0260] The proposed scheme can be used to reorder merge candidates. Normally, the index of the best merge candidate refers to its order in the merge candidate list. For the proposed scheme, the index of the best merge candidate (index_best_merge) refers to the priority order based on boundary matching costs.

[0261] For example, suppose the list of candidate merges consists of {cand0, cand1, cand2, cand3, cand4, cand5}. These predicted candidates are referred to as the predicted candidate group in this disclosure.

[0262] original bill:

[0263] index_best_merge 0 refers to cand0

[0264] index_best_merge 1 refers to cand1

[0265] index_best_merge 2 refers to cand2

[0266] ...

[0267] Recommended:

[0268] Calculate the boundary matching cost {cost_cand0, cost_cand1, cost_cand2, ...} for each merge candidate.

[0269] cost_cand0 is the boundary matching cost of cand0.

[0270] cost_cand1 is the boundary matching cost of cand1.

[0271] cost_cand2 is the boundary matching cost of cand2.

[0272] ...

[0273] Reorder {cand0, cand1, cand2, ...} based on boundary matching cost.

[0274] In one example

[0275] If cost_cand0>cost_cand1>cost_cand2>cost_cand3>cost_cand4>cost_cand5, then the reordered merge candidates form {cand5,cand4,cand3,cand2,cand1,cand0}.

[0276] After reordering,

[0277] `index_best_merge` 0 refers to `cand5` (the merge candidate with the lowest cost is signaled using the shortest codeword).

[0278] index_best_merge 1 refers to cand4

[0279] index_best_merge 2 refers to cand3

[0280] ...

[0281] Otherwise, if cost_cand0 < cost_cand1 < cost_cand2 < cost_cand3 < cost_cand4 < cost_cand5, the order of the merge candidates remains the same as the original order.

[0282] In another example,

[0283] If cost_cand0 < cost_cand1 < cost_cand2 < cost_cand3 < cost_cand4 < cost_cand5, the re-ordered merge candidates are formed as {cand5, cand4, cand3, cand2, cand1, cand0}.

[0284] After re-ordering,

[0285] index_best_merge 0 refers to cand5 (the merge candidate with the largest cost signals with the shortest codeword).

[0286] index_best_merge 1 refers to cand4

[0287] index_best_merge 2 refers to cand3

[0288] ……

[0289] Otherwise, if cost_cand0 > cost_cand1 > cost_cand2 > cost_cand3 > cost_cand4 > cost_cand5, the order of the merge candidates remains the same as the original.

[0290] In another embodiment, the index_best_merge with a smaller value is encoded with a shorter codeword length. For example, index_best_merge is encoded with a truncated unary codeword.

[0291] In another embodiment, re-ordering is applied to a subset of the merge candidate list.

[0292] - For example, the subset refers to the original first n candidates cand0, cand1, cand2. Then index_best_merge0 / 1 / 2 refers to the priority order based on boundary matching, and index_best_merge 3 / 4 / 5 refers to the original cand3, cand4, and cand5.

[0293] - For example, a subset refers to the original last n candidates cand3, cand4, and cand5. Then index_best_merge 3 / 4 / 5 refers to the priority order based on boundary matching, and index_best_merge 0 / 1 / 2 refers to the original cand0, cand1, and cand2.

[0294] -For example, a subset refers to a candidate for spatial merging.

[0295] In another embodiment, the best merge candidate is inferred to be the merge candidate with the minimum boundary matching cost among all merge candidates. Therefore, index_best_merge does not need to be signaled / parsed by the encoder / decoder and can be inferred to be 0.

[0296] In another embodiment, the best merge candidate is inferred to be the merge candidate with the highest boundary matching cost among all merge candidates. Therefore, index_best_merge does not need to be signaled / parsed by the encoder / decoder and can be inferred to be 0.

[0297] In another embodiment, the merge candidate in this section refers to a merge candidate that includes one or more of the following candidates:

[0298] 1) Spatial MVP from spatially adjacent CUs

[0299] 2) Time MVP from co-located CU

[0300] 3) Historical MVPs from FIFO tables

[0301] 4) Paired average MVP

[0302] 5) Zero MV.

[0303] In another embodiment, the merge candidates in this section refer to the merge candidates of CIIP. The predicted samples within the current block are generated according to the CIIP process.

[0304] In another embodiment, the merge candidate in this section refers to a merge candidate of sub-block merge candidates, such as an affine merge candidate. The predicted samples within the current block are generated according to an affine process.

[0305] BCW as a candidate mode

[0306] The proposed scheme can be used to reorder BCW candidates to improve the syntax of bcw_idx. These two bidirectional prediction candidates are referred to as the prediction candidate group in this disclosure. Below is the syntax table of BCW in the VVC standard.

[0307]

[0308] bcw_idx[x0][y0] specifies the weight index of the bidirectional prediction with CU weights. The array index x0, y0 specifies the position (x0, y0) of the top-left luminance sample of the coded block under consideration relative to the top-left luminance sample of the image.

[0309] The following is an example of BCW as a candidate pattern:

[0310] - Calculate the boundary matching cost for each BCW candidate weight.

[0311] -In one embodiment, after reordering according to cost

[0312] bcw_idx equal to 0 indicates that the BCW candidate weight is the smallest, and bcw_idx equal to 4 indicates that the BCW candidate weight is the largest.

[0313] In another approach, bcw_idx is implicit and uses BCW candidate weights (with minimum cost).

[0314] -In another embodiment, after reordering according to cost

[0315] bcw_idx equal to 0 indicates that the BCW candidate weight is the largest, and bcw_idx equal to 4 indicates that the BCW candidate weight is the smallest.

[0316] In another approach, bcw_idx is implicit and uses BCW candidate weights (with maximum cost).

[0317] In the embodiment where BCW is used as a candidate mode, the boundary matching cost is evaluated for a set of possible weights used to select the final combined prediction. The predictions associated with each combined BCW prediction (i.e., the selected weights) are called combined prediction members. The total number of combined prediction members corresponds to the total number of weights. bcw_idx is signaled / parsed or implicitly referenced for the final combined prediction. An ordered boundary matching cost should result in a more efficient representation of bcw_idx, i.e., a lower bit rate.

[0318] MHP as a candidate mode

[0319] For a block to which MHP is applied, one or more prediction hypotheses (i.e., prediction signals) are combined with existing prediction hypotheses to form the final (outcome) prediction for the current block. In other words, the final prediction is iteratively accumulated with each additional hypothesis to the prediction signal. An example of accumulation is shown below.

[0320] p n+1 =(1-α) n+1 )p n +α n+1 h n+1

[0321] The obtained prediction signal serves as the final p n (i.e., the p with the largest index n n ). p0 is the first (existing) prediction of the current block. In the case where MHP is a candidate mode, initially the weighted sum of the existing prediction and the prediction hypothesis is formed as the combined prediction. When there are two or more prediction hypotheses (i.e., n > 1), the current combined prediction is formed as the weighted sum of the previous combined prediction and the next prediction hypothesis. For example, if MHP is applied to a merge candidate, p0 is indicated by the existing merge index. The additional prediction is denoted as h and will be further combined with the previously accumulated prediction by a weight α.

[0322] Therefore, for each additional hypothesis of the prediction, a weight index is sent / parsed to indicate the weighted sum and / or for each hypothesis of the prediction, an inter-frame index is sent / parsed to indicate the motion candidate (for generating the prediction samples for that hypothesis). The syntax overhead of MHP can be a problem in some cases. For example, when the number of additional hypotheses of the prediction increases, the syntax overhead of MHP becomes larger. Another example is when the number of candidate weights for each additional hypothesis of the prediction increases, the syntax overhead of MHP becomes larger. Therefore, in order to make the signal of the MHP weights and / or the signal of the motion candidates more efficient, a reordering scheme needs to be proposed.

[0323] In one embodiment, a boundary matching cost for each MHP candidate weight is calculated for each additional prediction hypothesis. For the case of two candidate weights for a prediction hypothesis, an example of the proposed process is as follows:

[0324] Step 0: For the additional hypothesis of the prediction, calculate cost_w0 and cost_w1 as the costs of the first and second candidate weights respectively.

[0325] Step 1: For the prediction hypothesis, the candidate weights are reordered according to the cost.

[0326] In a sub - embodiment, the candidate weight with a smaller cost gets a higher priority.

[0327] If cost_w0 > cost_w1, weight index 0 refers to w1 and weight index 1 refers to w0.

[0328] Otherwise, no reordering is used and weight indices 0 and 1 refer to the original w0 and w1.

[0329] In another sub - embodiment, the candidate weight with a larger cost gets a higher priority.

[0330] If cost_w0 < cost_w1, then weight index 0 refers to w1 and weight index 1 refers to w0.

[0331] Otherwise, without reordering, weight indices 0 and 1 refer to the original w0 and w1.

[0332] In another sub-implementation, the candidate weights with the lowest cost are used for the current additional prediction hypothesis. In this case, the weight index of the current additional prediction hypothesis is inferred.

[0333] In another sub-implementation, the weight of each hypothesis or any subset of hypotheses is implicitly set based on the cost. The weight can be a scaled value of the cost or a scaled value of the multiplicative inverse of the cost. For example, if the cost equals 2, then the multiplicative inverse of the cost equals 1 / 2.

[0334] In another sub-implementation, the candidate weight with the highest cost is used for the current additional prediction hypothesis. In this case, the weight index of the current additional prediction hypothesis is inferred.

[0335] Repeat steps 0 and 1 for each additional prediction hypothesis and obtain the meaning of each weight index for each additional prediction hypothesis.

[0336] In another embodiment, a boundary matching cost is calculated for each MHP motion candidate. The following sub-example uses MHP as the merging method. MHP can be applied to other inter-frame modes, such as inter-frame AMVP and / or affine, and when using inter-frame AMVP or affine, "merge" in the following examples will be replaced with the name of that inter-frame mode.

[0337] In one sub-implementation, a boundary matching cost is calculated for each motion candidate. For example, candidate patterns include cand0 through cand4. Initially, index 0 (the shorter codeword) refers to cand0, and index 4 (the longer codeword) refers to cand4. Using the proposed method, the meaning of the indices follows priority. If the priority order (based on boundary matching cost) specifies that cand4 has the highest priority, then index 0 is mapped to cand4.

[0338] In another sub-implementation, the boundary matching cost for each motion candidate is calculated. For example, candidate patterns include cand0 to cand4. Initially, index 0 (the shorter codeword) refers to cand0, and index 4 (the longer codeword) refers to cand4. Using the proposed method, the meaning of the indices follows priority. If the priority order (based on the boundary matching cost) indicates that cand4 has the highest priority, then no index is sent / resolved, and the selected motion candidate is inferred to be the motion candidate with the highest priority.

[0339] In another sub-implementation, for each hypothesis, the current prediction of the motion candidate (used to calculate the boundary matching cost) is the motion compensation result generated by that motion candidate.

[0340] Examples combining the two sub-implementations above are shown below. For example, if the number of additive hypotheses is equal to 2, the top three motion candidates with higher priority are used to form the result prediction for the current MHP block. As another example, if the number of additional hypotheses is equal to 2, the existing hypotheses are retained, and the two motion candidates with higher priority are used to form the prediction for the additional hypotheses; the resulting prediction is formed by the existing hypotheses and the additional hypotheses.

[0341] In another sub-implementation, the current prediction of the motion candidate (used to calculate the boundary matching cost) is a combined prediction (i.e., a weighted average of the predictions from the motion candidate and the existing prediction (p0)). The weights are indicated by the weight index of the signal / parse.

[0342] In another sub-implementation, for each hypothesis, the current prediction of the motion candidate (used to calculate the boundary matching cost) is a combined prediction (a weighted average of the predictions from the motion candidate and the previously accumulated predictions). The weights are indicated by the weight index of the signal / parse. Figure 15 An example is shown, where prediction p0 1510 corresponds to the existing hypothesis, and predictions h1 1520 and h2 1530 correspond to additional hypotheses. When the proposed method is applied to reorder the motion candidate signals of h2, h2 has 4 motion candidates (including cand0 to cand3).

[0343] – The cost of CANDN is calculated as the weighted average of the p0 prediction, h1 prediction, and CANDN prediction.

[0344] That is,

[0345] For h2, the current prediction of candidate 0 is the weighted average of (p0 prediction, h1 prediction, and candidate 0 prediction).

[0346] For h2, the current prediction of candidate 1 is the weighted average of (p0 prediction, h1 prediction, and candidate 1 prediction).

[0347] o……

[0348] – When the proposed method is applied to reorder the signaling of the motion candidates of hl and hl has 4 motion candidates (including cand0 to cand3),

[0349] For h1, the current prediction of candidate 0 is the weighted average of (the prediction of p0 and the prediction of candidate 0).

[0350] For h1, the current prediction of candidate 1 is the weighted average of (the prediction of p0 and the prediction of candidate 1).

[0351] o……

[0352] In another embodiment, a boundary matching cost is calculated for each MHP combination (motion candidate and weight) for each prediction hypothesis. The following sub-example uses a merged MHP mode as an example; MHP can be applied to other inter-frame modes, such as inter-frame AMVP and / or affine. When using inter-frame AMVP or affine, "merge" in the following examples will be replaced with the name of that inter-frame mode.

[0353] In one sub-implementation, a combination refers to a motion candidate and its weights. If there are m motion candidates and n weights for each motion candidate, the number of combinations is m*n.

[0354] In another sub-implementation, the current prediction of the combination (used to calculate the boundary matching cost) is the combined prediction (a weighted average of predictions from motion candidates and the existing prediction (p0)). This method jointly determines the merge index and the weights representing additional prediction hypotheses. For example, the combination with the highest priority is the selected MHP combination. Signaling / resolving the merge index and weights to indicate additional prediction hypotheses is not required. In another example, the joint index is signaled / resolved to determine the MHP combination. The number of added hypotheses can be fixed in this sub-implementation.

[0355] In another embodiment, a boundary matching cost is calculated for each MHP motion candidate. The following example uses the merged MHP mode; MHP can be applied to other inter-frame modes, such as inter-frame AMVP and / or affine. When using inter-frame AMVP or affine, "merge" in the following text will be replaced with the name of that inter-frame mode. Therefore,

[0356] – Inferring merge index (used to indicate motion candidates for each hypothesis).

[0357] For example, merging candidates according to the order in the merge candidate list.

[0358] To merge cand0 for hypothesis 0, merge cand1 for hypothesis 1, ...

[0359] For example, it depends on cost.

[0360] First, use the merge candidates with lower costs.

[0361] For example, based on a predefined number of merge candidates

[0362] If the number of hypotheses is 4, then the 4 merge candidates from the merge candidate list are used as the motion candidates for each hypothesis.

[0363] √ Use the first 4 merge candidates, or

[0364] √ Use any 4 merge candidates from the merge candidate list.

[0365] – The weights (used to combine forecasting assumptions) are implicit based on the cost.

[0366] –MHP prediction is calculated using (weight 0)*(assumption 0) + (weight 1)*(assumption 1) + (weight 2)*(assumption 2) + ...

[0367] In one sub-implementation, a fixed number of prediction hypotheses are used. In other words, a fixed number of hypotheses are mixed and weights are implicitly derived using matching costs.

[0368] In another sub-implementation, the weight of a motion candidate (or hypothesis) with higher priority is greater than the weight of a motion candidate with lower priority.

[0369] In another sub-implementation, the current prediction of the motion candidate (used to calculate the boundary matching cost) is the motion compensation result generated by the motion candidate.

[0370] In another sub-implementation, the top n motion candidates in the merged candidate list are used to generate the hypothesis for prediction. Using this proposed method, no merge index signal is issued for MHP.

[0371] In another sub-implementation, all motion candidates in the merged candidate list are used to generate the hypothesis for the prediction. Weights determine whether a motion candidate is useful. If its weight is zero, the motion candidate is not actually used. Using this proposed method, no merge index signal is issued for the MHP.

[0372] In another sub-implementation, the weight of a motion candidate with higher priority is greater than the weight of a motion candidate with lower priority.

[0373] For example, the weights follow the cost ratio of different motion candidates. If there are two motion candidates and cost_cand0 = 2 * cost_cand1, then weight_cand0 = 2 * weight_cand1 or weight_cand0 = 1 / 2 * weight_cand1.

[0374] In another example, the cost of each motion candidate is first normalized to an interval [MIN_VALUE, MAX_VALUE]. MAX_VALUE is predefined, such as the number of prediction hypotheses. MIN_VALUE is predefined, such as 0. For example, (MAX_VALUE - normalized cost) could be the weight of the motion candidate, or the normalized cost could be the weight of the motion candidate.

[0375] In another example, the weight is a scaled value of the cost or a scaled value of the multiplicative inverse of the cost. For example, if the cost is equal to 2, then the multiplicative inverse of the cost is equal to 1 / 2.

[0376] - The scaling value represents the scaling factor multiplied by the original value.

[0377] - If the scaling factor = 1, then there is no scaling.

[0378] In another sub-implementation, weights and merge indexes are implicit in the proposed method.

[0379] The generation of the current predictions by this method can refer to any other method proposed in this invention.

[0380] In another embodiment, the proposed scheme is applied to a subset of all additional prediction hypotheses. In other words, steps 0 and 1 above are repeated for all subsets of additional prediction hypotheses. For example, only the candidate weights of the first additional prediction hypothesis combined with the existing prediction hypotheses are reordered using the proposed scheme.

[0381] In one sub-implementation, the subset is predefined in the standard.

[0382] In another sub-implementation, the subset depends on the width, height, or area of ​​the current block. For example, for blocks with an area greater than (or less than) a threshold, this subset includes more predictive assumptions.

[0383] In another sub-implementation, the reordering results from the subset can be reused for the remaining additional predictive hypotheses. For example,

[0384] - Based on the reordering results of the first prediction hypothesis, weight indices 0 and 1 refer to w1 and w0, respectively.

[0385] - For the following prediction assumptions, weight indices 0 and 1 also refer to w1 and w0, respectively.

[0386] In another embodiment, the proposed scheme is applied to a subset of all candidate weights for additional prediction hypotheses. In other words, steps 0 and 1 above are repeated for all subsets of candidate weights with additional prediction hypotheses. For example, consider a candidate weight count (for the additional hypothesis for prediction) of 4.

[0387] - For additional prediction assumptions, only the first two (or last) candidate weights are reordered using the proposed scheme.

[0388] In one sub-implementation, the subset is predefined in the standard.

[0389] In another sub-implementation, the subset depends on the width, height, or area of ​​the current block. For example, for blocks with an area greater than (or less than) a threshold, the subset includes more candidate weights.

[0390] In another embodiment, the prediction hypothesis may be a prediction signal derived from the results of single or dual prediction motion compensation.

[0391] Reordering schemes proposed for different tools (not limited to those in the examples below) can be standardized. For example, reordering schemes proposed for MHP, LM, BCW, MMVD, and / or merge candidates can be standardized with the same rules for calculating boundary matching costs.

[0392] The methods proposed in this invention can be enabled and / or disabled based on implicit rules (such as block width, height, or area) or explicit rules (such as syntax regarding blocks, slices, images, SPS, or PPS levels). For example, when a block area is smaller than a threshold, a suggested reordering is applied.

[0393] In this invention, the term "block" may refer to TU / TB, CU / CB, PU / PB, or CTU / CTB.

[0394] The term "LM" in this invention can be considered as a CCLM / MMLM mode or any other extension / variation of CCLM (e.g., the CCLM extension / variation proposed in this invention). Cross-component tools (e.g., the CCLM extension / variation proposed in this invention). The variation here means that when the block indication refers to the use of one of the cross-component modes (e.g., CCLM_LT, MMLM_LT, CCLM_L, CCLM_T, MMLM_L, MMLM_T and / or intra-prediction modes), some optional modes can be selected, which are not a traditional DC, planar, and angular mode for the current block. An example of using the Convolutional Cross-Component Mode (CCCM) as an optional mode is shown below. When this optional mode is applied to the current block, the cross-component information of the model (including nonlinear terms) is used to generate chroma predictions. The optional mode may follow the template selection of CCLM, thus the CCCM family includes CCCM_LT, CCCM_L, and / or CCCM_T.

[0395] The method proposed in this invention (for CCLM) can be used for any other LM mode.

[0396] Any combination of the methods proposed in this invention can be applied.

[0397] Any of the methods proposed above can be implemented in the encoder and / or decoder. For example, any of the proposed methods can be implemented in the intra-frame / inter-frame coding module of the encoder (e.g., Figure 1A Inter-frame prediction 112), motion compensation module (e.g.) Figure 1B In the MC 152), the candidate derivation module of the merge decoder. Alternatively, any of the proposed methods can be implemented as a circuit coupled to the intra / inter-frame coding module and / or motion compensation module of the encoder, and the candidate derivation module of the merge decoder.

[0398] Figure 16 A flowchart illustrating an exemplary video encoding / decoding system utilizing multiple prediction hypotheses according to an embodiment of the present invention is shown. The steps shown in the flowchart can be implemented as program code executable on one or more processors (e.g., one or more CPUs) at the encoder side. The steps shown in the flowchart can also be implemented in hardware, such as one or more electronic devices or processors arranged to perform the steps in the flowchart. According to the method, in step 1610, input data associated with the current block is received, wherein the input data includes pixel data of the current block to be encoded at the encoder side or encoded data associated with the current block to be decoded at the decoder side. In step 1620, combined prediction members are determined, wherein each combined prediction member includes a weight sum of a first prediction candidate and a second prediction candidate using target weights selected from a weight set. In step 1630, boundary matching costs associated with the combined prediction members are determined, wherein each boundary matching cost is determined for the combined prediction members with target weights using prediction samples of the current block and neighboring reconstructed samples based on the combined prediction members with target weights. In step 1640, the current block is encoded or decoded using a final combined prediction determined based on at least one boundary matching cost.

[0399] Figure 17 A flowchart illustrating another exemplary video coding system utilizing multiple prediction hypotheses according to an embodiment of the present invention is shown. According to the method, in step 1710, input data associated with the current block is received, wherein the input data includes pixel data of the current block to be encoded on the encoder side or encoded data associated with the current block to be decoded on the decoder side. In step 1720, prediction members associated with existing predictions are determined, wherein each prediction member corresponds to a weight sum of an existing prediction and a target combined prediction candidate from a set of combined prediction candidates, and wherein each combined prediction candidate corresponds to a combination of a target motion candidate and a target weight for combining the existing prediction and the target combined prediction candidate, the target motion candidate being from a set of m motion candidates and the target weight being from a set of n weights, where m and n are positive integers. In step 1730, boundary matching costs associated with the prediction members are determined, wherein each boundary matching cost for each prediction member is determined by using prediction samples of the current block based on each prediction member and neighboring reconstruction samples of the current block. In step 1740, the current block is encoded or decoded using a final prediction determined based on at least one boundary matching cost.

[0400] The flowchart shown is intended to illustrate an example of video encoding according to the present invention. Those skilled in the art can modify, rearrange, split, or combine each step to implement the invention without departing from its spirit. Specific syntax and semantics have been used in this disclosure to illustrate examples of implementing embodiments of the invention. Those skilled in the art can implement the invention by substituting equivalent syntax and semantics without departing from its spirit.

[0401] The above description is provided to enable those skilled in the art to practice the invention in the context of specific applications and their requirements. Various modifications to the described embodiments will be apparent to those skilled in the art, and the general principles defined herein can be applied to other embodiments. Therefore, the invention is not intended to be limited to the specific embodiments shown and described, but is accorded the widest scope consistent with the principles and novel features disclosed herein. In the above detailed description, various specific details have been illustrated to provide a thorough understanding of the invention. However, those skilled in the art will understand that the invention can be practiced.

[0402] The embodiments of the invention described above can be implemented in various hardware, software code, or combinations thereof. For example, one embodiment of the invention may be one or more circuits integrated into a video compression chip or program code integrated into video compression software to perform the processes described herein. Embodiments of the invention may also be program code to be executed on a digital signal processor (DSP) to perform the processes described herein. The invention may also relate to numerous functions performed by a computer processor, digital signal processor, microprocessor, or field-programmable gate array (FPGA). These processors may be configured to perform specific tasks according to the invention by executing machine-readable software code or firmware code that defines specific methods embodied in the invention. The software code or firmware code may be developed in different programming languages ​​and in different formats or styles. The software code may also be compiled for different target platforms. However, different code formats, styles, and languages ​​of the software code, as well as other ways of configuring the code to perform the tasks according to the invention, do not depart from the spirit and scope of the invention.

[0403] The invention may be embodied in other specific forms without departing from its spirit or essential characteristics. The described examples should be considered illustrative rather than restrictive in all respects. Therefore, the scope of the invention is indicated by the appended claims rather than by the foregoing description. All variations falling within the meaning and scope of the equivalents of the claims should be included within their scope.

Claims

1. A predictive encoding / decoding method, the method comprising: Receive input data associated with the current block, wherein the input data includes pixel data of the current block to be encoded on the encoder side or encoded data associated with the current block to be decoded on the decoder side; Determine prediction members associated with existing predictions, wherein each prediction member corresponds to the weighted sum of the existing predictions and at least one target prediction candidate from the combination of prediction members, wherein the weights and the at least one target prediction candidate are jointly determined by a joint index; The current block is encoded or decoded using the final prediction determined based on the combined index.

2. The predictive encoding / decoding method according to claim 1, characterized in that, The joint prediction index indicates a combination of a target motion candidate and a target weight, the combination being used to combine the existing predictions with the at least one target prediction candidate, the target prediction candidate being a prediction from the target motion candidate, the target motion candidate being a combination from m motion candidates, and the target weight being a combination from n weights, where m and n are positive integers.

3. The predictive encoding / decoding method according to claim 1, characterized in that, The joint index is associated with the final predicted target motion candidate and the target weight, and is either signaled in the bitstream at the encoder end or parsed from the bitstream at the decoder end.

4. The predictive encoding / decoding method according to claim 1, characterized in that, The joint index is associated with the final predicted target motion candidate and the target weight, and is implicitly determined.

5. The predictive encoding / decoding method according to claim 1, characterized in that, It further includes determining the boundary matching cost associated with the predicted member, wherein each boundary matching cost of each predicted member is determined using the predicted sample of the current block and the neighboring reconstructed samples of the current block, wherein the predicted sample of the current block is generated based on each predicted member, and wherein the final prediction is determined based on the boundary matching cost.

6. An apparatus for predictive encoding / decoding, the apparatus comprising one or more electronic devices or processors arranged to: Receive input data associated with the current block, where, The input data includes pixel data of the current block to be encoded on the encoder side or encoded data associated with the current block to be decoded on the decoder side; Determine prediction members associated with existing predictions, wherein each prediction member corresponds to the weighted sum of the existing predictions and at least one target prediction candidate from the combination of prediction members, wherein the weights and the at least one target prediction candidate are jointly determined by a joint index; The current block is encoded or decoded using the final prediction determined based on the combined index.

Images