Method and apparatus for mode selection in a video coding system

Abstract

A method and apparatus for video coding. According to the method, a set of candidates associated with coding modes or coding parameters is determined. Boundary match costs associated with the candidate set are determined by using reconstructed or predicted samples of a current block and neighboring reconstructed or predicted samples, wherein each of the boundary match costs is determined for a target candidate of the candidate set. The reconstructed or predicted samples of the current block are determined according to the target candidate. The candidate set is reordered according to the boundary match costs. The current block is then encoded or decoded using a final candidate selected from the candidate set based on the reordering of the candidate set.

Description

[0001] Cross-referencing

[0002] This application claims priority to U.S. Provisional Patent Application No. 63 / 294,426, filed on December 29, 2021, which is incorporated herein by reference in its entirety. Technical Field

[0003] This invention relates to video encoding and decoding systems. In particular, this invention relates to reordering mode candidates based on boundary matching costs to improve encoding and decoding efficiency. Background Technology

[0004] Universal Video Coding (VVC) is the latest international video coding standard jointly developed by the Joint Video Experts Group (JVET) of the ITU-T Video Coding Experts Group (VCEG) and the ISO / IEC Moving Picture Experts Group (MPEG). This standard has been published as an ISO standard: ISO / IEC 23090-3:2021, Information technology—Coding representation of immersive media—Part 3: Universal Video Coding, published in February 2021. VVC improves upon its predecessor, HEVC (High Efficiency Video Coding), by adding more coding and decoding tools to enhance efficiency and enabling it to handle various types of video sources, including 3D video signals.

[0005] Figure 1A An exemplary adaptive inter-frame / intra-frame video coding system incorporating loop processing is described. For intra-frame prediction 110, prediction data is derived from previously encoded video data in the current frame. For inter-frame prediction 112, motion estimation (ME) is performed on the encoder side, and motion compensation (MC) is performed based on the results of ME to provide prediction data derived from other frames and motion data. Switch 114 selects either intra-frame prediction 110 or inter-frame prediction 112, and the selected prediction data is provided to adder 116 to form a prediction error, also known as a residual. The prediction error is then processed by transform (T) 118 and subsequent quantization (Q) 120. The residuals from the transform and quantization are then encoded by entropy encoder 122 to be included in the video bitstream corresponding to the compressed video data. The bitstream associated with the transform coefficients is then packaged with auxiliary information (e.g., motion and coding modes associated with intra-frame and inter-frame prediction) and other information (e.g., parameters associated with loop filters applied to the underlying image regions). Auxiliary information associated with intra-frame prediction 110, inter-frame prediction 112, and intra-loop filter 130 is provided to entropy encoder 122, such as... Figure 1AAs shown. When using inter-frame prediction mode, one or more reference images must also be reconstructed at the encoder end. Therefore, the transformed and quantized residuals are processed by inverse quantization (IQ) 124 and inverse transform (IT) 126 to recover the residuals. Then, at reconstruction (REC) 128, the residuals are added back to the prediction data 136 to reconstruct the video data. The reconstructed video data can be stored in the reference image buffer 134 and used to predict other frames.

[0006] like Figure 1A As shown, the input video data undergoes a series of processing steps in the encoding system. Due to these processing steps, the reconstructed video data from REC128 may suffer various forms of degradation. Therefore, loop filters 130 are often applied to the reconstructed video data before it is stored in the reference picture buffer 134 to improve video quality. For example, deblocking filters (DF), sample adaptive offset (SAO), and adaptive loop filters (ALF) can be used. It may be necessary to incorporate loop filter information into the bitstream so that the decoder can correctly recover the required information. Therefore, loop filter information is also provided to the entropy encoder 122 for incorporation into the bitstream. Figure 1A In this process, the loop filter 130 is applied to the reconstructed video before the reconstructed samples are stored in the reference image buffer 134. Figure 1A The system described herein is intended to illustrate an exemplary architecture of a typical video encoder. It may correspond to a High Efficiency Video Coding (HEVC) system, VP8, VP9, ​​H.264, or VVC.

[0007] like Figure 1B As shown, the decoder can use similar or identical function blocks to the encoder, except for transform 118 and quantization 120, since the decoder only needs inverse quantization 124 and inverse transform 126. Instead of entropy encoder 122, the decoder uses entropy decoder 140 to decode the video bitstream into quantized transform coefficients and the required coding information (e.g., ILPF information, intra-frame prediction information, and inter-frame prediction information). Intra-frame prediction 150 on the decoder side does not require mode search. Instead, the decoder only needs to generate intra-frame predictions based on the intra-frame prediction information received from entropy decoder 140. Furthermore, for inter-frame prediction, the decoder only needs to perform motion compensation (MC 152) based on the inter-frame prediction information received from entropy decoder 140 without motion estimation.

[0008] According to VVC, similar to HEVC, the input image is divided into non-overlapping square block regions called CTUs (Coding Tree Units). Each CTU can be further divided into one or more smaller coding units (CUs). The resulting CU partitions can be squares or rectangles. Furthermore, VVC divides the CTUs into prediction units (PUs) as units for applying prediction processes, such as inter-frame prediction, intra-frame prediction, etc.

[0009] The VVC standard incorporates various new encoding tools to further improve encoding efficiency beyond the HEVC standard. Among these new encoding tools, some relevant to this invention are summarized below.

[0010] Inter-frame prediction overview

[0011] According to Section 3.4 of JVET-T2002 (Jianle Chen et al., “Algorithm description of Universal Video Coding and Test Model 11 (VTM11)”, Joint Video Experts Group (JVET) of ITU-T SG 16WP 3 and ISO / IEC JTC 1 / SC 29, 20th Meeting, Teleconference, October 7–16, 2020, Archive: JVET-T2002), for each inter-frame prediction CU, motion parameters include motion vectors, reference picture indexes, and a reference picture list using identifiers, as well as additional information required by the new coding features of VVC for inter-frame prediction sample generation. Motion parameters can be signaled explicitly or implicitly. When a CU is coded in skip mode, the CU is associated with a PU and has no significant residual coefficients, no encoded motion vector increments, and no reference picture indexes. A merge mode is specified where the motion parameters of the current CU are obtained from neighboring CUs, including spatial and temporal candidates, as well as additional scheduling introduced in VVC. The merge mode can be applied to any inter-frame prediction CU, not just skip mode. An alternative to the merged mode is the explicit transmission of motion parameters, in which motion vectors, the corresponding reference image index for each reference image list, the reference image list identifier, and other necessary information are explicitly sent per CU.

[0012] In addition to the inter-frame coding features in HEVC, VVC also includes many new and improved inter-frame predictive coding tools, as listed below:

[0013] –Extended merge prediction

[0014] – Merge mode with MVD (MMVD)

[0015] -Symmetric MVD (SMVD) signaling

[0016] – Affine motion compensated prediction

[0017] – Subblock-based temporal motion vector prediction (SbTMVP)

[0018] -Adaptive motion vector resolution (AMVR)

[0019] – Sports field storage: 1 / 16 luminance sample MV storage and 8x8 sports field compression

[0020] –Bi-prediction with CU-level weight (BCW)

[0021] – Bidirectional optical flow (BDOF)

[0022] Decoder-side motion vector refinement (DMVR)

[0023] – Geometric partitioning mode (GPM)

[0024] – Combined inter and intra prediction (CIIP)

[0025] The following description provides details of the inter-frame prediction methods specified in VVC.

[0026] Extended merge forecast

[0027] In VVC, the merge candidate list is constructed by sequentially including the following five categories of candidates:

[0028] 1) Spatial MVP from spatially adjacent CUs

[0029] 2) Time MVP from co-located CU

[0030] 3) Historical MVPs from FIFO tables

[0031] 4) Paired average MVP

[0032] 5) Zero MV.

[0033] The size of the merge list is signaled in the Sequence Parameter Set (SPS) header, and the maximum allowed size of the merge list is 6. For each CU encoded in merge mode, the index of the best merge candidate is encoded (TU) using truncated univariate binarization. The first binary string of the merge index (hereinafter referred to as bin) is context-encoded, and bypass encoding is used for the remaining bins.

[0034] This section provides the derivation process for merge candidates for each category. Similar to HEVC, VVC also supports parallel derivation of the merge candidate list (or merge candidate list) for all CUs within a given region.

[0035] Spatial candidate derivation

[0036] Except for swapping the positions of the first two merge candidates, the derivation of spatial merge candidates in VVC is the same as in HEVC. Figure 2 At most four merge candidates (B0, A0, B1, and A1) of the current CU210 are selected from the candidates at the indicated positions. The derived order is B0, A0, B1, A1, and B2. Position B2 is considered only if one or more adjacent CUs of positions B0, A0, B1, and A1 are unavailable (e.g., belonging to another slice or tile) or are intra-coded. After adding the candidate for position A1, redundancy checks are performed on the addition of the remaining candidates to ensure that candidates with the same motion information are excluded from the list, thereby improving coding efficiency. To reduce computational complexity, not all possible candidate pairs are considered in the redundancy check mentioned above. Instead, only... Figure 3 The pairs are linked by arrows, and a candidate is added to the list only if the corresponding candidate used for redundancy checking does not have the same motion information.

[0037] Time Candidate Derivation

[0038] In this step, only one candidate is added to the list. Specifically, in the derivation of this time-merging candidate for the current CU 410, based on belonging to... Figure 4 The scaled motion vector is derived from the collocated reference picture's collocated CU 420. The reference picture list and reference index used to derive the collocated CU are explicitly sent as signals in the slice header. For example... Figure 4As shown by the dashed line, the scaled motion vector 430 of the temporal merge candidate is obtained, which is scaled from the motion vector 440 of the co-located CU using POC (Picture Order Count) distances tb and td, where tb is defined as the POC difference between the reference image and the current image, and td is defined as the POC difference between the reference image and the co-located image. The reference image index of the temporal merge candidate is set to zero.

[0039] The position of the time candidate is selected between candidate C0 and C1, such as... Figure 5 As shown. If the CU at position C0 is unavailable, intra-coded, or outside the current CTU line, then position C1 is used. Otherwise, position C0 is used to derive time-merging candidates.

[0040] Historical Merger Candidate Derivation

[0041] Historically based MVP (HMVP) merge candidates are added to the merge list following the spatial MVP and TMVP. In this method, motion information from previous coded blocks is stored in a table and used as the MVP for the current CU. A table with multiple HMVP candidates is maintained during encoding / decoding. This table is reset (cleared) when a new CTU row is encountered. Whenever a non-sub-block inter-coded CU exists, the associated motion information is added as the last entry in the table as a new HMVP candidate.

[0042] The HMVP table size S is set to 6, indicating that a maximum of 5 history-based MVP (HMVP) candidates can be added to the table. When a new motion candidate is inserted into the table, a constrained First-In-First-Out (FIFO) rule is used, where a redundancy check is first applied to find if a duplicate HMVP already exists in the table. If found, the duplicate HMVP is removed from the table, and all subsequent HMVP candidates are shifted forward, with the duplicate HMVP inserted as the last entry in the table.

[0043] HMVP candidates can be used in the candidate list construction process. The latest few HMVP candidates in the table are checked sequentially, and inserted into the candidate list after the TMVP candidates. Redundancy checks are applied to HMVP candidates for spatial or temporal merging.

[0044] To reduce the number of redundant check operations, the following simplification is introduced:

[0045] 1. The last two entries in the table perform redundancy checks on candidates in space A1 and B1, respectively.

[0046] 2. Once the total number of available merge candidates reaches the maximum allowed number of merge candidates minus 1, the HMVP merge candidate list construction process terminates.

[0047] Derivation of Pairwise Average Merge Candidates

[0048] Pairwise averaging candidates are generated by averaging predefined candidate pairs in the existing merge candidate list using the first two merge candidates. The first merge candidate is defined as p0Cand, and the second merge candidate can be defined as p1Cand. The average motion vector is calculated for each reference list based on the availability of motion vectors for p0Cand and p1Cand. If both motion vectors are available in a list, they are averaged even if they point to different reference images, and their reference images are set to one of p0C and p1C; if only one motion vector is available, it is used directly; if no motion vector is available, the list remains invalid. Furthermore, if the half-pixel interpolation filter indices of p0Cand and p1Cand are different, they are set to 0.

[0049] If the merge list is not full after adding pairwise average merge candidates, insert zero MVPs at the end until the maximum number of merge candidates is encountered.

[0050] Merging estimation areas

[0051] Merge Estimation Region (MER) allows for the independent derivation of merge candidate lists for Cues (CUs) within the same MER. Candidate blocks within the same MER as the current CU are not included in the generation of the current CU's merge candidate list. Furthermore, merge candidate lists are only generated if (xCb+cbWidth)>>Log2ParMrgLevel is greater than xCb>>Log2ParMrgLevel and (yCb+cbHeight)>>Log2ParMrgLevel is greater than (yCb>>Log2ParMrgLevel), where (xCb, yCb) is the top-left luminance sample position of the current CU in the image, and (cbWidth, cbHeight) is the CU size. The MER size is selected at the encoder end and signaled as log2_parallel_merge_level_minus2 in the Sequence Parameter Set (SPS).

[0052] Merge pattern with MVD (MMVD)

[0053] In addition to the merging pattern in which implicitly derived motion information is directly used in the generation of prediction samples for the current CU, a merging pattern with motion vector difference (MMVD) is introduced in VVC. An MMVD flag is issued immediately after the regular merging flag to specify whether the MMVD pattern is used for the CU.

[0054] In MMVD, after selecting a merging candidate (referred to as a basic merging candidate in this invention), it is further refined by MVD information transmitted via signals. This further information includes a merging candidate identifier, an index specifying the motion amplitude, and an index indicating the motion direction. In MMVD mode, one of the first two candidates in the merging list is selected as the MV basis. An MMVD candidate identifier is issued to specify which one to use between the first and second merging candidates.

[0055] The distance index specifies motion amplitude information and indicates a predefined offset from the starting points (612 and 622) of L0 reference block 610 and L1 reference block 620. For example... Figure 6 As shown, offsets are added to the horizontal or vertical component of the starting MV, where different styles of small circles correspond to different offsets from the center. The relationship between the distance index and the predefined offsets is specified in Table 1.

[0056] Table 1 – Relationship between Distance Index and Predefined Offset

[0057]

[0058] The direction index indicates the direction of the MVD relative to the starting point. The direction index can represent the four directions shown in Table 2. It is important to note that the meaning of the MVD sign can vary depending on the information of the starting MV. When the starting MV is a non-predictive MV or a double-predictive MV, and both lists point to the same side of the current image (i.e., both reference POCs are greater than or less than the current image's POC), the sign in Table 2 specifies the sign of the MV offset added to the starting MV. When the starting MV is a bidirectional predictive MV, and the two MVs point to different sides of the current image (i.e., one reference POC is greater than the current image's POC, and the other reference POC is less than the current image's POC), and the difference in POCs in list 0 (which can be simply referred to as list 0 or L0) is greater than the difference in POCs in list 1 (which can be simply referred to as list 1 or L1), the sign in Table 2 specifies that the sign of the MV offset of the MV component in list 0 added to the starting MV has the opposite value to the sign of the MV in list 1. Otherwise, if the difference of POC in List 1 is greater than that in List 0, then the sign in Table 2 specifies that the sign of the MV offset of the List 1 MV component added to the starting MV has the opposite value to the sign of the List 0 MV.

[0059] MVD is scaled based on the difference in POC in each direction. If the difference in POC is the same in both lists, no scaling is needed. Otherwise, if the difference in POC in list 0 is greater than the difference in list 1, the MVD of list 1 is scaled by defining the difference in POC in L0 as td and the difference in POC in L1 as tb, as shown below. Figure 5As shown. If the POC difference of L1 is greater than that of L0, the MVD of list 0 is scaled in the same way. If the initial MV is unidirectionally predicted, the MVD is added to the available MV.

[0060] Table 2 – MV Offset Symbols Specified by Direction Index

[0061] Direction Index 00 01 10 11 X-axis + - N / A N / A Y-axis N / A N / A + -

[0062] Affine Motion Compensation Prediction

[0063] In HEVC, only the translational motion model is applied to motion compensation prediction (MCP). However, in the real world, there are many types of motion, such as zooming in / out, rotation, perspective motion, and other irregular motions. In VVC, block-based affine transformation motion compensation prediction is applied, as shown in the figure. Figure 7A In -B, the affine motion domain of block 710 is determined by... Figure 7A Motion information of two control points (4 parameters) or Figure 7B The motion vectors (6 parameters) of the three control points are described.

[0064] For the 4-parameter affine motion model, the motion vector at the sample position (x, y) in the block is derived as:

[0065]

[0066] For the 6-parameter affine motion model, the motion vector at the sample position (x, y) in the block is derived as:

[0067]

[0068] Where (mv 0x ,mv 0y (mv) is the motion vector of the top-left control point. 1x ,mv 1y (mv) is the motion vector of the upper right control point. 2x ,mv 2y ) is the motion vector of the bottom-left corner control point.

[0069] To simplify motion compensation prediction, a block-based affine transformation prediction is applied. To derive the motion vector for each 4×4 brightness sub-block, the motion vector of the center sample of each sub-block is as follows: Figure 8 As shown, calculations are performed according to the above equations, and rounded to 1 / 16 fractional precision. Then, a motion-compensated interpolation filter is applied to generate predictions for each sub-block with derived motion vectors. The sub-block size for the chroma components is also set to 4×4. The MV of the 4×4 chroma sub-block is calculated as the average of the MVs of the top-left and bottom-right luminance sub-blocks in the juxtaposed 8x8 luminance region.

[0070] For translational motion inter-frame prediction, there are also two affine motion inter-frame prediction modes: affine merging mode and affine AMVP mode.

[0071] Affine Merge Prediction (AF_MERGE)

[0072] The AF_MERGE mode can be applied to CUs with a width and height greater than or equal to 8. In this mode, the CPMV (Control Point MV) of the current CU is generated based on the motion information of spatially adjacent CUs. There can be up to five CPMV prediction candidates, and an index is signaled to indicate which one should be used for the current CU. The following three types of CPMV candidates are used to construct the affine merging candidate list:

[0073] – Inherited affine merge candidates inferred from the CPMV of adjacent CUs

[0074] – Constructing affine merge candidate CPMVPs using translational MV derivations of adjacent CUs

[0075] -Zero MV

[0076] In VVC, there are at most two inherited affine candidates, which come from the affine motion models of neighboring blocks: one from the left-neighboring CU and one from the top-neighboring CU. Candidate blocks and Figure 2 The same applies. For the left predictor, the scan order is A0->A1, and for the upper predictor, the scan order is B0->B1->B2. Only the first inherited candidate for each side is selected. No pruning check is performed between two inherited candidates. When identifying adjacent affine CUs, their control point motion vectors are used to derive CPMVP candidates from the affine merge list of the current CU. Figure 9 As shown, if the lower-left adjacent block A of the current block 910 is encoded in affine mode, then the motion vectors v2, v3, and v4 of the upper-left corner, upper-right corner, and lower-left corner containing block A of CU 920 are obtained. When block A is encoded using a 4-parameter affine model, the two CPMVs (i.e., v0 and v1) of the current CU are calculated based on v2 and v3. When block A is encoded using a 6-parameter affine model, the three CPMVs of the current CU are calculated based on v2, v3, and v4.

[0077] Constructing affine candidates refers to building candidates by combining the translational motion information of adjacent control points. For example... Figure 10As shown, the motion information of the control point is derived from the specified spatial and temporal neighbors of the current block 1010. CPMVk (k = 1, 2, 3, 4) represents the k-th control point. For CPMV1, check the B2->B3->A2 block and use the MV of the first available block. For CPMV2, check the B1->B0 block, and for CPMV3, check the A1->A0 block. If the TMVP is available, it is used as CPMV4.

[0078] After obtaining the motion signatures (MVs) of the four control points, affine merging candidates are constructed based on motion information. The following combinations of control point MVs are used for sequential construction:

[0079] {CPMV1,CPMV2,CPMV3},{CPMV1,CPMV2,CPMV4},{CPMV1,CPMV3,CPMV4},{CPMV2,CPMV3,CPMV4},{CPMV1,CPMV2},{CPMV1,CPMV3}

[0080] Combinations of three CPMVs construct a 6-parameter affine merging candidate, and combinations of two CPMVs construct a 4-parameter affine merging candidate. To avoid motion scaling, combinations of control point MVs are discarded if the reference indices of the control points are different.

[0081] After checking the affine merge candidates of inheritance and construction, if the list is still not full, insert zero MVs at the end of the list.

[0082] Affine AMVP Prediction

[0083] The affine AMVP mode can be applied to CUs with a width and height greater than or equal to 16. A CU-level affine identifier is signaled in the bitstream to indicate whether the affine AMVP mode is used, and another identifier is signaled to indicate whether a 4-parameter affine or a 6-parameter affine is used. In this mode, the difference between the current CU's CPVM and its predictor variable CPMVP is signaled in the bitstream. The affine AMVP candidate list is of size 2 and is generated sequentially from the following four CPVM candidates:

[0084] – Inherited affine AMVP candidates inferred from the CPMV of adjacent CUs

[0085] – The constructed affine AMVP candidate CPMVP is derived using the translation MV of adjacent CUs.

[0086] – Translation MV from adjacent CU

[0087] -Zero MV

[0088] The checking order for inherited affine AMVP candidates is the same as that for inherited affine merge candidates. The only difference is that for AVMP candidates, only affine CUs with the same reference picture as the current block are considered. No pruning process is applied when inserting inherited affine motion predictions into the candidate list.

[0089] The constructed AMVP candidates are from Figure 10 The specified spatial neighbors are shown in the diagram. The reference picture indices of adjacent blocks are also checked using the same checking order as in the affine merge candidate construction. In the checking order, the first block using inter-frame coding and having the same reference picture as in the current CU is selected. If the current CU is coded using a 4-parameter affine mode and both mv0 and mv1 are available, they are added as a candidate to the affine AMVP list. If the current CU is coded using a 6-parameter affine mode and all three CPMVs are available, they are added as a candidate to the affine AMVP list. Otherwise, the constructed AMVP candidate is set to unavailable.

[0090] If the number of affine AMVP candidates is still less than 2 after inserting valid inherited affine AMVP candidates and constructed AMVP candidates, then mv0, mv1, and mv2 are added as translational MVs to predict all control point MVs of the current CU when available. Finally, if the affine AMVP list is still not full, it is filled with zero MVs.

[0091] Affine motion information storage

[0092] In VVC, the CPMV of an affine CU is stored in a separate buffer. The stored CPMV is only used to generate inherited CPMVPs for the most recently coded CU in affine merge mode and affine AMVP mode. Subblock MVs derived from the CPMV are used for motion compensation, merging / AMVP lists of MVs, MV export, and deblocking.

[0093] To avoid additional CPMV image row buffers, affine motion data inherited from the CUs of the aforementioned CTU is treated differently, inheriting from normal adjacent CUs. If the candidate CU for affine motion data inheritance is in the aforementioned CTU row, the bottom left and bottom right sub-blocks of MV in the row buffer, instead of CPMV, are used for affine MVP derivation. Thus, CPMV is only stored in the local buffer. If the candidate CU is a 6-parameter affine code, the affine model degenerates into a 4-parameter model. Figure 11 As shown, along the top CTU boundary, the motion vectors of the lower left and lower right sub-blocks of the CU are used for affine inheritance of the CU in the bottom CTU. Figure 11In the diagram, rows 1110 and 1112 represent the x and y coordinates of the origin (0, 0) in the top-left corner of the image. Legend 1120 illustrates the meaning of various motion vectors, where arrow 1122 represents the CPMV used for affine inheritance in the local buffer, arrow 1124 represents inheritance in the row buffer for MC / merge / skip MVP / deblock / TMVP and affine sub-block vectors, and arrow 1126 represents the sub-block vector of MC / merge / skip MVP / deblock / TMVP.

[0094] Adaptive Motion Vector Resolution (AMVR)

[0095] In HEVC, when `use_integer_mv_flag` in the slice header is equal to 0, the motion vector difference (MVD) (between the CU's motion vector and the predicted motion vector) is transmitted as a signal in quarter-luminance samples. In VVC, a CU-level adaptive motion vector resolution (AMVR) scheme is introduced. AMVR allows the CU's MVD to be encoded with different precisions. Depending on the current CU's mode (normal AMVP mode or affine AVMP mode), the current CU's MVD can be adaptively selected as follows:

[0096] - Normal AMVP mode: quarter brightness sample, half brightness sample, integer brightness sample or four brightness sample.

[0097] – Affine AMVP mode: quarter brightness sample, integer brightness sample, or 1 / 16 brightness sample.

[0098] If the current CU has at least one non-zero MVD component, the CU-level MVD resolution indication is conditionally signaled. If all MVD components (i.e., the horizontal and vertical MVD of reference list L0 and reference list L1) are zero, the quarter-luminance sample MVD resolution is inferred.

[0099] For a CU with at least one non-zero MVD component, a first flag is signaled to indicate whether quarter-luminance sample MVD accuracy is used for the CU. If the first flag is 0, no further signaling is required, and quarter-luminance sample MVD accuracy is used for the current CU. Otherwise, a second flag is signaled to indicate whether half-luminance sample or other MVD accuracy (integer or quad-luminance sample) is used for the normal AMVP CU. In the case of half-luminance samples, a 6-tap interpolation filter is used at the half-luminance sample location instead of the default 8-tap interpolation filter. Otherwise, a third flag is signaled to indicate whether integer or quad-luminance sample MVD accuracy is used for the normal AMVP CU. In the case of affine AMVP CUs, the second flag indicates whether integer or 1 / 16 luminance sample MVD accuracy is used. To ensure that the reconstructed MV has the expected accuracy (quarter-luminance sample, half-luminance sample, integer luminance sample, or quad-luminance sample), the CU's motion vector predictor is rounded to the same MVD accuracy before being added to the MVD. The predicted motion vector values ​​are rounded to zero (i.e., negative motion vector predicted values ​​are rounded to positive infinity, and positive motion vector predicted values ​​are rounded to negative infinity).

[0100] The encoder uses RD checks to determine the current CU's motion vector resolution. To avoid always performing four CU-level RD checks for each MVD resolution, RD checks for MVD accuracy, except for quarter-luminance samples, are conditionally invoked only in VTM11. For normal AVMP mode, the RD costs for quarter-luminance sample MVD accuracy and integer luminance sample MVD accuracy are first calculated. Then, the RD costs for integer luminance sample MVD accuracy are compared with those for quarter-luminance sample MVD accuracy to determine if further checks of the four-luminance sample MVD accuracy's RD cost are necessary. The four-luminance sample MVD accuracy RD check is skipped when the RD cost for quarter-luminance sample MVD accuracy is significantly less than that for integer luminance sample MVD accuracy. Then, if the RD cost for integer luminance sample MVD accuracy is significantly greater than the optimal RD cost of the previously tested MVD accuracy, the half-luminance sample MVD accuracy check is skipped. For affine AMVP mode, if the affine inter-frame mode and affine AMVP mode are not selected after checking the rate-distortion cost of affine merge / skip mode, merge / skip mode, AMVP mode with normal quarter-luminance sample MVD accuracy, and quarter-luminance sample MVD accuracy, then the 1 / 16 luminance sample MV accuracy and 1 pixel MV accuracy affine inter-frame modes are not checked. Furthermore, the affine parameters obtained in the quarter-luminance sample MV accuracy affine inter-frame mode are used as the starting search point for the 1 / 16 luminance sample and quarter-luminance sample MV accuracy affine inter-frame modes.

[0101] Bidirectional prediction with CU-level weights (BCW)

[0102] In HEVC, dual prediction signals It is generated by averaging two prediction signals P0 and P1 obtained from two different reference images and / or using two different motion vectors. In VVC, the dual prediction mode is extended beyond simple averaging to allow for a weighted average of the two prediction signals.

[0103]

[0104] Weighted average bidirectional prediction allows five weights, w∈{-2,3,4,5,10}. For each bidirectionally predicted CU, the weight w is determined by one of two methods: 1) for non-merged CUs, the weight index is signaled after the motion vector difference; 2) for merged CUs, the weight index is inferred from neighboring blocks based on the merge candidate index. BCW is only applicable to CUs with 256 or more luminance samples (i.e., CU width multiplied by CU height is greater than or equal to 256). For low-latency images, all 5 weights are used. For non-low-latency images, only 3 weights are used (w∈{3,4,5}). At the encoder, a fast search algorithm is applied to find the weight index without significantly increasing the encoder's complexity. These algorithms are summarized below. Detailed information is disclosed in the VTM software and document JVET-L0646 (Yu-Chi Su et al., “CE4 related: Generalized bidirectional prediction improvement combining JVET-L0197 and JVET-L0296”, Joint Video Experts Group (JVET) of ITU-T SG 16WP 3 with ISO / IEC JTC1 / SC 29, 12th meeting: Macau, October 3-12, 2018, document: JVET-L0646).

[0105] When combined with AMVR, if the current image is a low-latency image, unequal weights are conditionally checked only for 1-pixel and 4-pixel motion vector precision.

[0106] When combined with affine, affine ME is performed on unequal weights if and only if the affine mode is selected as the current best mode.

[0107] When the two reference images in bidirectional prediction are the same, unequal weights are only conditionally checked.

[0108] Unequal weights are not searched when certain conditions are met, depending on the POC distance between the current image and its reference image, the encoding QP, and the time level.

[0109] The BCW weight index is encoded using a context-coded bin and a bypass-coded bin. The first context-coded bin indicates whether equal weights are used; if unequal weights are used, the bypass-coded bin signals the additional bin to indicate which unequal weights were used.

[0110] Weight Prediction (WP) is an encoding tool supported by the H.264 / AVC and HEVC standards that efficiently encodes video content with fading. Support for WP has also been added to the VVC standard. WP allows sending weight parameters (weights and offsets) for each reference picture in each of the reference picture lists L0 and L1. The weights and offsets of the corresponding reference pictures are then applied during motion compensation. WP and BCW are designed for different types of video content. To avoid interaction between WP and BCW, which would complicate the VVC decoder design, if a CU uses WP, the BCW weight index is not notified, and the weight w is inferred as 4 (i.e., equal weights are applied). For merged CUs, the weight index is inferred from neighboring blocks based on the merge candidate index. This can be applied to both normal merge mode and inherited affine merge mode. For constructed affine merge mode, the affine motion information is constructed based on the motion information of up to 3 blocks. The BCW index of a CU using a constructed affine merge mode is simply set to be equal to the BCW index of the first control point MV.

[0111] In VVC, CIIP and BCW cannot jointly request a CU. When a CU uses CIIP mode encoding, the BCW index of the current CU is set to 2, which is equivalent to w=4 for equal weights. Equal weights imply the default value of the BCW index.

[0112] Combined Inter-Frame and Intra-Frame Prediction (CIIP)

[0113] In VVC, when a CU is encoded in merged mode, if the CU contains at least 64 luma samples (i.e., the CU width multiplied by the CU height is equal to or greater than 64), and if both the CU width and height are less than 128 luma samples, an additional flag is sent to indicate whether Combined Inter-Frame / Intra-Frame Prediction (CIIP) mode is applied to the current CU. As its name suggests, CIIP prediction combines inter-frame prediction signals with intra-frame prediction signals. CIIP mode The inter-frame prediction signal is derived using the same inter-frame prediction process applied to the regular combining mode; and the intra-frame prediction signal... This is derived after a regular intra-frame prediction process with a planar pattern. Then, the intra-frame and inter-frame prediction signals are combined using a weighted average, where the weight value wt is based on the top and left adjacent blocks of the current CU 1210 (e.g., ...). Figure 12 The encoding pattern shown is calculated as follows:

[0114] If the top neighbor is available and intra-frame encoding is used, set isIntraTop to 1; otherwise, set isIntraTop to 0.

[0115] If the left neighbor is available and intra-frame coding is used, set isIntraLeft to 1; otherwise, set isIntraLeft to 0.

[0116] If (isIntraLeft+isIntraTop) equals 2, then wt is set to 3;

[0117] Otherwise, if (isIntraLeft+isIntraTop) equals 1, then wt is set to 2;

[0118] Otherwise, set wt to 1.

[0119] The CIIP predictions are as follows:

[0120]

[0121] CCLM (Cross-Component Linear Model)

[0122] The main idea behind CCLM mode (sometimes abbreviated as LM mode) is that there is often some correlation between the color components (e.g., Y / Cb / Cr, YUV, and RGB) of a color image. These colors may be referred to as the first color, the second color, and the third color in this disclosure. CCLM technology utilizes this correlation by predicting the chromaticity components of a block from juxtaposed reconstructed luminance samples using a linear model whose parameters are derived from the reconstructed luminance and chromaticity samples adjacent to the block.

[0123] In VVC, the CCLM mode utilizes inter-channel dependencies by predicting chromaticity samples from reconstructed luminance samples. This prediction is performed using a linear model of the following form.

[0124]

[0125] here, This represents the predicted chromaticity samples in the CU, while These represent reconstructed luminance samples from the same CU, downsampled for non-4:4:4 color formats. Model parameters a and b are derived from adjacent luminance and chrominance samples reconstructed at the encoder and decoder ends, without explicit signaling.

[0126] VVC specifies three CCLM modes: CCLM_LT, CCLM_L, and CCLM_T. These three modes differ in the location of the reference samples used for model parameter derivation. CCLM_T mode involves samples only from the top boundary, CCLM_L mode involves samples only from the left boundary, and CCLM_LT mode uses samples from both the top and left boundaries.

[0127] In general, the prediction process of the CCLM model includes three steps:

[0128] 1) Downsampling of the luminance block and its adjacent reconstructed samples to match the size of the corresponding chrominance block.

[0129] 2) Derivation of model parameters based on reconstructed neighboring samples, and

[0130] 3) Apply model equation (1) to generate intra-frame chroma prediction samples.

[0131] Luminance component downsampling: To match the chroma sample positions of 4:2:0 or 4:2:2 color format video sequences, two types of downsampling filters can be applied to the luminance samples, both with a 2:1 downsampling rate in both the horizontal and vertical directions. These two filters correspond to "Type-0" and "Type-2" 4:2:0 chroma format content, respectively, and are defined by...

[0132]

[0133] Based on SPS-level identification information, a two-dimensional 6th-order (f2) or 5th-order (f1) filter is applied to the luminance samples within the current block and their adjacent luminance samples. SPS-level refers to the Sequence Parameter Set level. An anomaly occurs if the row above the current block is a CTU boundary. In this case, a one-dimensional filter [1,2,1] / 4 is applied to the aforementioned adjacent luminance samples to avoid using multiple luminance queues above the CTU boundary.

[0134] Model parameter derivation process: The model parameters a and b from equation (5) are derived based on adjacent luminance and chrominance samples reconstructed at the encoder and decoder ends to avoid requiring any signaling overhead. In the initial CCLM model version, a linear minimum mean square error (LMMSE) estimator was used for parameter derivation. However, in the final design, only four samples are involved to reduce computational complexity. Figure 13 The relative sample positions of the M×N chroma block 1310, the corresponding 2M×2N luminance block 1320, and their adjacent samples (displayed as solid circles and triangles) of "Type 0" content are shown.

[0135] exist Figure 13 The example shows four samples used in CCLM_LT mode, labeled with triangles. They are located at the upper boundaries M / 4 and M·3 / 4, and the left boundaries N / 4 and N·3 / 4. In CCLM_T and CCLM_L modes, the upper and left boundaries are extended to the size of (M+N) samples, and the four samples used for model parameter derivation are located at (M+N) / 8, (M+N)·3 / 8, (M+N)·5 / 8, and (M+N)·7 / 8.

[0136] Once four samples are selected, four comparison operations are used to determine the two smallest and two largest brightness sample values. Let X l Let X represent the average of two maximum brightness sample values. s This represents the average of the two minimum brightness sample values. Similarly, let Y... l and Y s This represents the average value of the corresponding chromaticity sample values. Then, the linear model parameters are obtained according to the following equation:

[0137]

[0138]

[0139] In this equation, the division operation for parameter 'a' is performed using a lookup table. To reduce the memory required to store this table, the diff value—the difference between the maximum and minimum values—and parameter 'a' are represented using exponential notation. Here, the diff value is approximated with a 4-digit significant part and an exponent. Therefore, the diff table contains only 16 elements. This approach reduces both computational complexity and the memory required to store the table.

[0140] MMLM Overview

[0141] As the name suggests, the original CCLM model uses a linear model to predict the chromaticity samples of the entire CU's luminance samples, while MMLM (Multi-Model CCLM) can have two models. In MMLM, the neighboring luminance samples and neighboring chromaticity samples of the current block are divided into two groups, each group serving as a training set to derive the linear model (i.e., deriving specific α and β for a specific group). Furthermore, the samples of the current luminance block are also classified based on the same rules as the classification of neighboring luminance samples.

[0142] The threshold is the average value of adjacent reconstructed brightness samples. Neighbors with Rec′L[x,y] <= Threshold are classified into group 1; while neighbors with Rec′L[x,y] > Threshold are classified into group 2.

[0143] Accordingly, the chromaticity prediction is obtained using a linear model:

[0144]

[0145] Chroma intra-frame mode coding

[0146] For chroma intra-mode coding, a total of eight intra-modes are allowed. These modes include five traditional intra-modes and three cross-component linear model modes (CCLM, LM_A, and LM_L). The chroma mode signaling and derivation process are shown in Table 3. Chroma mode coding directly depends on the intra-prediction mode of the corresponding luma block. Due to the separate block partitioning structure for luma and chroma components enabled in the I-slice, one chroma block can correspond to multiple luma blocks. Therefore, for chroma-derived (DM) mode, the intra-prediction mode of the corresponding luma block covering the center position of the current chroma block is directly inherited.

[0147] Table 3. Deriving Chroma Prediction Mode from Luminance Mode when CCLM is Enabled

[0148]

[0149] As shown in Table 4, a single binary table is used regardless of the value of sps_cclm_enabled_flag.

[0150] Table 4. Unified Binarization Table for Colorimetric Prediction Mode

[0151]

[0152] The first bin indicates whether it is in normal mode (0) or LM mode (1). If it is in LM mode, the next bin indicates whether it is LM_CHROMA (0) or not (1). If it is not LM_CHROMA, the next bin indicates whether it is LM_L (0) or LM_A (1). In this case, when sps_cclm_enabled_flag is 0, the first bin of the binarization table corresponding to intra_chroma_pred_mode can be ignored before entropy encoding. Or, in other words, the first bin is inferred to be 0 and therefore not encoded. This single binarization table is used for cases where sps_cclm_enabled_flag equals 0 and 1. The first two bins are context-encoded using their own context model, and the remaining bins are side-encoded.

[0153] Multiple Hypothesis Prediction (MHP)

[0154] In the multi-hypothesis inter-frame prediction mode (JVET-M0425), in addition to the traditional dual prediction signal, one or more additional motion-compensated prediction signals are emitted. The final overall prediction signal is obtained by superimposing sample weights. Using the dual prediction signal p... bi The resulting prediction signal p3, obtained by combining the first additional inter-frame prediction signal / hypothesis h3, is as follows:

[0155]

[0156] According to the following mapping (Table 5), the weight factor α is specified by the new syntax element add_hyp_weight_idx:

[0157] Table 5. Mapping α to add_hyp_weight_idx

[0158] add_hyp_weight_idx α 0 1 / 4 1 -1 / 8

[0159] Similar to the above, more than one additional prediction signal can be used. The resulting overall prediction signal is iteratively accumulated together with each additional prediction signal.

[0160]

[0161] As the last p n (That is, p with the largest index n) n The resulting overall prediction signal is obtained. For example, up to two additional prediction signals can be used (i.e., n is limited to 2).

[0162] The motion parameters for each additional prediction hypothesis can be explicitly signaled by specifying a reference index, a motion vector prediction index, and a motion vector difference, or implicitly signaled by specifying a merging index. A single multi-hypothesis merging identifier distinguishes between these two signaling modes.

[0163] For inter-frame AMVP mode, MHP is applied only when non-equal weights are selected in BCW under bidirectional prediction mode. Detailed information on MHP for VVC can be found in JVET-W2025 (Muhammed Coban et al., “Algorithmic description of Enhanced Compression Model 2 (ECM 2)”, Joint Video Experts Group (JVET) of ITU-T SG 16WP 3 and ISO / IEC JTC 1 / SC29, 23rd meeting, teleconference, July 7-16, 2021, document: JVET-W2025).

[0164] Combining MHP and BDOF is possible, but BDOF is only applied to the double-predictive signal portion of the predicted signal (i.e., the ordinary first two assumptions). Summary of the Invention

[0165] A method and apparatus for video encoding and decoding are disclosed. According to the method, input data associated with a current block is received, wherein the input data includes pixel data of the current block to be encoded at the encoder side or encoded data associated with the current block to be decoded at the decoder side. A set of candidates associated with encoding / decoding modes or encoding / decoding parameters is determined. A boundary matching cost associated with the candidate set is determined by using reconstructed or predicted samples of the current block and neighboring reconstructed or predicted samples, wherein a boundary matching cost is determined for each target candidate in the candidate set. The reconstructed or predicted samples of the current block are determined based on said target candidate. The current block is then encoded or decoded using a final candidate selected from the candidate set based on at least one boundary matching cost.

[0166] In one embodiment, the set of candidates corresponds to a set of merge candidates associated with a merge coding pattern. In one embodiment, merge indices are assigned to the merge candidate set. A first merge index is assigned to the minimum index value of the first merge candidate that results in the minimum boundary match cost, and a second merge index is assigned to the maximum index value of the second merge candidate that results in the maximum boundary match cost. In one embodiment, merge indices are assigned to the merge candidate set, and the target merge index of the selected merge candidate is signaled on the encoder side or received on the decoder side. In another embodiment, merge indices are assigned to the merge candidate set, and the target merge index of the selected merge candidate that results in the minimum boundary match cost is implicitly determined on both the encoder and decoder sides.

[0167] In one embodiment, the candidate set corresponds to a set of LM (Linear Model) candidates associated with LM predictions between different color components. In one embodiment, the LM candidate set includes at least two LM candidates from a group including top-left LM, top-only LM, left-only LM, and MM (Multi-Model) LM. In one embodiment, an LM index is assigned to the LM candidate set, and a target LM index is assigned to the minimum index value of the target LM candidate, resulting in the minimum boundary matching cost. In another embodiment, the target LM candidate resulting in the minimum boundary matching cost is implicitly selected as the final candidate.

[0168] In one embodiment, the set of candidates corresponds to a set of MMVD (Merge Mode with MVD) candidates, where each MMVD candidate includes a combination of offset direction and offset distance from the base merge candidate. In one embodiment, an MMVD index is assigned to the set of MMVD candidates, and a target MMVD index is assigned the minimum index value for the target MMVD candidate, resulting in minimum boundary matching cost. In another embodiment, the target MMVD candidate that results in minimum boundary matching cost is implicitly selected as the final candidate.

[0169] In one embodiment, a set of equations for determining boundary matching costs is disclosed.

[0170] In one embodiment, a set of candidates is reordered based on boundary matching cost. Attached Figure Description

[0171] Figure 1A An exemplary adaptive inter-frame / intra-frame video coding system incorporating loop processing is illustrated.

[0172] Figure 1B The diagram shows... Figure 1A The corresponding decoder of the encoder in the code.

[0173] Figure 2 The diagram illustrates adjacent blocks used to derive spatial merge candidates for VVC.

[0174] Figure 3 The illustration shows possible candidate pairs for redundancy checks in VVC.

[0175] Figure 4 An example of temporal candidate derivation is illustrated, in which a scaled motion vector is derived based on the POC (picture order count) distance.

[0176] Figure 5 The diagram illustrates the position of the time candidate selected between candidate C0 and C1.

[0177] Figure 6 The diagram illustrates the distance offset from the starting MV in the horizontal and vertical directions according to the merge mode with MVD (MMVD).

[0178] Figure 7A An example of an affine motion field for a block described by motion information from two control points (4 parameters) is shown.

[0179] Figure 7B An example of an affine motion field of a block is illustrated, described by motion information from three control point motion vectors (6 parameters).

[0180] Figure 8 An example of block-based affine transformation prediction is illustrated, where the motion vector of each 4×4 luma subblock is derived from the control point MV.

[0181] Figure 9 The illustration shows an example of deriving inheritance affine candidates based on the control point MV derivation of adjacent blocks.

[0182] Figure 10 An example is shown of constructing an affine candidate by combining translational motion information from each control point in space and time.

[0183] Figure 11The illustration shows an example of affine motion information storage used for motion information inheritance.

[0184] Figure 12 The illustration shows an example of deriving weight values ​​for combining inter-frame and intra-frame prediction (CIIP) based on the coding patterns of the top and left adjacent blocks.

[0185] Figure 13 The illustration shows an example of model parameter derivation for CCLM (Cross Component Linear Model) using adjacent chroma samples and adjacent luminance samples.

[0186] Figure 14 The illustration shows an example of boundary samples and neighboring samples used to derive the boundary matching cost of candidate patterns.

[0187] Figure 15 The illustration shows a flowchart of an exemplary video coding system that utilizes coding mode candidates based on boundary matching cost according to an embodiment of the present invention. Detailed Implementation

[0188] It is readily understood that the components of the present invention, as generally described and illustrated in the accompanying drawings, can be arranged and designed in a variety of different configurations. Therefore, the following more detailed description of embodiments of the systems and methods of the present invention as illustrated in the drawings is not intended to limit the scope of the claimed invention, but only represents selected embodiments of the invention. References throughout the specification to “one embodiment,” “an embodiment,” or similar language mean that a particular feature, structure, or characteristic described in connection with that embodiment may include in at least one embodiment of the invention. Therefore, the phrases “in one embodiment” or “in an embodiment” appearing in various places throughout the specification do not necessarily refer to the same embodiment.

[0189] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. However, those skilled in the art will recognize that the invention can be practiced without one or more specific details, or using other methods, components, etc. In other instances, well-known structures or operations are not shown or not illustrated. Detailed description is provided to avoid obscuring aspects of the invention. The illustrated embodiments of the invention will be best understood by referring to the accompanying drawings, wherein like parts are represented by like numbers throughout. The following description is by way of example only and simply illustrates certain selected embodiments of devices and methods consistent with the invention as claimed herein.

[0190] With improvements in video encoding and decoding, more encoding and decoding tools have been developed. However, the encoding gain of new encoding and decoding tools is not additive. A possible reason is...

[0191] - Due to syntactic overhead, not all new encoding / decoding modes can be candidate modes for blocks.

[0192] - As the number of candidate modes in a block increases, longer codewords are needed to indicate the encoding / decoding mode from multiple candidate modes.

[0193] Taking merge mode as an example. Compared to HEVC, it is recommended to add new merge candidates (e.g., pairwise average merge candidates, HMVP merge candidates, etc.) to the merge candidate list. As mentioned earlier, the index of the best merge candidate is encoded / decoded to indicate the merge candidate selected for the current block. However,

[0194] - The number of merge candidates in the merge candidate list is limited to a predefined number, so not all merge candidates can be added to the merge candidate list.

[0195] - If the number of merge candidates in the merge candidate list increases, the codeword length of the best merge candidate index becomes longer.

[0196] Therefore, a scheme is proposed to adaptively reorder candidate patterns. The general rules are as follows.

[0197] - Calculate the cost of each candidate pattern, and may (but is not limited to) merge candidate patterns and / or extend to candidate patterns of any other tool.

[0198] Next, the priority of these candidate patterns will be determined based on cost.

[0199] In one embodiment, candidate patterns with lower costs receive higher priority.

[0200] In another embodiment, candidate patterns with lower costs are given lower priority.

[0201] Next, the candidate patterns are reordered according to priority.

[0202] Smaller index values ​​indicate higher-priority candidate patterns.

[0203] In other words, an index refers to a reordered candidate pattern.

[0204] Initially, the value of the index refers to the index of the candidate pattern.

[0205] After applying the suggested reordering scheme, the index value refers to the reordered index of the candidate pattern.

[0206] In another embodiment, shorter codewords are used for encoding / decoding candidate patterns with higher priority. In yet another embodiment, the highest priority candidate pattern is implicitly set as the encoding pattern for the current block. In this invention, the scheme for determining priority includes boundary matching. For each candidate pattern, a boundary matching cost is calculated. The priority depends on the boundary matching costs of these candidate patterns.

[0207] Boundary matching cost of candidate patterns

[0208] The boundary matching cost of a candidate pattern refers to the current prediction generated from the candidate pattern. Figure 14 The measurement of discontinuities (including top boundary matching and / or left boundary matching) between the predicted samples within the current block 1410 (shown as a thick outline) and adjacent reconstructions (reconstructed samples within one or more adjacent blocks) is shown in the figure. Top boundary matching refers to the comparison between the current top predicted sample and the adjacent top reconstructed sample 1420, and left boundary matching refers to the comparison between the current left predicted sample and the adjacent left reconstructed sample 1430.

[0209] In one embodiment, the boundary matching cost is calculated using a predefined subset of the current predictions. This is achieved using n1 rows of the top boundary within the current block and / or m1 columns of the left boundary within the current block, where n1 and m1 are integers greater than or equal to 0. Additionally, n2 rows of the top adjacent reconstruction and / or m2 columns of the left adjacent reconstruction are used, where n2 and m2 are integers greater than or equal to 0.

[0210] Here is an example of calculating the boundary matching cost for n1=2, m1=2, n2=2, and m2=2.

[0211]

[0212] In the equation above, calculate the cost of the top boundary (i.e., the first summation) and the left boundary (i.e., the second summation). The terms within the first pair of absolute symbols (i.e., | This corresponds to the difference between the pixel in the top row of the current block and its two vertically adjacent pixels (i.e., the pixel above (in the row above) and the pixel below (in the current block)). The terms within the second pair of absolute symbols (i.e., | This corresponds to the difference between the pixel in the top adjacent row of the current block and its two vertically adjacent pixels (i.e., the pixel above (in the upper adjacent row) and the pixel below (in the current block)). The terms within the third pair of absolute symbols (i.e., | This corresponds to the difference between the pixel in the left row of the current block and its two horizontally adjacent pixels (i.e., the left pixel (in the left adjacent row) and the right pixel (in the current block)). The fourth pair of terms within the absolute symbols (i.e., | This corresponds to the difference between the pixel in the left adjacent row of the current block and its two horizontally adjacent pixels (i.e., the left pixel (in the left adjacent row) and the right pixel (in the current block)). In the above equation, the weights (a, b, c, d, e, f, g, h, i, j, k, l) can be any positive integers, for example, a = 2, b = 1, c = 1, d = 2, e = 1, f = 1, g = 2, h = 1, i = 1, j = 2, k = 1, l = 1.

[0213] This is another example of calculating the boundary matching cost for n1=2, m1=2, n2=1, and m2=1.

[0214]

[0215] In the above equation, the weights (a,b,c,g,h,i) can be any positive integers, such as a=2, b=1, c=1, g=2, h=1, i=1.

[0216] This is another example of calculating the boundary matching cost for n1=2, m1=1, n2=2, and m2=2.

[0217]

[0218] In the above equation, the weights (d,e,f,j,k,l) ​​can be any positive integers, such as d=2, e=1, f=1, j=2, k=1, l=1.

[0219] This is another example of calculating the boundary matching cost for n1=1, m1=1, n2=1, and m2=1.

[0220]

[0221] In the above equation, the weights (a,c,g,i) can be any positive integers, such as a=1, c=1, g=1, i=1.

[0222] This is another example of calculating the boundary matching cost for n1=2, m1=1, n2=2, and m2=1.

[0223]

[0224] In the above equation, the weights (a,b,c,d,e,f,g,i) can be any positive integers, such as a=2, b=1, c=1, d=2, e=1, f=1, g=1, i=1.

[0225] This is another example of calculating the boundary matching cost for n1=1, m1=2, n2=1, and m2=2.

[0226]

[0227] In the above equation, the weights (a,c,g,h,i,j,k,l) ​​can be any positive integers, such as a=1, c=1, g=2, h=1, i=1, j=2, k=1, l=1.

[0228] The examples below for n1 and m1 can also be applied to n2 and m2.

[0229] For example, nl can be any positive integer, such as 1, 2, 3, 4, etc.

[0230] For example, m1 can be any positive integer, such as 1, 2, 3, 4, etc.

[0231] For example, n1 and / or m1 vary with the block width, height, or area.

[0232] As an example, for larger blocks (e.g., area > threshold), a larger m1 is used.

[0233] Threshold = 64, 128 or 256.

[0234] When the area exceeds the threshold, m1 increases to 2. (Initially, m1 is 1.)

[0235] When the area exceeds the threshold, m1 increases to 4. (Initially, m1 is 1 or 2.)

[0236] According to another example, for taller blocks (e.g., height > threshold * width), use a larger m1 and / or a smaller n1.

[0237] Threshold = 1, 2 or 4.

[0238] When the height exceeds the threshold multiplied by the width, m1 increases to 2. (Initially, m1 was 1.)

[0239] When the height exceeds the threshold multiplied by the width, m1 increases to 4. (Originally, m1 was 1 or 2.)

[0240] According to another example, for larger blocks (e.g., area > threshold), use a larger n1.

[0241] Threshold = 64, 128 or 256.

[0242] When the area exceeds the threshold, n1 increases to 2. (Initially, n1 is 1.)

[0243] When the area exceeds the threshold, n1 increases to 4. (Initially, n1 is 1 or 2.)

[0244] According to another example, for wider blocks (e.g., width > threshold * height), use a larger n1 and / or a smaller m1.

[0245] Threshold = 1, 2 or 4.

[0246] When the width exceeds the threshold multiplied by the height, n1 increases to 2. (Originally, n1 was 1.)

[0247] When the width exceeds the threshold multiplied by the height, n1 increases to 4. (Originally, n1 was 1 or 2.)

[0248] For another example, nl and / or ml can be defined in the standard or depend on the signaling / parsing of the syntax from CU / CB, PU / PB, TU / TB, CTU / CTB, slice level, picture level, SPS (sequence parameter set) level and / or PPS (picture parameter set) level.

[0249] In another embodiment, when the current block is located at the top boundary within a CTU row, top boundary matching is not used and / or only left boundary matching is used. (Adjacent reconstructed samples across CTU rows are not used.)

[0250] In another embodiment, when the current block is located at the left boundary within the CTU, left boundary matching is not used and / or only upper boundary matching is used.

[0251] In another embodiment, when the current block is high (e.g., height > threshold * width), only the left boundary is used for matching.

[0252] In another embodiment, when the current block is wider (e.g., width > threshold * height), only the top boundary is used for matching.

[0253] In another embodiment, the top-left adjacent reconstructed sample can be used for boundary matching. For example, the boundary matching cost is added to the following:

[0254]

[0255] Merging candidates as candidate patterns

[0256] The proposed scheme can be used to reorder merge candidates. Normally, the index of the best merge candidate refers to its order in the merge candidate list. For the proposed scheme, the index of the best merge candidate (index_best_merge) refers to the priority order based on boundary matching costs.

[0257] For example, suppose the list of candidate merges consists of {cand0, cand1, cand2, cand3, cand4, cand5}. These predicted candidates are referred to as the predicted candidate group in this disclosure.

[0258] original bill:

[0259] index_best_merge 0 refers to cand0

[0260] index_best_merge 1 refers to cand1

[0261] index_best_merge 2 refers to cand2

[0262] ...

[0263] Recommended:

[0264] Calculate the boundary matching cost {cost_cand0, cost_cand1, cost_cand2, ...} for each merge candidate.

[0265] cost_cand0 is the boundary matching cost of cand0.

[0266] cost_cand1 is the boundary matching cost of cand1.

[0267] cost_cand2 is the boundary matching cost of cand2.

[0268] ...

[0269] Reorder {cand0, cand1, cand2, ...} based on boundary matching cost.

[0270] In one example

[0271] If cost_cand0>cost_cand1>cost_cand2>cost_cand3>cost_cand4>cost_cand5, then the reordered merge candidates form {cand5,cand4,cand3,cand2,cand1,cand0}.

[0272] After reordering,

[0273] `index_best_merge` 0 refers to `cand5` (the merge candidate with the lowest cost is signaled using the shortest codeword).

[0274] index_best_merge 1 refers to cand4

[0275] index_best_merge 2 refers to cand3

[0276] ...

[0277] Otherwise, if cost_cand0 < cost_cand1 < cost_cand2 < cost_cand3 < cost_cand4 < cost_cand5, the order of the merge candidates remains the same as the original order.

[0278] In another example,

[0279] If cost_cand0 < cost_cand1 < cost_cand2 < cost_cand3 < cost_cand4 < cost_cand5, the re-ordered merge candidates are formed as {cand5, cand4, cand3, cand2, cand1, cand0}.

[0280] After re-ordering,

[0281] index_best_merge 0 refers to cand5 (the merge candidate with the largest cost signals with the shortest codeword).

[0282] index_best_merge 1 refers to cand4

[0283] index_best_merge 2 refers to cand3

[0284] ……

[0285] Otherwise, if cost_cand0 > cost_cand1 > cost_cand2 > cost_cand3 > cost_cand4 > cost_cand5, the order of the merge candidates remains the same as the original.

[0286] In another embodiment, the index_best_merge with a smaller value is encoded with a shorter codeword length. For example, index_best_merge is encoded with truncated unary codewords.

[0287] In another embodiment, re-ordering is applied to a subset of the merge candidate list.

[0288] - For example, the subset refers to the original first n candidates cand0, cand1, cand2. Then index_best_merge0 / 1 / 2 refers to the priority order based on boundary matching, and index_best_merge 3 / 4 / 5 refers to the original cand3, cand4, and cand5.

[0289] - For example, a subset refers to the original last n candidates cand3, cand4, and cand5. Then index_best_merge 3 / 4 / 5 refers to the priority order based on boundary matching, and index_best_merge 0 / 1 / 2 refers to the original cand0, cand1, and cand2.

[0290] -For example, a subset refers to a candidate for spatial merging.

[0291] In another embodiment, the best merge candidate is inferred to be the merge candidate with the minimum boundary matching cost among all merge candidates. Therefore, index_best_merge does not need to be signaled / parsed by the encoder / decoder and can be inferred to be 0.

[0292] In another embodiment, the best merge candidate is inferred to be the merge candidate with the highest boundary matching cost among all merge candidates. Therefore, index_best_merge does not need to be signaled / parsed by the encoder / decoder and can be inferred to be 0.

[0293] In another embodiment, the merge candidate in this section refers to a merge candidate that includes one or more of the following candidates:

[0294] 1) Spatial MVP from spatially adjacent CUs

[0295] 2) Time MVP from co-located CU

[0296] 3) Historical MVPs from FIFO tables

[0297] 4) Paired average MVP

[0298] 5) Zero MV.

[0299] In another embodiment, the merge candidates in this section refer to the merge candidates of CIIP. The predicted samples within the current block are generated according to the CIIP process.

[0300] In another embodiment, the merge candidate in this section refers to a merge candidate of sub-block merge candidates, such as an affine merge candidate. The predicted samples within the current block are generated according to an affine process.

[0301] MMVD as a candidate mode

[0302] The proposed scheme can be used to reorder MMVD candidates to improve the encoding and decoding efficiency of the syntax of mmvd_cand_flag, mmvd_distance_idx, and / or mmvd_direction_idx. The following table (Table 6) is the syntax table of MMVD in the VVC standard.

[0303] Table 6. Syntax table of MMVD in VVC standard

[0304]

[0305] In the table above, mmvd_cand_flag[x0][y0] specifies the first (0) or second (1) candidate in the merge candidate list, used together with the difference in motion vectors derived from mmvd_distance_idx[x0][y0] and mmvd_direction_idx[x0][y0]. The array indices x0, y0 specify the position (x0, y0) of the top-left luminance sample of the considered coded block relative to the top-left luminance sample of the image.

[0306] If mmvd_cand_flag[x0][y0] does not exist, it is inferred that it is equal to 0.

[0307] In the table above, mmvd_cand_flag[x0][y0] specifies the first (0) or second (1) candidate in the merge; mmvd_distance_idx[x0][y0] specifies the index used to derive MmvdDistance[x0][y0], as specified in Table 7. The array indices x0, y0 specify the position (x0, y0) of the top-left luminance sample of the coded block under consideration relative to the top-left luminance sample of the image.

[0308] Table 7 – MmvdDistance[x0][y0] Specification Based on mmvd_distance_idx[x0][y0]

[0309]

[0310]

[0311] In the table above, mmvd_direction_idx[x0][y0] specifies the index used to derive MmvdSign[x0][y0] as specified in Table 8. The array indices x0, y0 specify the top-left luminance sample of the coded block considered at position (x0, y0) relative to the top-left luminance sample of the image.

[0312] Table 8 – MmvdSign[x0][y0] based on mmvd_direction_idx[x0][y0]

[0313] mmvd_direction_idx[x0][y0] MmvdSign[x0][y0][0] MmvdSign[x0][y0][1] 0 +1 0 1 -1 0 2 0 +1 3 0 -1

[0314] Taking mmvd_cand_flag as an example below, the mode selection efficiency is improved. The signaling of mmvd_distance_idx and mmvd_direction_idx remains the same as the original method.

[0315] First, calculate the boundary matching cost for each MMVD mode, including

[0316] – MMVD mode 0: MMVD with the first candidate in the merge candidate list

[0317] – MMVD mode 1: MMVD with the second candidate in the merge candidate list

[0318] [In one embodiment, after reordering according to the cost,

[0319] – If the cost of MMVD mode 0 > the cost of MMVD mode 1, mmvd_cand_flag equals 0 representing MMVD mode 1, and mmvd_cand_flag equals 1 representing MMVD mode 0.

[0320] – In an alternative way, mmvd_cand_flag is implicit, and the first or second candidate (with the minimum cost) in the merge candidate list is used for MMVD.

[0321] In another embodiment, after reordering according to the cost,

[0322] – If the cost of MMVD mode 0 < the cost of MMVD mode 1, mmvd_cand_flag equals​​​​​​​​​​​​​​​​​​​

[0329] In the table above, cclm_mode_idx specifies which of the following chroma intra-frame prediction modes—INTRA_LT_CCLM, INTRA_L_CCLM, and INTRA_T_CCLM—is applied.

[0330] The following example uses cclm_mode_idx to improve mode selection efficiency.

[0331] First, calculate the boundary matching cost for each LM candidate mode (e.g., INTRA_LT_CCLM, INTRA_L_CCLM, and INTRA_T_CCLM chroma intra-prediction modes).

[0332] In one embodiment, after reordering according to cost

[0333] –cclm_mode_idx equal to 0 refers to the LM candidate mode with the lowest cost, while cclm_mode_idx equal to 2 refers to the LM candidate mode with the highest cost.

[0334] – In another approach, cclm_mode_idx is implicit and uses LM candidate modes (with minimal cost).

[0335] In another embodiment, after reordering according to cost,

[0336] –cclm_mode_idx equal to 0 refers to the LM candidate mode with the highest cost, while bcw_idx equal to 2 refers to the LM candidate mode with the lowest cost.

[0337] – In another approach, cclm_mode_idx is implicit and uses the LM candidate mode (with the maximum cost).

[0338] In another embodiment, in addition to the CCLM (deriving model parameters from luminance and CB for CB; deriving model parameters from luminance and CB for CR) described in the overview section, further CCLM variants are disclosed. Some examples are shown below.

[0339] One variant is for cr:

[0340] – The model parameters are derived by using neighboring reconstructed samples of cb and cr as inputs X and Y for model derivation.

[0341] –Then, CR predictions are generated using the exported model parameters and the CB reconstructed samples.

[0342] Another variant is MMLM.

[0343] Another change is that for cb (or cr), model parameters are derived from multiple juxtaposed brightness blocks.

[0344] The LM candidate patterns (in this section) can also include any LM extensions / variations. As the number of LM candidate patterns increases, the proposed coding performance improvements become more significant.

[0345] Reordering schemes proposed for different tools (not limited to those in the examples below) can be standardized. For example, reordering schemes proposed for MHP, LM, BCW, MMVD, and / or merge candidates can be standardized with the same rules for calculating boundary matching costs.

[0346] The methods proposed in this invention can be enabled and / or disabled based on implicit rules (such as block width, height, or area) or explicit rules (such as syntax regarding blocks, slices, images, SPS, or PPS levels). For example, when a block area is smaller than a threshold, a suggested reordering is applied.

[0347] In this invention, the term "block" may refer to TU / TB, CU / CB, PU / PB, or CTU / CTB.

[0348] The term "LM" in this invention can be considered as a CCLM / MMLM mode or any other extension / variation of CCLM (e.g., the CCLM extension / variation proposed in this invention). Cross-component tools (e.g., the CCLM extension / variation proposed in this invention). The variation here means that when the block indication refers to the use of one of the cross-component modes (e.g., CCLM_LT, MMLM_LT, CCLM_L, CCLM_T, MMLM_L, MMLM_T and / or intra-prediction modes), some optional modes can be selected, which are not a traditional DC, planar, and angular mode for the current block. An example of using the Convolutional Cross-Component Mode (CCCM) as an optional mode is shown below. When this optional mode is applied to the current block, the cross-component information of the model (including nonlinear terms) is used to generate chroma predictions. The optional mode may follow the template selection of CCLM, thus the CCCM family includes CCCM_LT, CCCM_L, and / or CCCM_T.

[0349] The method proposed in this invention (for CCLM) can be used for any other LM mode.

[0350] Any combination of the methods proposed in this invention can be applied.

[0351] Any of the methods proposed above can be implemented in the encoder and / or decoder. For example, any of the proposed methods can be implemented in the intra-frame / inter-frame coding module of the encoder (e.g., Figure 1A Inter-frame prediction 112), motion compensation module (e.g.) Figure 1B In the MC 152), the candidate derivation module of the merge decoder. Alternatively, any of the proposed methods can be implemented as a circuit coupled to the intra / inter-frame coding module and / or motion compensation module of the encoder, and the candidate derivation module of the merge decoder.

[0352] Figure 15 A flowchart illustrating an exemplary video codec system for reordering mode candidates based on boundary matching cost according to an embodiment of the present invention is shown. The steps shown in the flowchart can be implemented as program code executable on one or more processors (e.g., one or more CPUs) on the encoder side. The steps shown in the flowchart can also be implemented in hardware, such as one or more electronic devices or processors arranged to perform the steps in the flowchart. According to the method, in step 1510, input data associated with the current block is received, wherein the input data includes pixel data of the current block to be encoded on the encoder side or encoded data associated with the current block to be decoded on the decoder side. In step 1520, a set of candidates associated with a codec mode or codec parameter is determined, wherein each combined prediction member includes a weighted sum of a first prediction candidate and a second prediction candidate selected from a weighted set using a target weight. In step 1530, a matching cost associated with the combined prediction member candidate set is determined, wherein a boundary matching cost is determined for each target weighted candidate of the candidate combination using a prediction sample of the current block and reconstructed or predicted samples of adjacent blocks, and wherein a reconstructed or predicted sample of the current block is determined based on said target candidate. Step 1540: Encode or decode the current block using at least one final combination prediction selected from the candidate set based on boundary matching cost.

[0353] The flowchart shown is intended to illustrate an example of video encoding according to the present invention. Those skilled in the art can modify, rearrange, split, or combine each step to implement the invention without departing from its spirit. Specific syntax and semantics have been used in this disclosure to illustrate examples of implementing embodiments of the invention. Those skilled in the art can implement the invention by substituting equivalent syntax and semantics without departing from its spirit.

[0354] The above description is provided to enable those skilled in the art to practice the invention in the context of specific applications and their requirements. Various modifications to the described embodiments will be apparent to those skilled in the art, and the general principles defined herein can be applied to other embodiments. Therefore, the invention is not intended to be limited to the specific embodiments shown and described, but is accorded the widest scope consistent with the principles and novel features disclosed herein. In the above detailed description, various specific details have been illustrated to provide a thorough understanding of the invention. However, those skilled in the art will understand that the invention can be practiced.

[0355] The embodiments of the invention described above can be implemented in various hardware, software code, or combinations thereof. For example, one embodiment of the invention may be one or more circuits integrated into a video compression chip or program code integrated into video compression software to perform the processes described herein. Embodiments of the invention may also be program code to be executed on a digital signal processor (DSP) to perform the processes described herein. The invention may also relate to numerous functions performed by a computer processor, digital signal processor, microprocessor, or field-programmable gate array (FPGA). These processors may be configured to perform specific tasks according to the invention by executing machine-readable software code or firmware code that defines specific methods embodied in the invention. The software code or firmware code may be developed in different programming languages ​​and in different formats or styles. The software code may also be compiled for different target platforms. However, different code formats, styles, and languages ​​of the software code, as well as other ways of configuring the code to perform the tasks according to the invention, do not depart from the spirit and scope of the invention.

[0356] The invention may be embodied in other specific forms without departing from its spirit or essential characteristics. The described examples should be considered illustrative rather than restrictive in all respects. Therefore, the scope of the invention is indicated by the appended claims rather than by the foregoing description. All variations falling within the meaning and scope of the equivalents of the claims should be included within their scope.

Claims

1. A video encoding / decoding method, the method comprising: Receive input data associated with the current block, wherein the input data includes pixel data of the current block to be encoded on the encoder side or encoded data associated with the current block to be decoded on the decoder side; Identify a candidate set associated with encoding / decoding modes or encoding / decoding parameters; Determine the boundary matching cost associated with the candidate set, wherein the boundary matching cost is determined for each target candidate in the candidate set using a reconstructed or predicted sample of the current block and neighboring reconstructed or predicted samples of the current block, wherein the reconstructed or predicted sample of the current block is determined based on said target candidate; and Based on the minimum boundary matching cost in the boundary matching cost, the corresponding final candidate is selected from the candidate set, and the final candidate is used to encode or decode the current block.

2. The video encoding and decoding method according to claim 1, characterized in that, The candidate set corresponds to a set of merge candidates associated with the merge codec mode.

3. The video encoding and decoding method according to claim 2, characterized in that, Merge indices are assigned to the merge candidate set, wherein a first merge index is assigned the minimum index value for the first merge candidate that results in the minimum boundary matching cost, and a second merge index is assigned the maximum index value for the second merge candidate that results in the maximum boundary matching cost.

4. The video encoding and decoding method according to claim 2, characterized in that, The merge index is assigned to the merge candidate set, and the target merge index of the selected merge candidate is sent by signal on the encoder side or received on the decoder side.

5. The video encoding and decoding method according to claim 2, characterized in that, The merge index is assigned to the merge candidate set, and the target merge index of the selected merge candidate that results in the minimum boundary matching cost is implicitly determined on both the encoder and decoder sides.

6. The video encoding and decoding method according to claim 1, characterized in that, The candidate set corresponds to the linear model candidate set associated with the linear model predictions between different color components.

7. The video encoding and decoding method according to claim 6, characterized in that, The linear model candidate set includes at least two linear model candidates from combinations of top-left linear models, top-only linear models, left-only linear models, and multi-model linear models.

8. The video encoding and decoding method according to claim 6, characterized in that, The linear model index is assigned to the linear model candidate set, and the target LM linear model index is assigned the minimum index value for the target linear model candidate that results in the minimum boundary matching cost.

9. The video encoding and decoding method according to claim 6, characterized in that, The target linear model candidate that results in the minimum boundary matching cost is implicitly selected as the final candidate.

10. The video encoding and decoding method according to claim 1, characterized in that, The candidate set corresponds to a set of motion vector difference merging pattern candidates, wherein each motion vector difference merging pattern candidate includes a combination of offset direction and offset distance from the basic merging candidate.

11. The video encoding and decoding method according to claim 10, characterized in that, The motion vector difference merging pattern index is assigned to the motion vector difference merging pattern candidate set, and the target motion vector difference merging pattern index is assigned to the minimum index value of the target motion vector difference merging pattern candidate that results in the minimum boundary matching cost.

12. The video encoding and decoding method according to claim 10, characterized in that, The target motion vector difference merging pattern candidate that results in the minimum boundary matching cost is implicitly selected as the final candidate.

13. The video encoding and decoding method according to claim 1, characterized in that, Each boundary matching cost is determined as the sum of a first boundary matching cost, a second boundary matching cost, a third boundary matching cost, and a fourth boundary matching cost, or any combination thereof, wherein the first boundary matching cost is determined for the first column of pixels in the left column of the current block, the second boundary matching cost is determined for the second column of pixels in the left adjacent column of the current block, the third boundary matching cost is determined for the first row of pixels in the top adjacent row of the current block, and the fourth boundary matching cost is determined for the second row of pixels in the top adjacent row of the current block.

14. The video encoding and decoding method according to claim 13, characterized in that, Each first boundary matching cost is determined for each first column pixel based on a first weighted difference between each first column pixel and its two horizontally adjacent pixels. Each second boundary matching cost is determined for each second column pixel based on a second weighted difference between each second column pixel and its two horizontally adjacent pixels; each third boundary matching cost is determined for each first row pixel based on a third weighted difference between each first row pixel and its two vertically adjacent pixels; and each fourth boundary matching cost is determined for each second row pixel based on a fourth weighted difference between each second row pixel and its two vertically adjacent pixels.

15. The video encoding and decoding method according to claim 14, characterized in that, The cost of each first boundary matching is determined to be (2) The value of each first column pixel - the sum of the values ​​of the two horizontally adjacent pixels of each first column pixel); the cost of each second boundary matching is determined as (2 The value of each second column pixel is the sum of the values ​​of the two horizontally adjacent pixels of each second column pixel; the cost of each third boundary matching is determined as (2 The value of each first row pixel - the sum of the values ​​of the two vertically adjacent pixels of each first row pixel); the cost of each fourth boundary matching is determined as (2 The value of each second row pixel is the sum of the values ​​of the two horizontally adjacent pixels of each second row pixel; wherein, when the target pixel is in the current block, the predicted value is used for the target pixel, and when the target pixel is in the adjacent region of the current block, the reconstructed value is used for the target pixel.

16. The video encoding and decoding method according to claim 1, characterized in that, It also includes reordering the candidate set based on boundary matching costs.

17. An apparatus for video encoding and decoding, the apparatus comprising one or more electronic devices or processors arranged to: Receive input data associated with the current block, where, The input data includes pixel data of the current block to be encoded on the encoder side or encoded data associated with the current block to be decoded on the decoder side; Identify a candidate set associated with encoding / decoding modes or encoding / decoding parameters; Determine the boundary matching cost associated with the candidate set, wherein the boundary matching cost is determined for each target candidate in the candidate set using a reconstructed or predicted sample of the current block and neighboring reconstructed or predicted samples of the current block, wherein the reconstructed or predicted sample of the current block is determined based on said target candidate; and Based on the minimum boundary matching cost in the boundary matching cost, a final candidate is selected from the candidate set, and the final candidate is used to encode or decode the current block.

Images