Method and apparatus for implicit cross-component prediction in video coding system

Through the hybrid cross-color prediction sub-technology, the problem of inefficient inter prediction in video encoding and decoding systems is solved, and more efficient encoding and decoding and video quality recovery are achieved.

CN119013980APending Publication Date: 2024-11-22MEDIATEK INC
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202380033921.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2022-04-14
Filing Date
2023-04-13
Publication Date
2024-11-22

AI Technical Summary

Technical Problem

The existing video codec system has problems of inefficiency in the inter-frame prediction process, especially when reconstructing video data, the application of loop filters is complicated and information merging is difficult to effectively improve video quality.

Method used

The hybrid cross color predictor technology is used to improve the encoding and decoding efficiency by deciding the first and second predictors and generating the final predictors based on the target model parameters. The technique includes determining the first predictor and the second predictor, and generating the final predictor for encoding or decoding by implicitly derive the target model parameters.

Benefits of technology

It improves the efficiency of the video encoding and decoding system, simplifies the application of loop filters, enhances video quality recovery, and reduces the complexity of encoding and decoding.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119013980A_ABST
    Figure CN119013980A_ABST
Patent Text Reader

Abstract

A method and apparatus for video encoding and decoding are disclosed. According to the method, a first predictor including a prediction sample of the current block is determined for a second color block. Determining at least one second predictor for a second color block based on the first color block, where the one or more target model parameters are associated with at least one target prediction model corresponding to the at least one second predictor, the one or more target model parameters are implicitly derived by using neighboring samples of the second color block and / or neighboring samples of the first color block, and wherein the at least one second predictor corresponds to all samples or a subset of prediction samples of the current block. A final predictor is generated by mixing the first predictor and the at least one second predictor. Input data associated with the second color block is encoded or decoded using prediction data including a final predictor.
Need to check novelty before this filing date? Find Prior Art

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This disclosure is part of a non-provisional application that claims priority to U.S. Provisional Patent Application No. 63 / 330,827, filed on April 14, 2022. The contents of the above-listed applications are incorporated herein by reference in their entirety. Technical Field

[0003] The present disclosure generally relates to video coding systems and more particularly to hybrid cross-color prediction predictors to improve coding efficiency. Background Art

[0004] Versatile video coding (VVC) is the latest international video coding standard developed by the Joint Video Experts Team (JVET) of the ITU-T Video Coding Experts Group (VCEG) and the ISO / IEC Moving Picture Experts Group (MPEG). The standard was published as an ISO standard in February 2021: ISO / IEC23090-3:2021, Information technology - Codec representation of immersive media - Part 3: Versatile video coding. VVC is based on its previous generation High Efficiency Video Coding (HEVC) by adding more codec tools to improve codec efficiency and handle various types of video sources including three-dimensional (3D) video signals.

[0005] Figure 1AAn example adaptive inter / intra video codec system incorporating loop processing is shown. For intra prediction, prediction data is derived based on previously encoded video data in the current picture. For inter prediction 112, motion estimation (ME) is performed at the encoder end and motion compensation (MC) is performed based on the results of ME to provide prediction data derived from other pictures and motion data. Switch 114 selects intra prediction 110 or inter prediction 112, and the selected prediction data is provided to adder 116 to form a prediction error, also known as a residual. The prediction error is then processed by a transform (T) 118 followed by a quantization (Q) 120. The transformed and quantized residual is then encoded by an entropy encoder 122 to be included in a video bitstream corresponding to the compressed video data. The bitstream associated with the transform coefficients is then packaged with auxiliary information (such as auxiliary information such as motion and coding modes associated with intra prediction and inter prediction) and other information (parameters associated with loop filters applied to underlying image areas, etc.). As Figure 1A As shown, auxiliary information associated with intra prediction 110, inter prediction 112 and loop filter 130 is provided to entropy encoder 122. When inter prediction mode is used, one or more reference pictures must also be reconstructed at the encoder end. Therefore, the transformed and quantized residual is processed by inverse quantization (Inverse Quantization, referred to as IQ) 124 and inverse transformation (Inverse Transformation, referred to as IT) 126 to restore the residual. The residual is then added back to the prediction data 136 in reconstruction (REC) 128 to reconstruct the video data. The reconstructed video data can be stored in the reference picture buffer 134 and used for prediction of other frames.

[0006] like Figure 1A As shown, the input video data undergoes a series of processes in the encoding system. Due to the series of processes, the reconstructed video data from REC 128 may be subject to various damages. Therefore, before the reconstructed video data is stored in the reference picture buffer 134, a loop filter 130 is generally applied to the reconstructed video data to improve the video quality. For example, a deblocking filter (DF for short), a sample adaptive offset (SAO for short) and an adaptive loop filter (ALF for short) may be used. The loop filter information may need to be incorporated into the bitstream so that the decoder can correctly recover the required information. Therefore, the loop filter information is also provided to the entropy encoder 122 to be incorporated into the bitstream. In Figure 1A In , a loop filter 130 is applied to the reconstructed video before the reconstructed samples are stored in a reference picture buffer 134 . Figure 1A The system in is intended to illustrate an example structure of a typical video encoder. It may correspond to a High Efficiency Video Coding (HEVC) system, VP8, VP9, ​​H.264, or VVC.

[0007] like Figure 1B As shown, the decoder can use similar or partially identical functional blocks as the encoder, except for transform 118 and quantization 120, because the decoder only needs inverse quantization 124 and inverse transform 126. The decoder uses an entropy decoder 140 instead of an entropy encoder 122 to decode the video bitstream into quantized transform coefficients and required coding information (e.g., ILPF information, intra-frame prediction information, and inter-frame prediction information). The intra-frame prediction 150 on the decoder side does not need to perform a pattern search. Instead, the decoder only needs to generate an intra-frame prediction based on the intra-frame prediction information received from the entropy decoder 140. In addition, for inter-frame prediction, the decoder only needs to perform motion compensation (MC 152) based on the intra-frame prediction information received from the entropy decoder 140 without motion estimation.

[0008] According to VVC, the input picture is divided into non-overlapping square block areas called coding tree units (CTUs), similar to HEVC. Each CTU can be divided into one or more smaller-sized coding units (CUs). The generated CU partitions can be square or rectangular. In addition, VVC divides CTUs into prediction units (PUs) as a unit to apply prediction processing such as inter-frame prediction, intra-frame prediction, etc. Summary of the invention

[0009] A method and apparatus for video coding and decoding are disclosed. According to the method, input data associated with a first color block and a current block including a second color block is received, wherein the input data includes pixel data for the first color block and the current block, the pixel data to be encoded at the encoder side, and coded data associated with the first color block and the current block is received, the coded data to be decoded at the decoder side. A first predictor for the second color block is determined, wherein the first predictor corresponds to all samples or a subset of prediction samples of the current block. Based on the first color block, at least one second predictor of the second color block is determined, wherein one or more target model parameters are associated with at least one target prediction model corresponding to the at least one second predictor, the one or more target model parameters are implicitly derived by using one or more neighboring samples of the second color block and / or one or more neighboring samples of the first color block, and wherein the at least one second predictor corresponds to all samples or a subset of prediction samples of the current block. A final predictor is generated, wherein the final predictor includes a portion of the first predictor and a portion of the at least one second predictor. The input data associated with the second color block is encoded or decoded using the prediction data containing the final predictor.

[0010] In one embodiment, the first predictor corresponds to an intra predictor. In another embodiment, the first predictor corresponds to a cross color predictor. For example, the first predictor may be generated based on CCLM_LT, CCLM_L or CCLM_T.

[0011] In one embodiment, the at least one second predictor is generated based on a Multiple Model Cross Component Linear Model (Multiple Model CCLM (Cross Component Linear Model), MMLM for short) model.

[0012] In one embodiment, the portion of the first predictor is derived based on the first predictor having a first weight, and the portion of the at least one second predictor is derived based on the at least one second predictor having at least one second weight. The final predictor is derived as the sum of the portion of the first predictor and the portion of the at least one second predictor. The first weight, the at least one second weight, or both are determined by deriving for each sample of the second color block.

[0013] In one embodiment, the syntax is sent on the encoder side to indicate whether it is allowed to determine at least one second predictor, generate a final predictor, and encode or decode the current block using prediction data including the final predictor. In addition, the syntax can be sent on the encoder side or parsed on the decoder side at the block level, tile level, fragment level, picture level, sequence parameter set (SPS) level, or picture parameter set (PPS) level. In one embodiment, if the current block uses a predetermined cross-color mode, the syntax indication allows the determination of the at least one second predictor, the generation of the final predictor, and the encoding or decoding of the current block using prediction data including the final predictor. An example of a predetermined cross-color mode refers to a linear model (LM) mode. The LM mode can correspond to a CCLM_LT mode, a CCLM_L mode, or a CCLM_T mode.

[0014] In an embodiment, whether to allow determining the at least one second predictor, generating the final predictor, and encoding or decoding the current block using prediction data including the final predictor is implicitly determined.

[0015] In one embodiment, one or more model parameters of each prediction model of the candidate set are determined, and the cost of each prediction model of the candidate set is evaluated, and wherein a prediction model in the candidate set that achieves the minimum cost is selected as the at least one target prediction model, and the one or more model parameters associated with the one prediction model in the candidate set that achieves the minimum cost are selected as the one or more target model parameters.

[0016] In one embodiment, if the minimum cost is below a threshold, it is determined that the at least one second predictor, generating a final predictor, and encoding or decoding the current block using prediction data including the final predictor are permitted.

[0017] In one embodiment, a second color template including selected neighboring samples of the second color block and a first color template including corresponding neighboring samples of the first color block are determined, the one or more model parameters of each prediction model of the candidate set are determined based on reference samples of the first color template and reference samples of the second color template, and wherein the cost of each prediction model of the candidate set is determined based on the reconstructed samples and the predicted samples, and the predicted samples of the second color template are derived by applying the one or more model parameters determined for each prediction model to the first color template. In one embodiment, the second color template includes top neighboring samples of the second color block, left neighboring samples of the second color block, or both of the second color block, and the first color template includes top neighboring samples of the first color block, left neighboring samples of the first color block, or both of the first color block. In one embodiment, the current block includes a Cr block and a Cb block, the first color block corresponds to a Y block, and the second color block corresponds to a Cr block or a Cb block, wherein when the syntax indicates: determining the at least one second predictor, generating a final predictor, and encoding or decoding the current block using prediction data including the final predictor are allowed for one of the Cr block and the Cb block, then determining the at least one second predictor, generating the final predictor, and encoding or decoding the current block using prediction data including the final predictor are also allowed for the other of the Cr block and the Cb block.

[0018] In one embodiment, the cost of each prediction model of the candidate set corresponds to a boundary matching cost, the boundary matching cost is used to measure the discontinuity between the predicted sample of the second color block and the adjacent reconstructed sample of the second color block, and wherein the predicted sample of the second color block is derived based on the first color block using the one or more model parameters determined for each prediction model. In one embodiment, the boundary matching cost includes a top boundary matching cost, a left boundary matching cost, or both, the top boundary matching cost is compared between the top predicted sample of the second color block and the adjacent top reconstructed sample of the second color block, and the left boundary matching cost is compared between the left predicted sample of the second color block and the adjacent left reconstructed sample of the second color block.

[0019] In one embodiment, a second color template including selected neighboring samples of a second color block and a first color template including corresponding neighboring samples of a first color block are determined, the one or more model parameters of each prediction model of the candidate set are determined based on the second color template and the first color template, and wherein the cost of each prediction model of the candidate set is determined based on the reconstructed samples and the predicted samples of the second color template, and the predicted samples of the second color template are derived by applying the one or more model parameters determined for each prediction model to the first color template. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1A An example adaptive inter / intra video coding system incorporating loop processing is shown.

[0021] Figure 1B Show Figure 1A The corresponding decoder of the encoder in .

[0022] Figure 2 Neighboring blocks used to derive spatial merging candidates for VVC are shown.

[0023] Figure 3 Possible candidate pairs considered for redundancy checking in VVC are shown.

[0024] Figure 4 An example of temporal candidate derivation is shown, where scaled motion vectors are derived based on Picture Order Count (POC) distance.

[0025] Figure 5 The position of the temporal candidate selected between candidates C0 and C1 is shown.

[0026] Figure 6 The distance offset from the starting MV in the horizontal and vertical directions according to the Merge Mode with MVD (MMVD for short) is shown.

[0027] Fig. 7A An example of an affine motion field of a block described by motion information of two control points (4 parameters) is shown.

[0028] Figure 7B An example of an affine motion field of a block described by motion information of three control point motion vectors (6 parameters) is shown.

[0029] Figure 8 An example of block-based affine transform prediction is shown, where the motion vector for each 4×4 luma sub-block is derived from a control point MV.

[0030] Fig. 9 An example of derivation of inherited affine candidates based on control point MVs of neighboring blocks is shown.

[0031] Fig.10 An example of affine candidate construction by combining translational motion information of each control point from both spatial neighbors and temporal neighbors is shown.

[0032] Fig.11 An example of affine motion information storage for motion information inheritance is shown.

[0033] Fig.12An example of derivation of weight values ​​of Combined Inter and Intra Prediction (CIIP) according to the coding modes of the top and left neighboring blocks is shown.

[0034] Fig.13 An example of model parameter derivation for a cross-component linear model (CCLM) using neighboring chroma samples and neighboring luma samples is shown.

[0035] Fig.14 The intra-frame prediction mode adopted by the VVC video coding standard is shown.

[0036] Fig.15A -B shows an example of wide-angle intra prediction, where the width of the block is greater than the height ( Fig.15A ) and blocks that are taller than they are wide ( Fig. 15B ).

[0037] Fig.16 An example of two vertically adjacent prediction samples using two non-adjacent reference samples in the case of wide-angle intra prediction is shown.

[0038] Fig.17A An example of a selected template for a current block is shown, where the template includes T rows above the current block and T columns to the left of the current block.

[0039] Fig. 17B An example is shown in which T=3 and a Histogram of Gradient (HoG) is calculated for pixels in a middle row and pixels in a middle column.

[0040] Fig. 17C An example of the amplitude (ampl) of the angular intra prediction mode is shown.

[0041] Fig.18 An example of a hybrid process is shown where two intra modes (M1 and M2) and a planar mode are selected according to the index with the two highest bins of the histogram bin.

[0042] Fig.19 An example of a template-based intra mode derivation (TIMD) mode is shown, where TIMD implicitly derives the intra prediction mode of a CU using neighboring templates at the encoder and decoder.

[0043] Fig. 20 Examples of luminance and chrominance templates and reference samples of the templates used to derive model parameters and template matching distortion are shown.

[0044] Fig.21An example of boundary matching is shown, which measures the discontinuity measure between the current prediction and the adjacent reconstruction.

[0045] Fig. 22 Examples of luminance and chrominance templates used to derive model parameters and template matching distortion are shown.

[0046] Fig.23 A flowchart of an exemplary video coding and decoding system using a hybrid predictor according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0047] It is readily understood that the components of the present invention as generally described and illustrated in the figures herein may be arranged and designed in a variety of different configurations. Therefore, the following more detailed description of embodiments of the systems and methods of the present invention, as shown in the figures, is not intended to limit the scope of the claimed invention, but is merely representative of selected embodiments of the present invention. References in this specification to "an embodiment," "some embodiments," or similar language mean that specific features, structures, or characteristics described in conjunction with the embodiment may be included in at least one embodiment of the present invention. Therefore, the phrases "in an embodiment" or "in some embodiments" appearing in various places throughout this specification do not necessarily all refer to the same embodiment.

[0048] In addition, the described features, structures or characteristics may be combined in any suitable manner in one or more embodiments. However, those skilled in the art will recognize that the present invention may be implemented without one or more specific details or using other methods, components, etc. In other cases, well-known structures or operations are not shown or described in detail to avoid obscuring various aspects of the present invention. The illustrated embodiments of the present invention will be best understood by reference to the accompanying drawings, in which the same components are represented by the same numbers throughout. The following description is by way of example only, and simply describes some selected embodiments of the apparatus and method consistent with the present invention as claimed herein.

[0049] The VVC standard incorporates various new coding tools to further improve the coding efficiency of the HEVC standard. Among the various new coding tools, some coding tools related to the present invention are summarized as follows.

[0050] Inter-frame prediction overview

[0051] According to Section 3.4 of JVET-T2002 (Jianle Chen, et.al., "Algorithm description for Versatile Video Coding and Test Model 11 (VTM 11)", Joint Video Experts Team (JVET) of ITU-T SG 16WP 3 and ISO / IEC JTC 1 / SC 29, 20th Meeting, bytleconference, 7-16 October 2020, Document: JVET-T2002), for each inter-prediction CU, motion parameters consisting of motion vector, reference picture index and reference picture list usage index and additional information are used for the generation of inter-prediction samples. Motion parameters can be sent explicitly or implicitly. When the CU is encoded and decoded in skip mode, the CU is associated with one PU and has no significant residual coefficients, no coded motion vector increment or reference picture index. The merge mode means that the motion parameters of the current CU are obtained from neighboring CUs, including spatial and temporal candidates, as well as additional scheduling introduced in VVC. Merge mode can be used for any inter-predicted CU. An alternative to merge mode is explicit transmission of motion parameters, where the motion vector of each CU, the corresponding reference picture index and reference picture list usage flag of each reference picture list and other required information are explicitly sent.

[0052] In addition to the inter-frame coding and decoding functions in HEVC, VVC also includes many new and improved inter-frame prediction coding and decoding tools, as listed below:

[0053] - Extended Merger Forecast

[0054] -Merge mode with MVD (MMVD)

[0055] -Symmetric MVD (SMVD) transmission

[0056] -Affine motion compensated prediction

[0057] -Subblock-based temporal motion vector prediction (SbTMVP)

[0058] - Adaptive motion vector resolution (AMVR) - Motion field storage: 1 / 16 luminance sample MV storage and 8x8 motion field compression

[0059] - Bi-prediction with CU-level weight (BCW)

[0060] -Bi-directional optical flow (BDOF)

[0061] -Decoder side motion vector refinement (DMVR)

[0062] -Geometric partitioning mode (GPM)

[0063] - Combined inter and intra prediction (CIIP)

[0064] The following description provides details of those inter prediction methods specified in VVC.

[0065] Expanded Consolidated Forecast

[0066] In VVC, the merge candidate list is constructed by including the following five categories of candidates in sequence:

[0067] 1) Spatial MVP from spatially adjacent CUs

[0068] 2) Time MVP from the same CU

[0069] 3) History-based MVP from FIFO table

[0070] 4) Paired Average MVP

[0071] 5) Zero MV.

[0072] The size of the merge list is sent in the sequence parameter set (SPS) header and the maximum allowed size of the merge list is 6. For each CU encoded in merge mode, the index of the best merge candidate is encoded using truncated unary binarization. The first bin of the merge index is encoded and decoded using context, and bypass encoding and decoding is used for the remaining bins.

[0073] This section provides the derivation process of merge candidates for each category. As done in HEVC, VVC also supports parallel derivation of merge candidate lists (or merge candidate lists) for all CUs in a certain size region.

[0074] Spatial candidate derivation

[0075] The derivation of spatial merge candidates in VVC is the same as in HEVC, except that the positions of the first two merge candidates are swapped. Figure 2 A maximum of four merge candidates (B0, A0, B1 and A1) of the current CU 210 are selected from the candidates at the positions shown. The derived order is B0, A0, B1, A1 and B2. Position B2 is only considered when one or more neighboring CUs at positions B0, A0, B1 and A1 are not available (for example, belong to another fragment or tile) or are intra-encoded. After the candidate at position A1 is added, a redundancy check is performed on the addition of the remaining candidates to ensure that candidates with the same motion information are excluded from the list, thereby improving the encoding and decoding efficiency. In order to reduce computational complexity, not all possible candidate pairs are considered in the mentioned redundancy check. Instead, only the possible candidate pairs are considered. Figure 3 The pairs are connected by arrows in , and candidates are added to the list only if the corresponding candidates used for redundancy checking do not have the same motion information.

[0076] Time candidate derivation

[0077] In this step, only one candidate is added to the list. Specifically, in the derivation of this temporal merge candidate for the current CU 410, the scaled motion vector is based on the candidate belonging to Figure 4 The co-located CU 420 of the co-located reference picture shown is derived. The reference picture list and the reference index used to derive the co-located CU are explicitly sent in the slice header. As shown by the dotted line in Figure 44, the scaled motion vector 430 of the temporal merge candidate is obtained, which is scaled from the motion vector 440 of the co-located CU using the picture order count (Picture Order Count, POC) distances tb and td, where tb is defined as the POC difference between the reference picture of the current picture and the current picture, and td is defined as the POC difference between the reference picture of the co-located picture and the co-located picture. The reference picture index of the temporal merge candidate is set to zero.

[0078] The position of the temporal candidate is selected between candidates C0 and C1, such as Figure 5 As shown. If the CU at position C0 is not available, is intra-coded or is outside the current CTU row, then position C1 is used. Otherwise, position C0 is used to derive the temporal merge candidate.

[0079] History-based Merge Candidate Derivation

[0080] History-based MVP (HMVP) merge candidates are added to the merge list after spatial MVP and TMVP. In this method, the motion information of the previous codec block is stored in a table and used as the MVP of the current CU. During the encoding / decoding process, a table with multiple HMVP candidates is retained. When a new CTU row is encountered, the table will be reset (cleared). Whenever there is a non-sub-block inter-coded CU, the associated motion information will be added to the last entry of the table as a new HMVP candidate.

[0081] The HMVP table size S is set to 6, which indicates that a maximum of 5 history-based MVP (HMVP) candidates can be added to the table. When inserting a new motion candidate into the table, a constrained first-in-first-out (FIFO) rule is used, where first a redundancy check is applied to find if there is an identical HMVP in the table. If found, the identical HMVP is removed from the table and all subsequent HMVP candidates are moved forward, and the identical HMVP is inserted into the last entry of the table.

[0082] HMVP candidates can be used in the merge candidate list construction process. The latest few HMVP candidates in the table are checked in turn and inserted into the candidate list after the TMVP candidate. Redundancy check is applied to HMVP candidates to spatial or temporal merge candidates.

[0083] To reduce the number of redundant checking operations, the following simplifications are introduced:

[0084] 1. The last two entries in the table are checked for redundancy with respect to the A1 and B1 spatial candidates respectively.

[0085] 2. Once the total number of available merge candidates reaches the maximum allowed merge candidate - 1, the merge candidate list construction process of HMVP is terminated.

[0086] Pairwise average merge candidate derivation

[0087] A pairwise average candidate is generated by averaging a predetermined candidate pair in an existing merge candidate list using the first two merge candidates. The first merge candidate is defined as p0Cand, and the second merge candidate may be defined as p1Cand. The average motion vector is calculated for each reference list according to the availability of the motion vectors of p0Cand and p1Cand. If two motion vectors are available in one list, they are averaged and their reference pictures are set to one of p0C and p0C even if they point to different reference pictures; if only one motion vector is available, this motion vector is used directly; if no motion vector is available, this list is kept invalid. In addition, if the half-pixel interpolation filter index of p0Cand and p1Cand is different, it is set to 0.

[0088] When the merge list is not full after adding pairwise average merge candidates, zero MVPs are inserted at the end until the maximum number of merge candidates is reached.

[0089] Merge estimated areas

[0090] The merge estimation region (MER) allows independent derivation of merge candidate lists for CUs in the same merge estimation region (MER). Candidate blocks in the same MER as the current CU are not included in the generation of the merge candidate list for the current CU. In addition, the history-based update process of the motion vector predictor candidate list is updated only if (xCb+cbWidth)>>Log2ParMrgLevel is greater than xCb>>Log2ParMrgLevel and (yCb+cbHeight)>>Log2ParMrgLevel is greater than (yCb>>Log2ParMrgLevel), where (xCb, yCb) is the top-left luma sample position of the current CU in the picture and (cbWidth, cbHeight) is the CU size. The MER size is selected at the encoder side and emitted as log2_parallel_merge_level_minus2 in the Sequance Parameter Set (SPS).

[0091] Merge Mode with MVD (MMVD)

[0092] In addition to the merge mode in which the implicitly derived motion information is directly used for the prediction sample generation of the current CU, a merge mode with motion vector difference (MMVD) is introduced in VVC. The MMVD flag is sent immediately after the regular merge flag is sent to specify whether the MMVD mode is used for the CU.

[0093] In MMVD, after a merge candidate (referred to in this disclosure as a basic merge candidate) is selected, it is further refined by the MVD information sent. Further information includes a merge candidate flag, an index for specifying the magnitude of motion, and an index for indicating the direction of motion. In MMVD mode, one of the first two candidates in the merge list is selected for use as the MV basis. The MMVD candidate flag is sent to specify which one to use between the first and second merge candidates.

[0094] The distance index specifies the motion magnitude information and indicates a predetermined offset from the start points (612 and 622) of the L0 reference block 610 and the L1 reference block 620. Figure 6 As shown, the offset is added to the horizontal component or the vertical component of the starting MV, where small circles of different styles correspond to different offsets from the center. The relationship between the distance index and the predetermined offset is specified in Table 1.

[0095] Table 1 - Relationship between distance index and predetermined offset

[0096]

[0097] The direction index indicates the direction of the MVD relative to the starting point. The direction index can indicate four directions as shown in Table 2. It should be noted that the meaning of the MVD symbol can change according to the information of the starting MV. When the starting MV is an un-predictionMV or a bi-directional prediction MV, where both lists point to the same side of the current picture (i.e., both referenced POCs are greater than the POC of the current picture, or both are less than the POC of the current picture), the symbol in Table 2 specifies the sign of the MV offset added to the starting MV. When the starting MV is a bi-directional prediction MV, the two MVs point to different sides of the current picture (i.e., one referenced POC is greater than the POC of the current picture, and the other referenced POC is less than the POC of the current picture), and the difference in POC in List 0 is greater than the POC in List 1, the symbol in Table 2 specifies that the sign of the MV offset added to the List 0 MV component of the starting MV has the opposite value to the sign of the List 1 MV. Otherwise, if the difference in POC in List 1 is greater than List 0, the symbol in Table 2 specifies that the sign of the MV offset added to the List 1 MV component of the starting MV has the opposite value to the sign of the List 0 MV.

[0098] The MVD is scaled based on the difference in POC in each direction. If the difference in POC in both lists is the same, no scaling is required. Otherwise, if the POC difference in list 0 is greater than that in list 1, the MVD of list 1 is scaled by defining the POC difference of L0 as td and the POC difference of L1 as tb, as Figure 5 If the POC difference of L1 is greater than L0, the MVD of list 0 is scaled in the same way. If the starting MV is unidirectionally predicted, the MVD is added to the available MVs.

[0099] Table 2 - MV offset symbols specified by direction index

[0100] Directions to IDX 00 01 10 11 x-axis + - N / A N / A y-axis N / A N / A + -

[0101] Affine Motion Compensated Prediction

[0102] In HEVC, only the translational motion model is applied to motion compensation prediction (MCP). In the real world, there are many kinds of motion, such as zooming in / out, rotation, perspective motion, and other irregular motions. In VVC, block-based affine transformation motion compensation prediction is applied. Fig. 7A As shown in FIG. 1B , the affine motion field of block 710 is given by Fig. 7A The motion information of the two control points (4 parameters) in Figure 7B The three control point motion vectors (6 parameters) are described in .

[0103] For the 4-parameter affine motion model, the motion vector at the sample position (x, y) in the block is derived as:

[0104]

[0105] For the 6-parameter affine motion model, the motion vector at the sample position (x, y) in the block is derived as:

[0106]

[0107] Where (mv 0x ,mv 0y ) is the motion vector of the upper left control point, (mv 1x ,mv 1y ) is the motion vector of the upper right control point, (mv 2x ,mv 2y ) is the motion vector of the lower left control point.

[0108] To simplify motion compensated prediction, block-based affine transform prediction is applied. To derive the motion vector for each 4×4 luminance sub-block, the motion vector of the center sample of each sub-block is Figure 8 As shown, it is calculated according to the above equation and rounded to 1 / 16 fractional precision. Then, the motion compensated interpolation filter is applied to generate a prediction for each subblock with a derived motion vector. The subblock size of the chroma component is also set to 4×4. The MV of the 4×4 chroma subblock is calculated as the average of the MVs of the top left and bottom right luminance subblocks in the same 8x8 luminance region.

[0109] For translational motion inter prediction, there are also two affine motion inter prediction modes: affine merge mode and affine AMVP mode.

[0110] Affine Merge Predictions

[0111] AF_MERGE mode can be applied to CUs with width and height greater than or equal to 8. In this mode, the control point MV (CPMV) of the current CU is generated based on the motion information of the spatially adjacent CUs. There can be up to five CPMVP (CPMV prediction) candidates, and the index is sent to indicate which candidate to use for the current CU. The following three types of CPVM candidates are used to form the affine merge candidate list:

[0112] - Inherited affine merge candidates inferred from the CPMV of neighboring CUs

[0113] -Constructedaffine mergecandidate (CPMVP) derived from the translation MV of the adjacent CU

[0114] -Zero MV

[0115] In VVC, there are at most two inherited affine candidates, which come from the affine motion models of neighboring blocks, one from the left neighboring CU and one from the upper neighboring CU. Figure 2 The blocks shown are the same. For the left predictor, the scanning order is A 0- >A1, for the upper predictor, the scan order is B0->B1->B2. Only the first inherited candidate on each side is selected. No pruning check is performed between two inherited candidates. When a neighboring affine CU is identified, its control point motion vectors are used to derive the CPMVP candidate in the affine merge list of the current CU. Fig. 9As shown, if the lower left neighboring block A of the current block 910 is encoded and decoded in affine mode, the motion vectors v2, v3 and v4 of the upper left corner, upper right corner and lower left corner of the CU 920 containing block A are obtained. When block A is encoded and decoded using a 4-parameter affine model, the two CPMVs (i.e., v0 and v1) of the current CU are calculated based on v2 and v3. When block A is encoded and decoded using a 6-parameter affine model, the three CPMVs of the current CU are calculated based on v2, v3 and v4.

[0116] Constructing affine candidates means constructing candidates by combining the adjacent translation motion information of each control point. Fig.10 As shown, the motion information of the control point is derived from the specified spatial and temporal neighboring blocks of the current block 1010. CPMV k (k=1, 2, 3, 4) indicates the kth control point. For CPMV1, B2->B3->A2 blocks are checked and the MV of the first available block is used. For CPMV2, B1->B0 blocks are checked, and for CPMV3, A1->A0 blocks are checked. If TMVP is available, it is used as CPMV4.

[0117] After obtaining the MVs of the four control points, an affine merge candidate is constructed based on the motion information. The following combination of control point MVs is used to construct in order:

[0118] {CPMV1, CPMV2, CPMV3}, {CPMV1, CPMV2, CPMV4}, {CPMV1, CPMV3, CPMV4}, {CPMV2, CPMV3, CPMV4}, {CPMV1, CPMV2}, {CPMV1, CPMV3}

[0119] Combinations of 3 CPMVs construct 6-parameter affine merge candidates and combinations of 2 CPMVs construct 4-parameter affine merge candidates. To avoid motion scaling processing, the relevant combination of control point MVs is discarded if the reference indexes of the control points are different.

[0120] After the inherited affine merge candidates and the constructed affine merge candidates are checked, if the list is still not full, a zero MV is inserted at the end of the list.

[0121] Affine AMVP prediction

[0122] Affine AMVP mode can be applied to CUs with width and height greater than or equal to 16. A CU-level affine flag is sent in the bitstream to indicate whether affine AMVP mode is used, and then another flag is sent to indicate whether 4-parameter affine or 6-parameter affine is used. In this mode, the difference between the CPMV of the current CU and its predicted sub-CPMVP is sent in the bitstream. The affine AVMP candidate list size is 2, which is generated by the following four CPVM candidates in sequence:

[0123] - Inherited affine AMVP candidates inferred from the CPMV of neighboring CUs

[0124] -The constructed affine AMVP candidate CPMVP is derived using the translation MV of the neighboring CU -The translation MV from the neighboring CU

[0125] -Zero MV

[0126] The order in which inherited affine AMVP candidates are checked is the same as that of inherited affine merge candidates. The only difference is that for AVMP candidates, only affine CUs with the same reference picture as the current block are considered. When an inherited affine motion predictor is inserted into the candidate list, the pruning process is not applied.

[0127] The AMVP candidates constructed from Fig.10 The specified spatial neighboring blocks shown in are derived. The same check order as in the affine merge candidate construction is used. In addition, the reference picture index of the neighboring blocks is also checked. In the checking order, the first block that uses inter-frame coding and has the same reference picture in the current CU is used. When the current CU is encoded using 4-parameter affine mode, and both mv0 and mv1 are available, they are added to the affine AMVP list as a candidate. When the current CU is encoded using 6-parameter affine mode, and all three CPMVs are available, they are added to the affine AMVP list as a candidate. Otherwise, the constructed AMVP candidate is set to unavailable.

[0128] If the number of affine AMVP candidate lists is still less than 2 after inserting valid inherited affine AMVP candidates and constructed AMVP candidates, mv0, mv1, and mv2 are added as translation MVs to predict all control point MVs of the current CU when available. Finally, if the affine AMVP list is still not full, zero MVs are used to fill the affine AMVP list.

[0129] Affine motion information storage

[0130] In VVC, the CPMV of an affine CU is stored in a separate buffer. The stored CPMV is only used to generate inherited CPMVP for the most recently decoded CU in affine merge mode and affine AMVP mode. The sub-block MV derived from the CPMV is used for motion compensation, merged MV derive / translate MV's AMVP list and deblocking.

[0131] To avoid picture line buffers for extra CPMV, affine motion data inherited from CUs of the above CTUs is handled differently from that inherited from regular neighboring CUs. If the candidate CU for affine motion data inheritance is in the above CTU row, the bottom left and bottom right sub-block MVs in the line buffer are used for affine MVP derivation instead of CPMVs. In this way, CPMVs are only stored in the local buffer. If the candidate CU is a 6-parameter affine codec, the affine model degenerates to a 4-parameter model. Fig.11 As shown, along the top CTU boundary, the bottom left and bottom right sub-block motion vectors of the CU are used for affine inheritance of the CU in the bottom CTU. Fig.11 , columns 1110 and rows 1112 represent the x and y coordinates of the picture with the origin (0, 0) at the upper left corner. Legend 1120 shows the meaning of various motion vectors, where arrow 1122 represents the CPMV for affine inheritance in the local buffer, arrow 1124 represents the sub-block vector for MC / Merge / Skip / AMVP / Deblocking / TMVPs in the local buffer and the sub-block vector for affine inheritance in the row buffer, and arrow 1126 represents the sub-block vector for MC / Merge / Skip / AMVP / Deblocking / TMVPs.

[0132] Adaptive Motion Vector Resolution (AMVR)

[0133] In HEVC, when use_integer_mv_flag in the slice header is equal to 0, the motion vector difference (MVD) (between the CU's motion vector and the predicted motion vector) is sent in units of quarter brightness samples. In VVC, the CU-level adaptive motion vector resolution (AMVR) scheme is introduced. AMVR allows the CU's MVD to be encoded and decoded with different precisions. Depending on the current CU mode (normal AMVP mode or affine AVMP mode), the current CU's MVD can be adaptively selected as follows:

[0134] - Conventional AMVP modes: quarter brightness samples, half brightness samples, integer brightness samples, or quad brightness samples.

[0135] - Affine AMVP mode: quarter luma samples, integer luma samples, or 1 / 16 luma samples.

[0136] If the current CU has at least one non-zero MVD component, the CU-level MVD resolution indication is conditionally sent.If all MVD components (ie, horizontal and vertical MVD for reference list L0 and reference list L1) are zero, then quarter luma sample MVD resolution is inferred.

[0137] For CUs with at least one non-zero MVD component, a first flag is sent to indicate whether quarter luma sample MVD precision is used for the CU. If the first flag is 0, no further transmission is required, and quarter luma sample MVD precision is used for the current CU. Otherwise, a second flag is sent to indicate that half luma samples or other MVD precision (integer or four luma samples) are used for conventional AMVP CUs. In the case of half luma samples, a 6-tap interpolation filter is used for the half luma sample positions instead of the default 8-tap interpolation filter. Otherwise, a third flag is sent to indicate whether integer luma samples or four luma sample MVD precision is used for conventional AMVP CUs. In the case of affine AMVP CUs, the second flag is used to indicate whether integer luma samples or 1 / 16 luma sample MVD precision is used. To ensure that the reconstructed MV has the expected precision (quarter luma samples, half luma samples, integer luma samples, or four luma samples), the motion vector predictor of the CU will be rounded to the same precision as the MVD before being added to the MVD. The motion vector predictor is rounded towards zero (ie, a negative motion vector predictor is rounded towards positive infinity, and a positive motion vector predictor is rounded towards negative infinity).

[0138] The encoder uses the RD check to determine the motion vector resolution of the current CU. In VTM11, in order to avoid always performing four CU-level RD checks for each MVD resolution, the RD check of MVD precision other than quarter luma samples is only called conditionally. For the conventional AVMP mode, the RD cost of the quarter luma sample MVD precision and the integer luma sample MV precision is first calculated. Then, the RD cost of the integer luma sample MVD precision is compared with the RD cost of the quarter luma sample MVD precision to decide whether it is necessary to further check the RD cost of the four luma sample MVD precision. When the RD cost of the quarter luma sample MVD precision is much smaller than the RD cost of the integer luma sample MVD precision, the RD check of the four luma sample MVD precision is skipped. Then, if the RD cost of the integer luma sample MVD precision is significantly greater than the best RD cost of the previously tested MVD precision, the check of the half luma sample MVD precision is skipped. For the affine AMVP mode, if the affine inter mode is not selected after checking the rate-distortion costs of the affine merge / skip mode, merge / skip mode, quarter luma sample MVD accuracy regular AMVP mode, and quarter luma sample MVD accuracy affine AMVP mode, the 1 / 16 luma sample MV accuracy and 1 pixel MV accuracy affine inter modes are not checked. In addition, the affine parameters obtained in the quarter luma sample MV accuracy affine inter mode are used as the starting search points for the 1 / 16 luma sample and quarter luma sample MV accuracy affine inter modes.

[0139] Bi-Prediction with CU-level Weight (BCW)

[0140] In HEVC, the bidirectional prediction signal P bi-pred Generated by averaging two prediction signals P0 and P1 obtained from two different reference pictures and / or using two different motion vectors. In VVC, the bidirectional prediction mode is extended beyond simple averaging, allowing a weighted average of the two prediction signals.

[0141] P bi-pred =((8-w)*P0+w*P1+4)>>3 (3)

[0142] Weighted average bidirectional prediction allows five weights, w∈{-2,3,4,5,10}. For each bidirectionally predicted CU, the weight w is determined in one of two ways: 1) For non-merged CUs, the weight index is sent after the motion vector difference; 2) For merged CUs, the weight index is inferred from neighboring blocks based on the merge candidate index. BCW is only applicable to CUs with 256 or more luma samples (i.e., CU width multiplied by CU height is greater than or equal to 256). For low-latency pictures, all 5 weights are used. For non-low-latency pictures, only 3 weights (w∈{3,4,5}) are used. At the encoder, a fast search algorithm is used to find the weight index without significantly increasing the complexity of the encoder. These algorithms are summarized below. Detailed information is disclosed in the VTM software and document JVET-L0646 (Yu-Chi Su, et.al., "CE4-related: Generalized bi-prediction improvements combined from JVET-L0197and JVET-L0296", Joint Video Experts Team (JVET) of ITU-T SG 16WP 3and ISO / IEC JTC 1 / SC 29, 12th Meeting: Macao, CN, 3-12Oct.2018, Document: JVET-L0646).

[0143] - When combined with AMVR, unequal weights are only conditionally checked for 1-pixel and 4-pixel motion vector precision if the current picture is a low-latency picture.

[0144] - When combined with affine, if and only if the affine mode is selected as the current best mode,

[0145] Affine ME will be performed for unequal weights.

[0146] - Unequal weights are only checked conditionally when the two reference pictures in bidirectional prediction are the same.

[0147] - Unequal weights are not searched when certain conditions are met, which depend on the POC distance between the current picture and its reference pictures, the codec QP and the temporal level.

[0148] The BCW weight index is encoded using a context codec bin and a subsequent bypass codec bin. The first context codec bin indicates whether equal weights are used; if unequal weights are used, an additional bin is sent using the bypass codec to indicate which unequal weights are used.

[0149] Weighted prediction (WP for short) is a codec tool supported by the H.264 / AVC and HEVC standards for efficient encoding and decoding of video content with fading. Support for WP has also been added to the VVC standard. WP allows weighting parameters (weights and offsets) to be sent for each reference picture in each reference picture list L0 and L1. Then, during motion compensation, the weights and offsets of the corresponding reference pictures are applied. WP and BCW are designed for different types of video content. In order to avoid the interaction between WP and BCW, which complicates the VVC decoder design, if the CU uses WP, the BCW weight index is not sent, and the weight w is inferred to be 4 (ie, equal weights are applied). For the merged CU, the weight index is inferred from the neighboring blocks based on the merge candidate index. This can be applied to both the normal merge mode and the inherited affine merge mode. For the constructed affine merge mode, the affine motion information is constructed based on the motion information of up to 3 blocks. The BCW index of the CU using the constructed affine merge mode is simply set equal to the BCW index of the first control point MV.

[0150] In VVC, CIIP and BCW cannot be used jointly for a CU. When a CU is coded in CIIP mode, the BCW index of the current CU is set to 2, (ie, equal weight of w=4). Equal weight means the default value of the BCW index.

[0151] Combined Inter and Intra Prediction (CIIP)

[0152] In VVC, when a CU is encoded and decoded in merge mode, if the CU contains at least 64 luma samples (i.e., the CU width multiplied by the CU height is equal to or greater than 64), and if both the CU width and the CU height are less than 128 luma samples, an additional flag is sent to indicate whether the combined inter / intra prediction (CIIP) mode is applied to the current CU. As the name suggests, CIIP prediction combines the inter prediction signal with the intra prediction signal. The inter prediction signal in the CIIP mode Pinter is derived using the same inter prediction process as applied to the regular merge mode; and the intra prediction signal Pintra is derived following the regular intra prediction process with the planar mode. The intra and inter prediction signals are then combined using weighted averaging, where the weight value wt is calculated based on the top and left neighboring blocks of the current CU 1210 (such as Fig.12 The codec mode is calculated as follows:

[0153] - if the top neighboring block is available and is intra-coded, isIntraTop is set to 1,

[0154] Otherwise isIntraTop is set to 0;

[0155] - if the left neighboring block is available and is intra-coded, isIntraLeft is set to 1,

[0156] Otherwise isIntraLeft is set to 0;

[0157] - If (isIntraLeft + isIntraTop) is equal to 2, then wt is set to 3;

[0158] - Otherwise, if (isIntraLeft+isIntraTop) is equal to 1, wt is set to 2;

[0159] - Otherwise, wt is set to 1.

[0160] The CIIP forecast is formed as follows:

[0161] P CIIP =((4-wt)*P inter +wt*P intra +2)>>2 (4)

[0162] Cross Component Linear Model (CCLM)

[0163] The main idea behind the CCLM mode (sometimes abbreviated as LM mode) is that there is usually some correlation between the color components of a color picture (e.g., Y / Cb / Cr, YUV, and RGB). In this disclosure, these colors may be referred to as primary colors, secondary colors, and tertiary colors. The CCLM technique exploits the correlation by predicting the chrominance components of a block from the co-located reconstructed luminance samples using a linear model whose parameters are derived from the reconstructed luminance and chrominance samples adjacent to the block.

[0164] In VVC, the CCLM mode exploits inter-channel dependencies by predicting chrominance samples from reconstructed luma samples. This prediction is performed using a linear model of the following form:

[0165] P(i, j) = a·rec′ L (i,j)+b (5)

[0166] Here, P(i, j) represents the predicted chrominance sample in CU, and rec′ L (i, j) represents the reconstructed luma samples of the same CU, which are downsampled for non-4:4:4 color formats. The model parameters a and b are derived based on the adjacent luma and chroma samples reconstructed at the encoder and decoder sides and are not explicitly sent.

[0167] Three CCLM modes are specified in VVC, namely, CCLM_LT, CCLM_L and CCLM_T. These three modes differ in the location of the reference samples used for model parameter derivation. In CCLM_T mode, only samples from the top boundary are involved, and in CCLM_L mode, only samples from the left boundary are involved. In CCLM_LT mode, samples from both the top boundary and the left boundary are used.

[0168] In general, the forecasting process of CCLM model consists of three steps:

[0169] 1) downsampling the luma block and its adjacent reconstructed samples to match the size of the corresponding chroma block,

[0170] 2) derivation of model parameters based on reconstructed neighboring samples, and

[0171] 3) Apply the model equation (1) to generate chroma intra prediction samples.

[0172] Downsampling of the luma component: To match the chroma sample positions of a 4:2:0 or 4:2:2: color format video sequence, two types of downsampling filters can be applied to the luma samples, both of which have a 2:1 downsampling ratio in the horizontal and vertical directions. These two filters correspond to "type-0" and "type-2" 4:2:0 chroma format content, respectively, and are given by:

[0173]

[0174] Based on the SPS level flag information, a two-dimensional 6th order (i.e., f2) or 5th order (i.e., f1) filter is applied to the luma samples in the current block and its adjacent luma samples. The SPS level refers to the Sequence Parameter Set level. An exception occurs if the top row of the current block is a CTU boundary. In this case, a one-dimensional filter [1, 2, 1] / 4 is applied to the above adjacent luma samples to avoid using multiple luma lines above the CTU boundary.

[0175] Model parameter derivation process: The model parameters a and b from equation (5) are derived based on adjacent luminance and chrominance samples reconstructed at the encoder and decoder sides to avoid the need for any transmission overhead. In the initially adopted CCLM mode version, a linear minimum mean square error (LMMSE) estimator is used for parameter derivation. However, in the final design, only four samples are involved to reduce computational complexity. Fig.13The relative sample positions of an M×N chroma block 1310 of “Type-0” content, a corresponding 2M×2N luma block 1320 and its neighboring samples (shown as filled circles and triangles).

[0176] exist Fig.13 In the example of FIG. 1 , four samples used in CCLM_LT mode are shown, which are marked with triangles. They are located at the top boundary M / 4 and M·3 / 4, and the left boundary N / 4 and N·3 / 4. In CCLM_T and CCLM_L modes, the top boundary and the left boundary are extended to the size of (M+N) samples, and the four samples used for model parameter derivation are located at (M+N) / 8, (M+N)·3 / 8, (M+N)·5 / 8 and (M+N)·7 / 8.

[0177] Once four samples are selected, four comparison operations are used to determine the two smallest and two largest luma sample values. Let Xl denote the average of the two largest luma sample values, and let Xs denote the average of the two smallest luma sample values. Similarly, let Yl and Ys denote the averages of the corresponding chroma sample values. The linear model parameters are then obtained according to the following equations:

[0178]

[0179] b=Y s-a ·X s .(7)

[0180] In this equation, the division operation to calculate parameter a is implemented using a lookup table. To reduce the memory required to store the table, the diff value, which is the difference between the maximum and minimum values, and the parameter a are expressed in exponential notation. Here, the value of diff is approximated using a 4-bit significant part and an exponent. Therefore, the table for 1 / diff contains only 16 elements. This has the advantage of reducing both the complexity of the calculation and the size of the memory required to store the table.

[0181] MMLM Overview

[0182] As the name suggests, the original CCLM mode uses a linear model to predict the chrominance samples from the luma samples of the entire CU, while in MMLM (Multi-Model CCLM), there can be two models. In MMLM, the neighboring luma samples and neighboring chroma samples of the current block are divided into two groups, each group is used as a training set to derive a linear model (i.e., a specific α and β are derived for a specific group). In addition, the samples of the current luma block are also classified based on the same rules as the classification of the neighboring luma samples.

[0183] o The threshold is calculated as the average of the adjacent reconstructed luminance samples. Adjacent samples with Rec′L[x,y]<=threshold are classified into group 1; and adjacent samples with Rec′L[x,y]>threshold are classified into group 2.

[0184] o Accordingly, the prediction of chrominance is obtained using a linear model:

[0185]

[0186] Chroma Intra Mode Codec

[0187] For chroma intra mode coding and decoding, a total of 8 intra modes are allowed for chroma intra mode coding and decoding. These modes include five traditional intra modes and three cross-component linear model modes (CCLM, LM_A and LM_L). The chroma mode signaling and derivation process are shown in Table 3. The chroma mode coding and decoding depends directly on the intra prediction mode of the corresponding luminance block. Since separate block partitioning structures for luminance and chrominance components are enabled in the I segment, one chroma block can correspond to multiple luminance blocks. Therefore, for the chroma derived mode (DM) mode, the intra prediction mode of the corresponding luminance block covering the center position of the current chroma block is directly inherited.

[0188] Table 3. Chroma prediction modes derived from luma modes when CCLM is enabled

[0189]

[0190] As shown in Table 4, a single binarization table is used regardless of the value of sps_cclm_enabled_flag.

[0191] Table 4. Unified binarization table of chrominance prediction mode

[0192]

[0193]

[0194] The first bin indicates whether it is normal mode (ie 0) or LM mode (ie 1). If it is LM mode, the next bin indicates whether it is LM_CHROMA (ie 0) or not (ie 1). If it is not LM_CHROMA, the next bin indicates whether it is LM_L (ie 0) or LM_A (ie 1). For this case, when sps_cclm_enabled_flag is 0, the first bin of the binarization table corresponding to intra_chroma_pred_mode can be ignored before entropy encoding and decoding. In other words, the first bin is inferred to be 0 and therefore not encoded and decoded. This single binarization table is used for cases where sps_cclm_enabled_flag is equal to 0 and 1. The first two bins use their own context model for context encoding and decoding, and the remaining bins are bypass encoded and decoded.

[0195] Multi-Hypothesis prediction (MHP)

[0196] In the multiple hypothesis inter prediction mode (JVET-M0425), in addition to the traditional bidirectional prediction signal, one or more additional motion compensated prediction signals are sent. The final overall prediction signal is obtained by sample-wise weighted superposition. Using the bidirectional prediction signal pbi and the first additional inter prediction signal / hypothesis h3, the resulting prediction signal p3 is as follows:

[0197] p3=(1-α)p bi +αh3 (8)

[0198] The weight factor α is specified by a new syntax element add_hyp_weight_idx according to the following mapping (Table 5):

[0199] Table 5. Mapping α to add_hyp_weight_idx

[0200] add_hyp_weight_idx α 0 1 / 4 1 -1 / 8

[0201] Similar to the above, more than one additional prediction signal may be used. The resulting overall prediction signal is iteratively accumulated with each additional prediction signal.

[0202] p n+1 =(1-α n+1 ) n +α n+1 h n+1 (9)

[0203] The resulting overall prediction signal is obtained as the last pn (ie the pn with the largest index n). For example, at most two additional prediction signals (ie n is limited to 2) may be used.

[0204] The motion parameters for each additional prediction hypothesis can be signaled explicitly by specifying the reference index, motion vector predictor index and motion vector difference, or implicitly by specifying the merge index. A separate multi-hypothesis merge flag is used to distinguish between these two signaling modes.

[0205] For inter-frame AMVP mode, MHP is applied only when unequal weights in BCW are selected in bidirectional prediction mode. Details of MHP for VVC can be found in JVET-W2025 (Muhammed Coban, et.al., "Algorithm description of Enhanced Compression Model 2 (ECM 2)", Joint VideoExperts Team (JVET) of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29, 23rd Meeting, bytleconference, 7-16 July 2021, Document: JVET-W2025).

[0206] Intra-mode codec with 67 intra-prediction modes

[0207] To capture arbitrary edge directions that occur in natural video, the number of directional intra modes in VVC is extended from the 33 used in HEVC to 65. New directional modes not present in HEVC are indicated by dashed arrows in 14. Planar and DC modes remain unchanged. These denser directional intra prediction modes apply to all block sizes and for luma and chroma intra prediction.

[0208] In VVC, for non-square blocks, several traditional angular intra prediction modes are adaptively replaced with wide-angle intra prediction modes.

[0209] In HEVC, each intra-coded block has a square shape and the length of each of its sides is a power of 2. Therefore, no division operation is required to generate intra predictors using DC mode. In VVC, blocks can have a rectangular shape, which in general requires the use of division operations for each block. To avoid division operations for DC prediction, only the longer sides are used to calculate the average for non-square blocks.

[0210] In order to keep the complexity of most probable mode (MPM) list generation low, the intra mode codec with 6 MPMs is used by considering two available adjacent intra modes. The following three aspects are considered when constructing the MPM list:

[0211] - Default intra mode

[0212] - Adjacent Intra Mode

[0213] - Intra-mode for export

[0214] Regardless of whether the MRL and ISP codecs are applied, a unified 6-MPM list is used for intra blocks. The MPM list is constructed based on the intra modes of the neighboring blocks on the left and above. Assuming that the mode on the left is marked as Left and the mode of the block above is marked as Above, the unified MPM list is constructed as follows:

[0215] - When a neighboring block is not available, its intra mode is set to planar by default.

[0216] -If both Left and Above modes are non-angle modes:

[0217] --MPM list → {plane, DC, V, H, V-4, V+4}

[0218] - If one of Left and Above modes is an angle mode and the other is a non-angle mode:

[0219] - Set mode Max to the larger mode between Left and Above

[0220] --MPM list → {Plane, Max, DC, Max-1, Max+1, Max-2}

[0221] -If Left and Above are both angled and they are different:

[0222] - Set mode Max to the larger mode between Left and Above

[0223] - If the difference between the modes Left and Above is in the range 2 to 62, inclusive

[0224] MPM list → {Plane, Left, Above, DC, Max-1, Max+1}

[0225] -otherwise

[0226] MPM List → {Plane, Left, Above, DC, Max-2, Max+2}

[0227] -If Left and Above are both angled and they are the same:

[0228] --MPM list → {plane, Left, Left-1, Left+1, DC, Left-2}

[0229] In addition, the first bin of the MPM index codeword is decoded by the CABAC context. A total of three contexts are used, corresponding to whether the current intra block is MRL-enabled, ISP-enabled, or a normal intra block.

[0230] In the 6 MPM list generation processes, pruning is used to remove repeated patterns so that only unique patterns can be included in the MPM list. For the entropy coding and decoding of the 61 non-MPM modes, Truncated Binary Code (TBC) is used.

[0231] Wide-Angle Intra Prediction for Non-SquareBlocks

[0232] The conventional angular intra prediction direction is defined as clockwise from 45 degrees to -135 degrees. In VVC, several traditional angular intra prediction modes are adaptively replaced with wide-angle intra prediction modes for non-square blocks. The replaced modes are sent using the original mode index, which is remapped to the index of the wide-angle mode after parsing. The total number of intra prediction modes remains unchanged, i.e. 67, and the intra mode encoding and decoding method remains unchanged.

[0233] To support these prediction directions, a top reference of length 2W+1 and a left reference of length 2H+1 are respectively Fig.15A and Fig. 15B Definition shown.

[0234] The number of alternative modes in the wide-angle direction mode depends on the aspect ratio of the block. The alternative intra prediction modes are shown in Table 6.

[0235] Table 6 - Intra prediction modes replaced by wide angle mode

[0236] Aspect Ratio Alternative intra prediction modes W / H==16 Mode 12, 13, 14, 15 W / H==8 Mode 12, 13 W / H==4 Mode 2,3,4,5,6,7,8,9,10,11 W / H==2 Mode 2,3,4,5,6,7, W / H==1 none W / H==1 / 2 Mode 61,62,63,64,65,66 W / H==1 / 4 Mode 57,58,59,60,61,62,63,64,65,66 W / H==1 / 8 Mode 55,56 W / H==1 / 16 Modes 53, 54, 55, 56

[0237] like Fig.16As shown, in the case of wide-angle intra prediction, two vertically adjacent prediction samples (samples 1610 and 1612) can use two non-adjacent reference samples (samples 1620 and 1622). Therefore, a low-pass reference sample filter and edge smoothing are applied to wide-angle prediction to reduce the negative impact of the increased gap Δpα. It is assumed that the wide-angle mode represents a non-fractional offset. There are 8 modes in the wide-angle mode that meet this condition, namely [-14, -12, -10, -6, 72, 76, 78, 80]. When predicting blocks through these modes, the samples in the reference buffer are directly copied without applying any interpolation. Through this modification, the number of samples that need to be smoothed is reduced. In addition, it aligns the design of non-fractional modes in traditional prediction mode and wide-angle mode.

[0238] In VVC, 4:2:2 and 4:4:4 as well as 4:2:0 chroma formats are supported. The chroma derived mode (DM) derivation table for the 4:2:2 chroma format was originally ported from HEVC, expanding the number of entries from 35 to 67 to be consistent with the expansion of intra prediction modes. Since the HEVC specification does not support prediction angles below -135 degrees and above 45 degrees, the luma intra prediction modes from 2 to 5 are mapped to 2. Therefore, the chroma DM derivation table for the 4:2:2: chroma format is updated by replacing some values ​​of the mapping table entries to more accurately convert the prediction angles of chroma blocks.

[0239] Decoder Side Intra Mode Derivation (DIMD)

[0240] When DIMD is used, two intra modes are derived from reconstructed neighboring samples, and these two predictors are combined with a planar mode predictor with weights derived from the gradients. DIMD mode is used as an alternative prediction mode and is always checked in high complexity RDO mode.

[0241] In order to implicitly derive the intra prediction mode of a block, texture gradient analysis is performed on the encoder and decoder side. This process starts with an empty gradient histogram (HoG) with 65 entries, corresponding to 65 angular modes. The amplitudes of these entries are determined during texture gradient analysis.

[0242] In the first step, DIMD selects T=3 columns and rows of templates from the left and above the current block, respectively. This region is used as a reference for gradient-based intra prediction mode derivation.

[0243] In the second step, horizontal and vertical Sobel filters are applied to all 3×3 window locations, centered on the pixels in the template midline. At each window location, the Sobel filter computes the pure horizontal and vertical intensities Gx and Gy, respectively. The texture angle of the window is then calculated as:

[0244] angle=arctan(G x / G y ), (10)

[0245] This can be converted into one of the 65 angular intra prediction modes. Once the intra prediction mode index for the current window is derived as idx, the magnitude of its entry in HoG[idx] is updated by the following addition:

[0246] ampl=|G x |+|G y | ( 11 )

[0247] Fig.17A -C shows an example of the HoG calculated after applying the above operation to all pixel positions in the template. Fig.17A An example of a selected template 1720 is shown for a current block 1710. The template 1720 includes T rows above the current block and T columns to the left of the current block. For intra prediction of the current block, an area 1730 above and to the left of the current block corresponds to a reconstructed area, while an area 1740 below and to the right of the block corresponds to an unavailable area. Fig. 17B An example is shown where T=3 and HoG is calculated for pixel 1760 in the middle row and pixel 1762 in the middle column. For example, for pixel 1752, a 3x3 window 1750 is used. Fig. 17C An example of the amplitude (ampl) calculated based on equation (11) for the angular intra prediction mode as determined from equation (10) is shown.

[0248] Once the HoG is calculated, the indices with the two highest histogram bins are selected as the two implicitly derived intra prediction modes for the block, and further combined with the planar mode as the prediction for the DIMD mode. Prediction fusion is applied as a weighted average of the three predictors mentioned above. For this purpose, the weight of the plane is fixed to 21 / 64 (~1 / 3). The remaining weight 43 / 64 (~2 / 3) is then shared between the two HoG IPMs, proportional to the amplitude of their HoG bins. Fig.18 An example of mixed processing is shown. Fig.18As shown, two intra modes (M1 1812 and M2 1814) are selected based on the index of the two highest bars with the histogram bar 1810. Three predictors (1840, 1842 and 1844) are used to form a hybrid prediction. The three predictors correspond to applying M1, M2 and planar intra modes (1820, 1822 and 1824, respectively) to the reference pixel 1830 to form the corresponding predictor. The three predictors are weighted by corresponding weighting factors (ω1, ω2 and ω3) 1850. The weighted predictors are summed using an adder 1852 to generate a hybrid predictor 1860.

[0249] Furthermore, two implicitly derived intra modes are included in the MPM list so that DIMD processing is performed before building the MPM list. The primary derived intra mode of a DIMD block is stored with the block and used for MPM list construction for neighboring blocks.

[0250] Template-based Intra Mode Derivation (TIMD)

[0251] The template-based intra mode derivation (TIMD) mode implicitly derives the intra prediction mode of a CU using neighboring templates at the encoder and decoder, instead of sending the intra prediction mode to the decoder. Fig.19 As shown, the template prediction samples (1912 and 1914) of the current block 1910 are generated using the reference samples (1920 and 1922) of the template of each candidate mode. The cost is calculated as the sum of the absolute transformed differences (SATD) between the predicted samples and the reconstructed samples of the template. The intra prediction mode with the lowest cost is selected as the DIMD mode and used for intra prediction of the CU. The candidate mode can be 67 intra prediction modes as in VVC or extended to 131 intra prediction modes. Typically, MPM can provide clues to indicate the directional information of the CU. Therefore, in order to reduce the intra mode search space and utilize the characteristics of the CU, the intra prediction mode can be implicitly derived from the MPM list.

[0252] For each intra prediction mode in the MPM, the SATD between the predicted and reconstructed samples of the template is calculated. The first two intra prediction modes with the smallest SATD are selected as TIMD modes. These two TIMD modes are fused with weights after applying PDPC processing, and this weighted intra prediction is used to encode and decode the current CU. The position-dependent intra prediction combination (PDPC) is included in the derivation of the TIMD mode.

[0253] The costs of the two selection modes are compared to a threshold, and in the test a cost factor of 2 is applied as follows:

[0254] costMode2<2*costMode 1.

[0255] If this condition is true, fusion is applied, otherwise only mode 1 is used. The weight of the mode is calculated based on its SATD cost as follows:

[0256] weightl=costMode2 / (costMode1+costMode2)

[0257] weight2=1-weight1.

[0258] In order to improve the efficiency of video coding and decoding, more and more coding tools are designed to generate / refine the predictors of intra-frame and / or inter-frame blocks. Here, intra-frame and / or inter-frame are defined in the standard with mode types. For example, intra-frame refers to the mode type intra-frame, and inter-frame refers to the mode type inter-frame. The proposed method is not limited to improving blocks with traditional mode types, and can be used for blocks with any mode type defined in the standard. For traditional mechanisms, the inter-frame mode uses temporal information to predict the current block, and for intra-frame blocks, spatially adjacent reference samples are used to predict the current block. In the present invention, the coding tool uses cross-component information to predict or further improve the predictor of the current block. The concept of the coding tool is described as follows.

[0259] - First, the color components (eg, Y, Cb, and Cr) of the current block are divided into several groups, and one color component is selected as a representative color component of each group.

[0260] In one embodiment, Y is in the first group, and Cb and Cr are in the second group. For example, Y is the representative color component of the first group. For another example, one of Cb and Cr is the representative color component of the second group. For another example, for the second group, the information from the representative color component is the average information from Cb and Cr.

[0261] In another embodiment, Y is in the first group, Cb is in the second group, and Cr is in the third group. The representative color components of the first group, the second group, and the third group are Y, Cb, and Cr, respectively.

[0262] In another embodiment, Cb is in the first group, and Cr is in the second group. The representative color components of the first group and the second group are Cb and Cr, respectively.

[0263] - Second, neighboring samples (which may be neighboring reconstructed or predicted samples) of the first representative color component and the second (or third) representative color component are used to generate model parameters.

[0264] In another embodiment, the model is a linear model and the model parameters include α and β. Third, the model parameters are performed on samples (belonging to the first group) within the current block (which may be the current reconstruction or current prediction samples) to obtain the second (or third) setting.

[0265] P(i, j) = alpha·rec first_set (i, j)+beta

[0266] Or P(i, j) = alpha·pred first_set (i, j)+beta

[0267] O If the first group is for luma components and the second (or third) group is for chroma components, the downsampling process is applied to the first group.

[0268] - In another sub-embodiment, the cross-component predictor may be the final predictor of the second (or third) group.

[0269] - In another sub-embodiment, the cross-component predictor is mixed with the existing predictors of the second (or third) group. This is an example of mixing one additional prediction hypothesis on top of the existing prediction hypotheses. The proposed method is not limited to mixing only one additional prediction hypothesis, but can also be extended to mixing multiple prediction hypotheses.

[0270] OP final (i, j) = (w1·P existing (i,j)+w2·P+c)>>d

[0271] For example, w1 and w2 can be based on samples. Each sample has its own weight. When the template matching setting is used, one prediction is suggested from the upper template and the other prediction is suggested from the left template. The weight depends on the distance between the current sample and the upper template and / or the distance between the current sample and the left template. Samples close to the upper template have higher weights for the prediction of candidates suggested by the upper template. Samples close to the left template have higher weights for the prediction of candidates suggested by the left template. The proposed method can be used in a boundary matching setting and / or a model accuracy setting. In one embodiment, P existing P is generated by one mode suggested by a sub-template, and P is generated by another mode suggested by another sub-template. existing Indicated by signaling, and multiple Ps are generated by multiple modes suggested by the sub-templates.

[0272] For example, w1 and w2 are uniform for the current block. The weight depends on P and P existing When the template matching setting is used, predictions with smaller template matching costs have higher weights. When the boundary matching setting is used, predictions with smaller boundary matching costs have higher weights. When the model accuracy setting is used, predictions with smaller distortion have higher weights. In one embodiment, P existing is the mode with the smallest sample matching cost (or boundary matching cost / model accuracy distortion) and / or P is the mode with the second smallest sample matching cost (or boundary matching cost / model accuracy distortion). When more prediction hypotheses are mixed, more modes with small sample matching costs (or boundary matching costs / model accuracy distortion) will be used. In another embodiment, P existing Indicated by signaling, and the recommended settings are used to decide the weights and / or one or more Ps to be mixed.

[0273] For another example, w1 and w2 depend on the neighboring blocks. When the number of neighboring intra (or CCLM) blocks is greater than the number of neighboring inter (or non-intra or non-CCLM) blocks, w2 is greater than w1.

[0274] · Adjacent blocks refer to the top and left adjacent blocks.

[0275] The neighboring block refers to any predetermined 4x4 block around the left and top of the current block.

[0276] In the above example, the final predictor (i.e., P final (i, j)) includes part of the first predictor (i.e., w1·P existing (i, j)) and a portion of the at least one second predictor (i.e., w2·P). In one embodiment, P existing From the cross-component mode. In another embodiment, Pexisting It is intra prediction, inter prediction or third type prediction. existing The prediction type implies the mode type of the current block. existing When the prediction is intra, the current block is of mode type intra. existing When P is inter prediction, the current block is of mode type inter. existing When the prediction is the third type, the current block is of the third mode type. The third prediction can be generated by using an intra-block copy scheme to obtain a relative displacement from the position of the current block to the position of the reference block through (1) a displacement vector (referred to as a block vector or BV) indicating the relative displacement from the position of the current block to the position of the reference block and / or (2) a template matching mechanism for searching for a reference block in a predetermined search area, and / or the third mode type can refer to intra-block copy (IBC) or a special intra-mode type, such as intra-template matching prediction (intra-TMP). Although specific equations are used to illustrate combining two predictors to form a final predictor, the specific form should not be understood as a limitation of the present invention. For example, an offset can be added to the weighted sum of the first predictor and the second predictor before the shift operation (i.e., “>>d”). In addition, w1 and w2 can be represented as w1(i, j) and w2(i, j) because in one embodiment w1 and w2 can be based on samples.

[0277] In another sub-embodiment, the codec tool corresponds to CCLM or MMLM.

[0278] In another sub-embodiment, the codec tool corresponds to a tool that uses cross-component information to improve the predictor of the current block. The codec tool may include various candidate modes. Different modes may use different ways to derive model parameters. For example, the codec tool corresponds to CCLM, and the candidate mode corresponds to CCLM_LT, CCLM_L, CCLM_T or any combination of the above. For another example, the codec tool corresponds to MMLM and the candidate mode corresponds to MMLM_LT, MMLM_L, MMLM_T or any combination of the above. For another example, the codec tool corresponds to the LM series (including CCLM and MMLM), and the candidate mode corresponds to CCLM_LT, CCLM_L, CCLM_T, MMLM_LT, MMLM_L, MMLM_T or any combination of the above. For example, the convolutional cross-component mode (CCCM for short) is a cross-component mode. When the cross-component mode is applied to the current block, cross-component information with one or more models (including nonlinear terms and / or derived using a predetermined regression method) is used to generate chrominance prediction. This cross-component mode can follow the template selection of CCLM, so the CCCM series includes CCCM_LT CCCM_L and / or CCCM_T. For another example, the gradient linear model (GLM) that uses the brightness sample gradient to predict the chrominance sample is a cross-component mode. Candidates for the GLM mode can refer to different gradient filters and / or different variants of GLM. Different GLM variants can use one or more two-parameter models and / or one or more three-parameter models. When using a two-parameter GLM, the brightness sample gradient is used to derive the linear model. When using a three-parameter GLM, the chrominance sample can be predicted based on the brightness sample gradient and the downsampled brightness value with different parameters. The model parameters of the three-parameter GLM are derived as a predetermined regression method for CCCM. An example of a predetermined regression method is to use 6 rows and 6 columns of adjacent samples through a decomposition-based minimization method. For another example, for the cross-component mode, different candidates refer to different downsampling processes (e.g., downsampling filters). That is, for the cross-component mode, the brightness sample is first downsampled using a selected downsampling filter, and then used to derive model parameters and / or predict chrominance samples. When applying the template matching setting (or boundary matching setting / model accuracy setting) to select a downsampling filter, a template (or boundary) including N1 rows of neighboring samples adjacent to the top of the current chroma block and / or N2 rows of neighboring samples adjacent to the left of the current chroma block is predefined for measuring the cost of each candidate filter. The cost of a candidate filter is derived based on the reconstructed chroma samples in a predetermined template (or boundary) and the corresponding predictor of the candidate filter.Finally, the candidate filter with the minimum cost is selected as the downsampling filter to generate the prediction of the current block. N1 and N2 are any predefined integers, such as 1, 2, 4, 8, or adaptive adjustment values ​​depending on block width, block height and / or block area. For more settings of N1 and / or N2, please refer to the description of n and / or m lines in the boundary matching setting section.

[0279] In another embodiment, explicit rules are used to decide whether to enable or disable a codec tool and / or when a codec tool is enabled, explicit rules are used to decide candidate modes. For example, a flag is sent / parsed at the block level. If the flag is true, the codec tool is applied to the current block; otherwise, the codec tool is disabled for the current block.

[0280] In another embodiment, an implicit rule is used to decide whether to enable or disable the codec tool and / or when the codec tool is enabled, the implicit rule is used to decide the candidate mode. For example, the implicit rule depends on the template matching setting, the boundary matching setting or the model accuracy setting.

[0281] In another embodiment, Cb and Cr may use different candidate modes.

[0282] In another embodiment, the implicit rules for intra blocks and inter blocks can be unified. For example, when template settings are used as implicit rules, the derivation process of template settings for inter blocks is unified with the process of intra blocks (eg, TIMD blocks).

[0283] In another embodiment, the threshold used in template matching and / or boundary matching and / or model accuracy may depend on the block size of the current block, the sequence resolution, the neighboring blocks and / or the QP.

[0284] Template-matching settings

[0285] Step 0: When the template matching setting is used, the model parameters of each candidate pattern are derived based on the reference samples of the luminance and chrominance templates, and then the derived model parameters are performed on the current template (ie, the neighboring region). Fig. 20 Examples of templates for luminance and chrominance and reference samples of the templates used to derive model parameters and distortion are shown. Fig. 20 , block 2010 represents the current chrominance block (Cb or Cr) and block 2020 represents the corresponding luma block. Region 2012 corresponds to the chrominance template, and region 2014 corresponds to the reference sample of the chrominance template. Region 2022 corresponds to the luma template, and region 2024 corresponds to the reference sample of the luma template.

[0286] Take the LM series as an example:

[0287] - Different model parameters are derived from different LM modes (i.e., candidate sets).

[0288] O The model parameters (e.g., alpha and beta) of each LM mode (i.e., each candidate mode) are derived by using reference samples (reconstructed or predicted samples) of the luma and chroma templates.

[0289] The derived model parameters may then include the individual candidate modes:

[0290] ·alphaCCLM_LT_cb, betaCCLM_LT_cb, alphaCCLM_LT_cr, betaCCLM_LT_cr

[0291] ·alphaCCLM_L_cb, betaCCLM_L_cb, alphaCCLM_L_cr, betaCCLM_L_cr

[0292] ·alphaCCLM_T_cb, betaCCLM_T_cb, alphaCCLM_T_cr, betaCCLM_T_cr

[0293] ·alphaMMLM_LT_cb, betaMMLM_LT_cb, alphaMMLM_LT_cr, betaMMLM_LT_cr

[0294] ·alphaMMLM_L_cb, betaMMLM_L_cb, alphaMMLM_L_cr, betaMMLM_L_cr

[0295] ·alphaMMLM_T_cb, betaMMLM_T_cb, alphaMMLM_T_cr, betaMMLM_T_cr

[0296] Step 1: Take the reconstructed sample on the current block template as the golden data (i.e., the target data to be compared or matched).

[0297] Step 2: For each candidate mode, apply the derived model parameters to the template of the corresponding luma block to obtain the predicted sample within the current chroma block template

[0298] Step 3: For each candidate pattern, calculate the distortion between the golden data and the predicted sample on the template.

[0299] Step 4: Determine the mode of the current block based on the calculated distortion.

[0300] - In another sub-embodiment, the candidate mode with the smallest distortion is selected and used for the current block.

[0301] - In another sub-embodiment, the model parameters of the candidate mode with the smallest distortion are selected and used for the current block.

[0302] - In another sub-embodiment, regarding the enabling condition of the codec tool, when the minimum distortion is less than a predetermined threshold, the codec tool may be applied to the current block.

[0303] For example, the predetermined threshold is the T* template area. :

[0304] T can be any floating point value or 1 / N. (N can be any positive integer)

[0305] The template area is set to template width * current block height + template height * current block width.

[0306] For another example, the predetermined threshold is the distortion between the reconstructed samples of the current block template and the predicted samples of the template generated from the default mode (the original mode, without using the proposed codec tool improvement). When cross-component prediction is used for refinement of inter prediction, the default mode is the original inter mode, which can be any of normal, merge candidate, AMVP candidate, affine candidate, GPM candidate or merge candidate.

[0307] - In another sub-embodiment,

[0308] O If the minimum distortion of Cb is less than a predetermined threshold, the candidate mode with the minimum distortion is used for Cb.

[0309] O Otherwise, no candidate mode can be applied to Cb.

[0310] O If the minimum distortion of Cr is less than a predetermined threshold, the candidate mode with the minimum distortion is used for Cr.

[0311] O Otherwise, no candidate mode can be applied to Cr.

[0312] - In another sub-embodiment, it is decided whether to apply any candidate mode to both Cb and Cr simultaneously.

[0313] (Take LM as an example. When LM is applied to Cb, LM is also applied to Cr.)

[0314] If the minimum distortion of Cb and the minimum distortion of Cr are less than a predetermined threshold, LM is applied to Cb and Cr.

[0315] Otherwise, LM is not applicable to Cb and Cr.

[0316] If the minimum distortion of Cb or the minimum distortion of Cr is less than a predetermined threshold, LM is applied to Cb and Cr.

[0317] Otherwise, LM is not applicable to Cb and Cr.

[0318] - In another embodiment, the template size can be adjusted according to the description in the boundary matching settings. (e.g., the description of n and / or m rows in the boundary matching settings section)

[0319] As described in the TM-based method above, a second color template including selected adjacent samples of the second color block and a first color template including corresponding adjacent samples of the first color block are determined. For example, the first color can be a luminance signal and the second color can be one or both of the chrominance components. In another example, the first color can be one of the chrominance components (e.g., Cb / Cr) and the second color can be another of the chrominance components (e.g., Cr / Cb). Based on the reference samples of the first color template and the reference samples of the second color template, a set of model parameters (e.g., α and β) is determined for each prediction model of the candidate set. The candidate set may include some modes selected from CCLM_TL, CCLM_T, CCLM_L, MMLM_TL, MMLM_T, and MMLM_L. Fig. 20 An example of a template is shown in FIG. However, the template may include only the top template, only the left template, or the top template and the left template. In another example, the template selection may depend on the coding mode information of the current block or the candidate type of the candidate in the candidate set.

[0320] Boundary Matching Settings

[0321] like Fig.21 As shown, when the boundary matching setting is used, the boundary matching cost of the candidate mode refers to the discontinuity measurement (including top boundary matching and / or left boundary matching) between the current prediction (i.e., the prediction sample in the current block) generated from the candidate mode and the adjacent reconstruction (i.e., the reconstructed sample in one or more adjacent blocks), where predi,j refers to the prediction block, recoi,j refers to the adjacent reconstructed block, and block 2110 (as shown in the bold box) corresponds to the current block. The top boundary matching refers to the comparison between the current top prediction sample and the adjacent top reconstructed sample, and the left boundary matching refers to the comparison between the current left prediction sample and the adjacent left reconstructed sample.

[0322] In another sub-embodiment, the candidate pattern with the smallest boundary matching cost is applied to the current block.

[0323] In another sub-embodiment, regarding the enabling condition of the codec tool, when the minimum boundary matching cost is less than a predetermined threshold, the codec tool is applied to the current block. For example, the predetermined threshold is the boundary matching cost of the default mode (e.g., the original mode, without using the proposed codec tool improvement). When cross-component prediction is used to refine inter-frame prediction, the default mode is the original inter-frame mode, which can be any of conventional, merge candidate, AMVP candidate, affine candidate, GPM candidate or merge candidate.

[0324] - In another sub-embodiment,

[0325] O If the minimum distortion of Cb is less than a predetermined threshold, the candidate mode with the minimum distortion is used for Cb.

[0326] O Otherwise, no candidate mode can be applied to Cb.

[0327] O If the minimum distortion of Cr is less than a predetermined threshold, the candidate mode with the minimum distortion is used for Cr.

[0328] O Otherwise, no candidate mode can be applied to Cr.

[0329] - In another sub-embodiment, it is decided whether to apply any candidate mode to both Cb and Cr simultaneously.

[0330] (Take LM as an example. When LM is applied to Cb, LM is also applied to Cr.)

[0331] O If the minimum distortion of Cb and Cr is less than a predetermined threshold, LM is applied to Cb and Cr.

[0332] Otherwise, LM is not applicable to Cb and Cr.

[0333] If the minimum distortion of Cb or the minimum distortion of Cr is less than a predetermined threshold, LM is applied to Cb and Cr.

[0334] O Otherwise, LM is not used for Cb and Cr.

[0335] In one embodiment, a predetermined subset of the current prediction is used to calculate the boundary matching cost. The nth row of the top boundary in the current block and / or the mth row of the left boundary in the current block are used. (In addition, the nth row of the top adjacent reconstruction and / or the mth row of the left adjacent reconstruction are used.)

[0336] In the example of calculating the boundary matching cost, n=2, m=2, n2=2, m2=2:

[0337]

[0338] In the above formula, the weights (a, b, c, d, e, f, g, h, i, j, k, l) can be any positive integer, for example, a=2, b=1, c=1, d=2, e=1, f=1, g=2, h=1, i=1, j=2, k=1, and 1=1.

[0339] In another example of computing the cost of a boundary match, n=2, m=2, n2=1, and m2=1:

[0340]

[0341] In the above formula, the weights (a, b, c, g, h and i) can be any positive integers, for example, a=2, b=1, c=1, g=2, h=1, and i=1.

[0342] In yet another example of calculating the boundary matching cost, n=1, m=1, n2=2 and m2=2:

[0343]

[0344] In the above formula, the weights (d, e, f, j, k and l) can be any positive integers, for example, d=2, e=1, f=1, j=2, k=1, l=1.

[0345] In yet another example of calculating the boundary matching cost, n=1, m=1, n2=1 and m2=1:

[0346]

[0347] In the above equation, the weights (a, c, g and i) can be any positive integers, for example, a=1, c=1, g=1 and i=1.

[0348] In yet another example of calculating the boundary matching cost, n=2, m=1, n2=2 and m2=1:

[0349]

[0350] In the above formula, the weights (a, b, c, d, e, f, g and i) can be any positive integers, for example, a=2, b=1, c=1, d=2, e=1, f=1, g=1, i=1.

[0351] In yet another example of calculating the boundary matching cost, n=1, m=2, n2=1 and m2=2:

[0352]

[0353] In the above formula, the weights (a, c, g, h, i, j, k and l) can be any positive integers, for example, a=1, c=1, g=2, h=1, i=1, j=2, k=1, and 1=1.

[0354] (The following examples for n and m can also be applied to n2 and m2.)

[0355] For another example, n can be any positive integer, such as 1, 2, 3, 4, etc.

[0356] For another example, m can be any positive integer, such as 1, 2, 3, 4, etc.

[0357] For another example, n and / or m vary with block width, height, or area. In one embodiment, m becomes larger for larger blocks (e.g., area > threshold 2). For example,

[0358] OThreshold 2 = 64, 128 or 256.

[0359] When area > threshold 2, m increases to 2. (Initially, m is 1.)

[0360] When area > threshold 2, m increases to 4. (Initially, m is 1 or 2.)

[0361] In another example, for taller blocks, m becomes larger and / or n becomes smaller (e.g., height > threshold 2*width).

[0362] OThreshold 2 = 1, 2 or 4.

[0363] When height > threshold 2*width, m increases to 2. (Initially, m is 1.)

[0364] O When height > threshold 2*width, m increases to 4. (Initially, m is 1 or 2.)

[0365] In another embodiment, n becomes larger for larger blocks (area > threshold 2).

[0366] OThreshold 2 = 64, 128 or 256.

[0367] When area > threshold 2, n increases to 2. (Initially, n is 1.)

[0368] When area > threshold 2, n increases to 4. (Initially, n is 1 or 2.)

[0369] In another embodiment, for wider blocks (width > threshold 2*height) n becomes larger and / or m becomes smaller. For example,

[0370] OThreshold 2 = 1, 2 or 4.

[0371] When width > threshold 2*height, n increases to 2. (Initially, n is 1.)

[0372] When width > threshold 2*height, n increases to 4. (Initially, n is 1 or 2.)

[0373] As described in the above boundary matching-based method, the cost of each prediction model of the candidate set corresponds to the boundary matching cost, which is used to measure the discontinuity between the predicted samples of the second color block and the adjacent reconstructed samples of the second color. The predicted samples of the second color block are derived based on the first color block using a model parameter set determined for each prediction model. For example, the first color can be a luminance signal and the second color can be one or both of the chrominance components. In another example, the first color can be one of the chrominance components (e.g., Cb / Cr) and the second color can be another of the chrominance components (e.g., Cr / Cb). The model parameter set may include alpha and beta. The candidate set may include some modes selected from CCLM_TL, CCLM_T, CCLM_L, MMLM_TL, MMLM_T, and MMLM_L. Fig.21 An example of a boundary is shown in FIG. However, the boundary may include only the top boundary, only the left boundary, or both the top boundary and the left boundary. In another example, the boundary selection may depend on the coding mode information of the current block or the candidate type of the candidate in the candidate set.

[0374] Model accuracy settings

[0375] Step 0: When the model accuracy setting is used, the model parameters of each candidate mode are performed on the template (i.e., the neighboring region) of the current block. Fig. 22 An example of a template for deriving model parameters and distortion for luminance and chrominance is shown. Fig. 22 In FIG. 2 , block 2210 represents the current chrominance block (Cb or Cr) and block 2220 represents the corresponding luminance block. Region 2212 corresponds to the chrominance template. Region 2222 corresponds to the luminance template. Take the LM series as an example.

[0376] -Different model parameters are derived from different LM modes.

[0377] O applies each LM mode by using adjacent reconstructed luma samples and adjacent reconstructed chroma samples to obtain the model parameters (i.e., alpha and beta).

[0378] Then, the derived model parameters of each candidate mode may include:

[0379] ·alphaCCLM_LT_cb, betaCCLM_LT_cb, alphaCCLM_LT_cr, betaCCLM_LT_cr

[0380] ·alphaCCLM_L_cb, betaCCLM_L_cb, alphaCCLM_L_cr, betaCCLM_L_cr

[0381] ·alphaCCLM_T_cb, betaCCLM_T_cb, alphaCCLM_T_cr, betaCCLM_T_cr

[0382] ·alphaMMLM_LT_cb, betaMMLM_LT_cb, alphaMMLM_LT_cr, betaMMLM_LT_cr

[0383] ·alphaMMLM_L_cb, betaMMLM_L_cb, alphaMMLM_L_cr, betaMMLM_L_cr

[0384] ·alphaMMLM_T_cb, betaMMLM_T_cb, alphaMMLM_T_cr, betaMMLM_T_cr

[0385] Step 1: Take the reconstructed sample of the current block template as golden data.

[0386] Step 2: For each candidate mode, apply the derived model parameters to the reconstructed / predicted samples in the corresponding luma block template to obtain the predicted samples in the current chroma block template

[0387] Step 3: For each candidate pattern, calculate the distortion between the golden data and the predicted sample on the template.

[0388] There are many ways to calculate the distortion in step 3. In one embodiment, the template used in the distortion calculation is a template for model parameter derivation. In another embodiment, the template selection may depend on the coding mode information of the current block or the candidate type of the candidate in the candidate set.

[0389] For example, for CCLM_LT / MMLM_LT, the template used in the distortion calculation is the template containing the left template and the top template.

[0390] In another example, for CCLM_L / MMLM_L, the template used in the distortion calculation is the template including the left template.

[0391] In another example, for CCLM_T / MMLM_T, the template used in the distortion calculation is the template including the top template.

[0392] In another embodiment, the template used in the distortion calculation is a template including a left template and a top template.

[0393] Step 4: Determine the mode of the current block based on the calculated distortion.

[0394] - In another sub-embodiment, the candidate mode with the smallest distortion is used for the current block.

[0395] - In another sub-embodiment, regarding the enabling condition of the codec tool, when the minimum distortion is less than a predetermined threshold, the codec tool is applied to the current block.

[0396] - For example, the predetermined threshold is the T* template area.

[0397] OT can be any floating point value or 1 / N (N can be any positive integer).

[0398] O The template area is set to template width * current block height + template height * current block width.

[0399] - For example, the predetermined threshold is a distortion between a reconstructed sample of the current block template and a predicted sample of a template generated from a default mode. When cross-component prediction is used to refine inter prediction, the default mode is the original inter mode, which can be any of normal, merge candidate, AMVP candidate, affine candidate, GPM candidate or merge candidate.

[0400] - In another sub-embodiment,

[0401] O If the minimum distortion of Cb is less than a predetermined threshold, the candidate mode with the minimum distortion is used for Cb.

[0402] O Otherwise, no candidate mode can be applied to Cb.

[0403] O If the minimum distortion of Cr is less than a predetermined threshold, the candidate mode with the minimum distortion is used for Cr.

[0404] Otherwise, no candidate mode can be applied to Cr

[0405] - In another sub-embodiment, it is decided whether to apply any candidate mode to both Cb and Cr simultaneously.

[0406] (Take LM as an example. When LM is applied to Cb, LM is also applied to cr.)

[0407] If the minimum distortion of Cb and the minimum distortion of Cr are less than a predetermined threshold, LM is applied to Cb and Cr.

[0408] O Otherwise, LM is not used for Cb and Cr.

[0409] If the minimum distortion of Cb or the minimum distortion of Cr is less than a predetermined threshold, LM is applied to Cb and Cr.

[0410] O Otherwise, LM is not used for Cb and Cr.

[0411] As described in the above-mentioned model accuracy-based method, a second color template including selected neighboring samples of the second color block and a first color template including corresponding neighboring samples of the first color block are determined. For example, the first color can be a luminance signal and the second color can be one or both of the chrominance components. In another example, the first color can be one of the chrominance components (e.g., Cb / Cr) and the second color can be another of the chrominance components (e.g., Cr / Cb). A set of model parameters for each prediction model of the candidate set is determined based on the second color template and the first color template, and wherein the cost of each prediction model of the candidate set is determined based on the reconstructed samples and the predicted samples of the second color template. The predicted samples of the second color template are derived by applying the one or more model parameters determined for each prediction model to the first color template.

[0412] The method proposed in the present invention can be enabled and / or disabled according to implicit rules (such as block width, height or area) or according to explicit rules (such as syntax in block, tile, segment, picture, sequence parameter set (Sequance Parameter Set) or picture parameter set (Picture Parameter Set) level). For example, when the block width, height and / or area is less than a threshold, the proposed method is applied. For example, when the block width, height and / or area is greater than a threshold, the proposed method is applied.

[0413] The term "block" in the present invention may refer to TU / TB, CU / CB, PU / PB, a predetermined area or CTU / CTB. The following is an example of the current block referencing a CU. In single-tree partitioning, the current block refers to a CU containing Y, Cb, and Cr. When the proposed method is used for chrominance components to improve prediction, the corresponding luminance components may remain unchanged. That is, if the current CU is of inter-frame or IBC mode type, the luminance component still uses motion compensation or intra-frame block copying scheme to generate luminance prediction. In dual-tree partitioning, for the luminance dual tree, one luminance CU contains Y, and for the chrominance dual tree, the current block refers to one chrominance CU containing Cb and Cr.

[0414] The term "LM" in the present invention may be regarded as one of the CCLM / MMLM modes or any other extension / variant of CCLM (eg, the CCLM extension / variant proposed in the present invention).

[0415] The method proposed in the present invention (for CCLM) can be used for any other LM mode.

[0416] Any combination of the methods proposed in the present invention may be applied.

[0417] Any of the aforementioned proposed implicit cross-component prediction methods for codec tools using hybrid predictors may be implemented in an encoder and / or a decoder. For example, a hybrid predictor corresponds to two cross-component intra- or inter-predictors, which may be implemented in an inter / intra / prediction module of an encoder and / or an inter / intra / prediction module of a decoder. For example, on the encoder side, the required processing may be implemented as part of the inter-prediction unit 112 or the intra-prediction unit 110 as shown in FIG. 1A . However, the encoder may also use additional processing units to implement the required processing. For the decoder side, the required processing may be implemented as part of the inter-prediction unit 112 or the intra-prediction unit 110 as shown in FIG. 1A . Figure 1B The MC unit 152 shown or a part of the intra-frame prediction 150 is implemented. However, the decoder can also use additional processing units to implement the required processing. Alternatively, any of the proposed methods can be implemented as a circuit coupled to the inter-frame / intra-frame / prediction module of the encoder and / or the inter-frame / intra-frame / prediction module of the decoder to provide the information required by the inter-frame / intra-frame / prediction module. Although the inter-frame prediction 112 and intra-frame prediction 110 on the encoder side and the MC 152 and intra-frame prediction 150 on the decoder side are shown as separate processing units, they can correspond to executable software or firmware code stored on a medium such as a hard disk or flash memory, for a central processing unit (Central Processing Unit, for short) or a programmable device (such as a digital signal processor (Digital Signal Processor) or a field programmable gate array (Field Programmble Gate Array, for short FPGA)).

[0418] Fig.23A flowchart of an exemplary video coding system using a hybrid predictor according to an embodiment of the present invention is shown. The steps shown in the flowchart can be implemented as program codes that can be executed on one or more processors (e.g., one or more CPUs) on the encoder side. The steps shown in the flowchart can also be implemented based on hardware, such as one or more electronic devices or processors arranged to perform the steps in the flowchart. According to the method, in step 2310, input data associated with a first color block and a current block including a second color block is received, wherein the input data includes pixel data of the current block to be encoded on the encoder side or encoded and decoded data associated with the current block to be decoded on the decoder side. In step 2320, a first predictor of the second color block is determined, wherein the first predictor corresponds to all samples or a subset of the prediction samples of the current block. In step 2330, at least one second predictor of the second color block is determined based on the first color block, wherein one or more target model parameters associated with at least one target prediction model corresponding to the at least one second predictor are implicitly derived by using one or more neighboring samples of the second color block and / or one or more neighboring samples of the first color block, and wherein the at least one second predictor corresponds to all samples or a subset of the prediction samples of the current block. In step 2340, a final predictor is generated, wherein the final predictor includes a portion of the first predictor and a portion of the at least one second predictor. In step 2350, input data associated with the second color block is encoded or decoded using prediction data including the final predictor.

[0419] The flowchart shown is intended to illustrate an example of video coding according to the present invention. Those skilled in the art may modify each step, rearrange the steps, split the steps, or combine the steps to implement the present invention without departing from the spirit of the present invention. In the present disclosure, specific syntax and semantics are used to illustrate examples to implement embodiments of the present invention. A technician may implement the present invention by replacing the above syntax and semantics with equivalent syntax and semantics without departing from the spirit of the present invention.

[0420] The above description is presented to enable one of ordinary skill in the art to implement the present invention provided in the context of a specific application and its requirements. Various modifications to the described embodiments will be apparent to those skilled in the art, and the general principles defined herein may be applied to other embodiments. Therefore, the present invention is not intended to be limited to the specific embodiments shown and described, but to conform to the widest scope consistent with the principles and novel features disclosed herein. In the above detailed description, various specific details are described in order to provide a thorough understanding of the present invention. However, it will be understood by those skilled in the art that the present invention may be implemented.

[0421] The embodiments of the present invention as described above can be implemented in various hardware, software codes or a combination of the two. For example, one embodiment of the present invention can be one or more electronic circuits integrated into a video compression chip or integrated into video compression software to perform the program code described herein. The embodiment of the present invention can also be a program code to be executed on a digital signal processor (Digital Signal Processor, referred to as DSP) to perform the processing described herein. The present invention can also be related to many functions performed by a computer processor, a digital signal processor, a microprocessor or a field programmable gate array (field programmable gate array, referred to as FPGA). These processors can be configured to perform specific tasks according to the present invention by executing machine-readable software code or firmware code that defines the specific method embodied in the present invention. The software code or firmware code can be developed in different programming languages ​​and different formats or styles. The software code can also be compiled for different target platforms. However, the different code formats, styles and languages ​​of the software code and other ways of configuring the code to perform tasks according to the present invention will not depart from the spirit and scope of the present invention.

[0422] The present invention may be embodied in other specific forms without departing from its spirit or essential characteristics. The described examples are to be considered in all respects as illustrative only and not restrictive. Therefore, the scope of the present invention is indicated by the appended claims rather than by the foregoing description. All changes within the equivalent meaning and scope of the claims should be included within their scope.

Claims

1. A video encoding and decoding method for multiple color pictures, the method comprising: receiving input data associated with a first color block and a current block including a second color block, wherein the input data comprises pixel data of the first color block and the current block, the pixel data to be encoded at an encoder side, or coded-decoded data associated with the first color block and the current block, the coded-decoded data to be decoded at a decoder side; determining a first predictor for the second color block, wherein the first predictor corresponds to all or a subset of a plurality of prediction samples of the current block; determining at least one second predictor for the second color block based on the first color block, wherein one or more target model parameters associated with at least one target prediction model corresponding to the at least one second predictor are implicitly derived by using one or more neighboring samples of the second color block and / or one or more neighboring samples of the first color block, and wherein the at least one second predictor corresponds to all samples or a subset of the plurality of prediction samples of the current block; generating a final predictor, wherein the final predictor comprises a portion of the first predictor and a portion of the at least one second predictor; as well as The input data associated with the second color block is encoded at the encoder side using prediction data including the final predictor, or the input data associated with the second color block is decoded at the decoder side using the prediction data including the final predictor.

2. The video encoding and decoding method according to claim 1, characterized in that: The first predictor corresponds to an intra predictor.

3. The video encoding and decoding method according to claim 1, characterized in that: The first predictor corresponds to a cross-color predictor.

4. The video encoding and decoding method according to claim 3, characterized in that: The first predictor is generated based on CCLM LT, CCLM L or CCLM T.

5. The video encoding and decoding method according to claim 1, characterized in that: The at least one second predictor is generated based on a multi-model cross-component linear model pattern.

6. The video encoding and decoding method according to claim 1, characterized in that: The portion of the first predictor is derived based on the first predictor having a first weight, and the portion of the at least one second predictor is derived based on the at least one second predictor having at least one second weight.

7. The video encoding and decoding method according to claim 6, characterized in that: The final predictor is derived as a sum of the portion of the first predictor and the portion of the at least one second predictor.

8. The video encoding and decoding method according to claim 6, characterized in that: The first weight, the at least one second weight, or both may be determined by deriving from each sample of the second color block.

9. The video encoding and decoding method according to claim 1, characterized in that: At the encoder side, syntax is sent to indicate whether it is allowed to determine the at least one second predictor, generate the final predictor, and encode or decode the current block using the prediction data including the final predictor.

10. The video encoding and decoding method according to claim 9, characterized in that: The syntax is sent at the encoder side or parsed at the decoder side at block level, tile level, slice level, picture level, sequence parameter set level or at picture parameter set level.

11. The video encoding and decoding method according to claim 9, characterized in that: When the current block uses a predetermined cross color mode, the syntax is sent to indicate whether it is allowed to determine the at least one second predictor, generate the final predictor, and encode or decode the current block using the prediction data including the final predictor.

12. The video encoding and decoding method according to claim 11, characterized in that: The predetermined cross color mode corresponds to a CCLM LT mode, a CCLM L mode or a CCLM T mode.

13. The video encoding and decoding method according to claim 1, characterized in that: Whether to allow determining the at least one second predictor, generating the final predictor, and encoding or decoding the current block using the prediction data including the final predictor is implicitly determined.

14. The video encoding and decoding method according to claim 1, characterized in that: One or more model parameters corresponding to each prediction model of the candidate set are determined, and the cost of each prediction model of the candidate set is evaluated, and the prediction model that achieves the minimum cost in the candidate set is selected as the at least one target prediction model, and the one or more model parameters associated with the one prediction model that achieves the minimum cost in the candidate set are selected as the one or more target model parameters.

15. The video encoding and decoding method according to claim 13, characterized in that: If the minimum cost is below a threshold, it is determined that the at least one second predictor, generating the final predictor, and encoding or decoding the current block using the prediction data including the final predictor are permitted.

16. The video encoding and decoding method according to claim 15, characterized in that: The threshold depends on the block size of the current block, sequence resolution, neighboring blocks, quantization parameter, or any combination thereof.

17. The video encoding and decoding method according to claim 14, characterized in that: A second color template including multiple selected adjacent samples of the second color block and a first color template including multiple corresponding adjacent samples of the first color block are determined, one or more model parameters corresponding to each prediction model of the candidate set are determined based on multiple reference samples of the first color template and multiple reference samples of the second color template, and the cost of each prediction model of the candidate set is determined based on multiple reconstructed samples and multiple prediction samples of the second color template, and the multiple prediction samples of the second color template are derived by applying the one or more model parameters determined for each prediction model to the first color template.

18. The video encoding and decoding method according to claim 17, characterized in that: The second color template includes a plurality of top-neighboring samples, a plurality of left-neighboring samples, or both of the second color block, and the first color template includes a plurality of top-neighboring samples, a plurality of left-neighboring samples, or both of the first color block.

19. The video encoding and decoding method according to claim 17, characterized in that: The current block includes a Cr block and a Cb block, the first color block corresponds to a Y block and the second color block corresponds to the Cr block or the Cb block, wherein when the syntax indicates that determining the at least one second predictor, generating the final predictor, and encoding or decoding the current block using the prediction data including the final predictor are allowed for one of the Cr block and the Cb block, then determining the at least one second predictor, generating the final predictor, and encoding or decoding the current block using the prediction data including the final predictor are also allowed for the other of the Cr block and the Cb block.

20. The video encoding and decoding method according to claim 14, characterized in that: The cost of each prediction model of the candidate set corresponds to a boundary matching cost, the boundary matching cost is used to measure the discontinuity between multiple prediction samples of the second color block and multiple adjacent reconstructed samples of the second color block, and wherein the multiple prediction samples of the second color block are derived based on the first color block using the one or more model parameters determined for each prediction model.

21. The video encoding and decoding method according to claim 20, characterized in that: The boundary matching cost includes a top boundary matching cost, a left boundary matching cost, or both, the top boundary matching cost is used to compare between multiple top prediction samples of the second color block and multiple adjacent top reconstructed samples of the second color block, and the left boundary matching cost is used to compare between multiple left prediction samples of the second color block and multiple adjacent left reconstructed samples of the second color block.

22. The video encoding and decoding method according to claim 14, characterized in that: A second color template including multiple selected adjacent samples of the second color block and a first color template including multiple corresponding adjacent samples of the first color block are determined, and based on the first color template and the second color template, the one or more model parameters corresponding to each prediction model of the candidate set are determined, and wherein the cost of each prediction model of the candidate set is determined based on multiple reconstructed samples and multiple prediction samples of the second color template, and the multiple prediction samples of the second color template are derived by applying the one or more model parameters determined for each prediction model to the first color template.

23. An apparatus for video encoding and decoding, the apparatus comprising one or more electronic devices or processors, arranged to: receiving input data associated with a first color block and a current block including a second color block, wherein the input data comprises pixel data of the first color block and the current block, the pixel data to be encoded at an encoder side, or coded-decoded data associated with the first color block and the current block, the coded-decoded data to be decoded at a decoder side; determining a first predictor for the second color block, wherein the first predictor corresponds to all or a subset of a plurality of prediction samples of the current block; determining at least one second predictor for the second color block based on the first color block, wherein one or more target model parameters associated with at least one target prediction model corresponding to the at least one second predictor are implicitly derived by using one or more neighboring samples of the second color block and / or one or more neighboring samples of the first color block, and wherein the at least one second predictor corresponds to all samples or a subset of the plurality of prediction samples of the current block; generating a final predictor, wherein the final predictor comprises a portion of the first predictor and a portion of the at least one second predictor; as well as The input data associated with the second color block is encoded at the encoder side using prediction data including the final predictor, or the input data associated with the second color block is decoded at the decoder side using the prediction data including the final predictor.

Citation Information

Cited By

  • Methods and apparatus of prediction blending using neighbouring samples for video coding

    WO2026138864A1