Method and apparatus for explicit mode mixing in video coding and decoding system

Abstract

A method and apparatus for hybrid intra prediction mode. According to a decoder-side method, a first predictor is determined from a first set of intra prediction candidates, where the first predictor provides a first prediction for a second color block. A second predictor is determined from a second set of intra prediction candidates, wherein the second predictor provides a second prediction for a second color block. At least one of a first predictor and a second predictor is derived based on the first color block using a cross-component model. A final predictor is generated by mixing the first predictor and the second predictor. And encoding or decoding the second color block by using prediction data including the final predictor.

Description

[0001] Cross-references

[0002] The present invention is a non-provisional application of and claims priority to U.S. Provisional Patent Application No. 63 / 367,820 filed on July 7, 2022 and U.S. Provisional Patent Application No. 63 / 482,816 filed on February 2, 2023. The entire contents of the U.S. Provisional Patent Application are incorporated herein by reference. Technical Field

[0003] The present invention relates to a video coding system, and in particular to mixing two intra-frame prediction modes and signal coding modes and / or mixing weights to improve coding performance. Background Art

[0004] Versatile Video Coding (VVC) is the latest international video coding standard jointly developed by the Joint Video Experts Group (JVET) of the ITU-T Video Coding Experts Group (VCEG) and the ISO / IEC Moving Picture Experts Group (MPEG), which has been published as an ISO standard: ISO / IEC 23090-3:2021, Information technology - Coded representation of immersive media - Part 3: Versatile video coding, published in February 2021. VVC is an improvement over its predecessor HEVC (High Efficiency Video Coding) by adding more codec tools to improve codec efficiency, and can also handle various types of video sources, including 3D video signals.

[0005] Figure 1A An exemplary adaptive inter / intra video coding system including a loop process is described. For intra prediction, prediction data is derived from previously encoded video data in the current picture. For inter prediction 112, motion estimation (ME) is performed on the encoder side and motion compensation (MC) is performed based on the results of ME to provide prediction data derived from other pictures and motion data. Switch 114 selects intra prediction 110 or inter prediction 112 and the selected prediction data is provided to adder 116 to form a prediction error, also known as a residual. The prediction error is then processed by transform (T) 118 and subsequent quantization (Q) 120. The transformed and quantized residual is then encoded by entropy encoder 122 to be included in a video bitstream corresponding to the compressed video data. The bitstream associated with the transform coefficients is then packaged with auxiliary information (such as motion and coding modes associated with intra prediction and inter prediction) and other information (such as parameters associated with loop filters applied to underlying image areas). Auxiliary information associated with intra prediction 110, inter prediction 112, and in-loop filter 130 is provided to entropy encoder 122, such as Figure 1AAs shown. When using inter-frame prediction mode, one or more reference pictures must also be reconstructed at the encoder end. Therefore, the transformed and quantized residual is processed by inverse quantization (IQ) 124 and inverse transform (IT) 126 to recover the residual. The residual is then added back to the prediction data 136 at reconstruction (REC) 128 to reconstruct the video data. The reconstructed video data can be stored in a reference picture buffer 134 and used to predict other frames.

[0006] like Figure 1A As shown, the input video data undergoes a series of processes in the coding system. Due to a series of processes, the reconstructed video data from REC128 may be subject to various impairments. Therefore, loop filter 130 is often applied to reconstructed video data before the reconstructed video data is stored in reference picture buffer 134 to improve video quality. For example, deblocking filter (DF), sample adaptive offset (SAO) and adaptive loop filter (ALF) can be used. It may be necessary to merge loop filter information into the bitstream so that the decoder can correctly restore the required information. Therefore, loop filter information is also provided to entropy encoder 122 to be merged into the bitstream. Figure 1A In , a loop filter 130 is applied to the reconstructed video before the reconstructed samples are stored in a reference picture buffer 134 . Figure 1A The system in is intended to illustrate an exemplary structure of a typical video encoder. It may correspond to the High Efficiency Video Coding (HEVC) system, VP8, VP9, ​​H.264, or VVC.

[0007] like Figure 1B As shown, the decoder can use similar or identical functional blocks as the encoder, except for transform 118 and quantization 120, because the decoder only needs inverse quantization 124 and inverse transform 126. Instead of entropy encoder 122, the decoder uses entropy decoder 140 to decode the video bitstream into quantized transform coefficients and required encoding information (such as ILPF information, intra-frame prediction information, and inter-frame prediction information). Intra-frame prediction 150 on the decoder side does not need to perform mode search. Instead, the decoder only needs to generate intra-frame prediction based on the intra-frame prediction information received from entropy decoder 140. In addition, for inter-frame prediction, the decoder only needs to perform motion compensation (MC152) based on the inter-frame prediction information received from entropy decoder 140 without motion estimation.

[0008] According to VVC, similar to HEVC, the input picture is divided into non-overlapping square block areas called CTUs (Coding Tree Units). Each CTU can be divided into one or more coding units (CUs) of smaller size. The generated CU partitions can be square or rectangular. In addition, VVC divides CTUs into prediction units (PUs) as units for applying prediction processes such as inter-frame prediction, intra-frame prediction, etc.

[0009] The VVC standard incorporates various new coding tools to further improve coding efficiency over the HEVC standard. For example, to reduce cross-component redundancy, a cross-component linear model (CCLM) prediction mode is used in VVC, where a linear model is used to predict chrominance samples based on reconstructed luminance samples of the same CU. In addition, in the continued development of next-generation video coding, more new coding tools have been disclosed, such as mixing multiple prediction modes. In the present invention, a method and apparatus for mixing two intra-frame prediction modes are disclosed, wherein at least one mode is related to a cross-component prediction mode (e.g., CCLM or any other type of cross-component prediction mode) to improve coding performance. Summary of the invention

[0010] A method and apparatus for video encoding and decoding are disclosed. According to the method, input data associated with a current block including a first color block and a second color block is received, wherein the input data includes pixel data for the current block to be encoded at the encoder side or encoded data related to the current block to be decoded at the decoder side. A first predictor is determined from a first intra-frame prediction candidate set, wherein the first predictor provides a first prediction for the second color block. A second predictor is determined from a second intra-frame prediction candidate set, wherein the second predictor provides a second prediction for the second color block. At least one of the first predictor and the second predictor is derived based on the first color block using a cross-component model. A final predictor is generated by mixing the first predictor and the second predictor. The second color block is encoded or decoded by using prediction data including the final predictor.

[0011] In one embodiment, the first color block corresponds to a luma block or a chroma block, and the second color block corresponds to a remaining color component block of the current block.

[0012] In one embodiment, the first intra prediction candidate set or the second intra prediction candidate set includes one or more cross color prediction (CCP) modes, wherein the one or more CCP modes are based on obtaining a prediction of a current color component block in another color component block. In one embodiment, the one or more CCP modes include one or more CCLM types, one or more MMLM types, one or more GLM types, one or more CCCM types, or any combination thereof. The CCLM type may correspond to CCLM_LT, CCLM_L, CCLM_T, or any combination thereof. The MMLM type may correspond to MMLM_LT, MMLM_L, MMLM_T, or any combination thereof. The CCCM type may correspond to a CCCM model using different convolution filter shapes, a CCCM model using non-downsampled samples, a CCCM model with multiple downsampled filters, a hybrid CCCM model, a CCCM model derived using different numbers of reference lines, a CCCM model derived using templates at different positions, or any combination thereof.

[0013] In one embodiment, the second intra prediction candidate set is generated by excluding the first predictor from the first intra prediction candidate set.

[0014] In one embodiment, a first predictor or a second predictor is selected from the first intra-frame prediction candidate set or the second intra-frame prediction candidate set, respectively, based on the TIMD cost associated with the member candidate in the first intra-frame prediction candidate set or the second intra-frame prediction candidate set. In one embodiment, the target member candidate with the minimum TIMD cost is implicitly selected as the first predictor or the second predictor. In another embodiment, a list including k candidates with the minimum TIMD cost is generated, and an index is signaled in the bitstream or parsed from the bitstream to indicate the first predictor or the second predictor selected from the list, and wherein k is an integer less than the total number of candidates in the first intra-frame prediction candidate set or the second intra-frame prediction candidate set.

[0015] In one embodiment, the first predictor or the second predictor is selected from the first intra prediction candidate set or the second intra prediction candidate set, respectively. The first predictor or the second predictor is determined according to an index signaled in the bitstream or parsed from the bitstream.

[0016] In one embodiment, a flag of the current block is signaled in the bitstream or parsed from the bitstream to indicate whether to determine the first predictor, the second predictor, and the final predictor of the current block and whether to encode or decode the current block. The current block is determined by using prediction data including the final predictor. In one embodiment, the flag is signaled or parsed at the CU level, the PU level, or the CTU level.

[0017] In one embodiment, the first intra prediction candidate set consists of CCLM_LT, CCLM_L and CCLM_T, and the second intra prediction candidate set consists of MMLM_LT, MMLM_L and MMLM_T.

[0018] In one embodiment, the final predictor corresponds to the weighted sum of the first predictor and the second predictor. In one example, the weights of the weighted sum correspond to α and (1-α), where 0<α<1. In another example, the weights of the weighted sum correspond to w1 and w2, and where w1>0, w2>0, and w1+w2=1. In one embodiment, the weights are determined based on the first predictor TIMD cost and the second predictor TIMD cost. In this case, w1 can be equal to (first predictor TIMD cost / (first predictor TIMD cost+second predictor TIMD cost)), and w2 can be equal to (second predictor TIMD cost / (first predictor TIMD cost+second predictor TIMD cost)). BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1A An exemplary adaptive inter / intra video coding system incorporating loop processing is shown.

[0020] Figure 1B Shows Figure 1A The corresponding decoder of the encoder in .

[0021] Figure 2 The 67 intra-frame prediction modes used by the VVC video coding standard are marked.

[0022] Figure 3A -B shows blocks that are wider than they are tall ( Figure 3A ) and blocks that are taller than they are wide ( Figure 3B ) is an example of wide-angle intra prediction.

[0023] Figure 4 An example of the positions of the upper left sample and the samples of the current block involved in the LM_LA mode is shown.

[0024] Figure 5 An example of classifying adjacent samples into two groups according to the multimodal CCLM is shown.

[0025] Fig. 6A An example of the CCLM model is shown.

[0026] Figure 6B An example of the effect of the slope adjustment parameter "u" on the model update is shown.

[0027] Figure 7An example of Template-based Intra Mode Derivation (TIMD) mode is shown, where TIMD implicitly derives the intra prediction mode of a CU using neighboring templates at both the encoder and the decoder.

[0028] Figure 8 An example of the spatial part of a convolution filter is shown.

[0029] Fig. 9 An example of a reference area with filling for deriving filter coefficients is shown.

[0030] Fig.10 The 16 gradient modes of the gradient linear model (GLM) are marked.

[0031] Fig.11 A flow chart of an exemplary video coding system incorporating intra-prediction mode mixing according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0032] It will be readily understood that the components of the present invention as generally described and illustrated in the accompanying drawings herein may be arranged and designed in a variety of different configurations. Therefore, the following more detailed description of the embodiments of the system and method of the present invention as shown in the figures is not intended to limit the scope of the claimed invention, but only represents selected embodiments of the present invention. References throughout this specification to "embodiment", "one embodiment" or similar language mean that a particular feature, structure or characteristic described in conjunction with the embodiment may be included in at least one embodiment of the present invention. Therefore, the phrases "in one embodiment" or "in one embodiment" that appear throughout this specification do not necessarily all refer to the same embodiment.

[0033] In addition, the described features, structures or characteristics may be combined in one or more embodiments in any suitable manner. However, those skilled in the relevant art will recognize that the present invention may be practiced without one or more specific details, or using other methods, components, etc. In other cases, well-known structures or detailed descriptions of operations are not shown or are not shown to avoid obscuring aspects of the present invention. The illustrated embodiments of the present invention will be best understood with reference to the accompanying drawings, in which like parts are represented by like numbers throughout. The following description is intended to be exemplary only and simply illustrates certain selected embodiments of the apparatus and methods consistent with the present invention as claimed herein.

[0034] Intra-mode coding with 67 intra-prediction modes

[0035] To capture arbitrary edge directions present in natural videos, the number of directional intra modes in VVC is expanded from 33 used in HEVC to 65. New directional modes not present in HEVC are introduced in Figure 2Depicted as dashed arrows in , planar and DC modes remain unchanged. These more dense directional intra prediction modes apply to all block sizes and luma and chroma intra prediction.

[0036] In VVC, for non-square blocks, several traditional angular intra prediction modes are adaptively replaced with wide-angle intra prediction modes.

[0037] In HEVC, each intra-coded block has a square shape and the length of each of its sides is a power of 2. Therefore, no division operation is required to generate intra predictors using DC mode. In VVC, blocks can have a rectangular shape, which in general requires the use of division operations for each block. To avoid division operations for DC prediction, only the longer sides are used to calculate the average for non-square blocks.

[0038] In order to keep the complexity of the most probable mode (MPM) list generation low, an intra mode coding method with 6 MPMs is used by considering two available adjacent intra modes. The following three aspects are considered to construct the MPM list:

[0039] -Default intra mode

[0040] - Adjacent Intra Mode

[0041] - Intra mode for export.

[0042] A unified 6-MPM list is used for intra blocks, regardless of whether the MRL and ISP coding tools are applied. The MPM list is constructed based on the intra modes of the left and upper neighboring blocks. Assuming that the mode on the left is denoted as Left and the mode of the upper block is denoted as Above, the unified MPM list is constructed as follows:

[0043] - When a neighboring block is not available, its intra mode is set to planar by default.

[0044] -If both Left and Above modes are non-angle modes:

[0045] -MPM List → {plane, DC, V, H, V-4, V+4}

[0046] - If one of Left and Above modes is an angle mode and the other is a non-angle mode:

[0047] - Set mode Max to the larger mode between Left and Above

[0048] -MPM List → {Flat, Max, DC, Max-1, Max+1, Max-2}

[0049] -If Left and Above are both angled and they are different:

[0050] - Set mode Max to the larger mode between Left and Above

[0051] - If the difference between the modes Left and Above is in the range 2-62,

[0052] MPM list → {Plane, Left, Above, DC, Max-1, Max+1}

[0053] -otherwise:

[0054] MPM List → {Plane, Left, Above, DC, Max-2, Max+2}

[0055] -If Left and Above are both angled and they are the same:

[0056] -MPM List → {Plane, Left, Left-1, Left+1, DC, Left-2}

[0057] In addition, the first bin of the MPM index codeword is CABAC context coded. A total of three contexts are used, corresponding to whether the current intra block is MRL enabled, ISP enabled, or a normal intra block.

[0058] In the 6MPM list generation process, pruning is used to remove repeated patterns so that only unique patterns can be included in the MPM list. For entropy coding of the 61 non-MPM patterns, truncated binary code (TBC) is used.

[0059] Wide-angle intra prediction for non-square blocks

[0060] Conventional angular intra prediction directions are defined as clockwise from 45 degrees to -135 degrees. In VVC, several traditional angular intra prediction modes are adaptively replaced with wide-angle intra prediction modes for non-square blocks. The replaced modes are signaled using the original mode index, which is remapped to the index of the wide-angle mode after parsing. The total number of intra prediction modes remains unchanged, i.e. 67, and the intra mode encoding method remains unchanged.

[0061] To support these predictions, Figure 3A As shown in FIG. 3B , a top reference with a length of 2W+1 and a left reference with a length of 2H+1 are defined.

[0062] The number of alternative modes in wide-angle direction mode depends on the aspect ratio of the block. The alternative intra prediction modes are shown in Table 1.

[0063] Table 1 - Intra prediction modes replaced by wide angle mode

[0064]

[0065]

[0066] In VVC, 4:2:2, 4:4:4, and 4:2:0 chroma formats are supported. The chroma derived mode (DM) derivation table for the 4:2:2 chroma format was originally ported from HEVC, expanding the number of entries from 35 to 67 to keep consistent with the expansion of intra prediction modes. Since the HEVC specification does not support prediction angles below -135° and above 45°, the luma intra prediction modes are mapped from 2 to 5 to 2. Therefore, the chroma DM derivation table for the 4:2:2 chroma format is updated by replacing some values ​​of the mapping table entries to more accurately convert the prediction angles of the chroma blocks.

[0067] Cross-Component Linear Model (CCLM) Forecasting

[0068] In order to reduce cross-component redundancy, the cross-component linear model (CCLM) prediction mode is used in VVC, where a linear model is used to predict chrominance samples based on the reconstructed luminance samples of the same CU as follows:

[0069] pred c (i, j) = α·rec L ′(i,j)+β (1)

[0070] where pred C (i, j) represents the predicted chroma sample in CU, rec L ′(i, j) represents the downsampled reconstructed luma sample of the same CU.

[0071] The CCLM parameters (α and β) are derived from up to four adjacent chroma samples and their corresponding downsampled luma samples. Assuming the current chroma block dimension is W×H, then W′ and H′ are set to

[0072] - When LM_LA mode is applied, W′=W, H′=H;

[0073] - When LM_A mode is applied, W'=W+H;

[0074] - When LM_L mode is applied, H'=H+W.

[0075] The adjacent positions above are represented as S[0, -1]...S[W′-1, -1], and the adjacent positions on the left are represented as S[-1, 0]...S[-1, H′-1]. Then four samples are selected as:

[0076] -S[W′ / 4, -1], S[3*W′ / 4, -1], S[-1, H′ / 4], S[-1, 3*H′ / 4], when LM mode is applied and the neighboring samples above and to the left are available;

[0077] -S[W′ / 8, -1], S[3*W′ / 8, -1], S[5*W′ / 8, -1], S[7*W′ / 8, -1], when LM-A mode is applied or only upper adjacent samples are available;

[0078] -S[-1, H′ / 8], S[-1, 3*H′ / 8], S[-1, 5*H′ / 8], S[-1, 7*H′ / 8], when LM-L mode is applied or only left neighboring samples are available.

[0079] The four adjacent brightness samples at the selected position are downsampled and compared four times to find the two larger values: x 0 A and x 1 A , and two smaller values: x 0 B and x 1 B Their corresponding chrominance sample values ​​are represented by y 0 A ,y 1 A ,y 0 B and 1 B Then x A , x B ,y A and B Export as:

[0080] "Xa=(x0A+x1A+1)>>1;

[0081] Xb=(x0B+x1B+1)>>1;

[0082] Ya=(y0A+y1A+1)>>1;

[0083] Yb=(y0B+y1B+1)>>1(2)

[0084] Finally, the linear model parameters α and β are obtained according to the following equations.

[0085]

[0086] β=Y b -α·X b (3)

[0087] Figure 4 An example of the positions of the upper left sample and the current block sample involved in the LM_LA mode (Linear Mode Left and Above) is given. Figure 4 The relative sample positions of an NxN chroma block 410, a corresponding 2Nx2N luma block 420, and their neighboring samples (shown as solid circles) are shown.

[0088] The division operation to calculate the parameter α is implemented by a lookup table. To reduce the memory required to store the table, the diff value (the difference between the maximum and minimum values) and the parameter α are expressed in exponential notation. For example, diff is approximated by a 4-bit significant part and an exponent. Therefore, for 16 significant values, the table of 1 / diff is reduced to 16 elements, as shown below:

[0089] DivTable[]={0,7,6,5,5,4,4,3,3,2,2,1,1,1,1,0} (5)

[0090] This will help reduce the complexity of the calculations and the amount of memory required to store the required tables.

[0091] In addition to the upper template and the left template being used together to calculate the linear model coefficients, they can also be used alternately in 2 other LM modes, called LM_A and LM_L modes.

[0092] In LM_A mode, only the upper template is used to calculate the linear model coefficients. To obtain more samples, the upper template is expanded to (W+H) samples. In LM_L mode, only the left template is used to calculate the linear model coefficients. To obtain more samples, the left template is expanded to (H+W) samples.

[0093] In LM_LA mode, the left and upper templates are used to calculate the linear model coefficients.

[0094] In order to match the chroma sample positions of the 4:2:0 video sequence, two types of downsampling filters are applied to the luma samples to achieve a 2:1 downsampling ratio in the horizontal and vertical directions. The choice of downsampling filter is specified by the SPS level flag. The two downsampling filters are as follows, corresponding to "type-0" and "type-2" content, respectively.

[0095] Rec L ′(i, j)=[rec L (2i--1, 2j--1)+2·rec L (2i-1, 2j-1)+rec L (2i+1,2j-1)+rec L (2i-1, 2j)+2·rec L (2i,2j)+recL (2i+1,2j)+4]>>3 (6)

[0096] RecL′(i,j)=rec L (2i,2j--1)+rec L (2i--1, 2j)+4·rec L (2i,2j)+rec L (2i+1,2j)+rec L (2i, 2j+1)+4]>>3 (7)

[0097] Note that when the upper reference line is at a CTU boundary, only one luma line (general line buffer in intra prediction) is used for downsampling. An exception occurs if the top row of the current block is a CTU boundary. In this case, a one-dimensional filter [1, 2, 1] / 4 is applied to the above adjacent luma samples to avoid using more than one luma line above the CTU boundary.

[0098] This parameter calculation is performed as part of the decoding process, not just as an encoder search operation. Therefore, there is no syntax for conveying the α and β values ​​to the decoder.

[0099] For chroma intra mode coding, a total of 8 intra modes are allowed for chroma intra mode coding. These modes include five traditional intra modes and three cross-component linear model modes ({LM_LA, LM_L, and LM_A}, or {CCLM_LT, CCLM_L, and CCLM_T}). {LM_LA, LM_L, LM_A} and {CCLM_LT, CCLM_L, CCLM_T} are interchangeable in this article. The chroma mode signaling and derivation process are shown in Table 3. Chroma mode coding depends directly on the intra prediction mode of the corresponding luminance block. Since separate block partitioning structures for luminance and chrominance components are enabled in the I slice, one chroma block can correspond to multiple luminance blocks. Therefore, for the chroma derivation (Chroma DM) mode, the intra prediction mode of the corresponding luminance block at the center position of the current chroma block is directly inherited and overwritten.

[0100] Table 2 - Chroma prediction modes derived from luma mode when CCLM is enabled

[0101] As shown in Table 3, a single binary table is used regardless of the value of sps_cclm_enabled_flag.

[0102] Table 3 - Unified binary table for chroma prediction mode

[0103]

[0104] In Table 3, the first binary code indicates whether it is normal (0) or CCLM mode (1). If it is LM mode, the next binary code indicates whether it is LM_LA (0). If it is not LM_LA, the next binary code indicates whether it is LM_L (0) or LM_A (1). For this case, when sps_cclm_enabled_flag is 0, the first binary code of the binarization table of the corresponding intra_chroma_pred_mode can be discarded before entropy coding. Or, in other words, the first binary code is inferred to be 0 and therefore not encoded. This single binarization table is used for the cases where sps_cclm_enabled_flag is equal to 0 and 1. The first two binary codes in Table 4 are context encoded using their own context model, and the remaining binary codes are bypass encoded.

[0105] Additionally, to reduce luma-chroma latency in dual trees, when the 64x64 luma coding tree node is partitioned using NotSplit (and ISP is not used for 64x64 CU) or QT, the chroma CU tree nodes in 32x32 / 32x16 chroma coding can use CCLM in the following ways:

[0106] - If a 32x32 chroma node is not partitioned or partitioned QT partitioned, all chroma CUs in the 32x32 node can use CCLM;

[0107] - If a 32x32 chroma node adopts horizontal BT partitioning and the 32x16 child node is not partitioned or uses vertical BT partitioning, all chroma CUs in the 32x16 chroma node can use CCLM.

[0108] Under all other luma and chroma coding tree partitioning conditions, CCLM is not allowed for chroma CUs.

[0109] Multi-model CCLM (MMLM)

[0110] In JEM (J. Chen, E. Alshina, G. J. Sullivan, J.-R. Ohm, and J. Boyce, Algorithm Description of Joint Exploration Test Model 7, document JVET-G1001, ITU-T / ISO / IEC Joint Video Development Team (JVET), July 2017), a multiple model CCLM mode (MMLM) is proposed for predicting chrominance samples from luma samples of the entire CU using two models. In MMLM, the adjacent luma samples and adjacent chroma samples of the current block are divided into two groups, and each group is used as a training set to derive a linear model (i.e., a specific α and β are derived for a specific group). In addition, the samples of the current luma block are also classified based on the same rules as the classification of the adjacent luma samples.

[0111] Figure 5 An example of dividing neighboring samples into two groups is shown. The threshold is calculated as the average of neighboring reconstructed luminance samples. Neighboring samples with Rec′L[x, y] <= threshold are classified into group 1; and neighboring samples with Rec′L[x, y] > threshold are classified into group 2.

[0112]

[0113] CCLM slope adjustment

[0114] CCLM uses a 2-parameter model to map brightness values ​​to chromaticity values, such as Fig. 6A The slope parameter "a" and the deviation parameter "b" are defined and mapped as follows:

[0115] Chroma value = a*brightness value + b

[0116] Signaling an adjustment to the slope parameter “u” updates the model to the following form, Figure 6B As shown:

[0117] Chroma value = a'*brightness value + b'

[0118] in

[0119] a=a+u,

[0120] b'=bu*y r .

[0121] With this selection, the mapping function will be tilted or rotated around the point with brightness value yr. The reference brightness samples used in the model building are averaged to yr in order to make meaningful modifications to the model. Fig. 6A , 6B This process is shown.

[0122] Implementation of CCLM slope adjustment

[0123] The slope adjustment parameter is provided as an integer between -4 and 4, inclusive, and is signaled in the bitstream. The units of the slope adjustment parameter are (1 / 8) of the chroma sample value per luma sample value (for 10-bit content).

[0124] The adjustment applies to CCLM models that use reference samples above and to the left of the block (e.g., "LM_CHROMA_IDX" and "MMLM_CHROMA_IDX"), but not to the "one-sided" mode. This choice is based on a trade-off between coding efficiency and complexity. In the present invention, "LM_CHROMA_IDX" and "MMLM_CHROMA_IDX" refer to CCLM_LT and MMLM_LT. The "one-sided" mode in the present invention refers to CCLM_L, CCLM_T, MMLM_L, and MMLM_T.

[0125] When slope adjustment is applied to a multi-mode CCLM model, both models can be adjusted, so at most two slope updates can be signaled for a single chroma block.

[0126] Encoder method for CCLM slope adjustment

[0127] The proposed encoder method performs a Sum of Absolute Transformed Differences (SATD)-based search for the optimal value of the slope update for (Cr), and performs a similar SATD-based search for Cb. If either result is a non-zero slope adjustment parameter, the combined slope adjustment pair (SATD-based update for Cr, SATD-based update for Cb) will be included in the Rate-Distortion (RD) checklist of the TU.

[0128] Multiple Hypothesis Prediction (NHP)

[0129] In the multiple hypothesis inter prediction mode (JVET-M0425), in addition to the traditional bi-directional prediction signal, one or more additional motion compensated prediction signals are signaled. The final overall prediction signal is obtained by weighted superposition of samples. bi and the first additional inter-frame prediction signal / hypothesis h3, the resulting prediction signal p3 is as follows:

[0130] p3=(1-α)p bi +αh3 (9)

[0131] The weight factor α is specified by the new syntax element add_hyp_weight_idx according to the following mapping (Table 4):

[0132] Table 4. Mapping α to add_hyp_weight_idx

[0133]

[0134] Similar to the above, more than one additional prediction signal may be used. The resulting overall prediction signal is iteratively accumulated with each additional prediction signal.

[0135] p n+1 =(1-α n+1 ) n +α n+1 h n+1 (10)

[0136] The resulting overall prediction signal is obtained as the final p n (i.e., the p with the largest index n n ). For example, up to two additional prediction signals may be used (ie, n is limited to 2).

[0137] The motion parameters for each additional prediction hypothesis can be signaled explicitly by specifying the reference index, motion vector predictor index and motion vector difference, or implicitly by specifying the merge index. A separate multi-hypothesis merge flag distinguishes these two signaling modes.

[0138] For inter-frame AMVP mode, MHP is only applied when unequal weights in BCW are selected in bi-directional prediction mode. For detailed information on MHP for VVC, please refer to JVET-W2025 (Muhammed Coban et al., "Algorithmic Description of Enhanced Compression Model 2 (ECM 2)", Joint Video Experts Group (JVET) ISO / IEC JTC 1 / SC 29 of ITU-T SG 16WP 3, 23rd Meeting, via Teleconference, July 7-16, 2021, Document: JVET-W2025).

[0139] Template-based Intra Mode Derivation (TIMD)

[0140] The template-based intra mode derivation (TIMD) mode implicitly derives the intra prediction mode of a CU using neighboring templates at the encoder and decoder, instead of sending the intra prediction mode to the decoder. Figure 7As shown, for each candidate mode, the reference samples (720 and 722) of the template (712 and 714) are used to generate the prediction samples of the template of the current block 710. The cost is calculated as the sum of absolute transformed differences (SATD) between the predicted samples and the reconstructed samples of the template. The intra prediction mode with the smallest cost is selected as the TIMD mode for intra prediction of the CU. The candidate mode can be 67 intra prediction modes as in VVC or extended to 131 intra prediction modes. In general, MPM can provide clues indicating the directional information of the CU. Therefore, in order to reduce the intra mode search space and utilize the characteristics of the CU, the intra prediction mode can be implicitly derived from the MPM list.

[0141] For each intra prediction mode in MPM, the SATD between the predicted and reconstructed samples of the template is calculated. The first two intra prediction modes with the smallest SATD are selected as TIMD modes. These two TIMD modes are fused with weights after applying the PDPC process, and this weighted intra prediction is used to encode and decode the current CU. Position dependent intra prediction combination (PDPC) is included in the derivation of TIMD modes.

[0142] The costs of the two selected modes are compared to the threshold, and in the test, a cost factor of 2 is applied, as follows:

[0143] The cost of mode 2 is less than 2*the cost of mode 1.

[0144] If this condition is true, fusion is applied, otherwise only mode 1 is used. The weights of the modes are calculated based on their SATD costs as follows:

[0145] Weight 1 = Cost of Mode 2 / (Cost of Mode 1 + Cost of Mode 2)

[0146] Weight 2 = 1 - Weight 1.

[0147] Fusion of chroma intra prediction modes

[0148] In the development of emerging video coding systems, it is proposed that the DM mode and the four default modes can be merged with the MMLM_LT mode as follows:

[0149] pred=(w0*pred0+w1*pred1+(1<<(shift-1)))>>shift

[0150] In the above formula, pred0 is the predictor obtained by applying the non-LM mode, pred1 is the predictor obtained by applying the MMLM_LT mode, and pred is the final predictor of the current chroma block. The two weights w0 and w1 are determined by the intra-frame prediction mode of the adjacent chroma blocks, and the shift is set equal to 2. Specifically, when the upper and left adjacent blocks are both encoded in the LM mode, {w0, w1} = {1, 3}; when the upper and left adjacent blocks are both encoded in the non-LM mode, {w0, w1} = {3, 1}; otherwise, {w0, w1} = {2, 2}.

[0151] For syntax design, if non-LM mode is selected, a flag is signaled to indicate whether fusion is applied. And the proposed fusion is only applicable to I slices.

[0152] Convolutional cross-component model (CCCM)

[0153] In CCCM, a convolutional model is applied to improve the chrominance prediction performance. The convolutional model has a 7-tap filter consisting of a 5-tap plus sign shape spatial component, a nonlinear term, and a bias term. The input to the spatial 5-tap component of the filter consists of the center (C) luma sample, which is associated with the chroma sample to be predicted and its upper / north (N), lower / south (S), left / west (W), and right / east (E) neighbors, such as Figure 8 shown.

[0154] The nonlinear term (denoted as P) is expressed as the square of the center brightness sample C, and scaled to the sample value range of the content:

[0155] P = (C*C + midVal) >> bit depth.

[0156] For example, for 10-bit content, the nonlinear term is calculated as follows:

[0157] P=(C*C+512)>>10

[0158] The offset term (denoted as B) represents a scalar offset between the input and output (similar to the offset term in CCLM) and is set to the intermediate chrominance value (512 for 10-bit content).

[0159] The output of the filter is calculated as the convolution between the filter coefficients ci and the input value, and is clipped to the range of valid chroma samples:

[0160] predChromaVal=c0C+c1N+c2S+c3E+c4W+c5P+c6B

[0161] The filter coefficients ci are computed by minimizing the MSE between the predicted and reconstructed chroma samples in the reference region. Fig. 9 An example of a reference region consisting of 6 rows of chroma samples above and to the left of PU 910 is shown. The reference region extends one PU width to the right and one PU height below the PU boundary. The region has been adjusted to contain only the available samples. An extension to this region (denoted as "padding") is required to support Figure 8 The plus-shaped spatial filter in the middle "side samples" and fills in unavailable areas.

[0162] MSE minimization is performed by computing the autocorrelation matrix of the luma input and the cross-correlation vector between the luma input and the chroma output. The autocorrelation matrix is ​​subjected to an LDL decomposition and the final filter coefficients are calculated using inverse substitution. This process roughly follows the calculation of the ALF filter coefficients in ECM, but the LDL decomposition is chosen instead of the Cholesky decomposition to avoid the use of square root operations.

[0163] Gradient Linear Model (GLM)

[0164] Compared with CCLM, GLM uses the gradient of luma samples to infer the linear model instead of the downsampled luma values. Specifically, when applying GLM, the input of CCLM process, i.e., the downsampled luma samples L, is replaced by the gradient of luma samples G. The other parts of CCLM (e.g., parameter derivation, linear transformation of prediction samples) remain unchanged.

[0165] C=α·G+β

[0166] For signaling, when CCLM mode is enabled for the current CU, two flags will be transmitted for Cb and Cr respectively to indicate whether GLM is enabled for each component. If GLM is enabled for a component, a syntax element is further signaled to select 16 gradient filters ( Fig.10 One of 1010-1040 in is used for gradient calculation. GLM can be combined with an existing CCLM by sending an additional flag in the bitstream. When this combination is applied, the filter coefficients of the input luminance samples used to derive the linear model are calculated as a combination of the selected gradient filter of the GLM and the downsampling filter of the CCLM.

[0167] Proposed method

[0168] In order to improve the efficiency of video coding and decoding, some coding tools are disclosed for more efficient signaling and / or better predictors. For the traditional mechanism, the inter-frame mode uses temporal information to predict the current block. For the intra-frame block, the spatial adjacent reference samples are used to predict the current block. In the present invention, a coding tool is proposed. This coding tool creates a new hybrid mode to form a better predictor and more efficient signal transmission. The concept of this coding tool is described as follows.

[0169] In one embodiment, an additional intra prediction hypothesis (denoted as H2) is mixed with an existing intra prediction hypothesis (denoted as H1) by predefined weights to form a final prediction for the current block. H1 is generated by a mode selected from a first mode set (denoted as Set1), and H2 is generated by a mode selected from a second mode set (denoted as Set2). Let the weights of mixing H1 and H2 be w1 and w2,

[0170] Final prediction = w_1*H_1+w_2*H_2,

[0171] H1 and H2 can be any intra-frame mode. An intra-frame mode refers to a mode used to generate intra-frame block prediction, such as the 67 intra-frame prediction modes, CCLM mode, and MMLM mode mentioned above.

[0172] In another embodiment, H1 and H2 can be any cross-component prediction mode. The cross-component prediction mode refers to a mode of predicting the second color component of the block through a model that describes the relationship between the first color component of the block. For example, the first color component can be a luminance component, and the second color component can be Cb or Cr. For another example, the first color component can be Cb, and the second color component can be Cr. This model can be a linear model, a convolutional model, or an image model. Examples of cross-component prediction modes include CCLM, MMLM, GLM, CCCM, and variants of CCCM. CCLM and MMLM use linear models to predict chrominance components through luminance components. CCCM uses a convolutional model to predict chrominance components through luminance components. The image model includes a lookup table, which records the mapping relationship between the first color component value and the second color component value. The first color component value (e.g., sample intensity) is an index of the lookup table. In order to predict the second color component sample value in the current block, the juxtaposed first color component sample value is searched in the lookup table, and the corresponding second color component sample value in the table is used to predict the second color component sample value in the current block.

[0173] In another embodiment, H1 and H2 can be any convolutional cross-component model mode (CCCM) and CCCM variants. For example, CCCM models using different convolution filter shapes, CCCM models using non-downsampled samples, CCCM models with multiple downsampled filters, CCCM models with various terms (e.g., spatial terms, gradient terms, position terms, nonlinear terms, and bias terms), CCCM models derived using different numbers of reference lines, and CCCM models derived using templates at different positions. All CCCM modes described in this paragraph are referred to as CCCM types in this disclosure.

[0174] In another embodiment, H1 and H2 may be any IM mode. An IM mode is a mode that predicts the chrominance component of a block by juxtaposing reconstructed luminance samples according to a linear model, and its parameters are derived from reconstructed luminance and chrominance samples adjacent to the block. LM modes include CCLM and MMLM.

[0175] In another embodiment, H1 and H2 can be any IM mode and GLM mode.

[0176] In another embodiment, H1 can be any CCLM mode and H2 can be any MMLM mode

[0177] In one embodiment, the coding tool is applied to blocks of any color component. For example, it can be applied to luma components and / or chroma components.

[0178] In another embodiment, the encoding tool is applied to chroma component blocks.

[0179] In one embodiment, a flag is signaled / parsed at the block level to indicate whether the hybrid mode is applied to the current block. For example, the flag may be signaled at the CU level and / or the PU level and / or the CTU level. For another example, the flag may be signaled at the CB level and / or the PB level and / or the CTB level and / or the TU / TB level and / or any predefined region level.

[0180] In one embodiment, when the flag indicates that hybrid mode is disabled, the original syntax for signaling / parsing the intra prediction mode of the current block is followed. When this flag indicates that hybrid mode is enabled, the following syntax related to indicating the mode and / or weighting for generating H1 and H2 needs to be decided and signaled at the encoder or parsed at the decoder.

[0181] In one sub-embodiment, the hybrid mode and other intra prediction modes indicated by the original syntax are mutually exclusive, ie, if the hybrid mode is selected, other intra prediction modes cannot be selected, and vice versa. Only syntax related to the hybrid mode is signaled.

[0182] In a sub-embodiment, the hybrid mode and other intra prediction modes indicated by the original syntax are not mutually exclusive. The original syntax is then sent after the syntax associated with the hybrid mode. For example, another hypothesis H3 generated by the mode indicated by the original syntax can be further mixed with H1 and H2 through another pre-defined weight to produce a block predictor. The original syntax can also be sent before the syntax associated with the hybrid mode. For example, when the chroma intra prediction fusion flag is turned on, the DM mode and the four default modes can be fused with the hybrid mode instead of MMLM_LT (if the hybrid mode is also enabled).

[0183] In one embodiment, the flag is signaled after the flag enables / disables the LM mode. For example, the flag is signaled after the CclmEnabled flag. CclmEnabled specifies whether the cross-component chroma intra prediction mode is enabled (ie, TRUE) or not enabled (ie, FALSE) for the current chroma coding block.

[0184] For another embodiment, this flag is signaled before the flag that enables / disables the LM mode. For example, this flag is signaled before the CclmEnabled flag. CclmEnabled specifies whether the cross-component chroma intra prediction mode is enabled (ie, TRUE) or not enabled (ie, FALSE) for the current chroma coding block.

[0185] For another embodiment, the flag is signaled after a flag that specifies whether to use color space conversion to apply the decoded residual of the current coding unit. For example, the flag is signaled after cu_act_enabled_flag. cu_act_enabled_flag equal to 1 specifies to use color space conversion to apply the decoded residual of the current coding unit. cu_act_enabled_flag equal to 0 specifies to apply the decoded residual of the current coding unit without color space conversion. When cu_act_enabled_flag is not present, it is inferred to be equal to 0.

[0186] As previously described, H1 is generated by selecting a mode from a first mode set (denoted as Set1), and H2 is generated by selecting a mode from a second mode set (denoted as Set2). H1 and H2 can be any intra-frame mode. In one embodiment, Set1 includes CCLM_LT, CCLM_L, CCLM_T, MMLM_LT, MMLM_L, MMLM_T, or any subset of the above modes, or any extension of the LM family.

[0187] In another embodiment, Set1 includes CCLM_LT, CCLM_L, CCLM_T or any subset of the above modes. CCLM_LT, CCLM_L, CCLM_T are referred to as CCLM types in this disclosure.

[0188] In another embodiment, Set1 includes MMLM_LT, MMLM_L, MMLM_T, or any subset of the above modes. MMLM_LT, MMLM_L, MMLM_T are referred to as MMLM types in this disclosure.

[0189] In one embodiment, Set2 includes CCLM_LT, CCLM_L, CCLM_T, MMLM_LT, MMLM_L, MMLM_T, or any subset of the above modes, or any extension of the LM family.

[0190] In another embodiment, Set2 includes CCLM_LT, CCLM_L, CCLM_T, or any subset of the above modes.

[0191] In another embodiment, Set2 includes MMLM_LT, MMLM_L, MMLM_T, or any subset of the above modes. MMLM_LT, MMLM_L, MMLM_T are referred to as MMLM types in this disclosure.

[0192] In one embodiment, the number of candidates in set 2 is less than the number of candidates in set 1.

[0193] In one sub-embodiment, Set2 is generated from Set1 by excluding the pattern selected to generate H1. The number of candidates in Set2 is one less than the number of candidates in Set1.

[0194] In another sub-embodiment, when the weights used in the blend are implicitly determined (e.g., if equal weights are used), the size of Set2 can be further reduced. For example, if Set1 contains CCLM_LT, CCLM_L, CCLM_T, MMLM_LT, MMLM_L, and MMLM_T.

[0195] If CCLM_LT is used to generate H1, Set2 contains CCLM_L, CCLM_T, MMLM_LT, MMLM_L, and MMLM_T.

[0196] If CCLM_L is used to generate H1, Set2 contains CCLM_T, MMLM_LT, MMLM_L, and MMLM_T.

[0197] If CCLM_T is used to generate H1, Set2 contains MMLM_LT, MMLM_L, and MMLM_T.

[0198] If H1 is generated using MMLM_LT, Set2 contains MMLM_L and MMLM_T.

[0199] If MMLM_L is used to generate H1, Set2 contains MMLM_T.

[0200] In one embodiment, the mode selected from Set1 for generating H1 is indicated by an index signaled / parsed in the bitstream, represented as index 0. The mode selected from Set2 for generating H2 is indicated by an index signaled / parsed in the bitstream, represented as index 1.

[0201] In one sub-embodiment, let the number of candidates in Set1 be N1, and the range of index 0 is 0 to N1 minus 1.

[0202] In a sub-embodiment, index 0 is context encoded. The context is determined by the information of the neighboring blocks. It can be determined based on the neighboring mode information. For example, if more neighboring blocks are CCLM mode (e.g., CCLM_LT, CCLM_L, CCLM_T), we select a context, if more neighboring blocks are MMLM mode (e.g., MMLM_LT, MMLM_L, MMLM_T), we select another context. For another example, if more neighboring blocks are LT mode (e.g., CCLM_LT, MMLM_LT), we select a context, if more neighboring blocks are L mode (e.g., CCLM_L, MMLM_L), we select another context, if more neighboring blocks are T mode (e.g., CCLM_T, MMLM_T), we select another context.

[0203] In another sub-embodiment, index 0 is truncated unary coded. For example, if Set1 contains CCLM_LT, CCLM_L, CCLM_T, MMLM_LT, MMLM_L, and MMLM_T, and if the frequencies of the signaled modes are also in this order (i.e., CCLM_LT is the most frequently signaled and MMLM_T is the least frequently signaled), then the truncated unary code used can be designed as shown in Table 5:

[0204] Table 5. Truncated unary encoding example for index 0

[0205] CCLM_LT CCLM_L CCLM_T MMLM_LT MMLM_L MMLM_T 0 10 110 1110 11110 11111

[0206] In one sub-embodiment, if the number of candidates in Set2 is N2, then the index l ranges from 0 to N2 minus 1.

[0207] In a sub-embodiment, index l is context encoded. The context is determined by the information of the adjacent blocks. It can be determined based on the neighbor mode information. For example, if more adjacent blocks are CCLM mode (e.g., CCLM_LT, CCLM_L, CCLM_T), we select a context, if more adjacent blocks are MMLM mode (MMLM_LT, MMLM_L, MMLM_T), we select another context. For another example, if more adjacent blocks are LT mode (e.g., CCLM_LT, MMLM_LT), we select a context, if more adjacent blocks are L mode (e.g., CCLM_L, MMLM_L), we select another context, if more adjacent blocks are T mode (e.g., CCLM_T, MMLM_T), we select another context.

[0208] In another sub-embodiment, index l is truncated unary encoding. For example, if Set2 contains CCLM_L, CCLM_T, MMLM_LT, MMLM_L, and MMLM_T, and if the frequency of mode signaling is also in this order (CCLM is the most frequently signaled, and MMLM_T is the least frequently signaled), then the truncated unary encoding used can be designed as shown in Table 6:

[0209] Table 6. Truncated unary encoding example for index 1

[0210] CCLM_L CCLM_T MMLM_LT MMLM_L MMLM_T () 10 110 1110 1111

[0211] In one embodiment, since the number of candidates in Set2 is less than the number of candidates in Set1, and for the above embodiments regarding the encoding of index 0 and index 1, the codeword of index 1 is shorter than the codeword of index 0. For example, let Set1 include CCLM_LT, CCLM_L, CCLM_T, MMLM_LT, MMLM_L, and MMLM_T, both index 0 and index 1 use truncated unary codes, and CCLM_LT is selected to generate H1.

[0212] An example of truncated unary encoding of index 0 is shown in Table 7:

[0213] Table 7. Truncated unary encoding example for index 0

[0214] CCLM_LT CCLM_L CCLM_T MMLM_LT MMLM_L MMLM_T 0 10 110 1110 11110 11111

[0215] An example of truncated unary encoding of index 1 is shown in Table 8:

[0216] Table 8. Truncated unary encoding example for index 1

[0217] CCLM_L CCLM_T MMLM_LT MMLM_L MMLM_T 0 10 110 1110 1111

[0218] Six different codewords are needed to transmit the index 0 signal, and the maximum length of the codeword of index 0 is 5. Only five different codewords are needed to transmit the index 1 signal, and the maximum length of the codeword of index 1 is 4.

[0219] In another embodiment, the mode selected to generate H1 and H2 is indicated by a joint index (denoted as index 2) signaled / parsed in the bitstream.

[0220] In one sub-embodiment, let N3 represent the number of different pattern combinations that can be used to generate H1 and H2, i.e., all different pattern combinations selected from Set1 and Set2. Index 2 ranges from 0 to N3 minus 1. For example, let Set1 contain CCLM_LT, CCLM_L, CCLM_T, MMLM_LT, MMLM_L, and MMLM_T. N3 is equal to 15, and all different combinations are:

[0221] ·(CCLM_LT, CCLM_L), (CCLM_LT, CCLM_T), (CCLM_LT, MMLM_LT), (CCLM_LT, MMLM_L) and (CCLM_LT, MMLM_T).

[0222] ·(CCLM_L, CCLM_T), (CCLM_L, MMLM_LT), (CCLM_L, MMLM_L), and (CCLM_L, MMLM_T).

[0223] ·(CCLM_T, MMLM_LT), (CCLM_T, MMLM_L), and (CCLM_T, MMLM_T).

[0224] ·(MMLM_LT, MMLM_L) and (MMLM_LT, MMLM_T).

[0225] (MMLM_L, MMLM_T).

[0226] In a sub-embodiment, index 2 is context-coded. The context is determined by the information of the adjacent blocks. It can be determined based on the adjacent mode information. For example, if more adjacent blocks are CCLM mode (CCLM_LT, CCLM_L, CCLM_T), we select a context, if more adjacent blocks are MMLM mode (MMLM_LT, MMLM_L, MMLM_T), we select another context. For another example, if more adjacent blocks are LT mode (CCLM_LT, MMLM_LT), we select a context, if more adjacent blocks are L mode (CCLM_L, MMLM_L), we select another context, if more adjacent blocks are T mode (CCLM_T, MMLM_T), we select another context).

[0227] In another sub-embodiment, Index 2 is truncated unary. For example, let Set1 contain CCLM_LT, CCLM_L, CCLM_T, MMLM_LT, MMLM_L, and MMLM_T. N3 is equal to 15. Let the pattern combinations be arranged in descending order of the frequency with which the pattern combinations are sent: (CCLM_LT, CCLM_L), (CCLM_LT, CCLM_T), (CCLM_LT, MMLM_LT), (CCLM_LT, MMLM_L), (CCLM_LT, MMLM_T), ..., (MMLM_L, MMLM_T). The truncated unary code used can be designed as shown in Table 9:

[0228] Table 9. Truncated unary encoding example for index 2

[0229]

[0230] In one embodiment, the weight of the combination H1 and H2 is indicated by an index (denoted as index 3) that is signaled or parsed in the bitstream. Index 3 indicates a candidate in the weight set. For example, the weight set (w_h1, w_h2) can be an MHP-like weight, where w_h1 = (1-α), w_h2 = α, and α = 1 / 4 or -1 / 8,

[0231] Final predictor = (1-α)h1+αh2.

[0232] For another example, the weight set (w_h1, w_h2) may also be (3, 5), (5, 3), (-2, 10), (10, -2), (4, 4) or any subset of the above weights.

[0233] In another embodiment, the weights are determined implicitly. For example, the weights may be inferred to be equal weights.

[0234] In one sub-embodiment, the weights may be determined based on the block size. For example, the weights used may be determined based on whether the block width or height is greater than a predetermined value. The value may be 2, 4, 8, 16, or any other allowed block width / height value. For another example, the weights used may be determined based on whether the block area is greater than a predetermined value. The value may be 4*4, 8*8, or other allowed block area sizes.

[0235] In another sub-embodiment, the weight is determined by the mode information from the neighboring blocks. For example, if H1 is LT mode (CCIM_LT and MMIM_LT) and H2 is not LT mode, and more neighboring blocks are LT mode, then H1 has a larger weight. For another example, if H1 is CCLM mode (e.g., CCLM_LT, CCLM_L, CCLM_T) and H2 is MMLM mode (e.g., MMIM_LT, MMLM_L, MMLM_T), and more neighboring blocks are CCLM mode, then H1 has a larger weight.

[0236] In another sub-embodiment, the weight is determined by TIMD cost. TIMD cost is defined as follows. For each mode, Figure 7 As shown, the prediction samples of the template are generated using the reference samples of the template. For the case of the CCP mode, the CCP model is derived based on the reconstructed samples on the reference line. The number of reference lines can be more than 1. The derived CCP model is used to generate the prediction samples of the second color component of the template based on the first color component samples of the template. Take the CCLM mode as an example. The CCLM_LT model can be derived using the reconstructed luminance and chrominance samples on the top and left reference lines, the CCIM_L model can be derived using the reconstructed luminance and chrominance samples on the left reference line, and the CCLM_T model can be derived using the reconstructed luminance and chrominance samples on the top reference line. Then the derived linear model is used to generate the predicted chrominance samples of the template based on the reconstructed luminance samples of the template. The TIMD cost is calculated as the difference between the predicted sample and the reconstructed sample of the template. The difference can be SATD, SAD (sum of absolute differences), a weighted sum of SATD and SAD, or other difference measurements. The TIMD cost can be determined at the decoder and does not need to be signaled. For example, if the mode that generates H1 has a larger TIMD cost, H1 uses a smaller weight when mixing. If the mode that produces H2 has a larger TIMD cost, H2 is mixed with a smaller weight. For another example,

[0237] W h1 =TIMD_cost h2 / (TIMD_cost h1 +TIMD_cost h2 ))

[0238] W h2 =TIMD_cost h1 / (TIMD_cost h1 +TIMD_cost h2 ))

[0239] In another sub-embodiment, if the pattern used to generate H1 has a much larger TIMD cost than H2, then only H2 is used to form the final prediction.

[0240] In another sub-embodiment, if the pattern used to generate H2 has a much larger TIMD cost than H1, then only H1 is used to form the final prediction.

[0241] In another sub-embodiment, if the mode for generating H1 has a much larger TIMD cost than H2, H1 is generated via a predefined intra mode instead.

[0242] In another sub-embodiment, if the mode to generate H2 has a much larger TIMD cost than H1, H2 is instead generated via a predefined intra mode.

[0243] In one embodiment, when Set1, Set2, or a weight set contains only one candidate, the inference uses the only one candidate.

[0244] In one embodiment, the modes selected to generate H1 and H2 are explicitly signaled, and the weights of combining the two modes are implicitly determined. For example, the method of signaling the selected mode can signal the two modes separately by signaling index 0 and index 1. The method can also jointly signal the two modes by signaling index 2. The weights are implicitly determined by the TIMD cost.

[0245] W h1 =TIMD_cost h2 / (TIMD_cost h1 +TIMD_cost h2 ))

[0246] W h2 =TIMD_cost h1 / (TIMD_cost h1 +TIMD_cost h2 ))

[0247] In one embodiment, a flag is signaled / parsed at the block level to indicate whether the hybrid mode is applied to the current block. The selection of modes to produce H1 and H2 is implicitly determined, as is the weight of combining the two modes. For example, the mode is implicitly determined by the TIMD cost. The two modes with the lowest TIMD cost are selected. For another example, if the TIMD cost of one selected mode is much greater than the TIMD cost of another selected mode, only the mode with the smaller TIMD cost is used to form the final prediction. For example, if the sample above the current CU and the sample to the left of the current CU (such as Figure 7For example, if both the template above the current CU and the template to the left of the current CU are available, two default modes (e.g., CCLM_LT, CCLM_L) are selected. For example, if both the template above the current CU and the template to the left of the current CU are available, two default modes (e.g., CCLM_L, MMLM_L) are selected. For example, if both the template above the current CU and the template to the left of the current CU are available, two default modes (e.g., CCLM_T, MMLM_T) are selected. For example, the weight is implicitly determined by the TIMD cost.

[0248] W h1 =TIMD_cost h2 / (TIMD_cost h1 +TIMD_cost h2 ))

[0249] W h2 =TIMD_cost h1 / (TIMD_cost h1 +TIMD_cost h2 ))

[0250] In one embodiment, the mode selected to generate H1 and H2 is implicitly determined. Figure 7 ) are unavailable, the blending mode is disabled.

[0251] The intra-frame prediction mode mixing method described above can be implemented at the encoder side or the decoder side. For example, any of the proposed mode mixing methods can be implemented in the intra-frame coding module (e.g., Figure 1B The intra-frame prediction 150 in the example is implemented, or the intra-frame coding module is an encoder (for example, Figure 1A Any proposed cross-component prediction mode (e.g., CCLM or any other type of cross-component prediction mode) method can also be implemented as a circuit coupled to an intra-frame coding module at a decoder or encoder. However, the decoder or encoder can also use additional processing units to implement the desired mode mixing method. The intra-frame prediction unit (e.g., Figure 1A Unit 110 and Figure 1B The units 150 in the figure are shown as separate processing units, which may correspond to executable software or firmware program codes of a CPU (central processing unit) or a programmable device (such as a DSP (digital signal processor) or FPGA (field programmable gate array)) stored on a medium (such as a hard disk or flash memory).

[0252] Fig.11A flowchart of an exemplary video encoding system incorporating an intra-prediction hybrid method according to an embodiment of the present invention is shown. The steps shown in the flowchart can be implemented as program codes executable on one or more processors (e.g., one or more CPUs) on the codec side. The steps shown in the flowchart can also be implemented based on hardware, such as one or more electronic devices or processors arranged to perform the steps in the flowchart. According to the method, in step 1110, input data associated with a current block including a first color block and a second color block is received, wherein the input data includes data to be encoded on the encoder side or associated with the current block to be decoded on the decoder side. In step 1120, a first predictor is determined from a first intra-prediction candidate set, wherein the first predictor provides a first prediction of the second color block. In step 1130, a second predictor is determined from a second intra-prediction candidate set, wherein the second predictor provides a second prediction of the second color block, and wherein at least one of the first predictor and the second predictor is derived based on the first color block using a cross-component model. In step 1140, a final predictor is generated by mixing the first predictor and the second predictor. In step 1150 , the second color block is encoded or decoded by using the prediction data including the final predictor.

[0253] The flowchart shown is intended to illustrate an example of video encoding according to the present invention. Without departing from the spirit of the present invention, those skilled in the art may modify each step, rearrange the steps, split the steps, or combine the steps to implement the present invention. In the present disclosure, specific syntax and semantics have been used to illustrate examples to implement embodiments of the present invention. Without departing from the spirit of the present invention, those skilled in the art may implement the present invention by replacing syntax and semantics with equivalent syntax and semantics.

[0254] The above description is provided to enable those of ordinary skill in the art to practice the present invention provided in the context of a specific application and its requirements. Various modifications to the described embodiments will be apparent to those skilled in the art, and the general principles defined herein may be applied to other embodiments. Therefore, the present invention is not intended to be limited to the specific embodiments shown and described, but to conform to the widest scope consistent with the principles and novel features disclosed herein. In the above detailed description, various specific details are illustrated to provide a thorough understanding of the present invention. However, those skilled in the art will appreciate that the present invention may be implemented.

[0255] The embodiments of the present invention as described above can be implemented in various hardware, software codes or a combination of the two. For example, one embodiment of the present invention can be one or more circuits integrated into a video compression chip or a program code integrated into the video compression software to perform the processing described herein. The embodiments of the present invention can also be a program code to be executed on a digital signal processor (DSP) to perform the processing described herein. The present invention can also be related to many functions performed by a computer processor, a digital signal processor, a microprocessor or a field programmable gate array (FPGA). These processors can be configured to perform specific tasks according to the present invention by executing machine-readable software codes or firmware codes that define the specific methods embodied by the present invention. The software code or firmware code can be developed in different programming languages ​​and different formats or styles. The software code can also be compiled for different target platforms. However, the different code formats, styles and languages ​​of the software code and other ways of configuring the code to perform tasks according to the present invention will not depart from the spirit and scope of the present invention.

[0256] The present invention may be embodied in other specific forms without departing from its spirit or essential characteristics. The described examples should be considered in all respects as illustrative and not restrictive. Therefore, the scope of the present invention is indicated by the appended claims rather than by the foregoing description. All changes that fall within the meaning and scope of equivalents to the scope of the claims should be included within their scope.

Claims

1. A video encoding and decoding method for a color picture, the method comprising: Receiving input data associated with a first color block and a second color block, wherein the input data includes pixel data of a current block to be encoded at an encoder side or encoded data associated with a current block to be decoded at a decoder side; determining a first predictor from a first set of intra prediction candidates, wherein the first predictor provides a first prediction for the second color block; determining a second predictor from a second set of intra prediction candidates, wherein the second predictor provides a second prediction for the second color block, and wherein at least one of the first predictor and the second predictor uses a cross-component model based on the first color block; generating a final predictor by mixing the first predictor and the second predictor; as well as The second color block is encoded or decoded by using the prediction data including the final predictor.

2. The method according to claim 1, characterized in that The first color block corresponds to a luminance block or a chrominance block, and the second color block corresponds to a residual color component block of a current block.

3. The method according to claim 1, characterized in that The first intra prediction candidate set or the second intra prediction candidate set includes one or more cross color prediction (CCP) modes, wherein the one or more CCP modes derive prediction for one color component block based on another color component block.

4. The method according to claim 3, characterized in that The one or more CCP modes include one or more CCLM types, one or more MMLM types, one or more GLM types, one or more CCCM types, or any combination thereof.

5. The method according to claim 4, characterized in that The one or more CCLM types correspond to CCLM_LT, CCLM_L, CCLM_T or any combination thereof. The method of claim 4 , wherein the one or more MMLM types correspond to MMLM_LT, MMLM_L, MMLM_T, or any combination thereof.

7. The method according to claim 4, characterized in that The one or more CCCM types correspond to a CCCM model using different convolution filter shapes, a CCCM model using non-downsampled samples, a CCCM model with multiple downsampled filters, a hybrid CCCM model, a CCCM model derived using different numbers of reference lines, a CCCM model derived using templates at different positions, or any combination thereof.

8. The method according to claim 1, characterized in that: The second intra prediction candidate set is generated by excluding the first predictor from the first intra prediction candidate set.

9. The method according to claim 1, characterized in that: The first predictor or the second predictor is selected from the first intra prediction candidate set or the second intra prediction candidate set based on a TIMD cost associated with a member candidate in the first intra prediction candidate set or the second intra prediction candidate set, respectively.

10. The method according to claim 9, characterized in that The target member candidate with the smallest TIMD cost is implicitly selected as the first predictor or the second predictor.

11. The method according to claim 9, characterized in that Generate a list of k candidates with minimum TIMD costs, and signal an index in a bitstream or parse an index from the bitstream to indicate a first predictor or a second predictor selected from the list, wherein k is an integer less than the total number of candidates in the first intra-frame prediction candidate set or the second intra-frame prediction candidate set.

12. The method according to claim 1, characterized in that The first predictor or the second predictor is determined according to an index signaled in a bitstream or parsed from the bitstream.

13. The method according to claim 1, characterized in that A flag of the current block is signaled in a bitstream or parsed from the bitstream to indicate whether to determine a first predictor, a second predictor, and a final predictor of the current block and whether to encode or decode the current block by using prediction data including the final predictor.

14. The method according to claim 13, characterized in that The flag is signaled or parsed at CU level, PU level or CTU level.

15. The method according to claim 1, characterized in that The first intra prediction candidate set consists of CCLM_LT, CCLM_L, and CCLM_T, and the second intra prediction candidate set consists of MMLM_LT, MMLM_L, and MMLM_T.

16. The method according to claim 1, characterized in that The final predictor corresponds to a weighted sum of the first predictor and the second predictor.

17. The method according to claim 16, characterized in that The weights of the weighted sum correspond to α and (1-α), and wherein 0<α<1.

18. The method according to claim 16, characterized in that The weights of the weighted sum correspond to w1 and w2, and wherein w1>0, w2>0, and w1+w2=1.

19. The method according to claim 18, characterized in that The weight is determined based on the first predictor TIMD cost and the second predictor TIMD cost.

20. The method according to claim 19, characterized in that w1 is equal to (first predictor TIMD cost / (first predictor TIMD cost+second predictor TIMD cost)), and w2 is equal to (second predictor TIMD cost) / (first predictor TIMD cost+second predictor TIMD cost)).

21. An apparatus for video encoding and decoding, the apparatus comprising one or more electronic devices or processors configured to: Receiving input data associated with a current block including a first color block and a second color block, wherein the input data includes pixel data for a current block to be encoded at an encoder side or encoded data associated with a current block to be decoded at a decoder side; determining a first predictor from a first set of intra prediction candidates, wherein the first predictor provides a first prediction for the second color block; determining a second predictor from a second set of intra prediction candidates, wherein the second predictor provides a second prediction of the second color block, and wherein at least one of the first predictor and the second predictor uses a cross-component model based on the first color block; generating a final predictor by blending the first predictor and the second predictor; and The second color block is encoded or decoded by using the prediction data including the final predictor.

Images