Method and apparatus for hybrid intra prediction for video coding

Abstract

A method and apparatus for video coding using a matrix-based intra prediction (MIP) mode and various settings. According to the method, a target MIP mode is determined using MIP mode settings, and the MIP mode settings include a hybrid setting, a transpose setting, a representative mode setting, an inheritance setting, or a combination thereof. The MIP mode settings with hybrid settings include hybrid matrix candidates formed by combining two or more prediction hypotheses. The MIP mode settings with transpose settings include transpose matrix candidates generated by transposing one or more matrix coefficients or one or more reference samples. The MIP mode setting with the representative mode setting includes a representative mode generated according to the representative mode setting. Alternatively, the MIP mode setting with the inheritance setting includes the inherited MIP candidate. And encoding or decoding the current block using the target MIP mode.

Description

[0001] CROSS REFERENCE TO RELATED APPLICATIONS This application is a non-provisional of U.S. Provisional Patent Application No. 63 / 496,042 (filed April 14, 2023) and U.S. Provisional Patent Application No. 63 / 496,044 (filed April 14, 2023), and claims priority thereto. The above U.S. Provisional Patent Applications are incorporated herein by reference in their entirety. TECHNICAL FIELD The present disclosure relates to video coding systems. In particular, the present disclosure relates to hybrid intra matrix prediction modes with various settings to improve coding performance. BACKGROUND Versatile Video Coding (VVC) is the latest international video coding standard (MPEG) developed by the Joint Video Expert Team (JVET) of the ITU-T Video Coding Experts Group (VCEG) and the ISO / IEC Moving Picture Experts Group. The standard has been published as an ISO standard: ISO / IEC 23090-3:2021, Information technology - Coded representation of immersive media - Part 3: Versatile video coding, published in February 2021. VVC was developed by adding more coding tools based on its predecessor, HEVC (High Efficiency Video coding), to improve coding efficiency and handle various types of video sources, including 3-dimensional (3D) video signals.

[0004] Figure 1AAn exemplary adaptive Inter / Intra video coding system including in-loop processing is illustrated. For Intra prediction, prediction data is derived from previously coded video data in the current picture (hereinafter also referred to as picture). For Inter prediction 112, Motion Estimation (ME) is performed at the encoder side and Motion Compensation (MC) is performed based on the results of ME to provide prediction data derived from other pictures and motion data. A switch 114 selects either Intra prediction 110 or Inter prediction 112 and the selected prediction data is provided to an adder 116 to form prediction error, also referred to as residual. The prediction error is then processed by a transform (T) 118 and subsequent quantization (Q) 120. The transformed and quantized residual is then encoded by an entropy encoder 122 to be included in a video bitstream corresponding to compressed video data. The bitstream associated with the transform coefficients is then packed together with side information, such as motion and decoding mode associated with Intra and Inter prediction, and other information, such as parameters associated with in-loop filters applied to underlying image areas. The side information associated with Intra prediction 110, Inter prediction 112 and in-loop filters 130 is provided to the entropy encoder 122, as Figure 1A indicated. When Inter prediction mode is used, one or more reference pictures also have to be reconstructed at the encoder end. Therefore, the transformed and quantized residual is processed by inverse quantization (IQ) 124 and inverse transform (IT) 126 to recover the residual. The residual is then added back to the prediction data 136 at reconstruction (REC) 128 to reconstruct the video data. The reconstructed video data can be stored in a reference picture buffer 134 and used for predicting other frames.

[0005] As Figure 1AAs shown, the input video data goes through a series of processing in the encoding system. Due to the series of processing, the reconstructed video data from REC 128 can suffer from various impairments. Therefore, the in-loop filters 130 are often applied to the reconstructed video data before it is stored in the reference picture buffer 134 to improve the video quality. For example, deblocking filter (DF), Sample Adaptive Offset (SAO), and Adaptive Loop Filter (ALF) can be used. The in-loop filter information can need to be incorporated into the bitstream so that the decoder can properly recover the needed information. Therefore, the in-loop filter information is also provided to the entropy encoder 122 to incorporate into the bitstream. In Figure 1A In the system in Figure 1A The system in

[0006] As Figure 1B shown, in addition to the transform 118 and quantization 120, the decoder can use similar or identical functional blocks as the encoder because the decoder only needs inverse quantization 124 and inverse transform 126. Instead of the entropy encoder 122, the decoder uses an entropy decoder 140 to decode the video bitstream into quantized transform coefficients and the needed coding information (e.g., ILPF information, intra prediction information, and inter prediction information). The intra prediction 150 on the decoder side does not need to perform mode search. Instead, the decoder only needs to generate the intra prediction according to the intra prediction information received from the entropy decoder 140. Furthermore, for inter prediction, the decoder only needs to perform motion compensation (MC 152) according to the inter prediction information received from the entropy decoder 140 without the need for motion estimation.

[0007] Partitioning of CTU using tree structure In VVC, the coding tree scheme supports the ability of luma and chroma having separate block tree structures. For P and B slices, luma and chroma CTBs in a single CTU must share the same coding tree structure. However, for I slices, luma and chroma can have separate block tree structures. When separate block tree mode is applied, luma CTBs are partitioned into CUs by one coding tree structure, while chroma CTBs are partitioned into chroma CUs by another coding tree structure. This means that a CU in an I slice can consist of either a coding block of luma component or two coding blocks of chroma components, while a CU in a P or B slice always consists of coding blocks of all three color components, unless the video is monochrome.

[0008] Intra mode coding with 67 intra prediction modes To capture arbitrary edge directions presented in natural videos, the number of directional intra modes in VVC is extended from 33 used in HEVC to 65, while the planar and DC modes remain unchanged.

[0009] In VVC, several traditional angular intra prediction modes are replaced adaptively by wide-angle intra prediction modes for non-square blocks.

[0010] Intra mode coding To keep the complexity of the most probable mode (MPM) list generation low, an intra mode coding method with 6 MPMs (or called primary MPMs) is used, considering two available neighboring intra modes. The following three aspects are considered when constructing the MPM list: Default intra mode Neighboring intra modes Derived intra mode Decoder side intra mode derivation (DIMD) When DIMD is applied, two intra modes are derived from the reconstructed neighboring samples (templates), and these two predictors are combined with the planar mode predictor, the weights are derived from the gradient.

[0011] To derive the intra prediction mode of a block implicitly, texture gradient analysis is performed on both the encoder and the decoder side. This process starts from an empty histogram of gradient (HoG) which has 65 entries corresponding to the 65 angular modes. The amplitudes of these entries are determined during the texture gradient analysis.

[0012] In the first step, DIMD selects T=3 columns and rows templates from the current block on the left and above, respectively. This region is used as a reference for gradient-based intra prediction mode derivation.

[0013] In the second step, horizontal and vertical Sobel filters are applied to all 3x3 window positions, centered on the pixel in the middle line of the template. At each window position, the Sobel filters compute the intensity in the pure horizontal and vertical direction, respectively, as and Then, the texture angle of the window is computed as: , which can be converted to one of the 65 angular intra prediction modes. Once the intra prediction mode index for the current window is derived as idx, the amplitude of its entry in HoG[idx] is updated by adding .

[0014] Figure 2A -C illustrates an example of HoG computed after applying the above operations on all pixel positions in the template. Figure 2A illustrates an example of a selected template 220 for the current block 210. The template 220 includes T lines above the current block and T columns to the left of the current block. For the intra prediction of the current block, the area 230 above and to the left of the current block corresponds to the reconstructed area, while the area 240 below and to the right of the block corresponds to the unavailable area. Figure 2B illustrates an example for T = 3, and HoG is computed for the pixel 260 in the middle line and the pixel 262 in the middle column. For example, for the pixel 252, a 3x3 window 250 is used. Figure 2C illustrates an example of the amplitude (ampl) computed based on the above equation.

[0015] Once HoG is computed, the two indices with the highest histogram entries are selected as the two implicitly derived intra prediction modes for the block, and further combined with the planar mode as the prediction of the DIMD mode. The prediction fusion is applied as a weighted average of the above three predictors. For this, the weight of the planar is fixed to 21 / 64 (about 1 / 3). The remaining 43 / 64 (about 2 / 3) weight is then shared between the two HoG IPMs, proportionally to their HoG entry amplitudes.

[0016] Template-based intra mode derivation (TIMD) The template-based intra mode derivation (TIMD) mode implicitly derives the intra prediction mode of a CU using a neighboring template on both the encoder and decoder side, instead of passing the intra prediction mode to the decoder. As Figure 3As shown, the prediction samples of the template (312 and 314) of the current block 310 are generated using the reference samples (320 and 322) of the template of each candidate mode. The SATD (sum of absolute transformed differences) between the prediction samples and the reconstructed samples of the template is calculated as the cost. The intra prediction mode with the minimum cost is selected as the TIMD mode (as mode derivation of DIMD mode) and used for the intra prediction of the CU. The candidate modes can be the 67 intra prediction modes in VVC or extended to 131 intra prediction modes. Generally, the MPMs can provide a clue to indicate the directional information of the CU. Therefore, to reduce the intra mode search space and exploit the characteristics of the CU, the intra prediction mode can be implicitly derived from the MPM list.

[0017] For each intra prediction mode in the MPM, the SATD between the prediction samples and the reconstructed samples of the template is calculated (TIMD cost). The first two intra prediction modes with the minimum SATD are selected as the TIMD modes. These two TIMD modes are fused with a weight after applying the PDPC process and the current CU is coded using this weighted intra prediction. The position dependent intra prediction combination (PDPC) is included in the derivation of the TIMD modes.

[0018] The costs of the two selected modes are compared with a threshold, in the test, a cost factor 2 is applied as follows: costMode2<2 costMode1.

[0019] If this condition is true, fusion is applied, otherwise only mode1 is used (i.e. single mode case). The weights of the modes are calculated from their SATD costs as follows: weight1 = costMode2 / (costMode1+ costMode2), weight2 = 1 - weight1.

[0020] Intra sub-partition (ISP) Intra sub-partition (ISP) divides a luma intra prediction block vertically or horizontally into 2 or 4 sub-partitions according to the block size. For each sub-partition, the reconstructed samples are obtained by adding the residual signal to the prediction signal. Here, the residual signal is generated by processes such as entropy decoding, inverse quantization and inverse transform, etc. Therefore, the reconstructed sample values of each sub-partition can be used to generate the prediction of the next sub-partition, and each sub-partition is processed repeatedly. In addition, the first processed sub-partition is the one containing the top-left sample of the CU, then it continues down (for horizontal partitioning) or to the right (for vertical partitioning). The result is that the reference samples used to generate the sub-partition prediction signal are only located to the left and above the line. All sub-partitions share the same intra mode.

[0021] Template matching prediction (TMP) Template matching prediction (TMP) is a special intra prediction mode that copies the best prediction block from the reconstructed part of the current frame, whose L-shaped template matches the current template. For a predefined search range, the encoder searches for the template that is most similar to the current template in the reconstructed part of the current frame and uses the corresponding block as the prediction block (as reference block). Then, the encoder signals the use of this mode and the inverse operation is performed at the decoder side.

[0022] CCLM overview The main idea of the CCLM mode (sometimes abbreviated as LM mode) is as follows: The chroma components of a block can be predicted from co-located reconstructed luma samples by a linear model whose parameters are derived from the already reconstructed luma and chroma samples neighboring the block.

[0023] MMLM overview As the name suggests, the original CCLM mode uses one linear model to predict the chroma samples from the luma samples of the entire CU, while in MMLM there can be two models. In MMLM, the neighboring luma samples and the neighboring chroma samples of the current block are divided into two groups, each of which is used as a training set to derive a linear model (i.e., a particular a and b are derived for a particular group). Furthermore, the samples of the current luma block are also classified according to the same rule as the neighboring luma samples.

[0024] Convolutional cross-component model (CCCM) In CCCM, a convolutional model is applied to improve the chroma prediction performance. The convolutional model has a 7-tap filter consisting of a 5-tap plus-shaped spatial component, a non-linear term, and a bias term. The inputs of the spatial 5-tap component of the filter include the center luma sample co-located with the chroma sample to be predicted and its above / north, below / south, left / west, and right / east neighbors.

[0025] The filter coefficients are computed by minimizing the MSE between the predicted chroma samples and the reconstructed chroma samples in the reference region. Fig. 4 shows the reference region, which consists of 6 rows of chroma samples above and to the left of the PU. The reference region extends one PU width to the right and one PU height below the PU boundary. The region is adjusted to only contain available samples. Extensions (gray region) are needed to support the “side samples” of the plus-shaped spatial filter, and padding is applied when the region is not available.

[0026] The MSE minimization is performed by computing the autocorrelation matrix of the luma inputs and the cross-correlation vector between the luma inputs and the chroma outputs. The autocorrelation matrix is LDL-decomposed and the final filter coefficients are computed using back-substitution. This process roughly follows the computation of the ALF filter coefficients in ECM, but LDL-decomposition is chosen instead of Cholesky-decomposition to avoid the use of square root operations.

[0027] Gradient Linear Model (GLM) In contrast to CCLM, GLM does not use down-sampled luma values but exploits luma sample gradients to derive the linear model. Specifically, when GLM is applied, the input of the CCLM process, i.e., the down-sampled luma samples are replaced by luma sample gradients .

[0028] .

[0029] Matrix Weighted Intra Prediction (MIP) The Matrix Weighted Intra Prediction (MIP) method is a newly added intra prediction technique in VVC. To predict samples of a rectangular block with width and height , the Matrix Weighted Intra Prediction (MIP) takes as input a left line of H reconstructed neighboring boundary samples and an above line of W reconstructed neighboring boundary samples of the block. If the reconstructed samples are not available, they are generated in the way of traditional intra prediction. The generation of the prediction signal is based on three steps: averaging, matrix vector multiplication, and linear interpolation, as shown in Figure 5 . The left line of H reconstructed neighboring boundary samples 512 and the above line of W reconstructed neighboring boundary samples 510 of the block are represented as dotted small squares. After the averaging process, the boundary samples are down-sampled to the upper boundary line 514 and the left boundary line 516. The down-sampled samples are provided to the matrix vector multiplication unit 520 to generate the down-sampled prediction block 530. Then, an interpolation process is applied to generate the prediction block 540. The generation of the prediction signal is based on three steps: averaging, matrix vector multiplication, and linear interpolation, as shown below.

[0030] 1. Average neighboring samples Among the boundary samples, four samples or eight samples are selected by averaging according to the block size and shape. Specifically, the input boundary and are reduced to smaller boundary and by averaging neighboring boundary samples according to a predefined rule. Then, the two reduced boundary and are concatenated into one reduced boundary vector , which has a size of four for blocks with shape and eight for all other shapes of blocks.

[0031] 2. Matrix multiplication The average sample is taken as input, a matrix vector multiplication is performed, and then an offset is added. The result is a simplified prediction signal for the sub-sampled sample set in the original block. From the simplified input vector a simplified prediction signal is generated on a down-sampled block of width and height .

[0032] The simplified prediction signal is computed by computing a matrix vector product and adding an offset: .

[0033] Here, is a matrix that has 4 columns if and 8 columns in all other cases. is a vector of size .

[0034] 3. Interpolation The prediction signal for the remaining positions is generated by linear interpolation from the prediction signal on the sub-sampled set, which is a single step linear interpolation in each direction.

[0035] LFNST transform set selection The LFNST (low frequency non-separable transform) in ECM6.0 has 35 transform sets, each containing 3 non-separable transform matrices (kernels). The transform set idx lfnstTrSetIdx is defined according to the predModeIntra in Table 1.

[0036]

[0037] Determination of the LFNST transpose flag The LFNST transpose flag is determined by the predModeIntra as follows: • If the predModeIntra is less than or equal to 34, the LFNST transpose flag is set to 0; • Otherwise, the LFNST transpose flag is set to 1.

[0038] For MIP coded blocks, it is mapped to the PLANAR mode, the LFNST transform set 0 is used, and the LFNST transpose flag is always equal to 0.

[0039] In ECM6.0, the LFNST is in the enabled state for MIP coded blocks with width and height greater than or equal to 16.

[0040] Enhanced MTS for Intra-Frame Coding and Decoding In the current VVC, only the DST7 and DCT8 transform cores are used for MTS, which are used for intra-frame and inter-frame encoding and decoding.

[0041] Additional master transforms are employed, including DCT5, DST4, DST1, and the identity transform (IDT). Furthermore, the MTS set depends on the TU size and intra-frame mode information. Sixteen different TU sizes are considered, and for each TU size, five different categories are considered based on the intra-frame mode information. For each category, one, four, or six different transform pairs are considered. The number of intra-frame MTS candidates is adaptively selected based on the sum of the absolute values ​​of the transform coefficients (between one, four, and six MTS candidate values).

[0042] Chroma DM (Direct Mode) For chroma DM mode, the intra-prediction mode of the corresponding (i.e., co-bit) luma block covering the center position of the current chroma block is directly inherited.

[0043] Intra-block copying Intra-block copy (IBC) is a tool used in SCC by the HEVC extension. It is well known to significantly improve the encoding and decoding efficiency of screen content material. Since IBC mode is implemented as a block-level encoding / decoding mode, block matching (BM) is performed in the encoder to find the optimal block vector (or motion vector) for each CU. Here, the block vector is used to indicate the displacement from the current block to a reference block that has already been reconstructed within the current frame.

[0044] Inter-frame prediction overview According to Section 3.4 of JVET-T2002 (Jianle Chen et al., “Algorithm description for Versatile Video Coding and Test Model 11 (VTM 11),” ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29 Joint Video Expert Team (JVET), 20th Meeting, by teleconference, 7-16 Oct 2020, document: JVET-T2002), for each inter predicted CU, the motion parameters include the motion vector, the reference picture index and the reference picture list usage index, and additional information for VVC new coding features for inter prediction sample generation. The motion parameters can be conveyed in an explicit or implicit manner. When a CU is coded in skip mode, the CU is associated with one PU and has no significant residual coefficients, no coded motion vector delta or reference picture index. A merge mode is specified, in which the motion parameters of the current CU are obtained from neighboring CUs, containing spatial and temporal candidates, and additional plans introduced in VVC. The merge mode can be applied to any inter predicted CU, not limited to skip mode. An alternative to the merge mode is the explicit transmission of the motion parameters, in which the motion vector, the corresponding reference picture index for each reference picture list and the reference picture list usage flag and other required information are explicitly conveyed in each CU.

[0045] In addition to the inter coding functionality in HEVC, VVC contains many brand new and improved inter prediction coding tools, as follows: - Extended merge prediction - Merge mode with MVD (MMVD) - Symmetric MVD (SMVD) signaling - Affine motion compensation prediction - Subblock-based temporal motion vector prediction (SbTMVP) - Adaptive motion vector resolution (AMVR) - Motion field storage: 1 / 16 luma sample MV storage and 8x8 motion field compression - Bi-prediction with CU-level weights (BCW) - Bi-directional optical flow (BDOF) - Decoder-side motion vector refinement (DMVR) - Geometric partition mode (GPM) - Inter and intra combined prediction (CIIP) The following text provides detailed information on those inter prediction methods specified in VVC.

[0046] Extended merge prediction In VVC, the merge candidate list is constructed in order by including the following five types of candidates: 1) Spatial MVP from spatial neighboring CUs 2) Temporal MVP from collocated CUs 3) History-based MVP from FIFO table 4) Pairwise averaged MVP 5) Zero motion vector.

[0047] Spatial candidate derivation The spatial merge candidate derivation in VVC is the same as in HEVC, except that the positions of the first two merge candidates are swapped. At most four merge candidates are selected for the current CU 610, with the candidates located at the positions depicted in Figure 6 The derivation order is B0, A0, B1, A1, and B2. The position B2 is only considered when one or more of the positions B0, A0, B1, A1 are not available (e.g., because it belongs to another slice or tile) or is intra coded. After the candidate at position A1 is added, the addition of the remaining candidates is subject to a redundancy check to ensure that candidates with the same motion information are excluded from the list, thereby improving coding efficiency.

[0048] To reduce the computational complexity, not all possible pairs of candidates are considered in the above redundancy check. Instead, only the pairs linked by arrows in Figure 7 are considered, and a candidate is only added to the list if the corresponding candidate used for the redundancy check does not have the same motion information.

[0049] Temporal candidate derivation In this step, only one candidate is added to the list. In particular, when deriving this temporal merge candidate for the current CU 810, one scaled motion vector is derived based on a collocated CU 820 belonging to the collocated reference picture as shown in Figure 8 The reference picture list and the reference index used for deriving the collocated CU are explicitly signaled in the slice header. The scaled motion vector 830 of the temporal merge candidate is derived from the motion vector 840 of the collocated CU by scaling using the POC (Picture Order Count) distances tb and td, where tb is defined as the POC difference between the reference picture of the current picture and the current picture, and td is defined as the POC difference between the reference picture of the collocated picture and the collocated picture, as shown by the dashed line in Figure 8 The reference picture index of the temporal merge candidate is set to zero.

[0050] The position of the temporal candidate is chosen between the candidates C0 and C1, asFigure 9 As shown, where the CU at position C0 is not available, intra coded, or outside the current CTU row, position C1 is used. Otherwise, position C0 is used in the derivation of the temporal merge candidate.

[0051] History-based merge candidate derivation In this method, the motion information of previously coded blocks is stored in a table and used as the MVPs for the current CU. A table with multiple HMVP candidates is maintained during the encoding / decoding process. When a new CTU row is encountered, the table is reset (emptied). Whenever there is a non-subblock inter coded CU, the associated motion information is added as a new HMVP candidate to the last entry of the table.

[0052] The HMVP table size S is set to 6, which means that up to 5 history-based MVP (HMVP) candidates can be added to the table. When inserting a new motion candidate into the table, a restricted first-in-first-out (FIFO) rule is used, where a redundancy check is applied first to find if there is an identical HMVP in the table. If found, the identical HMVP is removed from the table, after which all the HMVP candidates are shifted forward and the identical HMVP is inserted to the last entry of the table.

[0053] The HMVP candidates can be used in the merge candidate list construction process. The latest few HMVP candidates in the table are checked in order and inserted into the candidate list after the TMVP candidate. A redundancy check is applied for the spatial or temporal merge candidates.

[0054] Non-adjacent spatial candidate The pattern of spatial merge candidates is shown as Figure 9 in the figure. The distance between the non-adjacent spatial candidate and the current coded block is based on the width and height of the current coded block. The row buffer restriction does not apply.

[0055] In the present invention, methods and apparatuses of hybrid intra matrix prediction mode with various settings are disclosed to improve performance. SUMMARY A method and apparatus for video coding using matrix-based intra prediction (MIP) modes with multiple settings. According to the method, input data related to a current block is received, wherein the input data comprises pixel data to be encoded at an encoder side or data related to the current block to be decoded at a decoder side. A target matrix-based intra prediction (MIP) mode is determined using a MIP mode setting, wherein the MIP mode setting comprises a hybrid setting, a transpose setting, a representative mode setting, an inheritance setting, or a combination thereof, wherein the MIP mode setting with the hybrid setting comprises a hybrid matrix candidate formed by combining two or more prediction hypotheses, wherein the MIP mode setting with the transpose setting comprises a transpose matrix candidate generated by transposing one or more matrix coefficients or one or more reference samples, wherein the MIP mode setting with the representative mode setting comprises a representative mode generated according to the representative mode setting, or wherein the MIP mode setting with the inheritance setting comprises an inherited MIP candidate. The current block is encoded or decoded using the target MIP mode.

[0057] In one embodiment, the target MIP mode is used as an additional mode or an alternative mode.

[0058] In one embodiment, when the hybrid setting is used, the one or more prediction hypotheses are generated according to one or more matrix information. In one embodiment, the combination weighting of the hybrid matrix candidate is block-based weighting. In another embodiment, the combination weighting of the hybrid matrix candidate is sample-based or position-based weighting. In yet another embodiment, the one or more matrix information is obtained by selecting from a set of candidate matrix information using a suggestion scheme, and wherein the set of candidate matrix information comprises all or any subset of inherited matrix information.

[0059] In one embodiment, the one or more matrix coefficients or the one or more reference samples are transposed into one or more additional candidate matrices of the current block.

[0060] In one embodiment, the representative mode is generated according to DIMD (Decoder-side Intra Mode Derivation) or TIMD (Template-based Intra Mode Derivation). In another embodiment, the representative mode is generated according to block width, block height, block area, or a combination thereof. In yet another embodiment, the representative mode is used to select a transform set of a primary or secondary transform of a current block that is separable or non-separable. In yet another embodiment, the representative mode is used by one or more subsequent blocks following the current block. In yet another embodiment, the representative mode is used as a chroma DM (Direct Mode) for a subsequent chroma block. In yet another embodiment, the representative mode is used as an MPM (Most Probable Mode) for one or more subsequent intra or inter blocks.

[0061] In one embodiment, when using inheritance settings, matrix information is recorded. In one embodiment, the matrix information includes matrix size, matrix mode index, transposition information, target information for a target MIP mode to determine one or more predictors, or a combination thereof. In one embodiment, an inheritance list is established for a current block. In one embodiment, an inheritance index is signaled, parsed, or inferred to indicate target matrix information in the inheritance list. In one embodiment, the target matrix information indicated by the inheritance index is used for the current block to generate a target MIP predictor. In one embodiment, if one or more matrix sizes of the target matrix information do not belong to a block size category of the current block, down-sampling or up-sampling settings are applied. BRIEF DESCRIPTION OF DRAWINGS Figure 1A An adaptive inter / intra video coding system is exemplarily shown, which includes a loop process.

[0063] Figure 1B A corresponding decoder of the encoder is shown. Figure 1A

[0064] Figure 2A An example of a selected template for a current block is shown, where the template includes T rows above the current block and T columns to the left of the current block.

[0065] Figure 2B An example of T = 3 is shown, and a Histogram of Gradient (HoG) is calculated for pixels in the middle row and pixels in the middle column.

[0066] Figure 2C An example of an amplitude (ampl) of an angular intra prediction mode is shown.

[0067] Figure 3 An example of template-based intra mode derivation is shown.

[0068] Figure 4 ​An example of a reference region with padding for deriving filter coefficients is shown.

[0069] Figure 5 An example of a processing flow of Matrix weighted Intra Prediction (MIP) is shown.

[0070] Figure 6 An example of a neighboring block for deriving spatial merge candidates for VVC is shown.

[0071] Figure 7 An example of possible pairs of candidates considering redundancy check in VVC is shown.

[0072] Figure 8 An example of temporal candidate derivation is shown, where scaled motion vectors are derived according to Picture Order Count (POC) distance.

[0073] Figure 9 An example of the position of a temporal candidate selected between candidates CO and CI is shown.

[0074] Figure 10 An example of a non-adjacent spatial merge candidate is shown.

[0075] Figure 11A An example of initial reference samples for regular MIP is shown.

[0076] Figure 11B An example of initial reference samples for IBC- / Intra-TMP- / Inter-MIP is shown, which contains the top N samples that are not adjacent to the current block and / or the left M samples that are not adjacent to the current block.

[0077] Figure 12A An example of initial reference samples for IBC- / Intra-TMP- / Inter-MIP is shown, which contains the top N samples that are not adjacent to the current block and / or the left M samples that are not adjacent to the current block.

[0078] Figure 12B An example of initial reference samples pointing to the current reconstructed samples of the corresponding region of the current block is shown.

[0079] FIG. 13 shows an example of further PDPC for adjusting the predictor of the current block.

[0080] FIG. 14A shows an example of applying histogram analysis on spatial neighboring reconstructed samples.

[0081] FIG. 14B shows an example of applying template analysis on spatial neighboring reconstructed samples.

[0082] FIG. 15A shows an example of templates including top template, left template, and top-left template.

[0083] FIG. 15B shows a template example including an extended top template and an extended left template.

[0084] FIG. 16 shows an example of DIMD / TIMD on current predictors as the intra prediction mode of transform, where the window center is applied to samples within the current (or down-sampled) block (gray area), but not located at the boundary of the current (or down-sampled) block.

[0085] Figure 17 An example of the intra prediction mode of transform is shown, where five prediction units in the reference region are predefined, and a scan order is applied to the predefined prediction units.

[0086] FIG. 18 shows an example of boundary cost evaluation between the current prediction within the current block and the neighboring reconstruction within one or more neighboring blocks.

[0087] FIG. 19 shows a flowchart of an example video coding system according to an embodiment of the present application, where one or more settings are used for the MIP mode.

DETAILED DESCRIPTION

[0089] Furthermore, described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It will further be appreciated that those of ordinary skill in the relevant arts will recognize that the

[0090] To improve the coding performance of matrix-based intra prediction, various settings are disclosed.

[0091] In the present invention, when the target mode is used for the current block, a down-sampling mechanism, an up-sampling mechanism, a transposing mechanism, an inheriting mechanism, a lateral mechanism, a mixing mechanism and / or a block partitioning mechanism are proposed to provide candidates for the target mode. The proposed mechanisms can slightly or completely not increase the number of circuits / tables / stores / matrices originally supported in the target mode function, and can derive more candidates for the target mode. In the present invention, the target mode is an example of a matrix (or matrix weighted) based prediction mode. However, the proposed method is not limited to only apply to matrix based prediction mode, and can be used for any prediction mode with Intra / Inter / Third mode type (e.g. IBC). For the target mode being a matrix based prediction mode, multiple matrices (or tables) are predefined, when one matrix is indicated to be selected for the current block, the selected matrix and one or more reference samples are used to generate the prediction for the current block. One example is shown in the following expression.

[0092] (1) The reference samples include: Reconstructed or predicted samples or any predefined information in spatial neighboring blocks adjacent or non-adjacent to the current block, e.g. left and / or above of the current block Reconstructed or predicted samples or any predefined information in a temporal collocated block, any predefined region of the current block, or one or more previous pictures or any subset of the above reference samples or both.

[0093] For example, when the reference samples refer to spatial neighboring reconstructed samples adjacent to the current block, and the current block is intra coded, the target mode refers to the regular “Matrix Weighted Intra Prediction” (MIP) mode. For another example, when the reference samples refer to spatial neighboring reconstructed samples adjacent to the current block, and the current block is inter coded, the target mode refers to the inter regular MIP mode. For another example, when the reference samples refer to spatial neighboring reconstructed samples non-adjacent to the current block, and the current block is intra coded, the target mode refers to the “Multiple Reference Line with Regular MIP” (MRL-MIP) mode. That is, the MRL based MIP mode uses reference samples consisting of spatial neighboring reconstructed samples non-adjacent to the current block. For another example, when the reference samples refer to spatial neighboring reconstructed or predicted samples of a collocated region of the current block and / or the current reconstructed or predicted samples, the target mode refers to: IBC-MIP if the collocated region is found by the block vector of the current block (similar to what IBC does) IntraTMP-MIP if the collocated region is found by template matching / searching within a predefined range (similar to what IntraTMP does) Inter-MIP if the collocated region is found by the motion information of the current block (similar to what Inter does) In the above MIP variants, the reference samples used by the IBC or IntraTMP based MIP mode include the reconstructed samples indicated by one or more block vectors of the current block; the reference samples used by the Inter based MIP mode include the reconstructed samples indicated by one or more motion vectors of the current block or the reconstructed samples in one or more previous pictures; and / or, for the ease of explanation, the reference samples are the reconstructed samples, but not limited to using the reconstructed samples as the reference samples. The steps of how to generate the predictor of the current block will be described in detail below. In the present invention, all the steps, any subset of the steps, or any extension of the steps can be used to generate the predictor of the target mode.

[0094] Step 0: Different matrices are predefined for different block sizes (width x height is N x M). In one embodiment, the matrix of the current block is predefined according to the block size and / or the block shape of the current block. The following shows an example of classifying the block size into 3 categories. The proposed method is not limited to be applied to this example. The number of categories can be any predefined positive integer.

[0095] For example, for a block with size (classified as S0), N is 4 (or the minimum block size), M is 4 (or the minimum block size), and K0 candidate matrices are predefined, each with dimension n0 x m0.

[0096] For another example, for a block with size (classified as S1), NxM is 4x8, 8x4, 8x8, 4xM, or Nx4, and K1 candidate matrices are predefined, each with dimension n1 x m1.

[0097] For yet another example, for a block with size (classified as S2), the remaining size is NxM, and K2 candidate matrices are predefined, each with dimension n2 x m2.

[0098] K0, K1, and K2 can be any predefined value, e.g., 1, 2, 3, 4, etc. m0, m1, and m2 are the matrix input size of S0, S1, and S2, respectively. n0, n1, and n2 are the matrix output size of S0, S1, and S2, respectively. The following table shows an example of different block sizes and different number of candidate matrices.

[0099] Different block sizes (NxM in width x height) and different matrices are predefined in Table 2 for classes S0, S1 and S2.

[0100]

[0101] In one embodiment, when only one candidate matrix is available for the current block, the matrix is not indicated by a signaling index, but derived.

[0102] Step 1: Derive the input reference samples of the matrix by using initial reference samples.

[0103] Step 1-0: Several embodiments are proposed to prepare the initial reference samples. In one embodiment, the initial reference samples are not filtered by a reference smoothing filter (such as [1, 2, 1] filter).

[0104] Figure 11A shows an example of initial reference samples for regular MIP, where block 1110 corresponds to the current block. The initial reference samples refer to the neighboring reconstructed samples adjacent to the current block. In Figure 11A, the initial reference samples include the first N rows of samples 1120 adjacent to the current block (from the reference line 0) and / or the first M columns of samples 1122 adjacent to the current block (from the reference line 0). In one embodiment (as shown in Figure 11B), the initial reference samples include the first N rows of samples 1130 non-adjacent to the current block (from the reference line greater than 0) and / or the first M columns of samples 1132 non-adjacent to the current block (from the reference line greater than 0). In another embodiment, the size of the initial reference samples in the top or left region can be extended to be less than or equal to (N+M) any value of k, where k can be any positive integer greater than or equal to 1. In another embodiment, the size of the initial reference samples in the top region can be extended to be less than or equal to N any value of k, where k can be any positive integer greater than or equal to 1. In another embodiment, the size of the initial reference samples in the left region can be extended to be less than or equal to M any value of k, where k can be any positive integer greater than or equal to 1. All or any subset of the samples in the top, left or both regions can be used as the reference samples for MIP.

[0105] FIG. 12A shows an example of initial reference samples for IBC- / intraTMP- / inter-MIP, where block 1210 corresponds to the collocated region. The collocated region can have the same size as the current block, and the initial reference samples are derived according to the regular MIP method described above (i.e., the initial reference samples include the top N rows of samples 1220 (starting from reference line 0) and / or the left M columns of samples 1222 (starting from reference line 0) that are adjacent to the current block). In one embodiment (as shown in FIG. 12B), the initial reference samples refer to the current reconstructed samples of the collocated region of the current block (1230 and 1232 in FIG. 12B). In one sub-embodiment, if the size of the collocated region is larger (or smaller) than the current block, a downsampling (or upsampling) operation is used to obtain the initial reference samples.

[0106] In one embodiment, when preparing the initial reference samples for the current block, the availability of neighboring reconstructed samples is checked, and then the unavailable samples are filled from the available samples. The operation in this step is unified with that in the regular intra prediction mode.

[0107] Step 1-1: The size of the input reference samples is related to the matrix size (m0, ml or m2). For S0, the size of the left and / or top = 2. Specifically, the size equal to 2 means 2 from the top and 2 from the left, so m0 is 4. In one embodiment, m0 can be set to any value smaller than 4. For example, for S0, the size of the left and top = 1. m0 can be set to 2. For S1 and / or S2, the size of the left and / or top = 4. The size equal to 4 means 4 from the top and 4 from the left, so ml and / or m2 is 8. In one embodiment, ml and / or m2 can be set to any value smaller than 8.

[0108] Step 1-2: If the size of the input reference samples is smaller than the size of the initial reference samples, a downsampling operation is used, and the downsampling factor is set to (size of the initial reference samples / size of the input reference samples). An example of the downsampling operation is shown in the following expression (2).

[0109] inputReferenceSample[ x ] = ( + ( 1<<( Log2( ) - 1 ) ))>>Log2( ), (2) Several embodiments are proposed to define different down-sampling operations. In one embodiment, a sub-sampling method is used as the down-sampling operation. That is, for each x, a representative sample value is set to the pre-defined value of the sample in initialReferenceSample[x downSampleFactor +i ] instead of averaging all samples in initialReferenceSample[x downSampleFactor +i ]. For example, the pre-defined value of the sample refers to the middle value / average value / max value / min value in initialReferenceSample[x downSampleFactor +i ]. For example, the pre-defined value of the sample refers to the middle value / average value / max value / min value in initialReferenceSample[x downSampleFactor +i ]. In another embodiment, a filtering method is used to generate the representative sample value in initialReferenceSample[x downSampleFactor +i ]. The filtering can refer to Gaussian filtering, cubic filtering, any low-pass filtering, and / or any filtering supported by the standard. In one sub-embodiment, the filtering tap is 2, 4, 6, 8, or any pre-defined value. The length of the filtering tap is different from (i.e., longer or shorter than) the intra and / or inter and / or third mode type prediction, or consistent with the intra and / or inter and / or third mode type prediction. The proposed method can be used for any other down / up-sampling operation in the present invention.

[0110] Step 1-2 is applied to the left and / or above, respectively. Then, all or any subset of the input reference samples from the left and / or above are concatenated as bdry red [i], where i is from 0 to (m0, m1, or m2) -1 (denoted as (m0 / m1 / m2) -1), for S0 / S1 / S2, respectively, and slightly adjusted to form a vector used as the matrix input reference samples as input[i].

[0111] Step 2: The input reference samples of the matrix are multiplied with the matrix coefficients for generating the initial predictor of the current block. For example, , where offset , or =((( )+ 32) ) + , The initial predictor is post-processed using clipping operation to make the predictor value within a predefined bit range. For example, the predefined bit range is 0 to (1 « bitDepth of luma or chroma). If the current block is a luma coded block, the bitDepth of luma is used; otherwise, the bitDepth of chroma is used.

[0112] Step 3: If n0 / n1 / n2 is smaller (or larger) than the size of the current block, the initial predictor is up-sampled (or down-sampled). For example, a bilinear interpolation filter is used, and a predefined order is set, either performing horizontal filtering first and then performing vertical filtering, or performing vertical filtering first and then performing horizontal filtering.

[0113] In one embodiment, when using the down-sampling and / or up-sampling mechanism to provide candidates for the target mode, one or more matrices corresponding to block size categories (different from the block size category of the current block) can be used as additional candidate matrices of the current block, or as replacement candidate matrices of the current block, to replace one or more existing matrices of the current block. For example, when the target block size associated with the target block size category is smaller than the regular block size associated with the regular block size category of the regular MIP mode, the reference samples above, left, or both of the current block will be down-sampled more than the regular MIP mode. For example, when the target block size associated with the target block size category is larger than the regular block size associated with the regular block size category of the regular MIP mode, the reference samples above, left, or both of the current block will be up-sampled more than the regular MIP mode.

[0114] For the case that the current block is classified as S0 (as regular case or regular MIP mode), only the matrix with size equal to n0 x m0 is available initially; by the proposed method, the matrix with size equal to n1 x m1 and / or the matrix with size equal to n2 x m2 are available. It is shown below that for the regular case, the target block size associated with the target block size category is larger than the regular block size associated with the regular block size category. When using the matrix from S1 and / or S2, the following methods are proposed. If the current block is 4x4 and m1 or m2 is set to 8, it means that the top is 4, the left is 4, all the neighboring reference samples are adjacent to the top and left of the current block as the initial and / or input reference samples. In one sub-embodiment, if m1 or m2 is set to a value larger than the size of the initial reference samples prepared for the current block, the initial reference samples prepared for the current block will be up-sampled to match m1 or m2. In another sub-embodiment, if m1 or m2 is set to a value larger than the size of the input reference samples of the current block, the input reference samples of the current block will be up-sampled to match m1 or m2. In another sub-embodiment, if n1 or n2 is set to a value larger than the size of the current block, the initial predictor will be down-sampled to match the size of the current block.

[0115] For one example, where the current block is classified as S1 (as regular case or regular MIP mode), only matrices of size equal to n1 x m1 are initially available; through the proposed method, matrices of size equal to n0 x m0 and / or size equal to n2 x m2 are available. When using matrices from S0 and / or S2, the following methods are proposed. In one sub-embodiment, if m0 is set to a value smaller than the size of the prepared initial and / or input reference samples of the current block, the prepared initial and / or input reference samples of the current block are down-sampled to match m0. If the current block is 8x8, and m0 is set to 4 (which means a down-sampling factor of 2), the top 2 reference samples and left 2 reference samples can be obtained, then more neighboring reference samples adjacent to the top and left of the current block will be down-sampled than the regular down-sampling to obtain the initial and / or input reference samples. When obtaining the input reference samples of the current block, the down-sampling factor of one side is set to (size of initial reference samples / 2 (instead of 4 originally)). That is, a larger down-sampling factor is used. In the original S1 category, for one side, an 8x8 block down-samples 8 samples to 4 samples (because m1 = 8 means there are 4 samples on one side), the down-sampling factor is 2, which can be calculated by 8 / 4. Using the proposed method of the S0 category, which is smaller than the original S1, for one side, an 8x8 block down-samples 8 samples to 2 samples (because m0 = 4 means there are 2 samples on one side), the down-sampling factor is 4, which can be calculated by 8 / 2. In another sub-embodiment, if n0 is set to a value smaller than n1, the initial predictor will be further up-sampled to match the size of the current block. In another sub-embodiment, if m2 is set to a value larger than the size of the prepared initial reference samples of the current block, the prepared initial reference samples of the current block will be up-sampled to match m2. In another sub-embodiment, if m2 is set to a value larger than the size of the input reference samples of the current block, the input reference samples of the current block will be up-sampled to match m2. In another sub-embodiment, if n2 is set to a value larger than the size of the current block, the initial predictor is down-sampled to match the size of the current block.

[0116] For one example, where the current block is classified as S2 (as regular case or regular MIP mode), initially only matrices of size equal to n2 x m2 are available; with the proposed method, matrices of size equal to n0 x m0 and / or of size equal to n1 x m1 are available. Below is shown a case where the target block size associated with the target block size category is smaller than the regular block size associated with the regular block size category. When using matrices from S0 and / or S1, the following method is proposed. In one sub-embodiment, if m0 or m1 is set to a value smaller than the size of the initial and / or input reference samples prepared for the current block, the initial and / or input reference samples prepared for the current block are down-sampled to match m0 or m1. If the current block is 16x16, and m0 is set to 4 (which means a down-sampling factor), then the top 2 reference samples and the left 2 reference samples can be obtained, then the neighboring reference samples adjacent to the top and left of the current block will be down-sampled more than the regular down-sampling to obtain the initial and / or input reference samples. When obtaining the input reference samples for the current block, the down-sampling factor for one side is set to (size of initial reference samples / 2 (instead of the original 4)). That is, a larger down-sampling factor is used. In the original S2 category, for one side, a 16x16 block is down-sampled from 16 samples to 4 samples (because m2 = 8 means there are 4 samples for one side), the down-sampling factor is 4, which can be calculated by 16 / 4. The proposed method uses the S0 category which is smaller than the original S2, for one side, a 16x16 block is down-sampled from 16 samples to 2 samples (since m0 = 4, means there are 2 samples for one side), the down-sampling factor is 8, which can be calculated by 16 / 2. In another sub-embodiment, if n0 or n1 is set to a value smaller than n2, the initial predictor will do more up-sampling to match the size of the current block.

[0117] In another embodiment, when providing candidates of the target mode using the transposition mechanism, one or more matrix coefficients or one or more reference samples are transposed to additional candidate matrices for the current block or as replacement candidate matrices for one or more existing matrices. When the transposition mechanism is applied to the reference samples, several embodiments are proposed. In one embodiment, if initially the top input reference samples are concatenated after the left input reference samples, now the left input reference samples are concatenated after the top input reference samples; or if initially the left input reference samples are concatenated after the top input reference samples, now the top input reference samples are concatenated after the left input reference samples. In another embodiment, the initial predictor is transposed. For example, , .

[0118] In another embodiment, when providing candidates of the target mode using the lateral mechanism, the reference samples are from the top or the left. Initially, the reference samples are from both the left and the top. With the proposed method, the reference samples focus on the information from one side. For example, two lines from the top are referenced. One line is the original line 0 from the top, and the other line is any pre-defined line i (i > 0) from the top. For another example, two lines from the left are referenced. One line is the original line 0 from the left, and the other line is any pre-defined line i (i > 0) from the left. The two lines can or can not be adjacent to each other.

[0119] In another embodiment, when providing candidates of the target mode using the inherited information, all or any subset or any combination of the following methods are applied. First, the encoding information of the previously coded blocks is recorded as follows. For each coded block using the target mode, the matrix information is recorded. The matrix information includes the matrix size, the matrix mode index, the transposition information, the target information of the target MIP mode used to determine one or more predictors, or a combination thereof. More specifically, the matrix information can include the matrix size, the matrix mode index (indicating which matrix is used), whether to transpose, all or any subset of the reference samples used in the matrix, any information used to obtain the predictors for the target mode, and / or any subset or extension of the above matrix information (e.g., the matrix coefficients or any information obtained from the matrix coefficients belonging to the matrix information). In one embodiment, one or more history-based tables are established to store the matrix information after each coded block uses the target mode. For example, different history-based tables are associated with different block sizes. For another example, different history-based tables are associated with different matrix sets. For another example, different history-based tables are associated with different block properties. For another example, only one history-based table is used. For the current block, the history-based matrix candidates in the one or more history-based matrix tables are added to the inherited list or set, and placed after a set of pre-defined matrix candidates. For example, the set of pre-defined matrix candidates includes the spatial candidates that are adjacent and / or non-adjacent, or any subset of the above. In this method, the matrix information of the previously coded blocks is stored in one or more tables, and used as the matrix information of the current block or used to derive the matrix information of the current block. During the encoding and / or decoding process, one or more tables containing multiple history-based matrix candidates will be maintained. When a new CTU row or any pre-defined range (the range is fixed in the standard, and / or determined by the block-level and / or high-level syntax in the bitstream) is encountered, the one or more tables will be reset (or emptied). The table size is any pre-defined value, which is fixed in the standard, and / or determined by the block-level and / or high-level syntax in the bitstream, and / or related to the size of the inherited list or set.

[0120] When inserting a new candidate into the table, a constrained first-in-first-out (FIFO) rule is used, where a redundancy check is first applied to find if there is an identical candidate in the table. If found, in one sub-embodiment, the identical candidate is removed from the table and all the candidates after it are moved forward and the identical candidate is inserted as the last entry of the table. These candidates can be used in the construction of the inheritance list or set. The latest few candidate matrices in the table are checked in order and inserted into the inheritance list or set after the predefined matrix candidate matrix set. For history-based candidate matrices, a full or partial redundancy check is performed on the previously existing candidate matrices in the inheritance list or set (e.g., the predefined matrix candidate matrix set). In another embodiment, to reduce the number of redundancy check operations, the following simplification methods are proposed. For example, the last K entries in the table are redundancy checked against the predefined spatial candidate matrix set (not all spatial candidate matrices). As another example, if the total number of inheritance candidate matrices reaches a certain number (i.e., the maximum allowed number of candidate matrices minus 1), the process of constructing the inheritance list or set from one or more history-based tables is terminated. In another embodiment, second, for the current block, the inheritance list or set is constructed by searching all or any subset of the coding blocks within a predefined range. In one embodiment, the predefined range refers to the current picture, tile, CTU, tile or any predefined region located in the current picture. In another embodiment, the predefined range is derived based on the current block. For example, the predefined range is the coding top range and / or the coding left range around the current block. As another example, the predefined range follows a non-adjacent candidate pattern. That is, the distance between the non-adjacent spatial candidate and the current block depends on the width and / or height of the current block. In another embodiment, the predefined range is limited by the CTU row. For example, if the current block is located in the top row within the CTU row, the range of the current block is smaller because there is no coding top range used. In another embodiment, the predefined range is not limited to be located in the current picture. The predefined range can be derived based on the location of the current block and / or can or can not be located in the current picture. When a coding block within the predefined range has matrix information and passes the validity check, the matrix information of the coding block is put into the inheritance list or set. In one embodiment, the validity check includes that the matrix input size (m) is larger than or equal to the current block size, which means that the reference samples of the current block are not enough to use the matrix. That is, the validity check depends on the input size of the matrix to be put into the list or set (if it passes the validity check, it can be inserted into the list or set) and / or the size of the current block. In another embodiment, the validity check includes that the matrix output size (n) is smaller than or equal to the current block size. That is, the validity check depends on the output size of the matrix to be put into the list or set and / or the size of the current block.In another embodiment, the validity check includes that the matrix information to be put in does not repeat with all or any subset of the existing members in the inheritance list or set. For example, full or partial pruning is used. In another embodiment, the order of the members in the inheritance list or set follows a search order within a predefined range. In another embodiment, the matrix information most selected by the coding blocks within a predefined range is put in the front of the inheritance list or set. Third, the inheritance index is signaled and / or parsed and / or derived to indicate one or more matrix information from the inheritance list or set. Then, the indicated matrix information is used for the current block to generate the predictor. If the matrix size of the indicated matrix information does not apply or does not belong to the block size category of the current block, then the proposed down-sampling and / or up-sampling mechanism can be applied here. In another embodiment, if a chroma block, for example, in the chroma DM mode, refers to a corresponding luma block, and the luma block is coded by the matrix-based mode, then the chroma block can inherit the matrix information from the corresponding luma block. When the color format is not 4:4:4 (e.g., 4:2:0), the size of the chroma block is different from the luma block. Therefore, the proposed down-sampling and / or up-sampling mechanism can be applied here to support the chroma block using the inherited matrix information.

[0121] In another embodiment, when candidates of the target mode are provided using a hybrid mechanism, the candidates using the hybrid mechanism are considered as additional sub-modes of the target mode or used to replace the prediction generation of the target mode. When the hybrid mechanism is applied to the current block, several embodiments of generating the predictor of the current block are proposed. In one embodiment, the predictor of the current block is formed by combining prediction hypotheses generated according to one or more matrix information. In one sub-embodiment, the combining weights are block-based weights (i.e., uniform for the whole block). In another sub-embodiment, the combining weights are sample-based weights (i.e., weights are derived for each sample) or position-based weights. In another embodiment, one or more matrix information is obtained by selecting from a set of candidate matrix information using a suggestion scheme. The set of candidate matrix information can include all or any subset of inherited matrix information. The set of candidate matrix information can include all or any subset of candidate matrix information of the current block. The suggestion scheme can refer to a boundary matching scheme, a template matching scheme (e.g., DIMD and / or TIMD), a pre-defined mapping table storing candidate information associated with different block sizes. For the example where the suggestion scheme is a boundary matching scheme, for each candidate matrix information in the set, a boundary matching cost is calculated, and the predictor of the current block is generated using the top K candidates with smaller boundary matching cost (as described later in this disclosure). The prediction hypotheses generated from the candidates with smaller cost have higher weights (similar to TIMD). For another example where the suggestion scheme is a boundary matching scheme, one or more matrix information is suggested from the top boundary and another one or more matrix information is suggested from the left boundary. If the current sample is close to the top boundary, the prediction hypotheses generated from the candidates suggested from the top boundary have higher weights. If the current sample is close to the left boundary, the prediction hypotheses generated from the candidates suggested from the left boundary have higher weights. The weights of the current sample can depend on the position of the current sample. One example of the weights is shown below: , .

[0122] If the matrix suggested from the left or top boundary corresponds to a larger boundary cost, the position-dependent weights are further adjusted to reduce the weights of the matrix with very large boundary matching cost.

[0123] In another embodiment, the predictor of the current block is further combined with one or more prediction hypotheses generated by one or more conventional intra prediction modes. In one sub-embodiment, the one or more conventional intra prediction modes are determined by the TIMD, boundary matching and / or DIMD suggestion. In another sub-embodiment, the one or more conventional intra prediction modes include at least any of the intra prediction modes in the planar, DC, horizontal, vertical, diagonal, 67 and / or 131 domains, any of the intra prediction modes supported in the standard or any subset of the above modes. In another sub-embodiment, if the size of the current block is greater or smaller than a predefined threshold (e.g., 4, 8, 16, 32, 64,...), the conventional intra prediction mode used is a fixed default mode (e.g., planar); otherwise, the conventional intra prediction mode used is determined by the suggestion. In another embodiment, PDPC is further used to adjust the predictor of the current block as shown in Figure 13 The horizontal neighboring reconstructed samples (R -1,y ) 1314 and / or the vertical neighboring reconstructed samples (R x,-1 ) 1316 are used to adjust the predictor (x, y) 1312. An example of the PDPC adjustment is shown as follows: .

[0124] General concept of transform process In the general transform process at the encoder, the residual of the current block is processed (or transformed) by a primary transform mode, and then becomes first temporary transform coefficients; the temporary transform coefficients can be further processed by a secondary transform mode, and then become second temporary transform coefficients; after the remaining operations of the residual coding, the bitstream containing the transform coefficient information is transmitted to the decoder. In the general transform process at the decoder, the second temporary transform coefficients are obtained from the received bitstream; the second temporary transform coefficients can be processed (or inverse transformed) by the secondary transform mode, and then the first temporary transform coefficients are obtained; the first temporary transform coefficients are processed by the first transform mode, e.g., the primary transform mode, and then the residual of the current block is obtained. In the above example, at most two transform modes (e.g., the primary transform mode and the secondary transform mode) are used for the current block. Note that this is not limited to this case.

[0125] In one embodiment, more than two transform modes are used for the current block. That is, in the encoder, more transform modes are applied after the secondary transform process; in the decoder, more transform modes are applied before the secondary transform process.

[0126] In another embodiment, only one transform mode (e.g., the primary transform mode) is used for the current block. For example, when NSPT is used as the primary transform, only the primary transform (i.e., without the secondary transform) is used for the current block.

[0127] In another embodiment, whether to use additional transform modes for the current block depends on a predefined rule. For example, the predefined rule is implicit. The implicit rule refers to block width, block height, block area, and / or neighboring coding information. For another example, the predefined rule is explicit. The explicit rule depends on block, SPS, PPS, tile, slice, picture, and / or sequence level flag. In one case, the explicit flag indicates the maximum number of transform modes for the current block. If the maximum number of transform modes is 2, the primary transform mode and the secondary transform mode are used for the current block. In another case, the explicit flag indicates whether to use one or more transform modes for the current block. For another example, the rule depends on block properties, such as the number of significant coefficients, the location of significant coefficients, and / or the distribution of significant coefficients. The rule can use a syntax related to the location of the last significant coefficient, a syntax related to whether a coding group within the current block contains any significant coefficients, and / or a syntax related to the location of the coding group in the current block.

[0128] In another embodiment, the proposed mechanism is to support more transform candidates for the target transform mode. The target transform mode refers to the primary transform mode, the secondary transform mode, and / or any transform mode (e.g., any transform mode with more than two transform modes).

[0129] In another embodiment, if the transform mode refers to a separable transform mode, the transform mode includes a transform type (e.g., matrix or kernel) for the horizontal transform and a transform type for the vertical transform; otherwise (the transform mode refers to a non-separable transform mode), a joint transform type is used. For example, the primary transform mode is a separable transform mode. For example, the primary transform mode is a non-separable transform mode. For example, the secondary transform mode is a separable transform mode. For example, the secondary transform is a non-separable transform mode. One example of a separable transform mode is setting {horizontal transform, vertical transform} to {DCT-II, DCT-II}, {MTS (multiple transform selection) of any transform type, MTS of any transform type}, {DCT-II, MTS of any transform type}, {MTS of any transform type, DCT-II}. One example of a non-separable transform is LFNST or NSPT (non-separable primary transform). For one example of NSPT (a primary transform mode that is similar to but different from the LFNST concept), each NSPT type (i.e., NPST4x4, NPST4x8, NPST8x4, and NPST8x8) uses 35 transform sets, and the transform set is determined by the intra mode. Each NSPT transform set includes three transform kernels and is designated by an explicitly signaled NSPT index. For the example of using DCT-II or MTS as the primary transform and LFNST as the secondary transform (when MTS is used, the secondary transform cannot be used), NSPT is used to replace DCT-II + LFNST for block sizes of 4x4, 4x8, 8x4, and 8x8. Thus, these block sizes will not test LFNST4 and LFNST8. However, they are still used for larger block sizes. NSPT can be extended or modified here to utilize additional non-separable transforms for 4x4, 4x8, 4x16, 8x4, 8x8, 8x16, 16x4, 16x8, and 16x16 intra blocks, and / or apply additional non-separable transforms in addition to the existing DCT-II + LFNST and adaptive MTS, and / or derive 67 transform kernels based on different intra prediction modes for each block size, and / or signal one additional flag (after the existing MTS flag) to indicate whether to apply a non-separable primary transform. If the flag is true, the MTS index is not signaled and NSPT is used. Otherwise, the MTS index is further signaled.

[0130] In another embodiment, a zero-out operation can be used in the transform mode. Whether the signaling of the transform mode is parsed at the decoder will check whether there are any significant coefficients located in the zero-out region. The check can use syntax related to the last significant coefficient position, a significant flag of the coding group (indicating whether the coding group contains any significant coefficients), and / or the position of the coding group within the current block. If yes, the signaling of the transform mode is bypassed or always indicates the disabling (bitstream conformance definition can be needed); otherwise, the signaling of the transform mode can be parsed at the decoder.

[0131] In another embodiment, the current block is coded by intra, inter or IBC. For example, the prediction type of the current block is MODE INTRA, MODE INTER or MODE IBC. For an example of the current block coded by intra, in the encoder, the residual is generated according to a prediction based on spatial neighboring reference samples and an intra prediction mode; in the decoder, the residual and the mentioned prediction are used to reconstruct the current block. For an example of the current block coded by inter, in the encoder, the residual is generated according to a prediction based on reference samples in a previously coded picture and inter motion information; in the decoder, the residual and the mentioned prediction are used to reconstruct the current block. For an example of the current block coded by IBC, in the encoder, the residual is generated according to a prediction based on reference samples in the same picture and block vector information; in the decoder, the residual and the mentioned prediction are used to reconstruct the current block.

[0132] Representative Prediction Mode in Transform It is proposed to determine a representative prediction mode of the regular intra mode, the special intra mode, the non-intra mode and / or any mode, e.g. any mode in annotation A.

[0133] The regular intra mode means that a predictor of the current block is generated using one or more traditional intra prediction modes and spatially neighboring reference samples (located in neighboring or non-neighboring reference lines of the current block). The regular intra mode can be used for luma and / or chroma components, the used traditional intra prediction mode can be indicated using syntax elements and / or a predefined implicit derivation method (e.g. DIMD applying a histogram analysis to suggest one or more intra prediction modes and / or TIMD applying a template analysis to suggest one or more intra prediction modes).

[0134] A special intra mode means that when the current block is coded using the special intra mode, a traditional intra prediction mode (e.g., one of the 67 intra prediction modes) is not used to generate the predictor of the current block, but an alternative (e.g., a matrix-based scheme and / or a cross-component information) is applied to the spatial neighboring reference samples and / or reference samples from co-located luma block to generate the predictor of the current block. For example, a special intra mode refers to a matrix-based intra prediction (MIP) or any intra mode that uses a derived or predefined matrix and input reference samples to generate the prediction of the current block. The special intra mode can be used for luma and / or chroma components. For example, a special intra mode for chroma refers to any kind of cross-component mode, which means that the current chroma block refers to the predicted / reconstructed samples of one or more co-located luma blocks or any predefined information to generate the prediction of the current chroma block. An example of cross-component mode is LM mode (e.g., CCLM and / or MMLM) and / or any LM variant (e.g., CCCM and / or GLM). For example, a special intra mode refers to a fusion of CCLM, MMLM, CCCM, GLM, any LM variant, or chroma intra prediction mode. For example, a special intra mode refers to TMP.

[0135] A non-intra mode means IBC, inter mode, and / or any mode type not equal to MODE TYPE INTRA. The non-intra mode can be used for luma and / or chroma components.

[0136] The modes in Note A are shown as follows.

[0137] SGPM HOR / VER PL (e.g., the design in JVET-AA0104 or other variants) For horizontal plane mode, only based on the left reference sample and the top-right reference sample, a horizontal linear interpolation is performed to predict the current sample as: , For vertical plane mode, only based on the above reference sample and the left-bottom reference sample, a vertical linear interpolation is performed to predict the current sample as: , When HOR / VER PL is used for the current block, other variants, e.g., a decoder-side derivation method, are used to decide which one of the two directional plane modes to use. Specifically, the horizontal gradient of each sample in the neighboring rows of the current block and the vertical gradient of each sample in the neighboring columns of the current block are calculated. If the sum of the absolute values of the horizontal gradients is greater than the sum of the absolute values of the vertical gradients, the vertical plane mode (or the horizontal plane mode) is used, otherwise the horizontal plane mode (or the vertical plane mode) is used.

[0138] In the transform / inverse transform stage and / or in the subsequent coded block, instead of using the planar mode directly, (1) the horizontal and vertical planar modes are modified to vertical (or horizontal) mode and horizontal (or vertical) mode, respectively, or (2) any of the proposed methods in this disclosure is used. In another embodiment, the selection between (1) and (2) depends on the block width / height / area. For example, if the block width / height / area is smaller than a pre-defined threshold (e.g., 2, 4, …, or any pre-defined value), (1) is used; otherwise, (2) is used.

[0139] Direct Block Vecto (DBV): a chroma prediction mode Variant 1: Signaling a bin to indicate whether DBV mode is used When the chroma dual tree is activated in Intra, for a chroma CU coded in DBV mode, if one of the collocated luma blocks is coded in IBC mode or IntraTMP mode, its block vector bvL is used to derive the chroma block vector bvC. The bv scaling process is determined according to template matching. If the luma block is coded in RRIBC (reconstruction reordering IBC), flip-aware bv adjustment is applied to bvL, same as JVET-AA0070.

[0140] By using the position of the current chroma block (xCb, yCb) and its bvC, the corresponding offset position (xCb+ bvC[0], yCb + bvC[1] ) is determined and block copy prediction is performed.

[0141] Variant 2: When the chroma dual tree is activated in Intra, for a chroma CU coded in DM mode, if one of the collocated luma blocks is coded in IBC mode or IntraTMP mode, its block vector bvL is used to derive the chroma block vector bvC. The bv scaling process is determined according to template matching. If the luma block is coded in RRIBC mode, flip-aware bv adjustment is applied to bvL, same as JVET-AA0070.

[0142] By using the position of the current chroma block (xCb, yCb) and its bvC, the corresponding offset position (xCb+ bvC[0], yCb + bvC[1] ) is determined and block copy prediction is performed.

[0143] In the transform / inverse transform stage and / or in the subsequent coding block, instead of using the planar mode, either (1) using the chroma DM or (2) using any of the proposed methods in this invention. In another embodiment, the selection between (1) and (2) depends on the block width / height / area. For example, if the block width / height / area is smaller than a pre-defined threshold (e.g. 2, 4, … or any pre-defined value), then (1) is used; otherwise, (2) is used.

[0144] Hybrid mode (e.g. CIIP / GPM) In the prediction stage, in order to improve the prediction efficiency, different modes can apply adjustment when generating the predictor or after generating the predictor. For regular intra modes, the final predictor can be a combined predictor based on multiple prediction hypotheses of multiple conventional intra prediction modes. Therefore, it is difficult to find a representative prediction mode directly among the multiple conventional intra prediction modes. For special intra modes and / or non-intra modes, the predictor is not generated based on (conventional) intra prediction modes. However, in some cases, the intra prediction mode of the current block will be needed.

[0145] In the following encoding / decoding stage after / before / in parallel with the prediction process, the intra prediction mode of the current block will be used.

[0146] In the transform / inverse transform stage, the intra prediction mode is used by the secondary transform, where the transform mode can be any transform mode mentioned in the section of General Concept of Transform Process. For example, this case can correspond to the selection of transform set and / or transposition flag for the secondary transform (e.g. LFNST). The secondary transform in this invention is not limited to LFNST, can be any pre-defined separable / non-separable and / or low-frequency / non-low-frequency transform, which is performed after (or before) the primary transform in the encoder (or decoder) and applied to the coefficients of the primary transform (or received dequantized) in the encoder (or decoder).

[0147] In the transform / inverse transform stage, the intra prediction mode can be used by the primary transform, where the transform mode can be any transform mode mentioned in the section of General Concept of Transform Process. For example, this case can correspond to the selection of transform kernel (e.g. default primary transform like DCT-II and / or MTS and / or NSPT) for the primary transform. The primary transform in this invention can be any pre-defined transform, which is performed on the residual (or inverse secondary transform coefficients or received dequantized transform coefficients) in the encoder (or decoder).

[0148] In the transform / inverse transform stage, the intra prediction mode can be used by any transform mode mentioned in the section of General Concept of Transform Process (i.e. can be referred by the primary transform, the secondary transform or any two or more transform modes).

[0149] For a subsequent coding block, the intra prediction mode information from one or more neighboring or previously coded blocks is used to derive the MPM list or any possible scheme that can refer to the intra prediction mode information from one or more neighboring or previously coded blocks.

[0150] If the current block is a luma block, the related chroma blocks can need the intra prediction mode from luma (e.g., to decide the chroma DM).

[0151] Therefore, when one of the above modes is applied to the current block, the proposed new mechanism can be used to determine a representative prediction mode for one or more of the above cases.

[0152] In one embodiment, the representative prediction mode is any one of the DIMD derived modes (e.g., applying histogram analysis on neighboring reconstructed samples that are spatially adjacent or non-adjacent to the current block), the TIMD derived modes (e.g., applying template analysis on neighboring reconstructed samples that are spatially adjacent or non-adjacent to the current block), DC, Planar, Horizontal, Vertical, Diagonal, or a predefined mode among any available intra prediction modes. Take the current block coded by MIP as an example (which can be replaced by any regular intra mode, special intra mode, non-intra mode, and / or any mode in Note A, and the predefined intra prediction mode is DIMD derived mode as an example. The DIMD derived mode is stored for MIP and / or can be used for any one or more of the following: a subsequent block, to derive the MPM list or any possible scheme that can refer to the intra prediction mode information from one or more neighboring or previously coded blocks a collocated chroma block, if the current block is a luma block, to decide its intra prediction mode (e.g., to derive the chroma DM) the current block, to decide the transform set and / or the transposed flag and / or the transform kernel of the transform stage In Figure 14A An example of applying histogram analysis on spatially neighboring reconstructed samples is shown in FIG. 14. The region 1420 corresponds to the neighboring region used as the template for the current block 1410. A window 1422 is applied to each or any subset of the neighboring region (template). The center of the window is located on the center sample of each or any subset of the neighboring region. The center sample is shown as a small circle 1424 filled with dots.

[0153] In Figure 14B An example of applying template analysis on spatially neighboring reconstructed samples is shown in FIG. 15. The TIMD cost (based on template) is calculated as the distortion between the reconstructed samples and the predicted samples in the neighboring region 1420 (i.e., template) of the current block 1410, where the predicted samples in the neighboring region (i.e., template) are generated according to the candidate prediction modes.

[0154] In one sub-embodiment, when using the template in the above example, the template can include any extension of the top template 1520, left template 1530 and top-left template 1540 as shown in Figure 15A

[0155] Figure 15B

[0155]

[0155] In another embodiment, the representative prediction mode is determined by applying a predefined process to all or any subset of the prediction samples of the current block. The predefined process suggests a kind of intra prediction mode as the representative prediction mode of the current block. In one sub-embodiment, the predefined process refers to DIMD and / or TIMD. In one sub-embodiment, when the current block is a chroma block and coded using cross-component mode or chroma intra prediction mode merging, the predefined process is applied to all or any subset of the reconstructed / predicted samples of one or more co-located luma blocks. Take DIMD as an example, the window used in DIMD is applied to the predictors within the current block. In another sub-embodiment, the prediction samples for applying the predefined process refer to down-sampled or reduced prediction samples, and the predefined process is applied to all or any subset of the down-sampled or reduced prediction samples. For example, the original size of the current block is 2Mx2N, by down-sampling from 2Mx2N, the reduced or down-sampled size of the current block is MxN. In another sub-embodiment, the predefined process is applied to all or any subset of the prediction samples of the current block, and the prediction samples of the current block are obtained by applying a predefined process to all or any subset of the reconstructed / predicted samples of one or more co-located luma blocks. Take DIMD as an example, the window used in DIMD is applied to the predictors within the current block. In another sub-embodiment, the prediction samples for applying the predefined process refer to down-sampled or reduced prediction samples, and the predefined process is applied to all or any subset of the down-sampled or reduced prediction samples. For example, the original size of the current block is 2Mx2N, by down-sampling from 2Mx2N, the reduced or down-sampled size of the current block is MxN. Figure 16 ​​As shown, the center of the window is applied to samples within the current (or reduced) block (gray area), but not at the boundary of the current (or reduced) block. In another sub-embodiment, the center of the window is applied to every sample within the current (or reduced) block. If any sample outside the current (or reduced) block is needed for the window, padding from the boundary is used instead of referencing samples outside the current (or reduced) block. Take TIMD as an example, the TIMD cost of a candidate prediction mode is calculated as the distortion between the final prediction samples (e.g., the final prediction sub-blocks are generated according to the block vector if the current block is coded by IBC) and the prediction samples (from the candidate prediction mode) of all or any subset of the current (or reduced) block.

[0156] Take the case that the current block is coded by MIP (which can be changed to any regular intra mode, special intra mode, non-intra mode, and / or any mode in Note A), and the representative prediction mode of the current block is from DIMD. The representative prediction mode is stored for MIP and / or can be used for any one or more of - a subsequent block for deriving the MPM list, or any scheme that can reference intra prediction mode information from one or more neighboring or previously coded blocks - a co-located chroma block (if the current block is a luma block) for determining its intra prediction mode (e.g., deriving the chroma prediction model) - the current block for determining the transform kernel of the transform stage In another embodiment, a representative prediction mode is predefined as the default mode. The default mode can be any predefined intra prediction mode in planar, DC, or the standard. In one sub-embodiment, the default mode is fixed to one intra prediction mode. In another sub-embodiment, the default mode is selected according to the coding mode of the current block. For example, the default mode of MIP is planar; the default mode of intra TMP is planar; the default mode of SGPM (spatial GPM, using intra prediction for two sub-blocks split from a GPM partition) / GPM / any GPM variant is any one of the modes in the mode according to partition angle or IPM (intra prediction mode) list; the default mode of chroma cross-component mode is chroma DM; the default mode of horizontal / vertical planar is horizontal / vertical; the default mode of DBV is chroma DM or DC or the same mode as the default mode of IBC; the default mode of CIIP (combined inter and intra prediction) is TIMD or planar or any intra prediction mode used in CIIP; and / or the default mode of IBC is DC or planar. For example, the default mode of SGPM is the mode according to partition angle; for example, there is a mapping table used to map from partition angle to default mode and / or the size of the mapping table depends on the number of candidate partition angles. If the number of candidate partition angles is 32, the size of the mapping table is 32. One possible mapping table is shown below, where the default mode is one of the 67 domain intra prediction modes: {50, 0, 44, 41, 34, 27, 0, 0, 18, 0, 0, 9, 66, 59, 56, 0, 50, 0, 44, 41, 34, 27, 0, 0, 18, 0, 0, 9, 66, 59, 56, 0}.

[0157] The numbers in the table refer to the default mode. The angle equal to 0 (i.e., the vertical partition direction) refers to the default mode 50 (i.e., the vertical intra prediction mode). The default mode numbers are symmetric. The first half of the table is the same as the second half of the table. In another example, the angle equal to 0 (i.e., the vertical partition direction) refers to the default mode 18 (i.e., the horizontal intra prediction mode). The size of the mapping table can be reduced by grouping the N candidate angles into multiple (< N) groups and using the group index to find the default mode from the mapping table. The size of the mapping table is the number of groups. For example, using angle symmetry (e.g., angles 0 and 16 are symmetric and both belong to the vertical partition direction), the symmetric angles are put into the same group.

[0158] In another embodiment, when the representative prediction mode is a DIMD derived mode (histogram analysis applied on spatially reconstructed samples neighboring or non- neighboring the current block), a TIMD derived mode (template analysis applied on spatially reconstructed samples neighboring or non-neighboring the current block) or any mode that obtains the best proposal (e.g. minimum cost, maximum histogram...) among all candidate modes, or when the representative prediction mode is determined by applying a predefined process on all or any subset of prediction samples on the current block, wherein the predefined process (e.g. DIMD, TIMD and / or any predefined method) proposes an intra prediction mode as the representative prediction mode for the current block, some embodiments aim at determining whether the proposal is reliable. If reliable, the proposed mode is used as the representative prediction mode for the current block; otherwise, the representative prediction mode for the current block is decided using a default way.

[0159] In one sub-embodiment, when the representative prediction mode is proposed using DIMD (Decoder-side Intra Mode Derivation), the proposal is reliable or not depends on the distribution of the histograms. For example, the proposal is reliable when the maximum histogram is greater than (T the second maximum histogram), where T is predefined to be any value greater than 1, e.g. 1.25, 1.5, 2.0. In implementation, a bit-shifting operation is used instead of multiplying by a floating value (T). For example, (2.0 the second maximum histogram) is implemented as (second maximum histogram « 1). Another example, (1.25 the second maximum histogram) is implemented as (second maximum histogram + second maximum histogram » 2). Yet another example, (1.5 the second maximum histogram) is implemented as (second maximum histogram + second maximum histogram » 1). In another implementation, a look-up table is used instead of multiplying by a floating value (T). By using a look-up table, T is approximated.

[0160] In another sub-embodiment, when the representative prediction mode is proposed using TIMD (Template-based Intra Mode Derivation), the proposal is reliable or not depends on the distribution of the TIMD costs. For example, the proposal is reliable when T2 the minimum cost is less than the second minimum cost (e.g. applying a similar rule or threshold to the histograms in DIMD), where T2is predefined to be any value greater than 1, e.g. 1.25, 1.5, 2.0 or any value greater than 1. In implementation, a bit-shifting operation is used instead of multiplying by a floating value (T2). For example, (2.0 the minimum cost) is implemented as (minimum cost « 1). Another example, (1.25 the minimum cost) is implemented as (minimum cost + minimum cost » 2). Yet another example, (1.5 The minimum cost is implemented as (minimum cost + minimum cost » 1). In another implementation, a look-up table is used instead of multiplying by a floating value (T2). By using a look-up table, T2 is approximated.

[0161] In another embodiment, when the representative prediction mode is a DIMD derived mode (i.e. histogram analysis is applied to reconstructed samples spatially neighboring or non-neighboring to the current block), a TIMD derived mode (i.e. template analysis is applied to reconstructed samples spatially neighboring or non-neighboring to the current block), or any mode that obtains the best suggestion (e.g. minimum cost, maximum histogram,...) among all candidate modes, or when the representative prediction mode is determined by applying a predefined process to the predicted samples of all or any subset of the current block, wherein the predefined process (e.g. DIMD, TIMD and / or any predefined method) suggests one intra prediction mode as the representative prediction mode for the current block, some embodiments are designed to generate one new transform matrix / set from multiple (e.g. 2) existing transform matrices / sets selected from the multiple (e.g. 2) suggested representative modes, and / or the new transform matrix / set is used for the current block. Multiple representative modes (e.g. 2) are introduced when DIMD is used to suggest the representative intra prediction mode and the histograms of the candidate modes are not distinguishable (e.g. the maximum histogram is similar to the second largest histogram). Multiple representative modes (e.g. 2) are introduced when TIMD is used to suggest the representative intra prediction mode and the costs of the candidate modes are not distinguishable (e.g. the minimum cost is similar to the second smallest cost).

[0162] In one sub-embodiment, the new transform matrix / set is generated by mixing multiple existing transform matrices / sets. An example of mixing two existing transform matrices / sets into a new transform matrix is shown below. First, one existing transform matrix / set is selected according to one suggested mode and another existing transform matrix / set is selected according to another suggested mode. Then, a new matrix / set is generated as the new transform matrix by weighted averaging or mixing all or any subset of transform matrix members in the two existing transform sets into a new transform set. The rule of weighted averaging or selecting members can or can not depend on the histogram / cost in the predefined process.

[0163] In another embodiment, one or more intra prediction modes are predefined as preferred modes. When the representative prediction mode is any of a DIMD-derived mode (i.e., histogram analysis is applied to reconstructed samples spatially neighboring or non-neighboring the current block), a TIMD-derived mode (i.e., template analysis is applied to reconstructed samples spatially neighboring or non-neighboring the current block), or any mode that obtains the best suggestion (e.g., minimum cost, maximum histogram,...) among all candidate modes, or when the representative prediction mode is determined by applying a predefined process to the prediction samples of all or any subset of the current block, wherein the predefined process (e.g., DIMD, TIMD, and / or any predefined method) suggests an intra prediction mode as the representative prediction mode for the current block, some embodiments are designed to give the preferred modes higher priority in the predefined process than other non-preferred modes. For example, when DIMD is used to suggest the representative prediction mode, the histogram of the preferred mode is adjusted by multiplying it by T3, where T3 = 1.25, 1.5, 2.0, or any value greater than 1. In implementation, a bit-shift operation is used instead of multiplying by a floating value (T3). For example, (2.0 Histogram) is implemented as (Histogram « 1). Another example, (1.25 Histogram) is implemented as (Histogram + Histogram » 2). Yet another example, (1.5 Histogram) is implemented as (Histogram + Histogram » 1). In another implementation, a lookup table is used instead of multiplying by a floating value (T3). By using a lookup table, T3 is approximated. Another example, when TIMD is used to suggest the representative prediction mode, the cost of the preferred mode is adjusted by multiplying it by T4, where T4 = 0.25, 0.5, or any value less than 1. In implementation, a bit-shift operation is used instead of multiplying by a floating value (T4) or dividing by 1 / T4. For example, (0.25 Cost) is implemented as (Cost » 2). Another example, (0.5 Cost) is implemented as (Cost » 1). In another implementation, a lookup table is used instead of multiplying by a floating value (T4) or dividing by 1 / T4. By using a lookup table, T4 is approximated.

[0164] In one sub-embodiment, different coding modes can have different preferred modes. For example, when the current block is coded using the SGPM coding mode, the preferred mode is defined according to the partition angle of the current block. For another example, when the current block is coded using the MIP coding mode, the preferred mode is defined as planar, DC, or any predefined intra prediction mode. For another example, when the current block is coded using the IBC or inter coding mode, the preferred mode is defined as planar, DC, any intra prediction mode based on the mode information, any predefined subset of prediction units (or coding blocks or any predefined region) in the reference region (including the corresponding reference block indicated by the block vector or motion vector and / or the neighboring template of the corresponding reference block), or any predefined intra prediction mode. For another example, when the current block is coded using the cross-component mode, the preferred mode is defined as chroma DM, planar, DC, or any predefined intra prediction mode. For another example, when the current block is coded using the DBV (direct block vector) coding mode, the preferred mode is defined as chroma DM, planar, DC, horizontal, vertical, any intra prediction mode based on the mode information, any predefined subset of prediction units (or coding blocks or any predefined region) in the reference region (including the corresponding reference block indicated by the block vector and / or the neighboring template of the corresponding reference block), or any predefined intra prediction mode. For another example, when the current block is coded using the coding mode horizontal planar mode, the preferred mode is defined as planar, DC, horizontal, vertical, or any predefined intra prediction mode. For another example, when the current block is coded using the coding mode vertical planar mode, the preferred mode is defined as planar, DC, horizontal, vertical, or any predefined intra prediction mode. For another example, when the current block is coded using the intra TMP coding mode, the preferred mode is defined as planar, DC, horizontal, vertical, any intra prediction mode based on the mode information, any predefined subset of prediction units (or coding blocks or any predefined region) in the reference region (including the corresponding reference block indicated by the block vector and / or the neighboring template of the corresponding reference block), or any predefined intra prediction mode. For another example, the preferred mode refers to the default mode.

[0165] In another sub-embodiment, when using the DIMD proposal to represent the prediction mode, the preferred mode is the first N modes of the DIMD proposal, where N = 1, 2, …, or any positive integer.

[0166] In another sub-embodiment, when using the TIMD proposal to represent the prediction mode, the preferred mode is the first N2 modes of the TIMD proposal, where N2 = 1, 2, …, or any positive integer.

[0167] In another way, it is proposed to use the mode information of the reference region to derive a representative prediction mode for any of the regular intra mode, the special intra mode, the non-intra mode and / or any mode in annotation A. The proposed alternative is simpler and / or can output the derived representative intra prediction mode earlier without waiting for the reconstruction or prediction stage to know the required prediction or reconstructed sample compared to performing texture analysis (DIMD or TIMD) on samples within the current block or spatially neighboring samples. Some embodiments below focus on using IBC for the current block. Note that the proposed method is not limited to be applied to IBC blocks and can be used for current blocks coded by any other mode, e.g. intra TMP or inter block. When the current block is coded using intra TMP, the reference block is found by template matching. When the current block is coded using inter mode, the reference block can be in a reference picture or a pre-defined picture indicated by the motion information of the current block and / or pre-defined in a picture different from the current picture coded previously. For example, for the motion information of the current block being bi-prediction, the reference picture is indicated by the reference index of list 0 and / or the reference index of list 1. For another example, for the motion information of the current block being uni-prediction, the reference picture is indicated by the reference index of list 0 or list 1. If multiple reference pictures are used, an order is used to define which reference picture is used first. One possible order is to use the reference picture closer to the current picture first (i.e. smaller POC). Another possible order is to use the reference picture from the pre-defined list (list 0 or list 1) first.

[0168] In one embodiment, the representative prediction mode is derived based on the mode information of any pre-defined subset of prediction units (or coding blocks or any pre-defined region) in the reference region including the corresponding reference block and / or the neighboring template of the corresponding reference block. The mode information includes the mode type, the intra prediction mode, the motion information, the block width, the block height, the block area, the block shape, the block ratio, the residual information, the transform information, the partition information and / or any subset / extension of the above.

[0169] In one sub-embodiment, if no representative prediction mode can be found from the pre-defined subset of prediction units, a default prediction mode is used as the representative prediction mode for the current block. The default prediction mode can be any available intra prediction mode, e.g. planar or normal DIMD mode, which is always derived at the decoder.

[0170] In another sub-embodiment where the current block is coded by IBC, one reference block is indicated by a block vector and one or more mode information (e.g. intra prediction mode) saved in the reference block is used to derive a representative prediction mode for the current block. If the reference block is any of the special intra modes and / or non-intra modes, a default intra prediction mode is used as the representative prediction mode; otherwise, the intra prediction mode of the reference block is used as the representative prediction mode for the current block.

[0171] In another sub-embodiment where the current block is coded by IBC, one or more prediction units in the reference region (may contain the reference block and / or one or more spatially adjacent / non-adjacent prediction units to the reference block) are predefined and a scan order is applied to the predefined prediction units. One example of the scan order is shown as follows: - The selection of intra prediction modes in prediction units P1 to P5 follows a predefined order: P1, P2, P3, (P4, P5), where the positions of P1-P5 are shown as Figure 17

[0172] P1 covers the middle position of the reference block.

[0173] P2 covers the bottom-right position of the reference block.

[0174] P3 covers the top-left position of the reference block.

[0175] P4 covers a predefined position (e.g. the middle position) of the reference block above the outside.

[0176] P5 covers a predefined position (e.g. the middle position) of the reference block left to the outside.

[0177] - If the height of the block is greater than the width, P5 is checked first, then P4.

[0178] - Otherwise, P4 is checked first, then P5.

[0179] According to the scan order, one of the intra prediction modes in the predefined prediction units can be used as the representative prediction mode. For example, the first mode with an intra prediction mode is used as the representative prediction mode. Another example is that an explicit index is signaled / parsed to indicate one of the intra prediction modes in the predefined prediction units as the representative prediction mode. When a prediction unit in the predefined prediction units is not a valid mode, the invalid prediction unit is skipped in the scan or a default prediction mode is set for the invalid prediction unit. For example, any of the non-intra (e.g. IBC), intra TMP, inter, MIP and / or annotation A modes are invalid.

[0180] ​In another sub-embodiment where the current block is coded by IBC, one or more prediction units in the reference region (possibly containing the reference block and / or one or more prediction units spatially neighboring / non-neighboring the reference block) are predefined and a voting method is applied to the predefined prediction units. The voting method (e.g. by majority method of counting) means using the most popular prediction mode as the representative prediction mode of the current block. When a prediction unit in the predefined prediction units is not a valid mode, skip the invalid prediction unit for counting, or set a default prediction mode for the invalid prediction unit for counting. For example, any mode in non-intra (e.g. IBC), intra-TMP, inter, MIP and / or annotation A is invalid.

[0181] In another embodiment, when the representative prediction mode (intra prediction mode) is used to decide the transform kernel of MTS, the MTS can be implicit or explicit and the current block can be regular intra mode, special intra mode or non-intra mode. Explicit MTS (e.g. enhanced MTS for intra coding) means that after a MTS set is selected for the current block, a MTS index is signaled to find a candidate (transform pair) from the MTS set. Implicit MTS means that a candidate is found using implicit mapping rules (not relying on syntax elements). With the proposed method, for explicit MTS, the selection of MTS set depends on the representative prediction mode; for implicit MTS, the candidate is decided according to the representative prediction mode (following the mapping rules). Table 3-5 shows an example of mapping rules.

[0182]

[0183]

[0184] In another embodiment, the proposed method can be used for color format 4:4:4. An example is shown below. When the color format is 4:4:4, MIP can be used for chroma. For chroma MIP block, the representative prediction mode can be used for primary transform and / or secondary transform to select the transform set and / or the transpose flag.

[0185] In another embodiment, when the current block is coded using special intra mode and / or non-intra mode, the representative prediction mode is stored in the buffer of intra prediction modes. In the following process, if any process needs to access the buffer of intra prediction modes, the buffer of intra prediction modes is readily available.

[0186] In another embodiment, the representative prediction mode can vary implicitly with block width, block height, block area or according to explicit rules (e.g. syntax at block, tile, slice, picture, SPS or PPS level).

[0187] In another embodiment, any of the proposed methods or any combination of the proposed methods can be applied to any intra mode, e.g. WAIP (Wide Angle Intra Prediction), Intra Angular mode, ISP, MIP, VVC or any intra mode specified in the background of HEVC or more development modes in note A.

[0188] In another embodiment, in order to reduce the delay caused by determining the representative prediction mode for the current block by the proposed method, some enabling conditions are designed for the proposed method, and the proposed method is only allowed when the enabling conditions are met.

[0189] In another sub-embodiment, the enabling conditions include size settings related to block width, block height or block area or block shape. The size settings mean that the size of the current block is first checked, and if the size of the current block is not suitable for hardware or not suitable for DIMD to calculate the histogram or TIMD to calculate the cost, the proposed method cannot be applied. For example, when the width or height of the current block is greater than a pre-defined threshold, the proposed method cannot be used. Another example is that when the width or height of the current block is less than a pre-defined threshold, the proposed method cannot be used. Another example is that if the area of the current block is greater than a pre-defined threshold, the proposed method cannot be used. Another example is that if the area of the current block is less than a pre-defined threshold, the proposed method cannot be used. Another example is that if the long side of the current block is much larger than the short side of the current block, the proposed method cannot be used. The pre-defined threshold can be any integer, such as 2, 4, 8, 16, …, or any pre-defined value. When the proposed method cannot be used, the default way is used to decide the representative prediction mode.

[0190] The proposed method in the present application can be enabled and / or disabled according to implicit rules (e.g. block width, height or area) or according to explicit rules (e.g. syntax at block, tile, slice, picture, SPS or PPS level). For example, the proposed method can be applied when the block area is less than / greater than a threshold.

[0191] The term "block" in the present application can refer to TU / TB, CU / CB, PU / PB, pre-defined region or CTU / CTB.

[0192] Any combination of the proposed methods in the present application can be applied.

[0193] Boundary-Matching Cost The boundary matching cost of a candidate pattern refers to a measure of discontinuity between the current prediction (the predicted sample within the current block generated by the candidate pattern) and the neighboring reconstructions (reconstructed samples within one or more neighboring blocks), including top boundary matching and / or left boundary matching. Top boundary matching refers to the comparison between the current top predicted sample and the neighboring top reconstructed sample, while left boundary matching refers to the comparison between the current left predicted sample and the neighboring left reconstructed sample. Figure 18 Displays the current prediction (prediction samples within the current block 1810). pred i,j An example of a measure of discontinuity between the reconstructed and neighboring structures (including top boundary matching and / or left boundary matching), where 0 ≤ i, j ≤ 3.

[0194] In one embodiment, a predefined subset of the current predictions is used to calculate the boundary matching cost.

[0195] Any of the proposed methods described above can be implemented in the encoder and / or decoder. For example, any proposed method can be implemented in the inter-frame and / or intra-frame and / or IBC and / or prediction and / or transform modules of the encoder, and / or in the inter-frame and / or intra-frame and / or IBC and / or prediction and / or transform modules of the decoder. Alternatively, any proposed method can be implemented as a circuit connected to both the inter-frame and / or intra-frame and / or IBC and / or prediction and / or transform modules of the encoder and the inter-frame and / or intra-frame and / or IBC and / or prediction and / or transform modules of the decoder to provide the information required by the inter-frame and / or intra-frame and / or IBC and / or prediction and / or transform modules.

[0196] The MIP mode with one or more settings described above can be implemented at the encoder or decoder end. For example, any MIP mode with one or more settings can be implemented in the intra / inter-frame codec module of the decoder (e.g., Figure 1B Implemented in intra-frame prediction 150 / MC 152 in the encoder, or in the intra-frame / inter-frame codec module in the encoder (e.g., Figure 1A The proposed MIP mode is implemented in the intra-frame prediction 110 / inter-frame prediction 112. Any proposed MIP mode with various settings can also be implemented as a circuit connected to the intra-frame / inter-frame codec module of the decoder or encoder. However, the decoder or encoder can also use additional processing units to implement the processing of the required proposed MIP mode with one or more settings. Although the intra-frame / inter-frame prediction unit (e.g., Figure 1A Units 110 / 112 and Figure 1BThe units 150 / 152 in FIG. 1 are shown as separate processing units, which can correspond to executable software or firmware code stored on a medium (e.g., a hard disk or flash memory) for a CPU (central processing unit) or programmable device (e.g., a DSP (digital signal processor) or FPGA (field programmable gate array)).

[0197] Figure 19 A flow diagram of an exemplary video coding system is shown in which MIP mode uses one or more settings according to an embodiment of the present application. The steps shown in the flow diagram can be executed as program code on one or more processors (e.g., one or more CPUs) at the encoder side. The steps shown in the flow diagram can also be implemented based on hardware, e.g., one or more electronic devices or processors are arranged to perform the steps in the flow diagram. According to the method, input data related to a current block is received in step 1910, where the input data includes pixel data to be encoded at the encoder side or data related to the current block to be decoded at the decoder side. A target matrix-based intra prediction (MIP) mode is determined in step 1920 using MIP mode settings, where the MIP mode settings include a hybrid setting, a transpose setting, a representative mode setting, an inheritance setting, or a combination thereof, where the MIP mode setting with the hybrid setting includes a hybrid matrix candidate formed by combining two or more prediction hypotheses, where the MIP mode setting with the transpose setting includes a transpose matrix candidate generated by transposing one or more matrix coefficients or one or more reference samples, where the MIP mode setting with the representative mode setting includes a representative mode generated according to the representative mode setting, or where the MIP mode setting with the inheritance setting includes an inherited MIP candidate. The current block is encoded or decoded using the target MIP mode in step 1930.

[0198] The flow diagram shown is intended to illustrate one example of video coding according to the present application. Those skilled in the art can modify each step, rearrange steps, split steps, or combine steps to practice the present application without deviating from the spirit of the present application. In this disclosure, specific syntax and semantics are used to illustrate examples of implementing embodiments of the present application. Those skilled in the art can practice the present application by substituting equivalent syntax and semantics for these syntax and semantics without deviating from the spirit of the present application.

[0199] The foregoing description is intended to enable any person skilled in the art to practice the present application as claimed. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein can be applied to other embodiments without the use of the innovative techniques. Thus, the present application is not intended to be limited to the

[0200] Embodiments of the present application as described above can be implemented in various hardware, software codes, or a combination of both. For example, an embodiment of the present application can be one or more circuits integrated into a video compression chip or program codes integrated into video compression software to perform the processing described herein. An embodiment of the present application can also be a program code to be executed on a digital signal processor (DSP), a microprocessor, or a field programmable gate array (FPGA) to perform the processing described herein. The present application can also involve the use of a computer processor, a digital signal processor, a microprocessor, or a field programmable gate array (FPGA) to execute a program code written to implement the processing described herein. The program code can be stored on a computer readable medium, such as a hard disk, a CD ROM, a RAM, a floppy, a USB drive, a smart card, or an optical disc, to name a few. The computer readable medium can be portable, installed in a computer or stationary, or be integral to the computer.

[0201] The present application can be carried out in other specific ways than those herein set forth without departing from the spirit and essential characteristics of the application. The embodiments described herein are illustrative rather than limiting. Thus, the scope of the present application should be determined by the appended claims and equivalents thereof rather than by the description of the embodiments. All changes that come within the meaning and range of equivalency of the claims are to be embraced within their scope.

Claims

1. A method of video coding, the method comprising: receiving input data related to a current block, wherein the input data comprises pixel data to be encoded at an encoder side or data related to the current block to be decoded at a decoder side; determining a target matrix-based intra prediction (MIP) mode using a MIP mode setting, wherein the MIP mode setting contains a hybrid setting, a transpose setting, a representative mode setting, an inheritance setting, or a combination thereof, wherein the MIP mode setting with the hybrid setting comprises a hybrid matrix candidate formed by combining two or more prediction hypotheses, wherein the MIP mode setting with the transpose setting comprises a transposed matrix candidate generated by transposing one or more matrix coefficients or one or more reference samples, wherein the MIP mode setting with the representative mode setting comprises a representative mode generated according to the representative mode setting, or wherein the MIP mode setting with the inheritance setting comprises an inherited MIP candidate; and encoding or decoding the current block using the target MIP mode.

2. The method of claim 1, wherein the target MIP mode is used as an additional mode or an alternative mode.

3. The method of claim 1, wherein the one or more prediction hypotheses are generated according to one or more matrix information when the hybrid setting is used.

4. The method of claim 3, wherein the combination weighting of the hybrid matrix candidate is block-based weighting.

5. The method of claim 3, wherein the combination weighting of the hybrid matrix candidate is sample-based or position-based weighting.

6. The method of claim 3, wherein the one or more matrix information is obtained by selecting from a set of candidate matrix information using a recommendation scheme, and wherein the set of candidate matrix information comprises all or any subset of inherited matrix information.

7. The method of claim 1, wherein the one or more matrix coefficients or the one or more reference samples are transposed into one or more additional candidate matrices of the current block.

8. The method of claim 1, wherein the representative mode is generated according to DIMD (decoder-side intra mode derivation) or TIMD (template-based intra mode derivation).

9. The method of claim 1, wherein the representative mode is generated according to a block width, a block height, a block area, or a combination thereof.

10. The method of claim 1, wherein the representative mode is used to select a transform set of a separable or non-separable primary transform or secondary transform of the current block.

11. The method of claim 1, wherein the representative mode is used by one or more subsequent blocks following the current block.

12. The method of claim 1, wherein the representative mode is used as a chroma DM (direct mode) for subsequent chroma blocks.

13. The method of claim 1, wherein the representative mode is used as an MPM (most probable mode) for one or more subsequent intra or inter blocks.

14. The method of claim 1, wherein matrix information is recorded when the inheritance setting is used.

15. The method of claim 14, wherein the matrix information comprises a matrix size, a matrix mode index, a transpose information, a target information for the target MIP mode to determine one or more predictors, or a combination thereof.

16. The method of claim 14, wherein an inheritance list is established for the current block.

17. The method of claim 16, wherein an inheritance index is signaled, parsed, or inferred to indicate target matrix information in the inheritance list.

18. The method of claim 17, wherein the target matrix information indicated by the inheritance index is used for the current block to generate a target MIP predictor.

19. The method of claim 18, wherein down-sampling or up-sampling settings are applied if one or more matrix sizes of the target matrix information do not belong to a block size category of the current block.

20. A video coding apparatus comprising one or more electronic devices or processors configured to: receive input data related to a current block, wherein the input data comprises pixel data to be encoded at an encoder side or data related to the current block to be decoded at a decoder side; determine a target matrix-based intra prediction (MIP) mode using MIP mode settings, wherein the MIP mode settings include a hybrid setting, a transpose setting, a representative mode setting, an inheritance setting, or a combination thereof, wherein the MIP mode setting with the hybrid setting includes a hybrid matrix candidate formed by combining two or more prediction hypotheses, wherein the MIP mode setting with the transpose setting includes a transpose matrix candidate generated by transposing one or more matrix coefficients or one or more reference samples, wherein the MIP mode setting with the representative mode setting includes a representative mode generated according to the representative mode setting, or wherein the MIP mode setting with the inheritance setting includes an inherited MIP candidate; and encode or decode the current block using the target MIP mode.

Images