Video coding and decoding method and device for improving cross-component prediction model propagation

Abstract

A method and apparatus for coding and decoding a color picture or video using a propagation cross-component prediction model are disclosed. According to the method, a current block is encoded or decoded using a non-cross component prediction (CCP) mode. One or more reference blocks pointed by one or more motion vectors or one or more block vectors of the current block are determined. If the one or more reference blocks have target cross component model (CCM) information, the target CCM information is copied and stored at the current block, where the target CCM information is stored in the current block and accessed by one or more subsequent blocks for CCP processing.

Description

[0001] Cross-references This invention is a non-provisional application and claims priority to U.S. Provisional Patent Application No. 63 / 579,532, filed August 30, 2023, and U.S. Provisional Patent Application No. 63 / 589,354, filed October 11, 2023. The entire contents of these U.S. Provisional Patent Applications are incorporated herein by reference. Technical Field

[0002] This invention relates to a video encoding and decoding system that uses encoding and decoding tools to include one or more modes associated with a cross-component model. In particular, this invention relates to chroma component encoding and decoding using a cross-component model for propagation.

[0003] Prior Technology Versatile Video Coding (VVC) is the latest international video coding standard developed by the Joint Video Experts Group (JVET) of the ITU-T Video Coding Experts Group (VCEG) and the ISO / IEC Moving Picture Experts Group (MPEG). This standard has been published as an ISO standard: ISO / IEC 23090-3:2021, Information technology – Coding representation of immersive media – Part 3: Versatile Video Coding, published in February 2021. VVC was developed based on its predecessor, High Efficiency Video Coding (HEVC), by adding more coding and decoding tools to improve coding and decoding efficiency and handle various types of video sources, including three-dimensional (3D) video signals.

[0004] FIG. 1AAn exemplary adaptive intra / inter video coding system incorporating in-loop processing is illustrated. For intra prediction 110, the prediction data is derived based on previously coded video data in the current picture. For inter prediction 112, motion estimation (ME) is performed at the encoder side and motion compensation (MC) is performed based on the results of ME to provide prediction data from other pictures and motion data. A switch 114 selects either intra prediction 110 or inter prediction 112 and provides the selected prediction data to a summer 116 to form prediction error, also known as residual. The prediction error is then processed by transform (T) 118 followed by quantization (Q) 120. The transformed and quantized residual is encoded by an entropy encoder 122 and included in a video bitstream corresponding to the compressed video data. The bitstream associated with the transform coefficients is then packaged with side information, such as motion and coding mode related to intra prediction 110 and inter prediction 112, and parameters related to in-loop filters applied to regions of the base picture. The side information related to intra prediction 110, inter prediction 112, and in-loop filters 130 is provided to the entropy encoder 122 as shown. FIG. 1A When inter prediction mode is used, the reference pictures must also be reconstructed at the encoder side. Therefore, the transformed and quantized residual is processed by inverse quantization (IQ) 124 and inverse transform (IT) 126 to recover the residual. The residual is then added to the prediction data 136 at reconstruction (REC) 128 to reconstruct the video data. The reconstructed video data can be stored in a reference picture buffer 134 and used for prediction of other frames.

[0005] As shown in FIG. 1A The incoming video data is processed in the encoding system through a series of processes. The reconstructed video data from REC 128 can be subject to various impairments due to the series of processes. Therefore, in-loop filters 130 are typically applied to the reconstructed video data before it is stored to the reference picture buffer 134 to improve the video quality. For example, a deblocking filter (DF), a sample adaptive offset (SAO), and an adaptive loop filter (ALF) can be used. The in-loop filter information can need to be included in the bitstream so that the decoder can properly recover the needed information. Therefore, the in-loop filter information is also provided to the entropy encoder 122 to be included in the bitstream. In FIG. 1AIn the process, the loop filter 130 is applied to the reconstructed video before the reconstructed samples are stored in the reference image buffer 134. FIG. 1A The system described herein is intended to illustrate an exemplary architecture of a typical video encoder. It may correspond to a High Efficiency Video Codec (HEVC) system, VP8, VP9, ​​H.264, or VVC.

[0006] like FIG. 1B As shown, the decoder can use functional modules similar to or partially identical to the encoder, except for transform 118 and quantization 120, since the decoder only needs inverse quantization 124 and inverse transform 126. The decoder uses entropy decoder 140 to decode the video bitstream into quantized transform coefficients and the required encoding / decoding information (e.g., ILPF information, intra-frame prediction information, and inter-frame prediction information). Intra-frame prediction 150 at the decoder end does not require mode search. Instead, the decoder only needs to generate intra-frame predictions based on the intra-frame prediction information received from entropy decoder 140. Furthermore, for inter-frame prediction, the decoder only needs to perform motion compensation (MC 152) based on the inter-frame prediction information received from entropy decoder 140, without needing to perform motion estimation.

[0007] Cross-component linear model (CCLM) prediction To reduce cross-component redundancy, a Cross-Component Linear Model (CCLM) prediction mode is used in VVC. In this mode, chroma samples are predicted using a linear model based on reconstructed luminance samples from the same CU, as shown below: (1) in This represents the predicted chromaticity samples in the CU. This represents the downsampled reconstructed luminance samples from the same CU.

[0008] CCLM parameters ( and The value is derived from at most four neighboring chroma samples and their corresponding downsampled luminance samples. Assuming the current chroma block size is W×H, then W' and H' are set as follows: When W' = W, H' = H is applied in LM_LA mode; W' = W + H when applying LM_A mode; H' = H + W when applying LM_L mode.

[0009] In this disclosure, the terms {LM_LA, LM_A, LM_L} and {CCLM_LT, CCLM_T, CCLM_L} are used interchangeably.

[0010] Multiple Model CCLM (MMLM) In JEM (J. Chen, E. Alshina, G. J. Sullivan, J.-R. Ohm, and J. Boyce, Algorithm description of Joint Exploration Test Model 7, document JVET-G1001, ITU-T / ISO / IEC Joint Video Exploration Team (JVET), July 2017), a Multiple Model CCLM mode (MMLM) is proposed for predicting chroma samples from luma samples of the whole CU using two models. In MMLM, neighboring luma samples and neighboring chroma samples of the current block are classified into two groups, each of which is used as a training set to derive a linear model (i.e., a particular a and b are derived for a particular group). In addition, samples of the current luma block are also classified based on the same neighboring luma sample classification rule.

[0011] The threshold is calculated as the average of the neighboring reconstructed luma samples. The neighboring samples with Rec' L [x,y] < threshold are classified into the 1st group; while the neighboring samples with Rec' L [x,y] > threshold are classified into the 2nd group.

[0012] (2) Convolutional Cross Component Model (CCCM) In CCCM, a convolutional model is applied to improve the chroma prediction performance. The convolutional model has a 7-tap filter, which consists of a 5-tap plus a sign-shaped spatial component, a non-linear term, and a bias term.

[0013] The output of the filter is calculated as the convolution between the filter coefficients and the input values, and clipped to the range of valid chroma samples.

[0014] The filter coefficients are calculated by minimizing the MSE between the predicted and reconstructed chroma samples in the reference region.

[0015] Gradient Linear Model (GLM) Compared to CCLM, GLM uses luma sample gradients to derive the linear model, instead of down-sampled luma values. Specifically, when GLM is applied, the input of the CCLM process, i.e., the down-sampled luma samples , are replaced by the luma sample gradients . Other parts of CCLM (e.g., parameter derivation, prediction sample linear transformation) remain unchanged: .

[0016] FIG. 2 The 16 gradient filters (210-240) used for gradient calculation are shown.

[0017] Intra-block copying Intra-Block Copy (IBC) is a tool employed by HEVC extensions in Screen Content Coding (SCC). It is well-known for significantly improving the coding and decoding efficiency of screen content material. Since IBC mode is implemented as a block-level coding mode, block matching (BM) is performed in the encoder to find the optimal block vector (or motion vector) for each CU. Here, the block vector indicates the displacement from the current block to a reference block that has been reconstructed within the current frame. The luma block vector of an IBC-encoded CU is integer precision. The chroma block vector is also rounded to integer precision. When combined with AMVR, IBC mode can switch between 1-pel and 4-pel motion vector precision. IBC-encoded CUs are considered a third prediction mode in addition to intra or inter-frame prediction modes. IBC mode is suitable for CUs with a width and height of 64 luma samples or less.

[0018] CCCM using unsubsampled brightness samples A CCCM mode using a 3x2 filter with unsampled luminance samples is employed. This mode consists of a 6-tap spatial term, four nonlinear terms, and a bias term. The 6-tap spatial term corresponds to the six neighboring luminance samples (L0, L1, …, L5) surrounding the chrominance sample (C) to be predicted. The four nonlinear terms are derived from samples L0, L1, L2, and L3, as shown below, where the positions of the unsampled luminance samples are indicated. FIG. 3 middle.

[0019] Cross-component residual model (CCRM) As described in JVET-AD0108 (Pekka Astola et al., “AHG12: Cross-component residual model (CCRM) for inter-frame prediction”, Joint Video Experts Group (JVET) of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29, 30th meeting, Antalya, Turkey, 21-28 April 2023, document: JVET-AD0108), when blocks use inter-frame prediction or intra-block copying (IBC), a cross-component residual model (CCRM) is applied to predict chroma samples from reconstructed luminance samples. FIG. 4The decoder side of the method is illustrated. The cross-component filter is derived using the luma and chroma prediction signals. The derived filter is applied to the reconstructed luma signal to generate the final chroma prediction. In step 420, filter coefficients are derived for each chroma component using the prediction signals (i.e. predY 410 and predCb 412 or predCr 414) respectively, and the filter is applied to the reconstructed luma signal in step 430 as FIG. 4

[0020] Chroma DM mode For the chroma DM mode, the intra prediction mode of the corresponding (co-located) luma block covering the center position of the current chroma block is directly inherited.

[0021] Decoder-side intra mode derivation (DIMD) To implicitly derive the intra prediction mode of a block, a texture gradient analysis is performed at both the encoder and the decoder side. This process starts with an empty histogram of gradients (HoG) having 65 entries, corresponding to the 65 angular modes. The magnitude of these entries is determined during the texture gradient analysis.

[0022] Intra template matching Intra template matching prediction (IntraTMP) is a special intra prediction mode that copies the best prediction block of an L-shaped template matching the current template from the reconstructed part of the current frame. For a predetermined search range, the encoder searches for the most similar template to the current template in the reconstructed part of the current frame and uses the corresponding block as the prediction block. Then, the encoder signals the use of this mode and performs the same prediction operation at the decoder side.

[0023] Extended merge prediction In VVC, the merge candidate list is constructed by including the following five types of candidates in order: Spatial MVP from spatial neighbor CUs Temporal MVP from co-located CU History-based MVP from FIFO table Paired average MVP ​Zero MV.

[0024] Spatial candidate derivation In VVC, the derivation of spatial merge candidates is the same as in HEVC, except that the positions of the first two merge candidates are swapped. For the current CU 510, the first two merge candidates are derived from the positions depicted in FIG. 5 . Up to four merge candidates (B 0, A 0, B1and A1). The order of derivation is B 0, A 0, B 1, A1and B2. Position B2 is only considered if one or more neighboring CUs of positions B0, A0, B1, A1are not available (e.g., belong to another slice or tile) or are intra coded. After adding the candidate of position A0, the addition of the remaining candidates is subject to a redundancy check to ensure that candidates with the same motion information are excluded from the list, thereby improving coding efficiency.

[0025] In addition to the spatial candidates described above, the non-adjacent spatial merge candidate in JVET-L0399 (Yu Han et al., “CE4.4.6: Improvement on Merge / Skip mode”, Joint Video Exploration Team (JVET) of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11, 12th Meeting: Macao, CN, 3-12 Oct 2018, Doc: JVET-L0399) is inserted in the regular merge candidate list after TMVP. An example of the mode of the spatial merge candidate is shown in FIG. 6 . The distance between the non-adjacent spatial candidate and the current coding block is based on the width and height of the current coding block. The line buffer restriction does not apply.

[0026] Temporal candidate derivation In this step, only one candidate is added to the list. In particular, when deriving this temporal merge candidate for the current CU 710, a collocated CU 720 based on a collocated reference picture is used to derive a scaled motion vector as shown in FIG. 7 . The reference picture list and the reference index used to derive the collocated CU are explicitly signaled in the slice header. The scaled motion vector 730 of the temporal merge candidate is scaled from the motion vector 740 of the collocated CU according to the POC (Picture Order Count) distances tband tdas shown by the dashed line in FIG. 7 , where tbis defined as the POC difference between the reference picture of the current picture and the current picture, and tdas the POC difference between the reference picture of the collocated picture and the collocated picture. The reference picture index of the temporal merge candidate is set to zero.

[0027] The position of the temporal candidate is selected between the candidates C0 and C1 as shown in FIG. 8 If the position C0 is not available, is intra coded, or outside the current CTU row, the position C1 is used. Otherwise, the position C0 is used when deriving the temporal merge candidate.

[0028] History-based merge candidate derivation The history-based MVP (HMVP) merge candidate is added to the merge list after the spatial MVP and TMVP. In this method, the motion information of previously coded blocks is stored in a table and used as the MVP for the current CU. A table with multiple HMVP candidates is maintained during the encoding / decoding process. When a new CTU row is encountered, the table is reset (emptied). Whenever there is a CU that is not inter-subblock coded, the relevant motion information is added to the last entry of the table as a new HMVP candidate.

[0029] Pairwise average merge candidate derivation The pairwise average candidate is generated by averaging a predefined pair of candidates in the existing merge candidate list, using the first two merge candidates. The first merge candidate is defined as p0Cand and the second merge candidate is defined as p1Cand. Depending on the availability of the motion vectors of p0Cand and p1Cand, an average motion vector is calculated for each reference list, respectively. If both motion vectors are available in one list, even if they point to different reference pictures, the two motion vectors are averaged with the reference picture set to that of p0Cand; if only one motion vector is available, it is used directly; if no motion vector is available, the list is kept invalid. In addition, if the half-pel interpolation filter indices of p0Cand and p1Cand are different, it is set to 0.

[0030] When the merge list is not full after adding the pairwise average merge candidate, a zero MVP is inserted at the end until the maximum number of merge candidates is reached.

[0031] Local boost cross component prediction (LB-CCP) The prediction samples of MM-CCLM / MM-CCCM can be filtered with neighboring samples. As shown in FIG. 9 a 3x3 low-pass filter is applied to filter the prediction samples generated by MM-CCLM / MM-CCCM. For samples located at the top / left boundary, the filtering window can involve neighboring reconstructed samples. For internal samples, the filtering window only involves prediction samples, which can be padded. A flag is used to indicate whether to apply the filtering for a block coded with MM-CCLM / MM-CCCM.

[0032] To improve the coding performance of a system using a cross-component model, methods and apparatuses using a propagated cross-component model are disclosed. SUMMARY

[0033] A method and apparatus for coding a color picture or video using a coding tool including one or more modes related to a cross-component model. According to the method, input data related to a current block is received, the current block including a first color block and a second color block, wherein the input data includes pixel data to be encoded at an encoder end or data related to the current block to be decoded at a decoder end, and the current block is coded in a non-intra mode. The current block is coded or decoded using a non-cross component prediction (CCP) mode. One or more reference blocks pointed to by one or more motion vectors or one or more block vectors of the current block are determined. If the one or more reference blocks have target cross-component model (CCM) information, the target CCM information is copied and stored in the current block, wherein the target CCM information is stored in the current block and accessed by one or more subsequent blocks for CCP processing.

[0034] In one embodiment, the one or more block vectors are related to the current first color block or the collocated first color block if the current first color block or the collocated first color block is coded in an intra block copy (IBC) or intra template matching prediction (IntraTMP) mode. In one embodiment, if the one or more reference blocks located according to the one or more block vectors or the one or more motion vectors are not coded with CCP but have first CCM information stored for the one or more reference blocks, the stored first CCM information is used as the target CCM information.

[0035] In one embodiment, if the one or more reference blocks correspond to a plurality of reference blocks and only one of the plurality of reference blocks has first CCM information, the first CCM information is used as the target CCM information. In one embodiment, if a plurality of reference blocks have first CCM information, the target CCM information is derived by combining corresponding CCP models related to all or a portion of the plurality of reference blocks.

[0036] In one embodiment, if multiple reference blocks have first CCM information, the target CCM information is copied from the first CCM information of a target reference block of the multiple reference blocks according to one or more predefined rules. In one embodiment, a reference block of a CCP codec is selected as the target reference block of the multiple reference blocks. In one embodiment, a reference block of an intra- or inter- inter- codec is selected as the target reference block of the multiple reference blocks. In one embodiment, a reference block with the shortest distance to the current block is selected as the target reference block of the multiple reference blocks according to a distance measurement. In one embodiment, the distance measurement corresponds to a Euclidean distance, a Manhattan distance, a Minkowski distance, a horizontal distance, or a vertical distance.

[0037] In one embodiment, if multiple reference blocks have first CCM information, the first CCM information of the multiple reference blocks is applied to the reconstructed current first color block to generate multiple predictions of the current second color block. In one embodiment, a distortion between the reconstructed current second color block and the multiple predictions is determined respectively, and target first CCM information related to a target prediction with the smallest distortion is selected to be copied to the current block.

[0038] In one embodiment, if the current block is not CCP encoded and a current slice or picture containing the current block is a non-intra slice or picture, the target CCM information is copied from a collocated block of the current block in a collocated picture. In one embodiment, if a picture order count (POC) distance between the collocated picture and a current picture containing the current block is greater than a predetermined threshold, the target CCM information is not copied from the collocated block of the current block in the collocated picture.

[0039] Figure illustrates a simple example FIG. 1A An exemplary adaptive intra / inter video coding system including a loop process is illustrated.

[0040] FIG. 1B An exemplary adaptive intra / inter video coding system including a loop process is illustrated. FIG. 1A A corresponding decoder of the encoder.

[0041] FIG. 2 Six gradient modes of a GLM are shown.

[0042] FIG. 3 A 6-tap spatial term corresponding to 6 neighboring luma samples (i.e., L0, L1, …, L5) around a chroma sample (i.e., C) to be predicted in a CCCM mode is illustrated.

[0043] FIG. 4 An exemplary system block diagram of a cross-component residual model (CCRM) is shown.

[0044] FIG. 5Five neighboring blocks used to derive VVC spatial merge candidates are illustrated.

[0045] FIG. 6 An exemplary pattern of one neighboring and non-neighboring spatial merge candidate is illustrated.

[0046] FIG. 7 An example of temporal candidate derivation is illustrated, where scaled motion vectors are derived according to picture order count (POC) distance.

[0047] FIG. 8 The position of the temporal candidate selected between candidates C0 and C1 is illustrated.

[0048] FIG. 9 Filters on MM-CCLM / MM-CCCM samples are illustrated.

[0049] FIG. 10A (Mode 1) and FIG. 10B (Mode 2) illustrate two different non-neighboring spatial neighboring candidate modes according to predefined positions and predefined orders.

[0050] FIG. 11 An example of CCM information propagation is illustrated, where the dashed blocks (i.e., A, E, G) are coded in cross-component models (e.g., CCLM, MMLM, GLM, CCCM, Chroma Fusion).

[0051] FIG. 12 Another example of CCM information propagation is illustrated.

[0052] FIG. 13 Another example of CCM information propagation is illustrated, where the current block C has a block vector available, and the reference block B has a motion vector available. The CCM information of block B is copied from block A.

[0053] FIG. 14A-FIG. 14B An n-tap pattern in an M x N window region around / containing the position (iL, jL) is illustrated to derive sourceTermSet0(i, j), where only the center (iL, jL) and the 5x5 cross (iL-2, jL-2) are used. FIG. 14A FIG. 14B ).

[0054] FIG. 15 An example of using Sobel filters to derive gradient information from prediction samples and / or reconstructed samples is illustrated.

[0055] FIG. 16A-FIG. 16B An m-tap pattern in an M2 x N2 window region around / containing the position (iC, jC) is illustrated to derive sourceTermSet1(i, j), where only the center (iC, jC) and the 5x5 cross (iC-2, jC-2) are used.​FIG. 16A ) and using 5x5 cross ( FIG. 16B ).

[0056] FIG. 17 An example of a weighted set reference region of a neighboring spatial region used as a self-derived cross-component model is illustrated.

[0057] FIG. 18 A flowchart of an exemplary video coding system including a cross-component prediction model propagated according to embodiments of the application is illustrated.

[0058] Embodiments It will be readily understood that the components of the present application, as generally described and illustrated in the Figures herein, can be arranged and designed in a wide variety of different configurations. Thus, the following more detailed description of the embodiments of the system and method of the application, as represented in the Figures, is not intended to limit the scope of the application, but is merely representative of selected embodiments of the application. The description of the embodiments of the application as set forth herein is not intended to be exhaustive or to be necessarily limited to any one embodiment of the application. The detailed description includes specific details for the purpose of providing an understanding of the embodiments of the application. It will be apparent to those skilled in the art that embodiments of the application can be practiced without these specific details. In some instances, well-known structures and components are not described in detail in order to avoid obscuring aspects of the application. The embodiments of the application will best be understood from the description and drawings and by practicing the application in accordance with the following claims.

[0059] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. One skilled in the art will understand that the application can be practiced without one or more of the specific details set forth herein, or with other methods, components, etc. In other instances, well-known structures or operations are not shown or described in detail in order to avoid obscuring aspects of the application. The embodiments of the application will best be understood from the description and drawings and by practicing the application in accordance with the following claims. The following description is provided solely to assist in understanding certain embodiments of the application. It is not intended to limit the broader aspects of the application.

[0060] To improve the coding performance of cross-component prediction, various schemes are disclosed.

[0061] Cross-component information is used to improve the prediction accuracy of intra blocks. To improve the prediction accuracy of the chroma components of an intra block, the luminance information from the corresponding luminance component of the current chroma block is used, and / or the chroma information from the current chroma block, and / or the chroma information from previously encoded chroma components.

[0062] A first scheme is to improve the prediction of Cb and / or Cr by applying a cross-component model to information (current reconstructed or predicted) from Y for a coding unit (under single tree partitioning) containing luminance (Y) and chroma (Cb and / or Cr) components.

[0063] The second approach is to improve the prediction of Cr by applying a cross-component model to information (current reconstructed or predicted) from Cb for a coding unit containing luma (Y) and chroma (Cb and / or Cr) components (under single tree partitioning) or a coding unit containing chroma (Cb and / or Cr) components (under chroma double tree partitioning).

[0064] In the following, several embodiments related to the first approach are proposed, using the inherited cross-component mode to perform the following steps for the current chroma block: Step (1) establish a candidate list (modelList) for the current block, which includes the cross-component model; Step (2) select one or more model information sets in the list; Step (3) generate multiple prediction hypotheses for the current chroma component (Cb or Cr) by applying the selected model information to the reconstructed or predicted samples of the corresponding luma component, using the model information (similar to the intra chroma cross-component mode).

[0065] When the selected model information refers to the traditional cross-component linear model, the proposed method is called the inter cross-component linear model (inter CCLM) mode. When the selected model information refers to the convolutional cross-component model derived by a regression-based method (e.g., CCCM), the proposed method is called the inter cross-component convolutional model (inter CCCM) mode.

[0066] In addition, in some embodiments, a self-derived (or re-derived) cross-component mode is proposed, which can be added to the candidate list in Step (1). In some embodiments, the self-derived cross-component mode is not added to the list, and a selection using the proposed inherited mode and / or using the proposed self-derived mode is designed. In some embodiments, the selection using the proposed inherited mode and / or using the proposed self-derived mode is determined according to explicit rules, implicit rules, or both. More details are described in the section titled “IV. Selection using the proposed inherited mode and / or using the proposed self-derived mode”.

[0067] In one embodiment, the proposed embodiments can also be used for the second approach by using the previously encoded chroma component (Cb) as the luma component in the first approach.

[0068] Storage and inheritance of the model In another embodiment, when the current intra block uses the model parameters of a self-derived cross component mode, the used model parameters can be saved and / or referenced by the subsequent coded block. For example, for a self-derived cross component that is CCRM, all or any subset of the model parameters can be saved. In one embodiment, if the subsequent coded block is intra, then the saved model parameters are allowed to be used. If the subsequent coded block is inter or any mode type (e.g., IBC), then the saved model parameters are allowed to be used. In another embodiment, if the subsequent coded block has a different mode type (e.g., not an intra block), then the saved model parameters are not allowed to be used.

[0069] In another embodiment, when the current intra block uses the model parameters of a self-derived cross component mode, the used model parameters can be saved and / or referenced by the subsequent coded block. For example, for a self-derived cross component that is CCRM, all or any subset of the model parameters can be saved. In one embodiment, if the subsequent coded block is intra, then the saved model parameters are allowed to be used. If the subsequent coded block is inter or any mode type (e.g., IBC), then the saved model parameters are allowed to be used. In another embodiment, if the subsequent coded block has a different mode type (e.g., not an intra block), then the saved model parameters are not allowed to be used.

[0070] In another embodiment, when the current intra block uses the model parameters of a self-derived cross component mode, the used model parameters can be saved and / or referenced by the subsequent coded block. For example, for a self-derived cross component that is CCRM, all or any subset of the model parameters can be saved. In one embodiment, if the subsequent coded block is intra, then the saved model parameters are allowed to be used. If the subsequent coded block is inter or any mode type (e.g., IBC), then the saved model parameters are allowed to be used. In another embodiment, if the subsequent coded block has a different mode type (e.g., not an intra block), then the saved model parameters are not allowed to be used.

[0071] Establishing a candidate list containing cross component models In one embodiment, when a merge-like candidate model list (modelList) is established, one or more of the following candidate model information sets are included. For each candidate in the list, it refers to a candidate model information. The definition of the model information can be found in the section titled "V.1. Inherited CCM information."

[0072] - Spatial model information from spatial neighboring blocks (corresponding to inter "spatial MVP from spatial neighboring CUs") - Temporal model information from collocated blocks (corresponding to inter "temporal MVP from collocated CUs") - History-based model information from FIFO table (corresponding to inter "history-based MVP from FIFO table") - Pairwise average model information (corresponding to inter "pairwise average MVP") - Default model information (corresponding to inter "zero MV") In one sub-embodiment of the above candidate type list, where the candidate type is "spatial model information from spatial neighboring blocks", the valid spatial neighboring blocks can come from spatial neighboring and non-neighboring neighbors (or any subset in the neighboring search region of the current block) that satisfy a predetermined condition. For the example of non-neighboring neighbors, the predetermined condition (e.g., validity / check availability) refers to that the non-neighboring neighbors are within the non-neighboring spatial candidate region. For example, the predetermined condition is that the neighbors are coded by cross-component modes or in conjunction with cross-component modes. Cross-component modes refer to CCLM, MMLM, CCCM, GLM, modes that inherit mode information from similar merge candidate list, MH CCLM, and / or any cross-component modes that belong to the cross-component branch (containing many cross-component modes) and do not belong to the traditional intra prediction modes. Cross-component modes in conjunction refer to chroma fusion (or called LM-assisted angular / plane mode), inter CCLM, inter CCCM, and / or any traditional modes whose syntax do not belong to the cross-component branch but use cross-component information to generate the prediction. In another sub-embodiment, when checking the validity of the neighboring coded blocks, when the mentioned validity check (e.g., the neighboring block is not cross-component mode or the neighboring block does not use / conjunction with cross-component mode), a second round of validity check is further used, the motion vectors and / or block vectors of the neighboring blocks can be used to find the cross-component model. The variation of how to use the motion vectors and / or block vectors to find the model can refer to the description of "temporal model information from collocated blocks" in the above candidate type list. If the model is found, the second round of validity check of the neighboring block is satisfied and the found model can be inserted into the list; otherwise, the neighboring block is not suitable to be inserted. When scanning the spatial neighboring blocks, if the candidate is valid, it is added to the list.

[0073] In another sub-embodiment, the candidate type is "temporal model information from collocated block", in the first case, the collocated block is from a block in the reference picture or a predefined collocated picture, in inter mode by using the current block position and / or the current block motion; in the second case, the collocated block is from a block in the reference picture or a predefined collocated picture, in inter mode by using the current block position and / or the neighboring block motion. In the first case, for example, when the current block is coded in inter prediction mode, the collocated block is referred by the motion information of the current block (containing the motion vector and the reference index indicated reference picture). If the current block is sub-block motion mode (e.g. affine mode), each sub-block in the current block has its own collocated temporal model information. All or any subset of the collocated temporal model information referred by the collocated temporal model information of different sub-block motion (each sub-block) is added to the list. Another example is when the reference index indicated reference picture is different from the predefined collocated picture, the collocated picture can be the collocated picture used for temporal motion vector prediction in inter mode or any collocated picture specified in any standard to keep the motion or cross component model information stored and available for the current block, the temporal information from the reference picture is prohibited to be used. Another example is when the reference index indicated reference picture is different from the predefined collocated picture, the collocated picture can be the collocated picture used for temporal motion vector prediction in inter mode or any collocated picture specified in any standard to keep the motion or cross component model information stored and available for the current block, the motion vector is scaled to refer to the predefined collocated picture and the scaled motion vector is used to find the collocated block in the collocated picture to obtain the cross component model in the collocated block. The scaling process is shown in the "Inherited temporal neighboring model parameters" section and the "Temporal candidate derivation" section. Some examples of the second case are described as follows. For example, the temporal model information can come from the collocated block referred by the motion information of the neighboring block. Similar to the first case, the prohibition method or the scaling method can be used in the second case. If the proposed method is applied to IBC block or any mode using block vector (in the first case, the current block is IBC; in the second case, the neighboring block is IBC), the block vector information is used as the motion vector, where the block vector information is determined by signal and / or template matching in a predefined search range, such as IntraTMP and / or any implicit or explicit predefined rule. More details can be found in the "Inherited temporal neighboring model parameters" section.

[0074] In another sub-embodiment, the candidate type is "history-based model information", a history-based table (FIFO table) is established and stores the model information from the previously coded blocks. The table can be reset at the beginning and / or end of CTU, slice, picture, tile and / or sequence. One or more history-based candidates can be added to the candidate list in the order from the head to the tail or the tail to the head of the table.

[0075] In another sub-embodiment, the candidate type is "pairwise average model information", the model information of the candidate is derived based on the model information of more than one previous candidate in the list. For example, it can average and / or modify the model parameters of more than one candidate as the model parameters to be applied. Another example is that it can combine more than one prediction as the final prediction, where each prediction is generated by applying one model in the candidate list.

[0076] In another sub-embodiment, if the list is not full after inserting all predefined candidates, a default model information is added. For example, the default model can be a CCLM model. The default alpha (or referred to as a or scaling parameter) is selected from {0, 1 / 8, -1 / 8, 2 / 8, -2 / 8, 3 / 8, -3 / 8,...}, the default beta (or referred to as b or offset parameter) is based on the selected default alpha, the average of the neighboring reconstructed luma sample values and the average of the neighboring reconstructed chroma (Cb / Cr) sample values. In another sub-embodiment, the default alpha (or referred to as a or scaling parameter) is selected from {0, 1 / 8, -1 / 8, 2 / 8, -2 / 8, 3 / 8, -3 / 8,...}, the default beta (or referred to as b or offset parameter) is based on the selected default alpha, the average of the neighboring reconstructed luma sample values and the average of the neighboring reconstructed chroma (Cb / Cr) sample values.

[0077] In another sub-embodiment, the candidate list for inter chroma blocks is unified with the candidate list for intra chroma blocks and / or can be generated based on the candidate list for intra chroma blocks, further containing inter specific candidates (e.g., temporal model information referred by the current motion) and / or can be any subset of the intra chroma block candidate list.

[0078] In another embodiment, when constructing the modelList, one or more self-derived cross-component candidates are included. The self-derived cross-component candidates are described in the section "Self-derived cross-component model". In another sub-embodiment, the self-derived cross-component candidates are added only when the list does not contain enough inherited candidates. For example, the self-derived candidates are added before the default candidates or are considered as default candidates. In another sub-embodiment, the self-derived cross-component candidates are added to any predefined position in the modelList. For example, the position is after the spatial neighboring candidates. Another example is that the position is after the spatial non-neighboring candidates. Another example is that the position is after all or any subset of the temporal candidates.

[0079] After constructing the list, in one embodiment, the list is reordered according to the methods defined in the section "Reordering the candidates in the list".

[0080] II. Signaling of enabling or disabling and selection of one or more model information in the list (if enabled) In this section, the term "inter CCLM" refers to "inter CCLM or inter CCCM". ​

[0081] When the proposed inter CCLM (or inter CCCM) is not applied, the prediction of the current block comes from the original inter prediction.

[0082] In another embodiment, the selection of whether to apply inter CCLM or not depends on a signalization.

[0083] In one sub-embodiment, the signal refers to an encoded TU / TB / CU / CB level flag. The flag can or can not be coded with context. Take the TU / TB flag as an example, the flag is only sent when the luma Cbf of the TU / TB is non-zero and the enabling flag is true. Take the CU / CB flag as an example, the flag is only sent when the luma Cbf of the CU / CB is non-zero and the enabling flag is true. For all inter modes, when the proposed inter CCLM (or inter CCCM) is supported, the enabling flag of the inter mode means that the predMode of the CU is MODE INTER. When the proposed inter CCLM (or inter CCCM) is supported for IBC, the enabling flag of IBC is checked first, and when the predMode of the CU is MODE IBC, the signal of inter CCLM (or inter CCCM) is coded / decoded. When the proposed inter CCLM (or inter CCCM) is supported for CIIP only, the enabling flag of CIIP is checked first, and when the CIIP flag is true, the signal of inter CCLM (or inter CCCM) is coded / decoded. When the proposed inter CCLM (or inter CCCM) is supported for merge only, the merge flag is checked first, and when the merge flag is true, the signal of inter CCLM (or inter CCCM) is coded / decoded. When the proposed inter CCLM (or inter CCCM) is supported for AMVP only, the merge flag is checked first, and when the merge flag is false, the signal of inter CCLM (or inter CCCM) is coded / decoded. The proposed inter CCLM (or inter CCCM) can be supported for any pre-defined subset of merge modes, any pre-defined subset of inter modes, or any pre-defined subset of non-intra modes only.

[0084] In another sub-embodiment, when the signal indicates to apply inter CCLM (or inter CCCM), an additional signal is used to select one or more models from the total candidates. The candidate index is referred to as modelldx in this disclosure. If the modelList contains the total candidates (e.g., the candidates described in the section titled "Constructing a candidate list containing cross-component models", CCLM LT, CCLM L, CCLM T, MMLM LT, MMLM L, MMLM T), or any subset of the candidates is reordered by the methods in the section "Reordering candidates in the list", the additional signal specifies the candidate index in the reordered list. For example, if one LM mode is selected, the LM prediction is generated by the selected one LM. For another example, if multiple LM modes are selected, the LM prediction is generated by the prediction hypothesis mixture of the multiple LM modes.

[0085] In another sub-embodiment, no additional signal is needed, and one or more models are selected according to an implicit rule. For example, the selected one or more models are implicitly determined, or the one or more models for the current block are determined without sending modelldx. For example, the first candidate in the list is used. If the list is reordered by template cost, the first candidate is the one with the minimum template cost.

[0086] In another embodiment, the original inter prediction (generated by motion compensation) is used for luma, and the prediction for chroma components is generated by CCLM and / or any other LM mode.

[0087] In one sub-embodiment, the current CU is treated as an inter CU, an intra CU, or a new type of prediction mode (i.e., neither intra nor inter).

[0088] In another embodiment, one or more LM modes (i.e., cross-component modes) for generating one or more prediction hypotheses for LM assisted angular / planar modes / inter CCLM / inter CCCM / MH CCLM are selected from a predefined merge candidate list (i.e., modelList). A modelldx is sent to select a candidate from the candidate list (modelList) and the selected candidate is used for the current block. The modelList contains one or more candidates, where each candidate refers to a model (or cross-component mode) information. If there is only one candidate in the list (i.e., the size of the list is 1), the modelldx is not sent and / or can be inferred to be 0 or a default value. In one embodiment, the modelldx or one or more models used for the current block is determined implicitly. For example, the first candidate in the list is used. If the list is reordered by template cost, the first candidate is the candidate with the smallest template cost. For another example, the candidate / model used is implicitly selected from the list according to a predefined rule, which depends on the coding information of the candidate to be used. This embodiment is referred to as "noteA".

[0089] In one embodiment, when constructing the modelList, one or more predefined candidates are added. The predefined candidates can include any subset / extensions of the following candidates and / or more candidates in the embodiments described in "noteA".

[0090] CCLM family: CCLM LT, CCLM L, CCLM T MMLM family: MMLM LT, MMLM L, MMLM T CCCM family: CCCM LT, CCCM L, CCCM T The proposed methods above can also be applied to IBC blocks or blocks with any IBC sub-modes (e.g., IBC merge or IBC AMVP or any IBC modes under IBC syntax). In this disclosure, the word "inter" can be changed to IBC. That is, for chroma components, the block vector prediction can be combined with or replaced by cross-component prediction.

[0091] III. Generating one or more prediction hypotheses for the current chroma component using model information III.1. Concept In one embodiment, one prediction hypothesis for the current chroma component is generated using a prediction or reconstruction based model.

[0092] In one sub-embodiment of the prediction-based linear model, the derived model parameters are applied to the predicted samples of the first component (Y) to obtain the predicted samples of the second or third component.

[0093] .

[0094] The predicted samples of the first component are down-sampled by a down-sampling filter, which can be fixed to a predefined filter or selected from a number of candidate filters.

[0095] In another sub-embodiment of the reconstruction-based linear model, the derived model parameters are applied to the reconstructed samples of the first component (Y) to obtain the predicted samples of the second or third component.

[0096] .

[0097] The reconstructed samples of the first component are down-sampled by a down-sampling filter, which can be fixed to a predefined filter or selected from a number of candidate filters.

[0098] The prediction- or reconstruction-based convolutional model is similar to the proposed prediction- or reconstruction-based linear model approach. The main difference is that the model coefficient pattern follows the Cross Component Model (CCCM) instead of the Cross Component Linear Model (CCLM), and the luma samples can or can not be down-sampled first.

[0099] In another embodiment, multiple hypotheses (MH) of cross-component prediction are mixed or multiple models are used to generate prediction hypotheses for the current block. A multiple hypotheses cross-component linear model (CCLM) is proposed to mix predictions from multiple CCLM approaches. The term “CCLM approach” can refer to all cross-component modes. The CCLM approaches to be mixed can come from (but not limited to) the CCLM approaches mentioned above (e.g., CCLM, MMLM, CCCM, GLM, CCRM, etc.) and / or the models defined in the embodiments described in note A. The mixing uses a weighting scheme. More variants of multiple hypotheses prediction from cross-component models are defined in the “Inherited multiple cross-component model” section.

[0100] III.2. CCLM for Inter-Frame Blocks The term “CCLM for inter-frame blocks” can also be referred to as “inter-frame CCLM”, and “CCLM” can be extended to (or replaced by) any linear model (LM) mode (or any cross-component mode). The CCLM for inter-frame blocks can also be referred to as inter-frame CCCM when a convolutional cross-component model is used that is derived based on a regression approach.

[0101] In one embodiment, for luma components, in addition to the original inter prediction (generated by motion compensation, can be single prediction and / or bi-prediction, multi-hypothesis prediction from multiple motion candidates, which can refer to one or more merge candidates, one or more AMVP candidates, any combination of the above, or just single prediction), one or more prediction hypotheses (generated by CCLM and / or any other LM mode) are used to generate the current prediction.

[0102] In one sub-embodiment, the current prediction is a weighted sum of the inter prediction and the CCLM prediction.

[0103] In another embodiment, the inter prediction can be generated by any of the above inter modes. For example, the inter mode can be regular merge mode. For another example, the inter mode can be CIIP mode. For another example, the inter mode can be GPM or any GPM variant (e.g., GPM Intra-Ref one uses an intra-predicted prediction unit).

[0104] In another embodiment, inter CCLM is supported only when one or more pre-defined inter modes are used, or when any one (or more) of the pre-defined inter mode’s enabling flag is indicated as enabled. Supporting inter CCLM means that the prediction of the current block can choose to apply inter CCLM or not.

[0105] When inter CCLM is applied, the prediction of the current block is generated by: In one sub-embodiment: blend one or more prediction hypotheses: (generated by CCLM and / or any other LM mode) with the original inter prediction Blend the existing inter mode’s chroma prediction and the prediction from LM Blend: Predfinal = ( wInter * PredInter + wLM * PredLM + 2 )>>2 Weighting rule: wInter and wLM, e.g., If top and left are both Intra (or any cross-component mode), (wInter, wLM) = (1, 3) Otherwise, if one of top and left is Intra, (wInter, wLM) = (2, 2) Otherwise, (wInter, wLM) = (3, 1) For another example, the weighting follows CIIP weighting rule.

[0106] For example, predInter = inter prediction after OBMC (if OBMC is used) For example, predInter = inter prediction before OBMC (OBMC can be applied after the blending) In another sub-embodiment: replace the original inter prediction with one or more prediction hypotheses (generated by CCLM and / or any other cross-component mode) For example, if CCLM mode is used to generate chroma prediction samples and the luma prediction comes from inter coding tool, use one flag to indicate whether the CCLM model used for chroma prediction is inherited from the CCLM model used in the previous coded block or from a predetermined CCLM mode. If the CCLM model is inherited from the CCLM model used in the previous coded block, use an index to indicate which model in the list is inherited or modified. Otherwise, use the predetermined CCLM mode to implicitly derive the CCLM model for the current chroma prediction.

[0107] IV. Selection of using the proposed inherited mode and / or using the proposed self-derived mode In one embodiment, it can be indicated / selected by a flag whether to use the re-derived model or not. If the flag is 0, the cross-component model used for encoding the neighboring merge candidate is inherited. If the flag is 1, the re-derived method is used.

[0108] In another embodiment, an implicit rule (without using an extra flag) is used to determine whether to use the re-derived model or not.

[0109] In another embodiment, if none of the models can be inherited during the construction of modelList, or the spatial neighboring / non-neighboring candidate, the history candidate, the temporal candidate or all or any subset (e.g., before the default candidate) mentioned in this invention are not available, the re-derived model is used.

[0110] In another embodiment, when the proposed inherited method is used, the candidate with the minimum cost (e.g., the first candidate in modelList) is implicitly selected to generate the cross-component prediction. As another example, one or more candidates in modelList are selected by an index. More details can be found in Section II.

[0111] V. Details of the cross-component mode (containing Model information) in the candidate list V.1. Inherited CCM information In one embodiment, the inherited cross-component model (CCM) information can be stored together with the inherited model parameters. The CCM information can be inherited together with the inherited model parameters. The prediction of the current block can be generated based on the inherited CCM information and the inherited model parameters. The CCM information can include, but is not limited to, the prediction mode (e.g., CCLM, MMLM, CCCM, 2- parameter GLM, 3-parameter GLM (GLM model with luminance term), model index to indicate which model shape is used in the convolution model, classification threshold of multi-model, information to indicate the use of non-subsampled samples in the convolution model, down-sampling filter flag (whether to do down-sampling), down-sampling filter index when multiple down-sampling filters are used, number of neighboring lines used to derive the model, template type used to derive the model, post-filter flag, and model parameters.

[0112] In one embodiment, a hybrid CCCM model composed of various terms (e.g., a spatial term, a gradient term, a position term, a nonlinearity term, and a bias term) can be inherited. In addition to storing the model parameters, the prediction mode can be stored in the CCM information to indicate that the inherited model is a hybrid CCCM model composed of various terms. If there are multiple types of hybrid CCCM models, a model index can also be stored in the CCM information to indicate which type of hybrid CCCM model is inherited. For example, the gradient and position based CCCM (GL-CCCM) proposed in JVET-AB0119 (Ramin G. Youvalari et al., “Non-EE2: Convolutional Cross-Component Model (GL-CCCM) for Intra Prediction Based on Gradient and Position,” Joint Video Exploration Team (JVET) of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11, 28th Meeting, Mainz, Germany, 20-28 October 2022, Doc: JVET-AB0119) is a hybrid CCCM model composed of one spatial term for the center position, two gradient terms for the horizontal and vertical directions, two position terms for the relative horizontal and vertical positions, one nonlinearity term, and one bias term. The prediction mode can be stored in the CCM information to indicate that the inherited model is a GL-CCCM model.

[0113] In another embodiment, a propagation flag can be stored in the CCM information. This information describes whether the CCM information is copied from the reference block, as described in section "V.6. Propagated CCM information". As described in section V.6, if a block is a CCP coded block, the CCM information of the current block is derived and stored (i.e., the CCM information containing the current block's coding information is stored). The propagation flag of the current block's CCM information is set to false. If a block is not a CCP coded block, the CCM information of the current block is derived by copying the CCM information from the reference block. The propagation flag of the current block's CCM information is then set to true.

[0114] In another embodiment, the CCM information can include information indicating the source of the propagated CCM information. For example, as described in section V.6, the reference block from which the CCM information is copied can be a block located by the block vector of the current block, a block located by the motion vector of the current block, or a collocated block of the current block.

[0115] In another embodiment, the CCM information can include information indicating whether to apply local boost (LB-CCP).

[0116] V.2. Inherited spatial neighboring model parameters In one embodiment, the inherited model parameters can come from a neighboring block. The models from the predetermined positions are added to the candidate list in a predetermined order. The predetermined order can be any possible order of the spatial neighboring blocks.

[0117] In one embodiment, the predetermined positions and the predetermined order can be the same as the spatial candidates for the intra-merge mode.

[0118] In one embodiment, the predetermined positions can be FIG. 5 the positions depicted in FIG. 6 (also described in section "Spatial candidate derivation"). The predetermined order can be B0, A0, B1, A1, and B2.

[0119] In one embodiment, assuming the position, width and height of the current block are (x, y), W and H, respectively, the predetermined positions can include the positions directly above the current block, e.g., (x + W»l, y - 1) or (x + (W+l)»l, y - 1), if W is greater than or equal to a threshold TH. The predetermined positions can also include the positions directly left of the current block, e.g., (x - 1, y + H»l) or (x - 1, y + (H+l)»l), if H is greater than or equal to the threshold TH. TH can be 2, 4, 8, 16, 32 or 64. The predetermined positions include the positions directly above (W»l) or ((W»l) - 1) when W is greater than or equal to TH, and the positions directly left (H»l) or ((H»l) - 1) when H is greater than or equal to TH.

[0120] In one embodiment, the maximum number of models inherited from spatial neighbors can be added to the candidate list, and the maximum number is less than the number of predetermined positions.

[0121] V.3. Inheriting temporal neighboring model parameters In one embodiment, if the current slice / picture is a non-intra slice / picture, the inherited model parameters can be from blocks in previously coded slices / pictures.

[0122] In one embodiment, the position of the current block is (x, y), and the block size is The inherited model parameters can be from blocks at certain predetermined positions of previously coded slices / pictures.

[0123] In one sub-embodiment, the predetermined positions can be the same as the predetermined positions of temporal candidates.

[0124] In one sub-embodiment, the predetermined positions can be or where Both value sets and are defined as: , and are all positive numbers.

[0125] For example, may be .

[0126] For example, . For example, .

[0127] For example, For example, and .

[0128] In one sub-embodiment, the predetermined positions are located within the respective area of the current coding block, i.e. and The predetermined positions can be .

[0129] In one sub-embodiment, the predefined positions are located outside the respective area of the current coding block, i.e. and The predefined positions can be .

[0130] In one embodiment, the model from the position closer to is added to the final merge candidate list first.

[0131] The previously coded picture from which the inherited parameter model is obtained is hereinafter referred to as the collocated picture.

[0132] In one embodiment, the previously coded picture (i.e. the collocated picture) from which the inherited parameter model is derived is one of the pictures in the reference list.

[0133] In one embodiment, the collocated picture can be the same as the collocated picture of the inter merge mode.

[0134] In one embodiment, the collocated picture is the same as the collocated reference picture of the TMVP of the inter merge mode, as described in the section“Temporal candidate derivation”. In another embodiment, the collocated picture can be different from the collocated reference picture of the TMVP of the inter merge mode.

[0135] In one embodiment, the collocated picture is signaled in the picture / slice header. The reference list and the reference index are signaled in the picture / slice header. For example, the collocated picture is selected as L0[0]. For another example, the collocated picture is selected as L1[0].

[0136] In one embodiment, the number of signaled collocated pictures can be more than one. For example, the number of signaled collocated pictures is 2. Temporal candidates from the two collocated pictures can be inherited.

[0137] In one embodiment, the collocated picture is selected as the picture in the reference list with the smallest POC difference between the respective picture and the current picture.

[0138] In another sub-embodiment, the co-located picture is selected as the picture with larger QP in the reference list. In one embodiment, the location in the previously coded slice / picture from which the model parameters are inherited is determined by the motion vectors of the neighboring blocks. Let Δχ and Δy be the horizontal and vertical displacements determined based on the motion vectors of the selected neighboring blocks, the current block position is (x, y), and the block size is The inherited model parameters can come from the block at position (x', y'), where x' = x + Δχ and y' = y + Δy, or x' = x + w / 2 + Δχ and y' = y + h / 2 + Δy.

[0139] V.4. Inheriting non-adjacent spatial neighboring models In one embodiment, the inherited model parameters can come from non-adjacent spatial neighboring blocks. The models from the predefined positions are added to the candidate list in a predefined order.

[0140] In one sub-embodiment, the predefined positions and the predefined order are the same as the non-adjacent spatial neighboring candidates for the intra-merge mode.

[0141] In one sub-embodiment, the predefined positions and the predefined order are as shown in FIG. 10A and FIG. 10B The positions of the numbered blocks are the predefined positions. The numbers within each block indicate the predefined order. The positions in mode 1 ( FIG. 10A ) are added to the list before the positions in mode 2 ( FIG. 10B ). The distance between each predefined position is proportional to the width and height of the current block.

[0142] In one embodiment, the maximum number of models inherited from non-adjacent spatial neighbors can be added to the candidate list, and the maximum number is less than the number of predefined positions.

[0143] V.5. Inheriting model parameters from a history table In one embodiment, the inherited model parameters can come from a cross-component model history table. The history table stores the CCM information of the valid previously coded blocks. A valid previously coded block refers to any block that contains valid CCM information. The cross-component models in the history table can be added to the candidate list in a predefined order. In one embodiment, the order of addition of the history candidates can be from the beginning of the table to the end of the table. In another embodiment, the order of addition of the history candidates can be from the end of the table to the beginning of the table.

[0144] In one embodiment, a cross-component model history table can be maintained to store previous cross-component models (i.e., CCM information), and the cross-component model history table can be reset at the beginning of a current picture, a current slice, a current tile, every M CTU rows, or every N CTUs, where N and M can be any value greater than 0. In another embodiment, the cross-component model history table can be reset at the end of a current picture, a current slice, a current tile, a current CTU row, or a current CTU.

[0145] In another embodiment, multiple history tables are used to store different types of cross-component models. For example, a first history table is used to store single models, and a second history table is used to store multiple models. For another example, a first history table is used to store gradient models, and a second history table is used to store non-gradient models. For another example, a first history table is used to store simple linear models (e.g., y = ax + b), and a second history table is used to store complex models (e.g., CCCM).

[0146] In one embodiment, when adding history candidates from multiple history tables to a candidate list, the order of addition can be from the beginning to the end of a certain table, and then the next history table is added in the same order or in reverse order.

[0147] V.6. Propagated CCM information In one embodiment, after a block is coded, the cross-component model (CCM) information of the current block is derived and stored in the current block. The stored CCM information can be referenced by subsequent coded blocks. The subsequent coded blocks can inherit the CCM information from the current block. The definition of CCM information is in the section of "Inherited CCM information". The stored CCM information can be inherited as, but not limited to, the following types of candidates: spatial candidates (as described in the section of "Inherited spatial neighboring model parameters"), non-adjacent candidates (as described in the section of "Inherited non-adjacent spatial neighboring model"), temporal candidates (as described in the section of "Inherited temporal neighboring model parameters"), and history candidates (as described in the section of "Inherited model parameters from history table").

[0148] In one embodiment, if the current block is not coded using cross-component prediction (CCP) (i.e., the block does not use a cross-component model, such as an inherited cross-component model, a self-derived cross-component model, a cross-component model used in chroma fusion, which means that the chroma prediction is based on adding one or more cross-component prediction hypotheses to one or more existing non-cross-component prediction hypotheses, or any combination of the above), and there is a block vector available in the current block (e.g., the current luma block is coded in IBC or IntraTMP mode, and the collocated luma block is coded in IBC or IntraTMP mode), the CCM information of the current block can be derived by copying the CCM information of the reference block located by the block vector. For example, as shown in FIG. 6, the CCM information of the current block 602 is derived by copying the CCM information of the reference block 604 located by the block vector 606. FIG. 11As shown, block B is not CCP encoded and has available block vectors in block B. The reference block A is located by the block vectors. The CCM information of the reference block A using the cross-component model is copied and stored in block B. In an embodiment, if the reference block located by the block vectors is also not CCP encoded, but has CCM information stored in the reference block, the CCM information of the current block can be derived by copying the CCM information stored in the reference block. That is, even if the reference block is not CCP encoded, as long as it has valid stored CCM information, the current block can reference the stored CCM information. For example, as shown in FIG. 11 As shown, the current block C has available block vectors, whose reference block B is not CCP encoded, but has CCM information stored. The CCM information of block B is copied and stored in block C. Since the CCM information stored in block B is copied from block A, the CCM information stored in block C is originally from block A (i.e., the CCM information of block A is propagated to block C). By accessing block B only, block C can retrieve the CCM information originally from block A. In an embodiment, if the reference block located by the block vectors is not CCP encoded and has no CCM information stored, the current block does not store CCM information. As mentioned above, the CCM information copied and stored in block B (not encoded in CCP mode) can be used by one or more subsequent blocks (e.g., block C) for CCP processing. The CCP processing in this disclosure refers to the related processing that uses CCM information to derive or improve cross-component prediction.

[0149] In an embodiment, the block vector used to derive the reference block is the block vector co-located with the center of the luma block. In another embodiment, the block vector used to derive the reference block is the block vector co-located with the top-left corner of the luma block.

[0150] In an embodiment, when the current block has multiple available block vectors (e.g., the block vectors can be bi-directional, the block can have multiple Intra TMP block vectors, or the current chroma block is co-located with multiple luma blocks, and multiple of the multiple luma blocks have block vectors), to derive the CCM information of the current block, if only one of the reference blocks located by the block vectors has CCM information, the CCM information of the reference block with CCM information is copied and stored in the current block. For example, as shown in FIG. 11 As shown, assume block F has two block vectors, and has two reference blocks G and H. Block G has CCM information, and block H does not. The CCM information of block G is copied and stored in block F.

[0151] In another embodiment, when the current block has multiple block vectors, and multiple of the multiple reference blocks located by the block vectors have CCM information, the CCM information of the current block is derived by combining all or part of the CCM models of its reference blocks (as described in the “Inherit multiple cross-component models” section).

[0152] In another embodiment, when the current block has multiple block vectors, and among the multiple reference blocks located by the block vectors, multiple have CCM information, one reference block is selected according to a set of predefined rules. The CCM information of the selected reference block is then copied and stored in the current block.

[0153] In a sub-embodiment, the reference block that is selected to be coded with CCP is selected.

[0154] In a sub-embodiment, the reference block that is selected to be intra coded is selected.

[0155] In a sub-embodiment, the reference block that is selected to be inter-inter coded is selected.

[0156] For one sub-embodiment, the reference block with the smallest distance to the current block is selected. The CCM information of the selected reference block is copied and stored in the current block. The distance between the reference block and the current block, located at (x r , y r ) and (x c , y c ) respectively, can be calculated by the Euclidean distance (x r - x r , y c - y c ). (x r , y r ) and (x c , y c ) can be the top-left, top-right, bottom-left, bottom-right or center position of the reference block and the current block. The distance measurement can also be Manhattan distance or Minkowski distance.

[0157] For one sub-embodiment, the reference block with the smallest horizontal distance |xr - xc| is selected. The CCM information of the selected reference block is copied and stored in the current block.

[0158] For another sub-embodiment, the reference block with the smallest vertical distance |yr - yc| is selected. The CCM information of the selected reference block is copied and stored in the current block.

[0159] For one sub-embodiment, the CCM information of all reference blocks is applied to the reconstructed luma samples of the current block to generate the prediction of the chroma samples of the current block. The distortion between the prediction and the reconstructed chroma samples is calculated. The reference block that is associated with the smaller distortion is selected.

[0160] For a sub-implementation, the aforementioned rules can be combined, and it is not necessary to apply all of them. For example, selecting a reference block for CCP encoding / decoding. If there are multiple reference blocks for CCP encoding / decoding, the block with the shortest distance to the current block is selected. If multiple reference blocks for CCP encoding / decoding have the smallest distance to the current block, the reference block with the smallest horizontal distance |xr - xc| is selected. Another example: selecting a reference block for CCP encoding / decoding. If there are multiple reference blocks for CCP encoding / decoding, the block with the shortest distance to the current block is selected. If multiple reference blocks for CCP encoding / decoding have the smallest distance to the current block, the block with the smallest vertical distance |y| is selected. r - y c The reference block. The CCM information of the selected reference block is copied and stored in the current block.

[0161] In one embodiment, if the current block is not CCP encoded and motion vectors are available in the current block (e.g., the current luma block is inter-frame encoded), the CCM information of the current block can be derived by copying the CCM information of a reference block located by the motion vectors of the current block in its reference image. For example, as FIG. 12 As shown, block B is not CCP-coded but has available motion vectors. Reference block A is located by motion vectors. The CCM information of reference block A, using the cross-component model, is copied and stored in block B. In one embodiment, if the reference block located by motion vectors is also not CCP-coded, but stores CCM information, the CCM information of the current block can be derived by copying the CCM information stored in the reference block. That is, even if the reference block is not CCP-coded, as long as it has valid stored CCM information, the current block can refer to the stored CCM information. For example, as... FIG. 12 As shown, current block C has available motion vectors, and its reference block B is not CCP-coded but stores CCM information. The CCM information of block B is copied and stored in block C. Since the CCM information stored in block B is copied from block A, the CCM information stored in block C originally came from block A (i.e., the CCM information of block A propagated to block C). By accessing only block B, block C can retrieve the CCM information originally from block A. In one embodiment, if the reference block located by the motion vector is not CCP-coded and does not store CCM information, then the current block does not store CCM information.

[0162] In one embodiment, the motion vector used to derive the reference block is the motion vector of the current block's center. In another embodiment, the motion vector used to derive the reference block is the motion vector of the current block's top-left corner.

[0163] In one embodiment, the motion vector used to derive the reference block is the MVP of the current block. In another embodiment, the motion vector used to derive the reference block is a combination of the MVP and MVD of the current block.

[0164] For one embodiment, when the current block is inter-inter coded with bi-prediction, to derive the CCM information of the current block, if only one of the reference blocks located by the motion vectors has CCM information, the CCM information of the reference block with CCM information is copied and stored in the current block. For example, as shown in FIG. 12 , assume that block F is inter-inter coded with bi-prediction. The two reference blocks located by the motion vectors are block G and block H. Block G has CCM information, while block H does not. The CCM information of block G is copied and stored in block F.

[0165] For another embodiment, when the current block is inter-inter coded with bi-prediction, and both of the reference blocks located by the motion vectors have CCM information, the CCM information of the current block is derived by combining all or part of the CCM models of its reference blocks (as mentioned in the section: "Inheriting multiple cross-component models").

[0166] For another embodiment, when the current block is inter-inter coded with bi-prediction, and both of the reference blocks located by the motion vectors have CCM information, one of the reference blocks is selected according to a set of predefined rules. The CCM information of the selected reference block is copied and stored in the current block.

[0167] For one sub-embodiment, the reference block that is inter-inter coded is selected.

[0168] For one sub-embodiment, the reference block that is intra coded is selected.

[0169] For one sub-embodiment, the reference block that is inter-inter coded is selected.

[0170] For one sub-embodiment, the reference block whose reference picture (i.e. the picture in which the reference block is located) has a smaller POC distance to the current picture is selected. The CCM information of the selected reference block is copied and stored in the current block. As shown in FIG. 12 , block F is inter-inter coded with bi-prediction. The two reference blocks located by the motion vectors are block G and block H. Assume that both block G and block H have CCM information, and |N1-N2| is smaller than |N3-N2|. Then block G is selected, and the CCM information of block G is copied and stored in block F.

[0171] For one sub-embodiment, the reference block whose reference picture has a smaller QP difference to the current picture is selected. The CCM information of the selected reference block is copied and stored in the current block. As shown in FIG. 12As shown, block F is inter-inter coded using bi-prediction. The two reference blocks located by the motion vectors are block G and block H. Assume that both block G and block H have CCM information stored. Assume that the QPs of the pictures of POC N1, POC N2 and POC N3 are 27, 32, 33 respectively. Since |33-32| is smaller than |27-32|, block H is selected and the CCM information of block H is copied and stored in block F.

[0172] For one sub-embodiment, the reference block whose reference picture has a smaller QP value is selected. For another sub-embodiment, the reference block whose reference picture has a larger QP value is selected.

[0173] For one sub-embodiment, the CCM information of the two reference blocks is applied to the reconstructed luma samples of the current block to generate the chroma sample prediction of the current block. The distortion between the prediction and the reconstructed chroma samples is calculated. The reference block associated with the smaller distortion is selected.

[0174] For one sub-embodiment, the aforementioned rules can be combined and not all of the aforementioned rules need to be applied. For example, the reference block that is CCP coded is selected. If both blocks are CCP coded, the block whose reference picture has a smaller POC distance to the current picture is selected. If both blocks are CCP coded and have the same POC distance to the current picture, the reference block whose reference picture has a smaller QP difference to the current picture is selected. If both blocks are CCP coded and have the same POC distance to the current picture and the same QP difference to the current picture, the reference block whose reference picture has a smaller QP value is selected. For another example, the block whose reference picture has a smaller POC distance to the current picture is selected. If both blocks have the same POC distance to the current picture, the reference block whose reference picture has a smaller QP difference to the current picture is selected. If both blocks have the same POC distance to the current picture and the same QP difference to the current picture, the reference block whose reference picture has a smaller QP value is selected.

[0175] In one embodiment, if the reference picture located by the motion vector has a POC distance to the current picture larger than a predetermined threshold, the motion vector and the reference block are considered unavailable.

[0176] In one embodiment, if the current block is inter-inter coded or there is a block vector available in the current block, the CCM information of the current block can be derived by copying the CCM information of the reference block located by the motion vector or the block vector. For example, as shown in FIG. 13 The current block C has a block vector available and its reference block B has a motion vector available. The CCM information of block B is copied from block A. Then the CCM information of block B is copied to block C. Thus, the CCM information of block A is propagated to the current block C.

[0177] In one embodiment, if the current block is not CCP coded and the current slice / picture is a non-intra slice / picture, the CCM information of the current block can be derived by copying the CCM information of its co-located block in the co-located picture. The definition of co-located picture can be found in the section of "Inherited Temporal Neighboring Model Parameters".

[0178] In one sub-embodiment, if the co-located picture is more than a pre-defined threshold away from the POC of the current picture, copying and storing CCM information from the co-located picture is not performed.

[0179] In one embodiment, the CCM information derivation process is performed after the current picture is coded / decoded when the current block is not CCP coded.

[0180] In one embodiment, when multiple types of propagated CCM information are available for the current block, the propagated CCM information to be stored in the current block is determined according to a set of pre-defined rules. For example, if both the reference block located by the motion vector and the co-located block in the co-located picture have valid CCM information, the CCM information of the reference block located by the motion vector is copied and stored in the current block. For another example, the CCM information of the co-located block is copied and stored in the current block. For another example, after the block is coded / decoded, the CCM information of the reference block located by the motion vector is copied and stored in the current block. After the picture is coded / decoded, the CCM information of the co-located block is copied and stored in the current block (i.e., replacing the CCM information of the reference block located by the motion vector).

[0181] For example, if both the reference block located by the block vector and the reference block of the co-located block in the co-located picture have valid CCM information, the CCM information of the reference block located by the block vector is copied and stored in the current block. For another example, the CCM information of the co-located block is copied and stored in the current block. For another example, after one block is coded / decoded, the CCM information of the reference block located by the block vector is copied and stored in the current block. After one picture is coded / decoded, the CCM information of the co-located block is copied and stored in the current block (i.e., the CCM information of the reference block located by the block vector is replaced).

[0182] V.7. Inheritance from Fusion Mode Fusion mode refers to a mode in which two predictions are fused to generate a final prediction. In the chroma intra-fusion mode, a chroma intra prediction generated using a non-cross component prediction (CCP) coding tool (e.g., CCLM, MMLM, CCCM) is fused with another chroma intra prediction generated using a cross component prediction coding tool. For example, a non-CCLM coded intra prediction and a CCLM coded intra prediction are fused together to obtain a final intra prediction.

[0183] In one embodiment, when inheriting cross-component model parameters from a position / block coded by the chroma intra merge mode, the model parameters used to obtain the CCP coded intra prediction are inherited and further refined.

[0184] In one embodiment, in addition to inheriting and refining the CCP model parameters, the merge weights and the coding mode of the non-CCP coded intra prediction are also inherited. That is, the chroma intra merge mode is inherited.

[0185] V.8. Inheriting multiple cross-component models The final prediction of the current block can be a combination of the predictions of multiple cross-component models, or a fusion of the prediction of a selected cross-component model and the prediction of a non-cross-component coding tool (e.g., intra angular prediction mode, intra planar / DC mode, or inter prediction mode). In one embodiment, if the size of the current candidate list is N, k candidates can be selected from the total of N candidates (where k ≤ N). Then, k predictions are generated respectively by applying the cross-component models of the selected k candidates to the corresponding luma reconstructed samples. The final prediction of the current block is the combination result of these k predictions. For example, if two candidate predictions (denoted as and ) are combined, the final prediction of the current block at (x, y) position is where is a weighting factor. In addition, the weighting factor can be predefined or implicitly derived according to the neighboring template cost (i.e., model error). For example, by using the template cost defined in the section of “Reordering the list of candidates”, the corresponding template costs of the two candidates are and , then is In another embodiment, if two candidate models are combined, the selected model is from the first two candidates in the list. In another embodiment, if i candidate models are combined, the selected model is from the first i candidates in the list.

[0186] In another embodiment, if the size of the current candidate list is N, k candidates can be selected from the total of N candidates (where k ≤ N). These k cross-component models can be merged into one final cross-component model by weighted averaging of the corresponding model parameters. For example, if a cross-component model has M parameters, the j-th parameter of the final cross-component model is the weighted average of the j-th parameters of the selected k candidates, where j is 1 … M. Then, the final prediction is generated by applying the final cross-component model to the corresponding luma reconstructed samples. For example, if two cross-component models are and The final cross-component model is where is a weighting factor that can be implicitly derived by a pre-defined or neighboring template cost, is the x-th model parameter of the y-th candidate. For example, by using the template cost defined in the section titled "Reordering candidates in the list", the corresponding template costs of two candidates are and then is For example, for two candidate models, one from a spatially neighboring neighboring candidate and the other from a non-neighboring spatial candidate or a history candidate. If the spatially neighboring neighboring candidate is not available, both candidate models are from a non-neighboring spatial candidate or a history candidate. In another embodiment, if merging two candidate models, the selected model is from the first two candidates in the list. In another embodiment, if merging i candidate models, the selected model is from the first i candidates in the list.

[0187] In another embodiment, two cross-component models are merged into one final model by a weighted average of the corresponding model parameters, where one of the two cross-component models is from an above spatial neighboring candidate and the other is from a left spatial neighboring candidate. The above spatial neighboring candidate is the neighboring candidate whose vertical position is less than or equal to the top boundary position of the current block. The left spatial neighboring candidate is the neighboring candidate whose horizontal position is less than or equal to the left boundary position of the current block. The weighting factor is determined according to the horizontal and vertical spatial positions within the current block. For example, if merging two candidate predictions (denoted as and ), the final prediction at the (x, y) position of the current block is where In another embodiment, the above spatial neighboring candidate is the first candidate in the list whose vertical position is less than or equal to the top boundary position of the current block. The left spatial neighboring candidate is the first candidate in the list whose horizontal position is less than or equal to the left boundary position of the current block.

[0188] In another embodiment, multiple cross-component models can be merged into one final cross-component model. For example, one model can be selected from one candidate and a second model can be selected from another candidate to form a multiple model mode. The selected candidate can be a CCLM / MMLM / GLM / CCCM coded candidate. The multiple model classification threshold can be an offset parameter of the two selected modes (e.g., offset / in CCLM or in CCCM). or the average of the neighboring luma and chroma samples of the current block. In one embodiment, if two candidate models are merged, the selected model is the first two candidates in the list. In another embodiment, the classification threshold is set to the average of the neighboring luma and chroma samples of the current block.

[0189] VI. Building the candidate list VI.1. Order of insertion of candidates into the list In one embodiment, candidates related to propagated CCM information (CCM information is copied from the reference block as described in the "Propagated CCM information" section) are inserted after candidates related to non-propagated CCM information.

[0190] In another embodiment, candidates related to propagated CCM information are only inserted into the candidate list when the current block is an inter block. In another embodiment, candidates related to propagated CCM information are only inserted into the candidate list when the current block is an intra block.

[0191] VI.2. Reordering candidates in the list Candidates in the candidate list can be reordered to reduce the syntax overhead in signaling the selected candidate index, or to bypass the syntax of signaling the candidate index by using an implicit rule to select one or more candidates.

[0192] In one embodiment, the reordering rule can rely on the coding information of neighboring blocks or model error. For example, if the above or left neighboring block is coded by MMLM, the MMLM candidate in the list can be moved to the beginning of the current list.

[0193] In one embodiment, the reordering rule is based on model error by applying the candidate model to the neighboring template of the current block, and then comparing the error with the reconstructed samples of the neighboring template.

[0194] VII. Self-deriving cross-component models In one embodiment, an example of a self-deriving cross-component model is CCRM. When self-deriving, the model (filter shape / pattern, parameter term) is unified with the cross-component model in regular intra modes. For example, the CCRM model can be unified with any pre-defined existing intra cross-component model (e.g., CCCM, GLM, MMLM using non-downsampled luma samples), and / or self-deriving simply means that the input to derive the model parameters comes from the current chroma and collocated luma samples (e.g., motion compensation results if the current block is inter).

[0195] In another embodiment, the self-deriving cross-component candidate refers to one or more models that are used to generate the cross-component prediction of the current block as follows. The cross-component prediction of the current block (used to generate the target prediction sample) is formed by combining one or more proposed source terms and models (referring to proposed weighting settings). As shown in equation (3), pred(i, j) is the target (prediction) sample in the current block, which can be obtained after our proposed mechanism, sourceTermSet0 contains one or more source terms from the luma component, sourceTermSet1 contains one or more source terms from the chroma component, and biasTermSet contains one or more bias terms.

[0196] Equation (3) is just an example, and our proposed mechanism can use any subset or extension of sourceTermSet0, sourceTermSet1, and biasTermSet. Each sample or any subset of samples in the current block obtains its target (prediction) sample according to equation (3). The content of sourceTermSet0 is described in Section VII.1, “Content of sourceTermSet0(i, j),” the content of sourceTermSet1 is described in Section VII.2, “Content of sourceTermSet1(i, j),” the content of biasTermSet is described in Section VII.3, “Content of biasTermSet,” and the predictor derivation of sample (i, j) using the proposed source terms and the proposed weighting settings is described in Section VII.4, “Predictor derivation of sample (i, j).” Several examples of our proposed mechanism are shown in Section VII.4, “Predictor derivation of sample (i, j).”

[0197] pred(i, j) = (sourceTermSet0(i, j) + sourceTermSet1(i, j) + … +biasTermSet) with proposed weight settings, where (i, j) is the sample location in the current block.

[0198] VII.1. Content of sourceTermSet0(i, j) sourceTermSet0(i, j) contains one or more luma source terms, denoted as sourceTerm00, sourceTerm01, …, and / or sourceTerm0n-1. The value of n represents the number of taps of the source term set.

[0199] In one embodiment, the source terms can be linear terms and / or non-linear terms, linear terms only, and / or non-linear terms only.

[0200] In another embodiment, n is a predefined value, e.g., 1, 2, …, or any positive integer. For example, the predefined value is fixed in the standard.

[0201] In another embodiment, n is determined by the coding information of the current block and / or the sample position (i, j). For example, when the current block is coded by a certain coding tool, n can be fixed at a predefined value of that certain coding tool.

[0202] In another embodiment, the pattern of the n-tap refers to any subset of the window region M x N defined to surround / contain the position (iL, jL), as shown in FIG. 14A. If the target sample is luma, (iL, jL) is (i, j). If the target sample is chroma (e.g., Cb or Cr), (iL, jL) is the co-located luma position from (i, j).

[0203] For example, only the center (iL, jL) of the window is used as FIG. 14A shown.

[0204] Another example is the 5x5 cross pattern that contains or excludes (iL, jL), as FIG. 14B shown.

[0205] For a source term in the set of source terms, the following embodiments are used to determine the source content generation.

[0206] In one embodiment, the source content is based on the prediction sample generated by the prediction mode and / or based on the reconstructed sample generated from the prediction sample and the reconstructed residual.

[0207] In another sub-embodiment, the source content is filtered source or source with any pre-processing. For example, the source content is the prediction / reconstructed sample filtered using a predefined model or filter.

[0208] In another sub-embodiment, the source content is gradient information from the prediction sample and / or the reconstructed sample. If the target sample (i, j) belongs to chroma, and the gradient information of the co-located luma sample (as the center circle) is calculated using any one of the Sobel filters (1510-1540) or any predefined filter in FIG. 15 . Each value around the center circle is multiplied by the corresponding prediction / reconstructed sample in the co-located luma block, and then added to form the gradient information of the source term for the target sample (i, j).

[0209] In another sub-embodiment, since the target sample belongs to a chroma sample (e.g., Cb or Cr), the prediction sample and / or the reconstructed sample is located within the co-located (luma) block of the current (chroma) block. The prediction sample and / or the reconstructed sample is treated as an initial sample and used as the source content to generate the target sample.

[0210] In another embodiment, the source term can further include position information. For example, if the target sample refers to luma, the horizontal position (i) of (i, j) is used for the source term, and the vertical position (j) of (i, j) is used for the source term; otherwise, the horizontal position of the co-located luma block of sample (i, j) is used for the source term, and the vertical position of the co-located luma block of sample (i, j) is used for the source term.

[0211] In another embodiment, the source term can further include position information. For example, if the target sample refers to luma, the horizontal position (i) of (i, j) is used for the source term, and the vertical position (j) of (i, j) is used for the source term; otherwise, the horizontal position of the co-located luma block of sample (i, j) is used for the source term, and the vertical position of the co-located luma block of sample (i, j) is used for the source term.

[0212] VII.2. Content of SourceTermSet1(i, j) SourceTermSet1(i, j) includes one or more chroma (Cb or Cr) source terms, denoted as sourceTerm00, sourceTerm01, …, and / or sourceTerm0m-1. The value of m represents the number of taps of the source term set. In one embodiment, the source term can be a linear term and / or a non-linear term, a linear term only, and / or a non-linear term only. In another embodiment, m is a pre-defined value, e.g., 1, 2, …, or any positive integer. For example, the pre-defined value is fixed in the standard.

[0213] In another embodiment, m is determined according to the coding information of the current block and / or the sample position (i, j). For example, when the current block is coded by a specific coding tool, m is fixed at a pre-defined value for that specific tool.

[0214] In another embodiment, the pattern of m taps refers to a pattern defined as any subset of the M2 x N2 window region surrounding / containing the position (iC, jC), as shown in FIG. 16A If the target sample is chroma (Cb or Cr), (iC, jC) is (i, j). If the target sample is luma, (iC, jC) is the co-located chroma position from (i, j).

[0215] For example, only the center (iC, jC) of the window is used, as shown in FIG. 16A

[0216] ​Another example is the 5x5 cross pattern: (including or excluding (iC, jC)), as shown in FIG. 16B

[0217] For a source term in the source term set, the following embodiments are used to determine the source content.

[0218] In one embodiment, the source content is based on the prediction sample generated by the prediction mode and / or based on the reconstructed sample generated based on the prediction mode and the reconstructed residual.

[0219] In another sub-embodiment, the source content is a filtered source or a source with any pre-processing. For example, the source content is the prediction / reconstructed sample filtered using a predefined model or filter.

[0220] In another sub-embodiment, the source content is gradient information from the prediction sample and / or the reconstructed sample. If the target sample (i, j) belongs to luma, the gradient information of the co-located chroma sample is calculated using any one Sobel filter or any predefined filter.

[0221] In another sub-embodiment, if the target sample belongs to chroma sample, the prediction sample and / or the reconstructed sample is located within the current block. The prediction sample and / or the reconstructed sample is treated as an initial sample and used as the source content to generate the target sample.

[0222] In another embodiment, the source term can further include position information. For example, if the target sample refers to chroma, the horizontal position (i) of (i, j) is used for the source term, and the vertical position (j) of (i, j) is used for the source term.

[0223] VII.3. Content of BiasTermSet The bias term is a predefined value. In one embodiment, the bias term is the median value according to the bit depth specified in the standard. For example, the bias term is set to In another embodiment, the bias term for each sample in the current block is the same. That is, the bias term is independent of the position (i, j).

[0224] VII.4. Predictor derivation for sample (i, j) VII.4.1. Proposed weighting setting ​The proposed weighting setup estimates the relationship between "predicted and / or reconstructed samples on the reference region of the current (chroma) block" and "predicted and / or reconstructed samples on the reference region of the corresponding luma block" using a predefined regression method (e.g., minimizing distortion), and generates weights (referring to model parameters) based on the regression method. The derived weights are then applied to the source terms to obtain the target (predicted) samples in the current block. In one embodiment, the predefined regression method may be the Linear Least Mean Squared Error (LMMSE) method for CCLM or any method unified with the regression method used for CCLM. In another embodiment, the predefined regression method may be the LDL decomposition method for CCCM or any method unified with the regression method used for CCCM. In yet another embodiment, the predefined regression method may be Gaussian elimination.

[0225] In one embodiment, such as FIG. 17 As shown, the reference region of the current block is the spatial neighbor region of the current block 1710. The spatial neighbor region of the current block includes the upper reference region 1720, the left reference region 1730, the upper-left reference region 1740, and / or any subset thereof. The size of the upper reference region is A. w xA H The size of the reference region on the left is L. w xL H The size of the upper left reference region is AL. W xAL H ,in A w = Current block width (W), k*W, W + current block height (H), any predefined value, or any adaptive value based on the current block's position, block width, block height, and / or block area.

[0226] A H or AL H = H, any predefined value (1, 2, 4, …), or any adaptive value based on the current block's position, block width, block height, and / or block area.

[0227] L W or AL W = W, any predefined value (1, 2, 4, …), or any adaptive value based on the current block's position, block width, block height, and / or block area.

[0228] L H = H, k*H, H + W, any predefined value, or any adaptive value based on the current block's position, block width, block height, and / or block area.

[0229] The reference region of the corresponding luma block is a spatial neighboring region of the corresponding luma block.

[0230] In another embodiment, the reference region of the current block is a vector collocated region of the current block, and the reference region of the corresponding luma block is a vector collocated region of the corresponding luma block. For inter-coded units containing luma and chroma blocks, the vector collocated region of the current block refers to the motion compensated result obtained using the motion information (motion vector and reference picture) of the current block, and the vector collocated region of the corresponding luma block refers to the motion compensated result obtained using the motion information (motion vector and reference picture) of the corresponding luma block. For intra block copy (IBC) or intra template matching prediction (IntraTMP), the vector collocated region of the current block refers to the motion compensated result obtained using the motion information (e.g. block vector and current picture) of the current block, and the vector collocated region of the corresponding luma block refers to the motion compensated result obtained using the motion information (e.g. block vector and current picture) of the corresponding luma block.

[0231] In another embodiment, both of the above-mentioned reference regions of the current block can be used simultaneously. For example, usually the samples in the vector collocated region of the current block are used as input samples when deriving the model parameters; however, for smaller blocks, the samples in the spatial neighboring reference region are used as additional input samples when deriving the model parameters.

[0232] In the present disclosure, the term "block" can refer to a TU / TB, a CU / CB, a PU / PB, or a CTU / CTB.

[0233] In the present disclosure, the term "LM" can be considered as a CCLM / MMLM mode or any other extension / variant of CCLM (e.g. the CCLM extension / variant proposed in the present disclosure). One variant is MMLM, which uses a threshold to decide different models for different samples in the current chroma component. Another variant is to derive the model parameters from multiple collocated luma blocks for Cb (or Cr). More possible variants are shown below. The CCLM variant here means that when a block indicates to use a cross-component mode (e.g. CCLM LT, MMLM LT, CCLM L, CCLM T, MMLM L, MMLM T and / or an intra prediction mode, not one of the traditional DC, planar and angular modes), some optional modes can be selected. An example of using a convolutional cross-component mode (CCCM) as an optional mode is shown below. When this optional mode is applied to the current block, the cross-component information using a model containing a non-linear term is used to generate the chroma prediction. The optional modes can follow the template selection of CCLM, so the CCCM family includes CCCM LT, CCCM L and / or CCCM T.

[0234] The method proposed in this disclosure (for CCLM) can be used for any other cross component mode.

[0235] Any combination of the methods proposed in this disclosure can be applied.

[0236] Any of the methods proposed above for cross component prediction by mixing multiple cross component prediction models can be implemented in an encoder and / or a decoder. For example, any of the proposed methods can be implemented in an inter, intra, prediction, IBC, transform, quantization module or their combination at the encoder side, and / or in an inter, intra / prediction, IBC, transform, quantization module or their combination at the decoder side. Alternatively, any of the proposed methods can be implemented as a circuit coupled to an inter, intra, prediction, transform, quantization module or their combination of an encoder and / or a decoder to provide the information required by the inter / intra / prediction / IBC / transform / quantization module.

[0237] The propagated cross component prediction model can be implemented at the encoder side or the decoder side as described above. For example, any of the proposed methods can be implemented in an intra / prediction module (e.g., Intra Pred. 150 / MC 152 in FIG. 1B ) at the decoder side, or in an intra / prediction module (e.g., Intra Pred. 110 / Inter Pred. 112 in FIG. 1A ) at the encoder side. Any of the proposed propagated cross component prediction can also be implemented as a circuit coupled to an intra / prediction module of a decoder or an encoder. However, a decoder or an encoder can also use an additional processing unit to implement the propagated cross component prediction processing. Although the intra / prediction units (e.g., units 110 / 112 in FIG. 1A and units 150 / 152 in FIG. 1B ) are shown as independent processing units, they can correspond to executable software or firmware code stored on a media (e.g., a hard disk or a flash memory) for a central processing unit (CPU) or a programmable device (e.g., a digital signal processor (DSP) or a field programmable gate array (FPGA)).

[0238] FIG. 18A flowchart of an exemplary video coding system that includes a propagated cross-component prediction model according to an embodiment of the present application is illustrated. The steps shown in the flowchart can be executed as program code on one or more processors (e.g., one or more CPUs) at the encoder or decoder side. The steps shown in the flowchart can also be implemented based on hardware, such as one or more electronic devices or processors arranged to perform the steps in the flowchart. According to this method, input data related to a current block is received in step 1810, the current block including a first color block and a second color block, wherein the input data includes pixel data to be encoded at the encoder side or data related to the current block to be decoded at the decoder side, and the current block is coded in a non-intra mode. In step 1820, the current block is encoded or decoded using a non-cross component prediction (CCP) mode. In step 1830, one or more reference blocks pointed to by one or more motion vectors or one or more block vectors of the current block are determined. In step 1840, if the one or more reference blocks have target cross component model (CCM) information, the target CCM information is copied and stored in the current block, wherein the target CCM information is stored in the current block and accessed by one or more subsequent blocks for CCP processing.

[0239] The flowchart shown is intended to illustrate one example of video coding according to the present application. A skilled person can modify each step, rearrange the steps, split the steps, or combine the steps to practice the present application without departing from the spirit of the present application. In this disclosure, specific syntax and semantics are used to illustrate examples of practicing the present application. A skilled person can practice the present application without departing from the spirit of the present application by replacing with equivalent syntax and semantics.

[0240] The above description is intended to enable the ordinary skilled person in the art to practice the application in the context of the specific applications and its intended use. Various modifications to the described embodiments will be apparent to those skilled in the art and the general principles defined herein can be applied to other embodiments. Therefore, the present application is not intended to be limited to the particular embodiments shown and described, but is to be accorded the widest scope consistent with the principles and novel features herein disclosed. In the above detailed description, various specific details are illustrated in order to provide a thorough understanding of the present application. However, the skilled person will understand that the present application can be practiced.

[0241] Embodiments of the present application as described above can be implemented in various hardware, software codes, or a combination of both. For example, an embodiment of the present application can be one or more circuit circuits integrated into a video compression chip or program codes integrated into video compression software to perform the processing described herein. An embodiment of the present application can also be program codes to be executed on a digital signal processor (DSP) to perform the processing described herein. The present application can also involve various processes so performed on a computer processor coupled to a computer memory that can contain program codes, software, firmware or any combination of them. These processes together with the appropriate program codes or software can implement the methods described herein. These processes together with the appropriate program codes or software can implement the methods described herein.

[0242] The present application can be embodied in other specific forms without departing from the spirit or essential characteristics thereof. The described examples are to be considered in all respects only as illustrative and not restrictive. The scope of the application is, therefore, indicated by the appended claims rather than by the foregoing description. All changes that come within the meaning of the claims are intended to be embraced therein.

Claims

1. A method for encoding and decoding a color image using an encoding / decoding tool, comprising one or more patterns related to a cross-component model, the method comprising: Receive input data related to the current block, which includes a current first color block and a current second color block, wherein the input data includes pixel data to be encoded at the encoder end or data related to the current block to be decoded at the decoder end, and the current block is encoded in a non-internal mode. Encode or decode the current block using the non-cross component prediction (CCP) mode; Determine one or more reference blocks to which one or more motion vectors or one or more block vectors of the current block point; as well as If the one or more reference blocks have target cross component model (CCM) information, the target CCM information is copied and stored in the current block, and is accessed by one or more subsequent blocks for CCP processing.

2. The method according to claim 1, characterized in that, If the current first color block or the co-located first color block is encoded or decoded in Inner Block Copy (IBC) or Inner Template Match Prediction (IntraTMP) mode, the one or more block vectors are associated with the current first color block or the co-located first color block.

3. The method according to claim 1, characterized in that, If the one or more reference blocks located based on the one or more block vectors or the one or more motion vectors are not CCP encoded, but have first CCM information stored for the one or more reference blocks, then the stored first CCM information is used as the target CCM information.

4. The method according to claim 1, characterized in that, If the one or more reference blocks correspond to multiple reference blocks and only one of the multiple reference blocks has first CCM information, then the first CCM information is used as the target CCM information.

5. The method according to claim 1, characterized in that, If multiple reference blocks have first CCM information, the target CCM information is derived by combining corresponding CCP models that are all or partly related to the multiple reference blocks.

6. The method according to claim 1, characterized in that, If multiple reference blocks have first CCM information, then the target CCM information is copied from the first CCM information of the target reference block of the multiple reference blocks according to one or more predefined rules.

7. The method according to claim 6, characterized in that, Select a CCP encoding / decoding reference block as the target reference block for the plurality of reference blocks.

8. The method according to claim 6, characterized in that, Select an internally encoded or externally encoded reference block as the target reference block for the plurality of reference blocks.

9. The method according to claim 6, characterized in that, Based on distance measurements, the reference block with the shortest distance to the current block is selected as the target reference block among the multiple reference blocks.

10. The method according to claim 9, characterized in that, This distance measurement corresponds to Euclidean distance, Manhattan distance, Minkowski distance, horizontal distance, or vertical distance.

11. The method according to claim 1, characterized in that, If multiple reference blocks have first CCM information, the first CCM information of the multiple reference blocks is applied to the reconstructed current first color block to generate multiple predictions for the current second color block.

12. The method according to claim 11, characterized in that, The distortion between the reconstructed current second color block and the multiple predictions is determined, and the target first CCM information associated with the target prediction with the least distortion is selected to be copied to the current block.

13. The method according to claim 1, characterized in that, If the current block is not CCP encoded and the current slice or picture containing the current block is a non-internal slice or picture, then the target CCM information is copied from the co-located block of the current block in the co-located picture.

14. The method according to claim 13, characterized in that, If the image sequence count (POC) distance between the co-located image and the current image containing the current block is greater than a predetermined threshold, then the target CCM information is not copied from the current block in the co-located image.

15. An apparatus for encoding and decoding color images or videos, using an encoding / decoding tool comprising one or more cross-component model correlation modes, the apparatus comprising one or more electronic circuits or processors configured to: Receive input data related to the current block, which includes a current first color block and a current second color block, wherein the input data includes pixel data to be encoded at the encoder end or data related to the current block to be decoded at the decoder end, and the current block is encoded in a non-internal mode. Encode or decode the current block using the non-cross component prediction (CCP) mode; Determine one or more reference blocks to which one or more motion vectors or one or more block vectors of the current block point; as well as If one or more reference blocks have target cross component model (CCM) information, the target CCM information is copied and stored in the current block, wherein the target CCM information stored in the current block is accessed by one or more subsequent blocks for CCP processing.

Images