Method and apparatus for improving transform information coding in video coding according to intra chroma cross-element prediction model

Abstract

A method and apparatus for video coding using a coding tool including one or more cross-element model correlation modes. In accordance with a method at a decoder end, codec syntax data for one or more syntax associated with transform codec or residual codec applied to a current block is parsed. In one embodiment, the one or more syntax data is decoded using one or more context models to generate the one or more syntax according to information related to Cross-Component Prediction (CCP for short), and the one or more syntax data is decoded using one or more context models to generate the one or more syntax. And decoding the current block by using the CCP, wherein the decoding comprises transform coding of the current block or residual coding of the current block by using the one or more syntax. At the encoder end, one or more syntax related to transform coding or residual coding is encoded to generate coding syntax data, and one or more context models are used according to information related to CCP.

Description

[0001]

Cross-reference

[0002] This application is a non-provisional application based on and claims priority to U.S. Provisional Patent Application No. 63 / 479,556, filed January 12, 2023, and U.S. Provisional Patent Application No. 63 / 487,633, filed March 1, 2023. The aforementioned U.S. Provisional Patent Applications are incorporated herein by reference in their entirety.

Technical field

[0003] The present invention relates to a video coding and decoding system, and in particular to a coding and decoding syntax for performing transform or residual coding and decoding using one or more context models based on information related to cross-element prediction in the video coding and decoding system. [Background Technology]

[0004] Versatile Video Coding (VVC) is the latest international video codec standard developed by the Joint Video Experts Team (JVET) of the ITU-T Video Coding Experts Group (VCEG) and the ISO / IEC Moving Picture Experts Group (MPEG). The standard was published as ISO standard ISO / IEC 23090-3:2021, Information technology — Coded representation of immersive media — Part 3: Versatile video codec, in February 2021. VVC builds on its predecessor, High Efficiency Video Coding (HEVC), by adding additional codec tools to improve codec efficiency and handle various video sources, including three-dimensional (3D) video signals.

[0005] Figure 1AAn exemplary adaptive inter / intra video coding system including loop processing is shown. For intra prediction 110, prediction data is derived based on previously encoded video data in the current picture. For inter prediction 112, motion estimation (ME) is performed at the encoder end, and motion compensation (MC) is performed based on the results of ME to provide prediction data derived from other pictures and motion data. Switch 114 selects intra prediction 110 or inter prediction 112, and the selected prediction data is provided to adder 116 to form a prediction error, also known as a residual. The prediction error is then processed by transform (T) 118 and then quantized (Q) 120. The transformed and quantized residual is then encoded by entropy encoder 122 for inclusion in the video bitstream corresponding to the compressed video data. The bitstream associated with the transform coefficients is then packaged with additional information such as motion and codec modes associated with intra and inter prediction, as well as other information such as parameters of the loop filter applied to the underlying image region. Additional information associated with intra prediction 110, inter prediction 112, and loop filter 130 is as follows: Figure 1A The image is provided to the entropy encoder 122. When using inter-frame prediction mode, the reference picture or pictures must also be reconstructed at the encoder end. Therefore, the transformed and quantized residual undergoes inverse quantization (IQ) 124 and inverse transformation (IT) 126 to recover the residual. The residual is then added back to the predicted data 136 at reconstruction (REC) 128 to reconstruct the video data. The reconstructed video data can be stored in the reference picture buffer (RPB) 134 and used for prediction of other frames.

[0006] like Figure 1AAs shown, the input video data undergoes a series of processing in the encoding system. The reconstructed video data from REC128 may be subjected to various impairments due to the series of processing. Therefore, a loop filter 130 is usually applied to the reconstructed video data before the reconstructed video data is stored in the reference picture buffer 134 to improve the video quality. For example, a deblocking filter (DF), a sample adaptive offset (SAO), and an adaptive loop filter (ALF) can be used. The loop filter information may need to be incorporated into the bitstream so that the decoder can correctly recover the required information. Therefore, the loop filter information is also provided to the entropy encoder 122 to be incorporated into the bitstream. In Figure 1A In , a loop filter 130 is applied to the reconstructed video before the reconstructed samples are stored in a reference picture buffer 134 . Figure 1A The system in Figure 1 is intended to illustrate an exemplary structure of a typical video encoder. It may correspond to a High Efficiency Video Coding (HEVC) system, VP8, VP9, ​​H.264, or VVC.

[0007] like Figure 1B As shown, the decoder can use the same or partially the same functional modules as the encoder, except for transform 118 and quantization 120, because the decoder only needs inverse quantization 124 and inverse transform 126. The decoder uses an entropy decoder (EntropyDecoder) 140 to decode the video bitstream into quantized transform coefficients and required codec information (such as ILPF information, intra-frame prediction information, and inter-frame prediction information). The intra-frame prediction 150 at the decoder side does not need to perform a pattern search. Instead, the decoder only needs to generate an intra-frame prediction based on the intra-frame prediction information received from the entropy decoder 140. In addition, for inter-frame prediction, the decoder only needs to perform motion compensation (MC 152) based on the inter-frame prediction information received from the entropy decoder 140, without the need for motion estimation.

[0008] According to VVC, the input image is divided into non-overlapping square areas called Coding Tree Units (CTUs), similar to HEVC. Each CTU can be divided into one or more smaller-sized Coding Units (CUs). The generated CU partitions can be square or rectangular in shape. In addition, VVC divides the CTU into Prediction Units (PUs), which serve as units for applying prediction processes such as inter-frame prediction, intra-frame prediction, etc.

[0009] The VVC standard incorporates various new coding and decoding tools to further improve coding and decoding efficiency over the HEVC standard. Some of the new tools related to the present invention are described below.

[0010] Use tree structure to divide CTUs

[0011] In HEVC, CTU is divided into CUs using a quadtree (QT) structure, called a codec tree, to accommodate various local characteristics. The decision on whether to use inter-frame (temporal) or intra-frame (spatial) prediction to encode a picture area is made at the leaf-CU level. Each leaf CU can be further divided into one, two or four PUs according to the PU partition type. Within a PU, the same prediction process is applied, and the relevant information is transmitted to the decoder on a PU basis. After obtaining the residual block after applying the prediction process according to the PU partition type, the leaf CU can be divided into transform units (TUs) according to another quadtree structure similar to the codec tree of the CU. A key feature of the HEVC structure is that it has a multiple partitioning concept that includes CU, PU and TU.

[0012] In VVC, the concept of multiple types of segmentation units is replaced by a segmentation structure using binary and ternary segmented quadtrees and nested multi-type trees. That is, in addition to the need to handle the maximum transform length of CUs that are too large, the separation of the concepts of CU, PU, ​​and TU is removed, and more flexibility in the shape of CU segmentation is supported. In the codec tree structure, CU can be square or rectangular. The codec tree unit (CTU) is first segmented by a quadtree (also known as a quadtree) structure. Then, the quadtree leaf nodes can be further segmented by a multi-type tree structure. Figure 2 As shown, there are four types of splits in the multi-type tree structure: vertical binary split (SPLIT_BT_VER 210), horizontal binary split (SPLIT_BT_HOR 220), vertical ternary split (SPLIT_TT_VER 230), and horizontal ternary split (SPLIT_TT_HOR 240). The leaves of the multi-type tree are called codec units (CUs). Unless the size of the CU is so large that it exceeds the maximum transform length, this split is used for prediction and transform processing without further splitting. This means that in most cases, the CU, PU, ​​and TU have the same block size in the quadtree codec block structure with nested multi-type trees. The exception occurs when the maximum supported transform length is less than the width or height of the color components of the CU.

[0013] Figure 3A signaling mechanism for splitting information in a quadtree codec tree structure with nested multi-type trees is presented. The codec tree unit (CTU) is considered the root of the quadtree and is first split through the quadtree structure. Each quadtree leaf node (when large enough) is then further split through the multi-type tree structure. In the quadtree codec tree structure with nested multi-type trees, for each CU node, a first flag (split_cu_flag) is signaled to indicate whether the node is further split. If the current CU node is a quadtree CU node, a second flag (split_qt_flag) is signaled to indicate whether it is QT split mode or MTT split mode. When the node is split in MTT split mode, a third flag (mtt_split_cu_vertical_flag) is signaled to indicate the split direction, and then a fourth flag (mtt_split_cu_binary_flag) is signaled to indicate whether the split is binary or ternary. According to the values ​​of mtt_split_cu_vertical_flag and mtt_split_cu_binary_flag, the multi-type tree split mode (MttSplitMode) of CU is shown in Table 1.

[0014] Table 1 – Derivation of MttSplitMode based on multi-type tree syntax elements

[0015] MttSplitMode mtt_split_cu_vertical_flag mtt_split_cu_binary_flag SPLIT_TT_HOR 0 0 SPLIT_BT_HOR 0 1 SPLIT_TT_VER 1 0 SPLIT_BT_VER 1 1

[0016] Figure 4 The figure shows a CTU being partitioned into multiple CUs with a quadtree and nested multi-type tree codec block structure, where bold block edges indicate quadtree partitioning and the remaining edges indicate multi-type tree partitioning. The quadtree with nested multi-type tree partitioning provides a content-adaptive codec tree structure composed of CUs. The CU size can be as large as the CTU or as small as 4×4 in units of luma samples. For the 4:2:0 chroma format, the maximum chroma CB size is 64×64, and the minimum size chroma CB consists of 16 chroma samples.

[0017] In VVC, the maximum supported luma transform size is 64 × 64, and the maximum supported chroma transform size is 32 × 32. When the width or height of the CB is larger than the maximum transform width or height, the CB is automatically split horizontally and / or vertically to meet the transform size limit in that direction.

[0018] The following parameters are defined for the quadtree codec tree scheme with nested multi-type trees. These parameters are specified by the Sequence Parameter Set (SPS) syntax element and can be further refined by the picture header syntax elements.

[0019] –CTU size: the root node size of the quadtree

[0020] –MinQTSize: Minimum allowed quad tree leaf node size

[0021] –MaxBtSize: Maximum allowed binary tree root node size

[0022] –MaxTtSize: Maximum allowed ternary tree root node size

[0023] –MaxMttDepth: The maximum allowed hierarchical depth of a multi-type tree split starting from a quadtree leaf node

[0024] –MinCbSize: The minimum allowed encoding block node size

[0025] In one example of a quadtree codec tree structure with nested multi-type trees, the CTU size is set to 128×128 luma samples with two corresponding 64×64 blocks of 4:2:0 chroma samples, MinQTSize is set to 16×16, MaxBtSize is set to 128×128, MaxTtSize is set to 64×64, MinCbsize (width and height) is set to 4×4, and MaxMttDepth is set to 4. Quadtree splitting is first applied to the CTU to generate quadtree leaf nodes. The size of the quadtree leaf node can range from 16×16 (i.e., MinQTSize) to 128×128 (i.e., CTU size). If the leaf QT node is 128×128, it will not be further split by the binary tree because the size exceeds MaxBtSize and MaxTtSize (i.e., 64×64). Otherwise, the leaf quadtree node can be further split by the multi-type tree. Therefore, the quad leaf node is also the root node of the multi-type tree, and its multi-type tree depth (mttDepth) is 0. When the multi-type tree depth reaches MaxMttDepth (i.e. 4), no further splits are considered. When the width of the multi-type tree node is equal to MinCbsize, no further horizontal splits are considered. Similarly, when the height of the multi-type tree node is equal to MinCbsize, no further vertical splits are considered.

[0026] In VVC, the codec tree scheme supports independent block tree structures for luma and chroma. For P and B slices, the luma and chroma CTBs in a CTU must share the same codec tree structure. However, for I slices, luma and chroma can have independent block tree structures. When independent block tree mode is applied, the luma CTB is divided into CUs through one codec tree structure, while the chroma CTBs are divided into chroma CUs through another codec tree structure. This means that a CU in an I slice may consist of a codec block for the luma component or a codec block for two chroma components, while a CU in a P or B slice always consists of codec blocks for all three color components unless the video is monochrome.

[0027] Virtual Pipe Data Units (VPDUs)

[0028] Virtual Pipe Data Units (VPDUs) are defined as non-overlapping units in an image. In a hardware decoder, multiple pipeline stages process consecutive VPDUs simultaneously. The size of a VPDU is roughly proportional to the buffer size in most pipeline stages, so it is important to keep the VPDU size small. In most hardware decoders, the VPDU size can be set to the maximum transform block (TB) size. However, in VVC, ternary tree (TT) and binary tree (BT) partitioning may lead to an increase in the size of VPDUs.

[0029] To keep the VPDU size to 64x64 luma samples, the following canonical partitioning restrictions (with accompanying syntax signaling modifications) are applied in the VTM, such as Figure 5 As shown:

[0030] For CUs with width or height, or width and height equal to 128, TT splitting is not allowed (e.g. Figure 5 (as indicated by the “X” in the figure).

[0031] For 128xN CU with N≤64 (i.e., width equals 128 and height less than 128), horizontal BT is not allowed.

[0032] For Nx128 CUs with N≤64 (i.e., height equal to 128 and width less than 128), vertical BT is not allowed. Figure 5 In the example, the luminance block size is 128x128. The dotted line indicates a block size of 64x64. Figure 5 As shown in the various examples (510-580), they are represented by "X".

[0033] Intra-frame chroma partitioning and prediction constraints

[0034] In typical hardware video encoders and decoders, when an image has small intra blocks, processing throughput decreases due to data dependencies between sample processing samples of adjacent intra blocks. Predictor generation for an intra block requires reconstructed samples from the top and left boundaries of neighboring blocks. Therefore, intra prediction must be processed sequentially, block by block.

[0035] In HEVC, the smallest intra CU is 8x8 luma samples. The luma component of the smallest intra CU can be further split into four 4x4 luma intra prediction units (PUs), but the chroma component of the smallest intra CU cannot be further split. Therefore, the worst-case hardware processing throughput occurs when processing a 4x4 chroma intra block or a 4x4 luma intra block. In VVC, to improve the worst-case throughput, the partitioning of chroma intra CBs is restricted, prohibiting chroma intra CBs with a width less than 16 chroma samples (sizes of 2x2, 4x2, and 2x4) and chroma intra CBs with a width less than 4 chroma samples (size of 2xN).

[0036] In a single codec tree, a minimum chroma intra prediction unit (SCIPU) is defined as a codec tree node with a chroma block size greater than or equal to 16 chroma samples and at least one luma sub-block smaller than 64 luma samples, or a codec tree node with a chroma block size other than 2xN and at least one luma sub-block of 4xN luma samples. Within each SCIPU, all CBs are required to be inter, or all CBs are non-inter, i.e., intra or intra block copy (IBC). In the case of a non-inter SCIPU, the chroma of the non-inter SCIPU must not be further split, while the luma of the SCIPU may be further split. This eliminates small chroma intra CBs smaller than 16 chroma samples or of size 2xN. Furthermore, chroma scaling is not applied in the case of a non-inter SCIPU. No additional syntax signaling is used; whether a SCIPU is non-inter can be inferred from the prediction mode of the first luma CB in the SCIPU. If the current slice is an I slice or the current SCIPU has a 4x4 luma partition after being further split once (because inter 4x4 is not allowed in VVC), the type of SCIPU is inferred to be non-inter; otherwise, the type of SCIPU (inter or non-inter) is indicated by a flag before parsing the CUs in the SCIPU.

[0037] For dual trees in intra pictures, 2xN intra chroma blocks are removed by disabling vertical binary and vertical ternary splits for 4xN and 8xN chroma partitions. Small chroma blocks of sizes 2x2, 4x2, and 2x4 are also removed by partition restriction.

[0038] In addition, image size restrictions are taken into account to avoid 2x2 / 2x4 / 4x2 / 2xN intra chroma blocks in the image corners, considering that the image width and height are multiples of max(8,MinCbSizeY).

[0039] The intra mode codec has 67 intra prediction modes.

[0040] To capture arbitrary edge directions present in natural videos, the number of directional intra modes in VVC is expanded from 33 used in HEVC to 65. New directional modes not present in HEVC are Figure 6 Indicated by the red dashed arrows in , while the planar and DC modes remain unchanged. These more dense directional intra prediction modes are applicable to all block sizes and for luma and chroma intra prediction.

[0041] In VVC, several traditional angular intra prediction modes are adaptively replaced with wide-angle intra prediction modes for non-square blocks.

[0042] In HEVC, each intra-coded block is square, with each side length being a power of 2. Therefore, no division operation is required to generate the intra predictor using the DC mode. In VVC, blocks can be rectangular, and in general, a division operation is required for each block. To avoid the division operation for DC prediction, only the longer side is used to calculate the average value for non-square blocks.

[0043] In order to keep the complexity of the most probable mode (MPM) list generation low, an intra mode codec with 6 MPMs is used, taking into account the two available adjacent intra modes. The following three aspects are considered when building the MPM list:

[0044] – Default intra mode

[0045] – Neighboring intra mode

[0046] – Derived intra mode.

[0047] Regardless of whether MRL and ISP codecs are applied, intra blocks use a unified 6-MPM list. The MPM list is constructed based on the intra modes of the neighboring blocks on the left and above. Assuming the mode on the left is denoted as Left and the mode of the block above is denoted as Above, the unified MPM list is constructed as follows:

[0048] When a neighboring block is not available, its intra mode is set to planar by default.

[0049] If both Left and Above modes are non-angle modes:

[0050] MPM List → {Planar, DC, V, H, V-4, V+4}

[0051] If one of Left and Above modes is angular and the other is non-angular:

[0052] Set the larger mode to Max

[0053] MPM list → {Planar, Max, Max-1, Max+1, Max-2, Max+2}

[0054] If Left and Above are both angle modes and are different:

[0055] Set the larger mode to Max

[0056] If Max-Min is equal to 1:

[0057] MPM List → {Planar, Left, Above, Min-1, Max+1, Min-2}

[0058] Otherwise, if Max-Min is greater than or equal to 62:

[0059] MPM List → {Planar, Left, Above, Min+1, Max-1, Min+2}

[0060] Otherwise, if Max-Min is equal to 2:

[0061] MPM List → {Planar, Left, Above, Min+1, Min-1, Max+1}

[0062] otherwise:

[0063] MPM List → {Planar, Left, Above, Min-1, Min+1, Max-1}

[0064] If Left and Above are both in angle mode and are the same:

[0065] MPM list → {Planar, Left, Left-1, Left+1, Left-2, Left+2}

[0066] In addition, the first bin of the MPM index codeword is CABAC context coded. A total of three contexts are used, corresponding to whether the current intra block is enabled with MRL, ISP or a normal intra block.

[0067] During the 6MPM list generation process, pruning is used to remove repeated patterns so that only unique patterns are included in the MPM list. For entropy coding and decoding of the 61 non-MPM patterns, truncated binary code (TBC) is used.

[0068] Wide-angle intra prediction for non-square blocks

[0069] Traditional angular intra prediction directions are defined as running clockwise from 45 degrees to -135 degrees. In VVC, several traditional angular intra prediction modes are adaptively replaced with wide-angle intra prediction modes for non-square blocks. The replaced modes are signaled using the original mode indices, which are then parsed and remapped to the wide-angle mode indices. The total number of intra prediction modes remains unchanged at 67, and the intra mode encoding and decoding method remains unchanged.

[0070] To support these prediction directions, a top reference of length 2W+1 and a left reference of length 2H+1 are defined, as Figure 7A and Figure 7B shown.

[0071] The number of alternative modes in the wide angle direction mode depends on the aspect ratio of the block. The alternative intra prediction modes are described in Table 2.

[0072] Table 2 – Intra prediction modes replaced by wide angle modes

[0073]

[0074]

[0075] In VVC, 4:2:2 and 4:4:4 chroma formats are supported, as well as 4:2:0. The chroma derivation mode (DM) derivation table for the 4:2:2 chroma format was originally ported from HEVC, with the number of entries expanded from 35 to 67 to align with the expansion of intra prediction modes. Since the HEVC specification does not support prediction angles below -135 degrees and above 45 degrees, the range of luma intra prediction modes is mapped from 2 to 5 to 2. Therefore, the chroma DM derivation table for the 4:2:2 chroma format is updated by replacing some values ​​of the mapping table entries to more accurately convert the prediction angles of chroma blocks.

[0076] Cross-Component Linear Model (CCLM) prediction

[0077] In order to reduce cross-element redundancy, VVC uses a cross-element linear model (CCLM) prediction mode, in which the chroma samples are predicted using a linear model based on the reconstructed luminance samples of the same CU, as shown below:

[0078] pred C (i,j)=α·rec L ′(i,j)+β(1)

[0079] In a coding unit (CU), pred C (i,j) represents the predicted chrominance sample, rec L′(i,j) represents the downsampled reconstructed luma sample of the same CU.

[0080] The CCLM parameters (α and β) are derived from up to four neighboring chroma samples and their corresponding downsampled luma samples. Assuming the size of the current chroma block is W×H, when CCLM_LT mode is applied, W' and H' are set to

[0081] W'=W,H'=H;

[0082] When CCLM_T mode is applied, W'=W+H;

[0083] When CCLM_L mode is applied, H'=H+W.

[0084] The adjacent positions above are represented as S[0,-1]…S[W'-1,-1], and the adjacent positions on the left are represented as S[-1,0]…S[-1,H'-1]. Then four samples are selected as

[0085] – When CCLM_LT mode is applied and both the above and left neighbor samples are available, select S[W' / 4,

[0086] -1], S[3*W' / 4,-1], S[-1,H' / 4], S[-1,3*H' / 4];

[0087] – When CCLM_T mode is applied or only the above neighboring samples are available, select S[W' / 8,-1], S[3*W' / 8,-1], S[5*W' / 8,-1], S[7*W' / 8,-1];

[0088] – When CCLM_L mode is applied or only left neighbor samples are available, select S[-1,H' / 8], S[-1,3*H' / 8], S[-1,5*H' / 8], S[-1,7*H' / 8].

[0089] The four neighboring brightness samples at the selected position are downsampled and compared four times to find the two larger values: x 0 A and x 1 A , and two smaller values: x 0 B and x 1 B The corresponding chrominance sample value is represented by y 0 A ,y 1 A ,y 0 B and y 1 B Then x A, x B ,y A and y B is derived as:

[0090] x A =(x 0 A +x 1 A +1)>>1;

[0091] x B =(x 0 B +x 1 B +1)>>1;

[0092] y A =(y 0 A +y 1 A +1)>>1;

[0093] y B =(y 0 B +y 1 B +1)>>1 (2)

[0094] Finally, the linear model parameters α and β are obtained according to the following equations.

[0095]

[0096] β=y B -α·x B (4)

[0097] Figure 8 An example of the positions of the left and above samples and the current block samples involved in CCLM_LT mode is shown. Figure 8 The relative sample positions of an NxN chroma block 810, a corresponding 2Nx2N luma block 820, and their neighboring samples (shown as filled circles) are shown.

[0098] The division operation for calculating the parameter α is implemented using a lookup table. To reduce the memory required to store the table, the difference (the difference between the maximum and minimum values, diff) and the parameter α are expressed as exponents. For example, the difference is approximated using a 4-bit significant part and an exponent. Therefore, the table for 1 / diff is reduced to 16 elements, suitable for 16 significant values, as shown below:

[0099] DivTable[]={0,7,6,5,5,4,4,3,3,2,2,1,1,1,1,0} (5)

[0100] This will help reduce the complexity of the calculations as well as the memory size required to store the required tables.

[0101] In addition to the templates above and on the left being used together to calculate linear model coefficients, they can also be used alternately in two other LM modes, called CCLM_T and CCLM_L modes.

[0102] In CCLM_T mode, only the upper template is used to calculate the linear model coefficients. To obtain more samples, the upper template is expanded to (W+H) samples. In CCLM_L mode, only the left template is used to calculate the linear model coefficients. To obtain more samples, the left template is expanded to (H+W) samples.

[0103] In CCLM__LT mode, the left and upper templates are used to calculate the linear model coefficients.

[0104] To match the chroma sample positions of a 4:2:0 video sequence, two types of downsampling filters are applied to the luma samples to achieve a 2 to 1 downsampling ratio in both the horizontal and vertical directions. The choice of downsampling filter is specified by the SPS level flag. These two downsampling filters correspond to "type-0" and "type-2" content, respectively.

[0105] Rec L ′(i,j)=[rec L (2i-1,2j-1)+2·rec L (2i,2j-1)+rec L (2i+1,2j-1)+rec L (2i-1,2j)+2·rec L (2i,2j)+rec L (2i+1,2j)+4]>>3 (6)

[0106] Rec L ′(i,j)=rec L (2i,2j-1)+rec L (2i-1,2j)+4·rec L (2i,2j)+rec L (2i+1,2j)+rec L (2i,2j+1)+4]>>3 (7)

[0107] Note that when the upper reference line is at a CTU boundary, only one luma line (generic line buffer in intra prediction) is used to generate the downsampled luma samples.

[0108] This parameter calculation is performed as part of the decoding process, not just as an encoder search operation. Therefore, no syntax is used to pass the α and β values ​​to the decoder.

[0109] For chroma intra mode encoding and decoding, a total of 8 intra modes are allowed for chroma intra mode encoding and decoding. These modes include five traditional intra modes and three cross-element linear model modes (CCLM_LT, CCLM_T, and CCLM_L). The chroma mode signal transmission and derivation process are shown in Table 3. Chroma mode encoding and decoding directly depends on the intra prediction mode of the corresponding luminance block. Since the independent block partitioning structure of luminance and chrominance elements is enabled in the I slice, one chroma block may correspond to multiple luminance blocks. Therefore, for the chroma DM mode, the intra prediction mode of the corresponding luminance block covering the center position of the current chroma block is directly inherited.

[0110] Table 3 Derivation of chroma prediction mode from luma mode when CCLM is enabled

[0111]

[0112] Regardless of the value of sps_cclm_enabled_flag, a single binarization table is used, as shown in Table 4.

[0113] Table 4 – Unified binarization table for chroma prediction mode

[0114]

[0115]

[0116] In Table 4, the first bin indicates whether it is normal mode (0) or CCLM mode (1). If it is LM mode, the next bin indicates whether it is CCLM_LT (0) or not. If it is not CCLM_LT, the next bin indicates whether it is CCLM_L (0) or CCLM_T (1). For this case, when sps_cclm_enabled_flag is 0, the first bin of the corresponding intra_chroma_pred_mode binarization table can be discarded before entropy coding and decoding. In other words, the first bin is inferred to be 0 and therefore not encoded. This single binarization table is used for the cases where sps_cclm_enabled_flag is equal to 0 and 1. The first two bins in Table 4 are context coded using their own context model, and the remaining bins are bypass coded.

[0117] Additionally, to reduce luma-chroma latency in dual trees, CCLM is allowed for chroma CUs in 32x32 / 32x16 chroma codec tree nodes when the 64x64 luma codec tree node is not split (and ISP is not used for 64x64 CUs) or when QT is split, as follows:

[0118] – If a 32x32 chroma node is not split or split into QT splits, all chroma CUs in the 32x32 node can use CCLM

[0119] – If a 32x32 chroma node is split into horizontal BT and the 32x16 child node is not split or uses vertical BT split, all chroma CUs in the 32x16 chroma node can use CCLM.

[0120] Under all other luma and chroma codec tree split conditions, chroma CUs are not allowed to use CCLM.

[0121] Multi-model CCLM (MMLM)

[0122] In JEM (J. Chen, E. Alshina, G. J. Sullivan, J.-R. Ohm, and J. Boyce, Algorithmic Description of Joint Exploration Test Model 7, document JVET-G1001, ITU-T / ISO / IEC Joint Video Exploration Team (JVET), July 2017), a multi-model CCLM mode (MMLM) is proposed for predicting the chrominance samples of the entire CU from the luma samples using two models. In MMLM, the neighboring luma samples and the neighboring chroma samples of the current block are classified into two groups, each group is used as a training set to derive a linear model (i.e., a specific α and β are derived for a specific group). In addition, the samples of the current luma block are also classified based on the same rules to classify the neighboring luma samples. Three MMLM model modes (MMLM_LT, MMLM_T, and MMLM_L) are allowed to select neighboring samples from the left and top, only the top, and only the left, respectively.

[0123] Figure 9 An example of classifying neighboring samples into two groups is shown. The threshold is calculated as the average of the neighboring reconstructed luminance samples. L Neighboring samples with [x,y]<=threshold are classified as group 1; and those with Rec′ L Neighboring samples with [x,y]>threshold are classified into group 2.

[0124]

[0125] Therefore, MMLM uses two models according to the sample level of neighboring samples.

[0126] Slope adjustment of CCLM

[0127] CCLM uses a two-parameter model to map luminance values ​​to chrominance values, such as Figure 10A The slope parameter "a" and the bias parameter "b" define the mapping as follows:

[0128] chromaVal=a*lumaVal+b

[0129] The adjustment of the slope parameter "u" is transmitted by signal updating model in the following form, such as Figure 10B As shown:

[0130] chromaVal=a'*lumaVal+b'

[0131] in

[0132] a'=a+u,

[0133] b'=bu*y r .

[0134] With this choice, the mapping function is centered around the brightness value y r The average value of the reference brightness samples used for model creation is y r , so that meaningful modifications can be made to the model. Figure 10A and 10B The process is explained.

[0135] Implementation of CCLM slope adjustment

[0136] The slope adjustment parameter is provided as an integer between -4 and 4 and is signaled in the bitstream. The unit of the slope adjustment parameter is (1 / 8) chroma sample value per luma sample value (for 10-bit content).

[0137] The adjustment applies to CCLM models that use reference samples from both above and to the left of the block (e.g., LM_CHROMA_IDX and MMLM_CHROMA_IDX), but not to "one-sided" modes. This choice is based on a trade-off between codec efficiency and complexity. LM_CHROMA_IDX and MMLM_CHROMA_IDX are referred to as CCLM_LT and MMLM_LT in this disclosure. "One-sided" modes are referred to as CCLM_L, CCLM_T, MMLM_L, and MMLM_T in this disclosure.

[0138] When slope adjustment is applied to a multi-mode CCLM model, two models may be adjusted, so a maximum of two slope updates are signaled for a single chroma block.

[0139] Encoder method for CCLM slope adjustment

[0140] The proposed encoder method performs a SATD (Sum of Absolute Transform Differences)-based search to find the optimal slope update value for Cr, and a similar SATD-based search for Cb. If either result is a non-zero slope adjustment parameter, the combined slope adjustment pair (SATD-based update for Cr, SATD-based update for Cb) is included in the RD (rate-distortion) check list of the TU.

[0141] Convolutional Cross-Component Model (CCCM) - Single Model and Multi-Model

[0142] In CCCM, a convolutional model is applied to improve chrominance prediction performance. The convolutional model has a 7-tap filter consisting of a 5-tap plus-shaped spatial component, a nonlinear term, and a bias term. The input to the spatial 5-tap component of the filter includes the center (C) luma sample co-located with the chroma sample to be predicted and its upper / north (N), lower / south (S), left / west (W), and right / east (E) neighbors, such as Figure 11 shown.

[0143] The nonlinear term (denoted as P) is expressed as the square of the center luminance sample C, scaled to the sample value range of the content:

[0144] P=(C*C+midVal)>>bitDepth.

[0145] For example, for 10-bit content, the nonlinear term is calculated as:

[0146] P=(C*C+512)>>10

[0147] The bias term (denoted as B) represents a scalar offset between the input and output (similar to the offset term in CCLM) and is set to the middle chroma value (512 for 10-bit content).

[0148] The output of the filter is calculated as the filter coefficient c i Convolution between the input and the input, clipped to the range of valid chroma samples:

[0149] predChromaVal=c0C+c1N+c2S+c3E+c4W+c5P+c6B

[0150] Filter coefficient c i Computed by minimizing the MSE between the predicted and reconstructed chroma samples in the reference region. Figure 12 An example of a reference region is shown, consisting of 6 rows of chroma samples above and to the left of the PU. The reference region is extended by one PU width to the right and one PU height downward. The region is adjusted to include only available samples. The extension of the region (denoted as "padding") is to support Figure 11"Side samples" of the plus-shaped spatial filter in the middle, and filling in the unusable areas.

[0151] Minimizing the mean squared error (MSE) is achieved by calculating the autocorrelation matrix of the luma input and the cross-correlation vector between the luma input and the chroma output. The autocorrelation matrix undergoes an LDL decomposition, and the final filter coefficients are calculated by back substitution. This process roughly follows the calculation of the ALF filter coefficients in ECM, but LDL decomposition is chosen instead of Cholesky decomposition to avoid the use of square root operations. In newer ECMs, MSE minimization for CCM uses a method based on Gaussian elimination.

[0152] In addition, similar to CCLM, CCCM has the option of single model or multi-model variants. The multi-model variant uses two models, one model for samples above the average brightness reference value and another model for the remaining samples (following the spirit of CCLM design). For PUs with at least 128 reference samples available, the multi-model CCCM mode can be selected.

[0153] Gradient Linear Model (GLM)

[0154] Compared to CCLM, GLM uses the gradient of luma samples to infer the linear model instead of the downsampled luma values. Specifically, when applying GLM, the input of the CCLM process, i.e., the downsampled luma samples L, is replaced by the luma sample gradient G. The other parts of CCLM (e.g., parameter inference, linear transformation of prediction samples) remain unchanged.

[0155] C=α·G+β

[0156] For signal transmission, when CCLM mode is enabled for the current CU, two flags are transmitted for Cb and Cr components respectively to indicate whether GLM is enabled for each component. If GLM is enabled for a component, a syntax element is further transmitted to select the 4 gradient filters ( Figure 13 The GLM can be used in conjunction with the existing CCLM by transmitting an additional flag in the bitstream. When this combination is applied, the filter coefficients used to derive the linear model input luma samples are calculated as a combination of the gradient filter selected by the GLM and the downsampling filter of the CCLM.

[0157] Spatial candidate derivation

[0158] The spatial merge candidate derivation in VVC is the same as that in HEVC, except that the positions of the first two merge candidates are swapped. 0, A 0, B1 and A1) from Figure 14 The order of derivation is B0, A 0, B 1, A1 and B2. Position B2 is considered only when one or more neighboring CUs of positions B0, A0, B1, A1 are not available (for example, belong to another slice or tile) or are intra-coded. After the candidate at position A1 is added, the remaining candidates are added with a redundancy check to ensure that candidates with the same motion information are excluded from the list, thereby improving encoding and decoding efficiency. To reduce computational complexity, not all possible candidate pairs are considered in the above redundancy check. Instead, only the Figure 15 The pairs are connected by arrows in , and candidates are added to the list only if the corresponding candidates used for redundancy checking do not have the same motion information.

[0159] Time candidate derivation

[0160] In this step, only one candidate is added to the list. Specifically, in the temporal merge candidate derivation of the current CU 1610, a scaled motion vector is derived based on the co-located CU 1620 belonging to the co-located reference image, such as Figure 16 The reference image list and reference index used for co-located CU derivation are explicitly transmitted in the slice header. The scaled motion vector 1630 of the temporal merge candidate is shown in Figure 16 As shown by the dashed line in , it is obtained by scaling the motion vector 1640 of the co-located CU using the POC (Picture Order Count) distances tb and td, where tb is defined as the POC difference between the reference image of the current image and the current image, and td is defined as the POC difference between the reference image of the co-located image and the co-located image. The reference image index of the temporal merging candidate is set to zero.

[0161] The position of the temporal candidate is selected between candidates C0 and C1, such as Figure 17 If the CU at position C0 is not available, is intra-coded, or is outside the current CTU row, position C1 is used. Otherwise, position C0 is used in the derivation of temporal merge candidates.

[0162] Non-adjacent spatial candidates

[0163] During the development of the VVC standard, a coding tool called non-adjacent motion vector prediction (NAMVP) was proposed, see JVET-L0399 (Yu Han et al., "CE4.4.6: Improvement on Merge / Skipmode", Joint Video Exploration Team (JVET) of ITU-T SG16WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11, 12th Meeting: Macau, CN, October 3-12, 2018, document: JVET-L0399). According to the NAMVP technology, non-adjacent spatial merge candidates are inserted after TMVP (i.e., temporal MVP) in the regular merge candidate list. The mode of spatial merge candidates is as follows: Figure 18 The distance between non-adjacent spatial candidates and the current codec block is based on the width and height of the current codec block. Figure 18 In [1], each small square corresponds to a NAMVP candidate, and the candidates are sorted by distance (as indicated by the numbers inside the squares). No line buffer restrictions apply. In other words, NAMVP candidates far from the current block may need to be stored, which may require a larger buffer.

[0164] In the present invention, a method and apparatus for improving cross-component predictive coding and decoding performance are disclosed. [Summary of the invention]

[0165] A method and apparatus for video encoding and decoding, using a codec tool including one or more cross-component model-related modes. According to a decoder-side method, codec data related to a current block is received, including a first color block and a second color block, wherein the current block is encoded and decoded using cross-component prediction (Cross-Component Prediction, CCP). Codec syntax data of one or more syntaxes related to transform coding or residual coding applied to the current block is parsed. Based on information related to the CCP, the codec syntax data is decoded using one or more context models to generate the one or more syntaxes. The current block is decoded using the CCP, wherein the decoding includes transform coding or residual coding of the current block using the one or more syntaxes.

[0166] In one embodiment, the one or more syntaxes are from the group consisting of root_cbf, cb_cbf, cr_cbf, transform_skip_flag, lfnst_idx, mts_idx, tu_joint_cbcr_residual_flag, last_significant_position, cu_qp_delta_abs, cu_qp_delta_sign_flag, cu_chroma_qp_offset_flag, cu_chroma_qp_offset_idx, or combinations thereof.

[0167] In one embodiment, the one or more context models correspond to a plurality of context models, and selection among the plurality of context models depends on the CCP type of the current block. In another embodiment, selection among the plurality of context models depends on the prediction mode of a neighboring block of the current block. For example, the prediction mode of the neighboring block corresponds to a CCP mode. In another embodiment, selection among the plurality of context models depends on the number of neighboring blocks encoded or decoded in the CCP mode.

[0168] In one embodiment, the one or more context models correspond to a CCP single model, a CCP multi-model, a convolutional cross-component model (CCCM), a gradient linear model (GLM), or a combination thereof.

[0169] A method corresponding to the encoder side is also disclosed.

[0170] According to another method, a current block is encoded or decoded using a CCP, wherein the encoding or decoding uses information including chroma syntax corresponding to one or more codec block flags, the one or more codec block flags including a root chroma codec block flag.

[0171] In one embodiment, the CCP candidates inherited from the neighboring blocks are included in the merge candidate list of the current block, and the merge flag indicates whether the current block inherits one or more cross-component model parameters from the inherited CCP candidates.

[0172] In one embodiment, the one or more codec block flags further include two or more element codec block flags corresponding to the two or more chroma blocks. In one embodiment, if a root chroma codec block flag is false, each element codec block flag corresponding to the two or more chroma blocks is equal to 0; or if a root chroma codec block flag is true, at least one of the two or more element codec block flags corresponding to the two or more chroma blocks is equal to 1.

[0173] In one embodiment, the one or more codec block flags further include two or more element codec block flags corresponding to the two or more chroma blocks. In one embodiment, if a root chroma codec block flag is false, each element codec block flag corresponding to the two or more chroma blocks is equal to 0; or if a root chroma codec block flag is true, at least one of the two or more element codec block flags corresponding to the two or more chroma blocks is equal to 1.

[0174] According to another method, a merge candidate list is generated for a current block, wherein the merge candidate list includes CCP candidates inherited from neighboring blocks. A merge flag is signaled or parsed to indicate whether the current block inherits one or more cross-component model parameters from the inherited CCP candidates. A skip flag is signaled or parsed before the merge flag, wherein the merge flag is inferred when the skip flag is true. The current block is encoded or decoded using information including the merge candidate list.

[0175] In one embodiment, when the skip flag is true, an index associated with the inherited CCP candidate is signaled or parsed. In one embodiment, when the merge flag is inferred to be 1, the current block inherits one or more cross-element model parameters from the inherited CCP candidate. In one embodiment, when the merge flag is inferred to be 0, a CCP mode index is signaled or parsed.

[0176] In one embodiment, a high-level syntax is signaled or parsed in a Sequence Parameter Set (SPS), Picture Parameter Set (PPS), Picture Header (PH), or Slice Header (SH) to indicate whether the skip flag is allowed for the current sequence, picture, or slice.

Brief Description of the Drawings

[0177] Figure 1A An adaptive inter-frame / intra-frame video encoding and decoding system including loop processing is exemplarily presented.

[0178] Figure 1B Shown Figure 1A The corresponding decoder of the encoder in .

[0179] Figure 2 Examples of multi-type tree structures corresponding to vertical binary split (SPLIT_BT_VER), horizontal binary split (SPLIT_BT_HOR), vertical ternary split (SPLIT_TT_VER), and horizontal ternary split (SPLIT_TT_HOR) are shown.

[0180] Figure 3 An example of the signaling mechanism for partition split information in a quadtree with a nested multi-type tree codec tree structure is presented.

[0181] Figure 4 Shows an example of a CTU being partitioned into multiple CUs using quadtree and nested multi-type tree codec block structures, where bold block edges represent quadtree partitions and the remaining edges represent multi-type tree partitions.

[0182] Figure 5 Shows some examples of TT splitting that is prohibited when the width or height of the luma codec block is greater than 64.

[0183] Figure 6 Demonstrates the intra-frame prediction mode adopted by the VVC video codec standard.

[0184] Figure 7A -B shows an example of wide-angle intra prediction, where Figure 7A For blocks whose width is greater than their height, Figure 7B A block whose height is greater than its width.

[0185] Figure 8 Shows an example of the positions of the left and above samples and the current block sample involved in CCLM_LT mode.

[0186] Figure 9 An example of classifying neighboring samples into two groups is shown.

[0187] Figure 10A An example of the CCLM model is shown.

[0188] Figure 10B An example of the effect of the slope tuning parameter “u” for model updating is shown.

[0189] Figure 11 Shows an example of the spatial part of a convolutional filter.

[0190] Figure 12 Shows an example of a padded reference region used to derive filter coefficients.

[0191] Figure 13The four gradient modes of the Gradient Linear Model (GLM) are shown.

[0192] Figure 14 Shows the neighboring blocks used to derive VVC spatial merge candidates.

[0193] Figure 15 Possible candidate pairs for redundancy checking in VVC are shown.

[0194] Figure 16 An example of temporal candidate derivation is shown, where scaled motion vectors are derived based on the POC (Picture Order Count) distance.

[0195] Figure 17 The positions of the temporal candidates selected between candidates C0 and C1 are shown.

[0196] Figure 18 An exemplary pattern of non-adjacent spatial merging candidates is shown.

[0197] Figure 19 An example of CCM information propagation is shown, where the dashed blocks (i.e., A, E, G) are encoded in cross-element patterns (e.g., CCLM, MMLM, GLM, CCCM).

[0198] Figure 20 An example showing the inheritance of temporal proximity model parameters is shown.

[0199] Figure 21A -B shows two search modes that inherit the non-adjacent spatial proximity model.

[0200] Figure 22A -B shows the history table of regions starting from the same starting geometry as the current region. Figure 22A ) or from a history table of regions containing the geometric location of the current region center ( Figure 22B ) Example of building a history table for the current region.

[0201] Figure 23 An example of mapping motion information of a reference position in an unavailable area to a predefined position is shown, where the predefined position is located in the row above the first CTU row.

[0202] Figure 24 An example of mapping motion information of a reference position in an unavailable area to a predefined position is shown, where the predefined position is located at the bottom row of the corresponding CTU row.

[0203] Figure 25An example of mapping motion information of a reference position in an unavailable area to a predefined position is shown, where the predefined position is located at the bottom row or the center row of the corresponding CTU row.

[0204] Figure 26 An example of mapping motion information of a reference position in an unavailable area to a predefined position is shown, where the predefined position is located at the bottom row of the corresponding CTU row or the previous CTU row of the corresponding CTU row.

[0205] Figure 27 Shows an example of a neighborhood template used to calculate model error.

[0206] Figure 28 An example of inheriting candidates from candidates in a candidate list of a neighboring block is shown.

[0207] Figure 29 Shows an example of subsampling inter-codec or CCM information at the top left position of each 2x2 grid in the CTU-level buffer before the information is saved to the picture-level buffer.

[0208] Figure 30 An embodiment of the present invention is shown in which if the skip flag is true, the merge flag is inferred to be 1, and both the CBF of the Cb component and the CBF of the Cr component are inferred to be 0.

[0209] Figure 31 An embodiment of the present invention is shown in which if the skip flag is true, the merge flag is inferred to be 1, and the CBF of the Cb component and the CBF of the Cr component are both inferred to be 0.

[0210] Figure 32 A flow chart illustrating an exemplary video decoding system for decoding syntax data using one or more context models based on information related to cross-element prediction is provided, according to one embodiment of the present invention.

[0211] Figure 33 A flow chart illustrating an exemplary video encoding system for encoding syntax data using one or more context models based on information related to cross-element prediction, according to one embodiment of the present invention.

[0212] Figure 34 A flow chart illustrating an exemplary video encoding and decoding system according to an embodiment of the present invention is provided for encoding or decoding a current block using information including chroma syntax corresponding to a coded block flag shared by one or more chroma blocks.

[0213] Figure 35A flow chart of an exemplary video coding system according to an embodiment of the present invention is shown, which uses a merge flag to indicate whether a current block inherits one or more cross-element model parameters from an inherited CCP candidate. [Specific implementation method]

[0214] It will be readily understood that the components of the present invention, as generally described and illustrated in the figures, may be arranged and designed in a variety of different configurations. Accordingly, the following more detailed description of embodiments of the present systems and methods, as illustrated, is not intended to limit the scope of the claimed invention, but is merely representative of selected embodiments of the present invention. References throughout this specification to "one embodiment," "an embodiment," or similar language mean that a particular feature, structure, or characteristic may be included in at least one embodiment of the present invention in the description relating to that embodiment. Therefore, the phrases "in one embodiment" or "in an embodiment" appearing throughout this specification are not necessarily all referring to the same embodiment.

[0215] Furthermore, the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. However, one skilled in the relevant art will recognize that the present invention may be practiced without one or more of the specific details, or using other methods, components, etc. In other cases, well-known structures or operations are not shown or described in detail to avoid obscuring aspects of the invention. The embodiments of the present invention will be best understood by reference to the accompanying drawings, in which like parts are numbered throughout. The following description is by way of example only and illustrates only certain selected apparatus and method embodiments consistent with the invention claimed herein.

[0216] In order to improve the encoding and decoding performance of cross-element prediction, various schemes are disclosed.

[0217] In ECM, various cross-component prediction (CCP) methods are proposed to improve the efficiency of intra-frame chroma coding and decoding. For example,

[0218] -CCLM_L,CCLM_T,CCLM_LT,MMLM_L,MMLM_T,MMLM_LT

[0219] -CCCM_L,CCCM_T,CCCM_LT,MMCCCM_L,MMCCCM_T,MMCCCM_LT

[0220] -GLM_L,GLM_T,GLM_LT,MMGLM_L,MMGLM_T,MMGLM_LT

[0221] -GLM2_L,GLM2_T,GLM2_LT,MMGLM2_L,MMGLM2_T,MMGLM2_LT (GLM2 is a GLM with a corresponding brightness term)

[0222] - For each GLM mode, 4 gradient kernels can be selected.

[0223] With the help of these cross-element models, intra-frame chrominance prediction becomes more accurate and the entropy of the residual becomes smaller. Based on this observation, the present invention proposes the following method to improve the encoding and decoding performance of intra-frame chrominance prediction:

[0224] Improved CBF codec

[0225] In previous codec standards, because intra codecs are generally not perfectly predictable, the root_cbf (i.e., indicating whether any transform block for all color components has transform coefficients) is not signaled when the current block is coded in intra mode. However, by using various cross-element models that utilize luma reconstructed samples as model input, chroma components can be predicted more accurately, and the probability of the CBF (codec block flag) or coding flag equal to 0 for all chroma components in intra codec mode is increasing. Based on this assumption, various methods have been proposed to improve CBF codec efficiency.

[0226] In one embodiment, the root_cbf syntax may be used for intra-coded blocks to indicate whether any transform blocks for each color component have any transform coefficients.

[0227] In one embodiment, a new flag, root_chroma_cbf, is proposed to indicate whether any transform block of a chroma component has any transform coefficients. For example, if the root_chroma_cbf flag is true, it means that at least one of the Cb or Cr components has a CBF flag equal to 1. Therefore, when the CBF of the Cb component is 0, the CBF of the Cr component is implicitly inferred to be 1 and not signaled. Otherwise, if the root_chroma_cbf flag is false, it means that both the CBF of the Cb component and the CBF of the Cr component are equal to 0.

[0228] Improved transform and residual coding by adding context models for intra chroma CCP mode and inherited cross-component models

[0229] Since intra chroma prediction using CCP mode can be more accurate than using non-CCP mode, and the inherited cross-element model is also a CCP mode, the syntax in transform codecs and residual codecs can be improved by adding one or more context models and / or additional context models based on CCP mode related information, depending on whether the current block uses the inherited cross-element model.

[0230] In one embodiment, syntax related to transform coding and residual coding may include, but is not limited to, root_cbf, CBF of the Cb component, CBF of the Cr component, transform_skip_flag, lfnst_idx, mts_idx, tu_joint_cbcr_residual_flag, last_significant_position, cu_qp_delta_abs, cu_qp_delta_sign_flag, cu_chroma_qp_offset_flag and / or cu_chroma_qp_offset_idx.

[0231] In one embodiment, all syntax elements in transform and residual codecs may have one or more context models based on CCP mode related information. In another embodiment, some syntax elements in transform and residual codecs may have one or more context models based on CCP mode related information. In another embodiment, all syntax elements in transform and residual codecs may have additional context models based on whether the current block uses an inherited cross-element model.

[0232] In another embodiment, some syntax elements in transform coding and residual coding may have one or more context models depending on whether the current block uses an inherited cross-element model.

[0233] In one embodiment, when the current block is coded in intra mode and the intra prediction mode is one of all CCP modes or an inherited cross-element model, an additional context model of syntax related to transform codec and residual codec is used.

[0234] In another embodiment, when the current block is coded in intra mode and the intra prediction mode is one of a subset of all CCP modes, an additional context model of syntax related to transform codec and residual codec is used.

[0235] In another embodiment, there are multiple additional context models, and the decision of the context model can depend on the CCP type of the current codec block. For example, if the current block is encoded and decoded by the CCP single model, the first additional context model is used; if the current block is encoded and decoded by the CCP multi-model, the second additional context model is used. Another example is that if the current block is encoded and decoded by the LM model, the first additional context model is used; if the current block is encoded and decoded by the Convolutional Cross-Component Mode (CCCM), the second additional context model is used; if the current block is encoded and decoded by the Gradient Linear Model (GLM), the third additional context model is used.

[0236] In another embodiment, there are multiple additional context models, and the context model decision of the syntax related to transform coding and residual coding can be related to the prediction mode of the adjacent codec block. For example, if only the upper block is coded and decoded in CCP mode, the first additional context model is used; if only the left block is coded and decoded in CCP mode, the second additional context model is used; if both the upper block and the left block are coded and decoded in CCP mode, the third additional context model is used.

[0237] In another embodiment, there are multiple additional context models, and the context model decision can depend on the number of neighboring blocks encoded and decoded in CCP mode. For example, if the number of neighboring blocks encoded and decoded in CCP mode is equal to the first number, the first additional context model is used; if the number of neighboring blocks encoded and decoded in CCP mode is equal to the second number, the second additional context model is used, and so on.

[0238] In order to improve the prediction accuracy or encoding and decoding performance of cross-element prediction, various solutions related to inheriting cross-element models are disclosed.

[0239] Guided parameter sets for refining cross-component model parameters

[0240] According to this approach, a guided parameter set is used to refine the derived model parameters using a specified CCLM mode. For example, the guided parameter set is explicitly signaled in the bitstream, and after the model parameters are derived, the guided parameter set is added to the derived model parameters as the final model parameters. The guided parameter set contains at least a differential scaling parameter (dA), a differential offset parameter (dB), and a differential shift parameter (dS). For example, equation (1) can be rewritten as:

[0241] pred C (i,j)=((α′·rec L ′(i,j))>>s)+β,

[0242] If dA is signaled, the final prediction is:

[0243] pred C (i,j)=(((α′+dA)·rec L ′(i,j)}>>s)+β.

[0244] Similarly, if dB is signaled, the final prediction is:

[0245] pred C (i,j)=((α′·rec L ′(i,j))>>s)+(β+dB).

[0246] If dS is signaled, the final prediction is:

[0247] pred c (i,j)=((α′·rec L ′(i,j))>>(s+dS))+β.

[0248] If dA and dB are signaled, the final prediction is:

[0249] pred C (i,j)=(((α′+dA)·rec L ′(i,j))>>s)+(β+dB).

[0250] A guided parameter set can be signaled for each color component. For example, one guided parameter set can be signaled for the Cb component and another for the Cr component. Alternatively, a single guided parameter set can be signaled and shared across color components. The signaled dA and dB values ​​can be positive or negative. When signaling dA, a bin is signaled to indicate the sign of dA. Similarly, when signaling dB, a bin is signaled to indicate the sign of dB.

[0251] For another embodiment, dA and dB can be the least significant bits (LSB) part of the final scaling and offset parameters. For example, if m bits are required to represent the final scaling parameter, dA is the LSB part of the final scaling parameter, and n bits (m>n) are used to represent dA, where the most significant bit (mn bit) of the final scaling parameter is implicitly derived. In other words, for the final scaling parameter, the most significant bit of the final scaling parameter is taken from the most significant bit part of α′, and the LSB part of the final scaling parameter comes from the signaled dA. Similarly, if p bits are required to represent the final offset parameter, dB is the LSB part of the final offset parameter, and q bits (p>q) are used to represent dB, where the most significant bit (pq bit) of the final offset parameter is implicitly derived. In other words, for the final offset parameter, the most significant bit of the final offset parameter is taken from the most significant bit part of β, and the LSB part of the final offset parameter comes from the signaled dB.

[0252] For another embodiment, if dA is signaled, dB can be implicitly derived from the average of neighboring (e.g., L-shaped) reconstructed samples. For example, in VVC, four neighboring luma and chroma reconstructed samples are selected to derive the model parameters. Assuming that the average values ​​of the neighboring luma and chroma samples are lumaAvg and chromaAvg, respectively, then β is derived by β = chromaAvg - (α′ + dA) · lumaAvg. The average value of the neighboring luma samples (i.e., lumaAvg) can be obtained by taking all selected luma samples, the luma DC mode value of the current luma CB, or the average value of the maximum and minimum luma samples (e.g., or Similarly, the average value of neighboring chroma samples (i.e., chromaAvg) can be obtained by taking all selected chroma samples, the chroma DC mode value of the current chroma CB, or the average of the maximum and minimum chroma samples (e.g., or Note that for non-4:4:4 color subsampling formats, the selected neighboring luma reconstruction samples can come from the output of the CCLM downsampling process.

[0253] For another embodiment, the shift parameter s may be a constant value (eg, s may be 3, 4, 5, 6, 7, or 8), and dS may be equal to 0 and need not be signaled.

[0254] In another embodiment, in an MMLM, guidance parameter sets can also be signaled per model. For example, one guidance parameter set can be signaled for one model, and another guidance parameter set can be signaled for another model. Alternatively, one guidance parameter set can be signaled and shared between linear models. Alternatively, one guidance parameter set can be signaled for only one selected model, while the other model is not further refined using the guidance parameter set.

[0255] In another embodiment, the MSB portion of α' is selected based on the cost of a possible final scaling parameter. That is, a possible final scaling parameter is derived based on the signaled dA and a possible MSB value of α'. For each possible final scaling parameter, a cost defined by the sum of the absolute differences between adjacent reconstructed chroma samples and the corresponding chroma values ​​generated by the CCLM model is calculated, and the final scaling parameter is the one with the smallest cost. In one embodiment, the cost function is defined as the sum of squared errors.

[0256] Inherit neighboring model parameters to refine cross-component model parameters

[0257] The final scaling parameters of the current block are inherited from the neighboring blocks and further refined by dA (e.g., the derivation or signaling of dA can be similar or identical to the method described above in “Guided parameter set for refining cross-element model parameters”). Once the final scaling parameters are determined, the offset parameters (e.g., β in CCLM) are derived based on the inherited scaling parameters and the average of the neighboring luminance and chrominance samples of the current block. For example, if the final scaling parameters are inherited from the selected neighboring block and the inherited scaling parameters are α′ nei , then the final scaling parameter is (α′ nei +dA). For another embodiment, the final scaling parameter is inherited from a history list and further refined by dA. For example, the history list records the last j final scaling parameter entries of the previous CCLM coded block. Then, the final scaling parameter is inherited from a selected entry in the history list, α′ list , the final scaling parameter is (α′ list + dA). For another embodiment, the final scaling parameter is inherited from the history list or the neighboring block, but only the MSB (most significant bit) part of the inherited scaling parameter is taken, and the LSB (least significant bit) of the final scaling parameter comes from dA. For another embodiment, the final scaling parameter is inherited from the history list or the neighboring block, but is not further refined by dA.

[0258] For another embodiment, after inheriting the model parameters, the offset can be further optimized by dB. For example, if the final offset parameter is inherited from the selected neighboring block, and the inherited offset parameter is β′ nei , then the final scaling parameter is (β′ nei+dB). For another embodiment, the final offset parameter is inherited from a history list and further optimized by dB. For example, the history list records the last j final scaling parameter entries of the previous CCLM coded block. Then, the final scaling parameter is inherited from a selected entry in the history list, β′ list , and the final scaling parameter is (β′ list +dB).

[0259] For another embodiment, if the inherited neighboring block is coded using a convolutional cross-component mode (CCCM), the filter coefficients (c i ) is inherited. The offset parameters (e.g., c6×B or c6 in CCCM) can be re-derived based on the inherited parameters and the average of the luma and chroma samples at the corresponding adjacent locations of the current block. For another embodiment, only some of the filter coefficients are inherited (e.g., only n of the 6 filter coefficients are inherited, where 1≤n<6), and the remaining filter coefficients are further re-derived using the adjacent luma and chroma samples of the current block.

[0260] For another embodiment, if the inherited candidate applies a Gradient Linear Model (GLM) gradient mode to its luma reconstruction samples, the current block should also inherit the candidate's GLM gradient mode and apply it to the current luma reconstruction samples.

[0261] In yet another embodiment, if the inherited neighboring block is encoded using multiple cross-element models (e.g., a multi-model linear model (MMLM) or a CCCM multi-model), the classification threshold is also inherited to classify the neighboring samples of the current block into multiple groups, and the inherited multiple cross-element model parameters are further assigned to each group. In another embodiment, the classification threshold is the average of the neighboring reconstructed luma samples, and the inherited multiple cross-element model parameters are further assigned to each group. Similarly, once the final scaling parameters for each group are determined, the offset parameters for each group are re-derived based on the inherited scaling parameters and the average of the neighboring luma and chroma samples for each group of the current block. For another example, if the CCCM multi-model is used, once the final coefficient parameters for each group are determined (e.g., c0 to c5, except c6 in CCCM), the offset parameters for each group (e.g., c6×B or c6 in CCCM) are re-derived based on the inherited coefficient parameters and the neighboring luma and chroma samples for each group of the current block.

[0262] For yet another embodiment, inherited model parameters may depend on the color component. For example, the Cb and Cr components may inherit model parameters or model derivation methods from the same or different candidates. For another example, only one color component inherits model parameters, and the other color component derives model parameters based on the inherited model derivation method (for example, if the inherited candidate is encoded using MMLM or CCCM, the current block also derives model parameters based on MMLM or CCCM using the current neighboring reconstructed samples). For yet another example, only one color component inherits model parameters, and the other color component derives its model parameters using the current neighboring reconstructed samples.

[0263] For another example, if the Cb and Cr components can inherit model parameters or model derivation methods from different candidates, the inherited model of Cr can depend on the inherited model of Cb. For example, possible situations include but are not limited to: (1) if the inherited model of Cb is CCCM, then the inherited model of Cr should be CCCM; (2) if the inherited model of Cb is CCLM, then the inherited model of Cr should be CCLM; (3) if the inherited model of Cb is MMLM, then the inherited model of Cr should be MMLM; (4) if the inherited model of Cb is CCLM, then the inherited model of Cr should be CCLM or MMLM; (5) if the inherited model of Cb is MMLM, then the inherited model of Cr should be CCLM or MMLM; (6) if the inherited model of Cb is GLM, then the inherited model of Cr should be GLM.

[0264] For another embodiment, after decoding a block, the cross-element model (CCM) information of the current block is derived and stored for use when reconstructing neighboring blocks using inherited neighboring model parameters. The CCM information mentioned in this disclosure includes but is not limited to prediction mode (e.g., CCLM, MMLM, CCCM), GLM mode index, model parameters or classification threshold. For example, even if the current block is encoded by inter-frame prediction, the cross-element model parameters of the current block can be derived by using the current luminance and chrominance reconstruction or prediction samples. Thereafter, if another block is predicted by using the inherited neighboring model parameters, it can inherit the model parameters from the current block. Another example is that the current block is encoded by cross-element prediction, and the cross-element model parameters of the current block are re-derived by using the current luminance and chrominance reconstruction or prediction samples. For another example, the stored cross-element model can be CCCM, LM_LA (i.e., a single model LM that uses the upper and left neighboring samples to derive the model) or MMLM_LA (a multi-model LM that uses the upper and left neighboring samples to derive the model). For another example, even if the current block is encoded by non-cross-element intra prediction (e.g., DC, planar, intra-frame angle mode, MIP, or ISP), the cross-element model parameters of the current block can be derived by using the current luminance and chrominance reconstruction or prediction samples. As another example, even if the current block is encoded by cross-element prediction, the cross-element model parameters of the current block are re-derived by using the current luminance and chrominance reconstruction or prediction samples. Thereafter, the re-derived model parameters are combined with the original cross-element model used to reconstruct the current block. In order to combine with the original cross-element model, the model combination method mentioned in the sections entitled “Model generated based on other inherited models” and “Inheriting multiple cross-element models” can be used. For example, assuming that the original cross-element model parameters are The re-exported cross-component model parameters are The final cross-component model is where α is a weighting factor that can be predefined or implicitly derived from the neighboring template cost.

[0265] For another embodiment, when a cross-element model is inherited from a neighboring merge candidate encoded using a cross-element mode (e.g., CCLM and CCCM, etc.), a flag may be signaled to indicate / select whether to use the re-derived model. If the flag is 0, the cross-element model used to encode the neighboring merge candidate is inherited. If the flag is 1, the cross-element model re-derived based on the luma and chroma reconstruction or prediction samples of the neighboring merge candidate is inherited.

[0266] For another example, when the current slice is a non-intra slice (e.g., a P slice or a B slice), the cross-element model of the current block is derived and stored for use in reconstructing neighboring blocks using inherited neighboring model parameters. For another embodiment, when the current block is inter-coded, the CCM information of the current inter-coded block is derived by copying the CCM information in the reference image having CCM information from its reference block. For example, Figure 19 As shown, block B in P / B picture 1920 is inter-coded, and the CCM information of block B is then obtained by copying the CCM information from reference block A in I picture 1910. It should be noted that the current block can also copy the CCM information from an intra-coded block in a P / B picture. For example, Figure 19 As shown, block D in P / B picture 1930 is inter-coded, and the CCM information of block D is obtained by copying the CCM information from the reference block E that is intra-coded in P / B picture 1920. For another embodiment, if the reference block in the reference picture is also inter-coded, the CCM information of the reference block is obtained by copying the CCM information from another reference block in another reference picture. Figure 19 As shown, current block C in current P / B image 1930 is inter-coded, and its reference block B is also inter-coded. Since the CCM information for block B is obtained by copying the CCM information from block A, the CCM information of block A is also propagated to current block C. In another embodiment, when the current block is inter-coded using bidirectional prediction, if one of its reference blocks is intra-coded and has CCM information, the CCM information for the current block is obtained by copying the CCM information from an intra-coded reference block in the reference image. For example, assume that block F is inter-coded using bidirectional prediction and has reference blocks G and H. Block G is intra-coded and has CCM information. The CCM information for block F is obtained by copying the CCM information from block G, which is coded in a CCM mode. In another embodiment, when the current block is inter-coded using bidirectional prediction, the CCM information for the current block is a combination of the CCM models of its reference blocks (as discussed in the section titled "Inheriting Multiple Cross-Component Models").

[0267] When deriving a cross-element model for the current block using the current luma and chroma reconstruction or prediction samples, in one embodiment, if the error of the currently derived model is greater than a threshold, the currently derived model is discarded and not stored. For example, the current luma reconstruction sample can be input into the model, the distortion between the model output and the current chroma reconstruction sample can be calculated, and the calculated distortion can then be normalized by the size of the current block or the number of samples used to calculate the distortion. If the normalized distortion is greater than or equal to the threshold, the currently derived model is discarded and not stored.

[0268] Whether to export a cross-component model for the current block may depend on the size or area of ​​the current block. For example, for a small block (e.g., a block width / height less than or equal to a threshold, or a block area less than or equal to a threshold), exporting a cross-component model is not allowed. For another example, for a large block (e.g., a block width / height greater than or equal to a threshold, or a block area greater than or equal to a threshold), exporting a cross-component model is not allowed.

[0269] Inherit CCM information

[0270] In one embodiment, cross-element model (CCM) information of the inherited cross-element model can be stored along with the inherited model parameters. As previously described in this disclosure, the CCM information includes, but is not limited to, a prediction mode (e.g., CCLM, MMLM, CCCM), a model index indicating which model shape to use in the convolutional model, a classification threshold for multiple models, a downsampling filter flag, a downsampling filter index, a number of neighboring lines used to derive the model, a template type used to derive the model, a post-filtering flag, or model parameters.

[0271] In one embodiment, the CCLM model may be inherited. In addition to storing model parameters, a prediction mode may also be stored in the CCM information to indicate that the inherited model is a CCLM model.

[0272] In another embodiment, a CCLM model with nonlinear terms may be inherited. In addition to storing model parameters, a prediction mode may be stored in the CCM information to indicate that the inherited model is a CCLM model with nonlinear terms.

[0273] In one embodiment, the CCCM model can be inherited. In addition to storing model parameters, the prediction mode can also be stored in the CCM information to indicate that the inherited model is a CCCM model. The luma offset and chroma offset used to adjust the CCCM model input can also be stored in the CCM information.

[0274] In another embodiment, CCCM models with different convolution filter shapes can be inherited. In addition to the model parameters and the prediction mode, a CCCM mode index can also be stored in the CCM information to indicate the convolution filter shape of the CCCM model to be inherited. For example, CCCM models with different convolution filter shapes can only include spatial items in the horizontal direction. For another example, CCCM models with different convolution filter shapes can only include spatial items in the vertical direction. For another example, CCCM models with different convolution filter shapes can only include spatial items in the diagonal direction. For another example, CCCM models with different convolution filter shapes can only include spatial items in the anti-diagonal direction. For another example, CCCM models with different convolution filter shapes can include X-shaped spatial items.

[0275] In another embodiment, a CCCM model using non-subsampled samples may be inherited. In addition to storing model parameters, a prediction mode may be stored in the CCM information to indicate that the inherited model is a CCCM model using non-subsampled samples.

[0276] In another embodiment, a CCCM model with multiple downsampling filters can be inherited. In addition to storing model parameters, a prediction mode can be stored in the CCM information to indicate that the inherited model is a CCCM model with multiple downsampling filters. A model index can also be stored in the CCM information to indicate which variant of the CCCM model with multiple downsampling filters is inherited.

[0277] In another embodiment, a hybrid CCCM model consisting of various terms (e.g., spatial terms, gradient terms, position terms, nonlinear terms, and bias terms) can be inherited. The gradient terms can be calculated in the downsampled or non-downsampled domain. The position terms can be calculated relative to the upper-left coordinate of the current block or image. In addition to storing model parameters, the prediction mode can also be stored in the CCM information to indicate that the inherited model is a hybrid CCCM model consisting of various terms. If there are multiple types of hybrid CCCM models, a model index can also be stored in the CCM information to indicate which type of hybrid CCCM model is inherited. For example, the gradient and position-based CCCM (GL-CCCM) proposed in JVET-AB0119 is a hybrid CCCM model that includes a spatial term for the center position, two gradient terms for the horizontal and vertical directions, two position terms X and Y for the relative horizontal and vertical positions, a nonlinear term, and a bias term. In addition to storing model parameters, the prediction mode can also be stored in the CCM information to indicate that the inherited model is a GL-CCCM model.

[0278] In one embodiment, the GLM model can be inherited. In addition to storing the model parameters, the prediction mode can also be stored in the CCM information to indicate that the inherited model is a GLM model, and the downsampling filter index can also be stored in the CCM information to indicate the gradient downsampling filter used for the inherited GLM model.

[0279] In another embodiment, a Gradient Linear Model (GLM) with a luma term may be inherited. In addition to storing model parameters, a prediction mode may be stored in cross-component model (CCM) information to indicate that the inherited model is a GLM with a luma term. Furthermore, a downsampling filter index may be stored in the CCM information to indicate a gradient downsampling filter for the inherited GLM with a luma term.

[0280] In one embodiment, any type of cross-component multi-model can be inherited. In addition to storing model parameters and prediction modes, a multi-model on / off flag can also be stored in the CCM information to indicate whether the inherited CCM model is a multi-model. If the multi-model on / off flag is true, the multi-model classification threshold is also stored in the CCM information.

[0281] In one embodiment, the CCM information may include information to indicate how the inherited model is derived. For example, the CCM information may include the number of adjacent rows used to derive the cross-element model and / or the type of template used to derive the model. For example, a set of templates may be used to derive a convolutional cross-element model (CCCM) model. The set of templates includes templates with different positions, sizes, and shapes. The CCM information may store the index of the template on which the inherited CCCM model is based. For example, the inherited CCCM model may be derived based on only the top template, only the left template, or both the left and top templates. As another example, the inherited CCCM model may be derived based on a 6-row template or a 2-row template.

[0282] In one embodiment, a post-filtering flag can be stored in the CCM information. This information describes how the inherited model is used in its source block. If the post-filtering flag is turned on, it means that a filter is applied to the prediction of the inherited model source block.

[0283] Optimization of inheritance model parameters

[0284] In one embodiment, the inherited model parameters can be further optimized based on the inherited CCM information. The inherited CCM information may include how the inherited model was derived, such as the type of template and / or the number of neighboring rows used to derive the model. The optimized parameters are derived based on local information. The optimization process can follow the way the inherited model was derived and use the same type of template and / or the same number of neighboring rows. For example, if the inherited model is a cross-chroma luminance model (CCLM) and is derived based on only the left template (i.e., the inherited model is CCLM_L), then the offset parameter β can be derived from the average of the neighboring left template reconstructed samples of the current block. For example, if the inherited model is a CCLM and is derived based on only the top template (i.e., the inherited model is CCLM_T), then the offset parameter β can be derived from the average of the neighboring top template reconstructed samples of the current block. As another example, if the inherited model is a CCCM and is derived based on a 2-row template, then the offset value (e.g., c6 in CCCM) can be re-derived based on the 2-row template reconstructed samples of the current block. Another example is that if the inherited model is a multi-model (MMLM, CCCM with multi-model) and derived based on the left template only, the classification threshold can be re-derived based on the left template reconstructed samples of the current block.

[0285] In one embodiment, the inherited model parameters are further optimized using different types of templates and / or different numbers of rows, and the final model parameters are determined by the template cost. The template cost is calculated by applying the candidate optimized model parameters to neighboring templates to predict template samples and calculating the difference (e.g., SAD or SATD) between the predicted samples and the reconstructed samples. For example, if the inherited model is CCLM, the optimized offset parameter β′ is L ,β′ T ,β′ LT The left template, top template and left top template of the reconstructed sample of the current block are used to derive them respectively. If β′ is applied L The template cost is in β′ L ,β′ T ,β′ LT is the smallest, then choose β′ L as the final offset parameter.

[0286] In another embodiment, the inherited model parameters can be further optimized by predefined values. The template cost is used to determine whether the inherited model parameters are further optimized. The template cost is calculated by applying the candidate optimized model parameters to the neighboring template to predict the template samples and calculating the difference (e.g., SAD or SATD) between the predicted samples and the reconstructed samples. For example, for CCCM mode, for each inherited model parameter c i , this value is passed through dc i Optimize and compare application c i +dc i and c i The template cost is used to determine which is the final model parameter value.

[0287] Inheriting spatial proximity model parameters

[0288] For another embodiment, the inherited model parameters may be from a directly adjacent block. Models from blocks at predefined locations are added to the candidate list in a predefined order. For example, the predefined location may be Figure 14 For the positions depicted in FIG, the predefined order may be B0, A0, B1, A1, and B2, or A0, B0, B1, A1, and B2.

[0289] For another embodiment, the predefined positions include those positions that are (W>>1) or ((W>>1)-1) positions immediately above, if W is greater than or equal to TH, and those positions that are (H>>1) or ((H>>1)-1) positions immediately to the left, if H is greater than or equal to TH, where W and H are the width and height of the current block, and TH is a threshold that can be 4, 8, 16, 32, or 64. For another embodiment, the maximum number of models inherited from spatial neighbors is less than the number of predefined positions. For example, if the predefined positions are such as Figure 14 As shown, there are 5 predefined positions. If the predefined order is B 0, A 0, B 1, A1 and B2, and the maximum number of models inherited from spatial neighbors is 4, then the model from B2 is added to the candidate list only if one of the previous blocks is not available or not encoded in the cross-element model.

[0290] Inherited temporal proximity model parameters

[0291] For another embodiment, if the current slice / image is a non-intra slice / image, the inherited model parameters may be from blocks in a previously encoded slice / image. Figure 20 As shown, the current block is at position (x, y) and has a block size of w×h. The inherited model parameters can come from a block at position (x', y'), (x', y'+h / 2), (x'+w / 2, y'), (x'+w / 2, y'+h / 2), (x'+w, y'), (x', y'+h), or (x'+w, y'+h) in a previously coded slice / image, where x' = x + Δx and y' = y + Δy. In one embodiment, if the prediction mode of the current block is intra, Δx and Δy are set to 0. If the prediction mode of the current block is inter, Δx and Δy are set to the horizontal and vertical motion vectors of the current block. In another embodiment, if the current block is inter-predicted, Δx and Δy are set to the horizontal and vertical motion vectors in reference picture list 0. In another embodiment, if the current block is inter-predicted, Δx and Δy are set to the horizontal and vertical motion vectors in reference picture list 1.

[0292] For another embodiment, if the current block is inter-frame bi-directionally predicted, the inherited model parameters can come from the block in the previously coded slice / picture in the reference list. For example, if the horizontal and vertical motion vectors in reference picture list 0 are Δx L0 and Δy L0 , the motion vector can be scaled to other reference images in reference lists 0 and 1. If the motion vector is scaled to i in reference list 0 th The reference image is (Δx L0,i0 ,Δy L0,i0 ). The model can be from the reference list 0 i th The block in the reference image, and Δx and Δy are set to (Δx L0,i0 ,Δy L0,i0 For another example, if the horizontal and vertical motion vectors in reference picture list 0 are Δx L0 and Δy L0 , and the motion vector is scaled to i in reference list 1 th The reference image is (ΔxL0,i1 ,Δy L0,i1 ). The model can be from i in reference list 1 th The block in the reference image, and Δx and Δy are set to (Δx L0,i1 ,Δy L0,i1 ).

[0293] Inheriting non-adjacent spatial neighbor models

[0294] For another embodiment, inherited model parameters can be from spatially adjacent blocks. Models from blocks at predefined locations are added to the candidate list in a predefined order. For example, the pattern of location and order can be as follows: Figure 18 As shown, the distance between each position is the width and height of the current coding block. For another embodiment, the distance between positions closer to the current coding block is smaller than the distance between positions farther from the current block.

[0295] For another embodiment, the maximum number of models inherited from non-adjacent spatial neighbors is less than the number of predefined locations. For example, if the predefined locations are Figure 21A -B, which shows two modes ( Figure 21A Mode 2110 and Figure 21B 2120 in ). If the maximum number of models inherited from non-adjacent spatial neighbors is N, search mode 2 is used only if the number of available models from search mode 1 is less than N.

[0296] Inherit model parameters from history table

[0297] In one embodiment, the inherited model parameters may come from a cross-component model history table. The cross-component models in the history table may be added to the candidate list in a predefined order. In one embodiment, the order of adding the historical candidates may be from the beginning of the table to the end of the table. In another embodiment, the order of adding the historical candidates may be from a certain predefined position to the end of the table. In another embodiment, the order of adding the historical candidates may be from the end of the table to the beginning of the table. In another embodiment, the order of adding the historical candidates may be from a certain predefined position to the beginning of the table. In another embodiment, the order of adding the historical candidates may be in an interleaved manner (e.g., the first candidate added is from the beginning of the table, the second candidate added is from the end of the table, and so on).

[0298] In one embodiment, a single cross-element model history table may be maintained to store previous cross-element models, and the cross-element model history table may be reset at the beginning of the current picture, the current slice, the current tile, every M CTU rows, or every N CTU, where N and M may be any values ​​greater than 0. In another embodiment, the cross-element model history table may be reset at the end of the current picture, the current slice, the current tile, the current CTU row, or the current CTU.

[0299] In another embodiment, a picture can be divided into multiple regions, and a history table is maintained for each region. During the encoding / decoding process, history table 0 and an additional history table are updated. The additional history table can be determined by the current position. For example, if the current CU is in the second region, the additional history table to be updated is history table 2.

[0300] In another embodiment, multiple history tables are used for different update frequencies. For example, the first history table is updated every CU, the second history table is updated every two CUs, the third history table is updated every four CUs, and so on.

[0301] In another embodiment, multiple history tables are used to store different types of cross-component models. For example, a first history table is used to store a single model, while a second history table is used to store multiple models. In another example, a first history table is used to store gradient models, while a second history table is used to store non-gradient models. In another example, a first history table is used to store simple linear models (e.g., y = ax + b), while a second history table is used to store complex models (e.g., CCCM).

[0302] In another embodiment, multiple history tables are used for different reconstructed luma intensities. For example, if the average of the reconstructed luma samples in the current block is greater than a predefined threshold, the cross-element model is stored in a first history table; otherwise, the cross-element model is stored in a second history table. In another embodiment, multiple history tables are used for different reconstructed chroma intensities. For example, if the average of the neighboring reconstructed chroma samples in the current block is greater than a predefined threshold, the cross-element model is stored in a first history table; otherwise, the cross-element model is stored in a second history table.

[0303] In one embodiment, when adding historical candidates to the candidate list from multiple historical tables, the order of addition may be from the beginning of a certain table to the end of a certain table, and then the next historical table is added in the same order or in the reverse order. In another embodiment, the order of addition may be from the end of a certain table to the beginning of a certain table, and then the next historical table is added in the same order or in the reverse order. In another embodiment, the order of addition may be from a predefined position of a certain table to the end of a certain table, and then the next historical table is added in the same order or in the reverse order. In another embodiment, the order of addition may be from a predefined position of a certain table to the beginning of a certain table, and then the next historical table is added in the same order or in the reverse order. In another embodiment, the order of addition of historical candidates may be performed in an interleaved manner in a certain historical table (for example, the first candidate added is from the beginning of a certain historical table, the second candidate added is from the end of a certain historical table, and so on), and then the next historical table is added in the same order or in the reverse order.

[0304] In another embodiment, the order of addition may be from the beginning of each historical table to the end of each historical table. In another embodiment, the order of addition may be from the end of each historical table to the beginning of each historical table. In another embodiment, the order of addition may be from a predefined position of each historical table to the end of each historical table. In another embodiment, the order of addition may be from a predefined position of each historical table to the beginning of each historical table. In another embodiment, the order of addition of historical candidates may be performed in an interleaved manner in each particular historical table (e.g., the first candidate added is from the beginning of all historical tables, the second candidate added is from the end of all historical tables, and so on).

[0305] In one embodiment, multiple cross-element model history tables are used, but not all history tables are used to create the candidate list. Only history tables whose regions are close to the current block region can be used to create the candidate list.

[0306] In one embodiment, if historical candidates are used, the range of selecting non-adjacent candidates can be reduced by using smaller distances between non-adjacent candidate positions. In another embodiment, if historical candidates are used, the number of non-adjacent candidates can be reduced by measuring the distance from the upper left position of the current block to the candidate position and then excluding candidates with distances greater than a predefined threshold. In another embodiment, if historical candidates are used, the number of non-adjacent candidates can be reduced by skipping candidates that are not located in the same area. In another embodiment, if historical candidates are used, the number of non-adjacent candidates can be reduced by skipping candidates that are not located in adjacent areas. The range of the adjacent area is predefined and can be an M by N area, where M and N can be any values ​​greater than 0. In another embodiment, if historical candidates are used, the range of selecting non-adjacent candidates can be reduced by skipping the second search mode.

[0307] In another embodiment, a picture can be divided into multiple regions, and at least one history table is retained in each region. For a region of the current picture, it can use or combine the history table of one or more regions in the previous coded picture as the initial history table. For example, if a picture is divided into N regions, it can implicitly or explicitly select the history table of one of the N regions in the previous coded picture as the initial history table. The index of one of the N regions can be transmitted by the corresponding region signal in the previous coded picture or implicitly derived. Figure 22A -B, where the current picture 2220 is a P / B coded picture and the previous picture 2210 is an intra-frame coded picture. Each picture is divided into 4 regions, as shown in 4 rectangular boxes. According to one embodiment of the present invention, the corresponding region in the previous coded picture can be as follows: Figure 22A The region 2212 shown has the same starting geometric position as the current region 2222, or as shown in FIG. Figure 22B The region shown contains the geometric location of the center of the current region 2222. For another example, it is possible to combine multiple history tables from the previous coded region / picture to construct the history table for the current region (e.g., the method in the section entitled "Inheriting candidates from neighbor's candidate lists").

[0308] Available areas for non-adjacent spatial candidates

[0309] In order to limit the required buffer / storage resources, the available range containing non-adjacent spatial candidates should be limited. In one embodiment, only the Cross-Component Model (CCM) information in the current CTU can be referenced by non-adjacent spatial candidates. In another embodiment, only the CCM information in the current CTU or the M CTUs to the left can be referenced by non-adjacent spatial candidates. M can be any integer greater than 0. In another embodiment, only the CCM information in the current CTU row can be referenced by non-adjacent spatial candidates. In another embodiment, only the to-be-referenced positions within the current CTU row or the N CTU rows above can be referenced. N can be any integer greater than 0. Note that the CCM information mentioned in this disclosure includes, but is not limited to, prediction modes (e.g., CCLM, MMLM, CCCM), GLM mode indexes, model parameters, or classification thresholds.

[0310] In another embodiment, the CCM information in the current CTU, the current CTU row, the current CTU row + N CTU rows above, the current CTU + M CTUs on the left, or the current CTU + N CTU rows above + M CTUs on the left can be referenced without restriction. In addition, CCM information in other areas can only be referenced through larger predefined units. For example, the CCM information in the current CTU row is stored in a 4x4 grid, while other CCM information outside the current CTU row is stored in a 16x16 grid. In other words, a 16x16 area only needs to store one CCM information, so the position to be referenced should be rounded to the 16x16 grid or changed to the nearest position of the 16x16 grid.

[0311] In another embodiment, the CCM information in the current CTU row or the current CTU row + M CTU rows can be referenced without restriction, and for the position to be referenced in the upper CTU row, the position will be mapped to a row above the current CTU, or the current CTU row + M CTU rows for reference. This design can retain most of the encoding and decoding efficiency and will not increase too much buffering to store the CCM information of the upper CTU row. For example, the CCM information in the current CTU row (2310) and the first upper CTU row (2312) can be referenced without restriction; and for the position to be referenced in the second upper (2320), third upper (2322), fourth upper CTU row, etc., the position will be mapped to a row (2330) above the first upper CTU row (2312) for reference (such as Figure 23 shown). Figure 23, dark circles represent unavailable candidates 2340, dot-filled circles represent available candidates 2342, and empty circles represent mapped candidates 2344. For example, unavailable candidate 2350 in the third (2322) CTU row above is mapped to available candidate 2352 in a row (2330) above the first (2312) CTU row above.

[0312] In the above example, the area that can be referenced without restriction is close to the current CTU (for example, the current CTU row or the first CTU row above). However, the area according to the present invention is not limited to the above exemplary area. The area can be larger or smaller than the above example. In general, the area can be limited to one or more predefined distances from the current CTU in the vertical direction, horizontal direction, or both. In the above example, the area is limited to 1 CTU height in the vertical direction above, and can be extended to 2 or 3 CTU heights if necessary. When using M CTUs on the left, it is limited to the M CTU width of the current CTU row. The horizontal position of the position to be referenced and the horizontal position of the mapped predefined position can be the same (for example, position 2350 and position 2352 are at the same horizontal position). However, other horizontal positions can also be used.

[0313] In another embodiment, the CCM information in the current CTU row or the current CTU row + M CTU rows can be referenced without restriction. In addition, for the position to be referenced in the above CTU row, the position will be mapped to the last row of the corresponding CTU row for reference. For example, Figure 24 As shown, the CCM information in the current CTU row (2310) and the first CTU row (2312) above can be referenced without restriction. For the position to be referenced in the second CTU row (2320) above, the position will be mapped to the bottom line (2330) of the second CTU row (2320) above. For the position to be referenced in the third CTU row (2322) above, the position will be mapped to the bottom line (2420) of the third CTU row (2322) above. For example, the unavailable candidate 2350 in the third CTU row (2322) above is mapped to the mapping candidate 2430 in the bottom line (2420) of the third CTU row (2322) above. Figure 24 The legend of the candidate types (i.e. 2340, 2342 and 2344) is the same as Figure 23 In this example, the unconstrained region may include one or more upper CTU rows (e.g., Figure 24 The second CTU row above is located above the unconstrained region. The third CTU row above is also called the upper-upper CTU row because it is located above the CTU row above the unconstrained region (i.e., the second CTU row above).

[0314] In another embodiment, the CCM information in the current CTU row or the current CTU row + M CTU rows can be referenced without restriction, and for the position to be referenced in the upper CTU row, the position will be mapped to the last row or bottom line or center line of the corresponding CTU row according to the position of the CCM information to be referenced for reference. Figure 25 As shown, the CCM information in the current CTU row (2310) and the first CTU row (2312) above can be referenced without restriction. For position 1 to be referenced in the second CTU row (2320) above, the position will be mapped to the bottom line (2330) of the second CTU row above before reference. However, for position 2 to be referenced in the second CTU row above, the position will be mapped to the center line (2510) of the second CTU row (2320) above before reference because it is closer to the center line (2510) than the bottom line (2330). Figure 25 The legend of the candidate types (i.e. 2340, 2342 and 2344) is the same as Figure 23 The same as in .

[0315] In another embodiment, the CCM information in the current CTU row or the current CTU row + M CTU rows can be referenced without restriction, and for the position to be referenced in the upper CTU row, the position will be mapped to the last row or bottom line of the corresponding CTU row according to the position of the CCM information to be referenced for reference. Figure 26 As shown, the CCM information in the current CTU row (2310) and the first CTU row (2312) above can be referenced without restriction. For the position 1 to be referenced in the second CTU row (2320) above, the position will be mapped to the bottom line (2330) of the second CTU row (2320) above before the reference. However, for the position 2 to be referenced in the second CTU row (2320) above, the position will be mapped to the bottom line (2420) of the third CTU row (2322) above before the reference because it is closer to the bottom line (2420) of the third CTU row (2320) above than the bottom line (2330) of the second CTU row (2320) above, as shown in FIG. Figure 26 The legend of candidate types (i.e. 2340, 2342 and 2344) is as follows Figure 23 The same as in .

[0316] In another embodiment, the CCM information in the current coding tree unit (CTU) or the CCM information in the current CTU plus the N CTUs on the left can be referenced without restriction. For the left CTU, the position to be referenced will be mapped to the rightmost row closest to the current CTU or the current CTU plus the N CTUs on the left. For example, the CCM information in the current CTU and the first left CTU can be referenced without restriction. If the position to be referenced is in the second left CTU, these positions will be mapped to a row to the left of the first left CTU. If the position to be referenced is in the third left CTU, these positions will be mapped to a row to the left of the first left CTU. For example, the CCM information in the current CTU and the first left CTU can be referenced without restriction. If the position to be referenced is in the second left CTU, these positions will be mapped to the rightmost row of the second left CTU. If the position to be referenced is in the third left CTU, these positions will be mapped to the rightmost row of the third left CTU.

[0317] In another embodiment, when the available range is restricted to include non-adjacent candidates, if the position of the non-adjacent candidate exceeds the available range, the candidate will be skipped and not inserted into the candidate list. The available area can be the current CTU, the current CTU row, the current CTU row plus N CTU rows above, the current CTU plus M CTUs to the left, or the current CTU plus N CTU rows above plus M CTUs to the left.

[0318] Models generated based on other inherited models

[0319] In another embodiment, a single cross-component model can be generated from multiple cross-component models. For example, if a candidate is encoded using multiple cross-component models (e.g., MMLM or CCCM with multiple models), a single cross-component model can be generated by selecting the first or second cross-component model from the multiple cross-component models.

[0320] Candidate list construction

[0321] In one embodiment, the candidate list is constructed by adding candidates in a predefined order until the maximum number of candidates is reached. The added candidates may include all or part of the above-mentioned candidates, but are not limited to the above-mentioned candidates. For example, the candidate list may include spatially adjacent candidates, temporally adjacent candidates, historical candidates, non-adjacent adjacent candidates, single-model candidates generated based on other inheritance models or combination models (as mentioned in the following section: inheritance of multiple cross-component models). In another case, the candidate list may include the same candidates as the previous example, but the candidates are added to the list in a different order.

[0322] In another embodiment, if all predefined neighboring and historical candidates have been added but the maximum candidate number has not been reached, some default candidates are added to the candidate list until the maximum candidate number is reached.

[0323] In a sub - embodiment, the default candidates include, but are not limited to, the candidates described below. The final scaling parameter α comes from the set {0, 1 / 8, -1 / 8, +2 / 8, -2 / 8, +3 / 8, -3 / 8, +4 / 8, -4 / 8}, and the offset parameter β = 1 / (1 << bit_depth) is derived based on neighboring luminance and chrominance samples. For example, if the average values of neighboring luminance and chrominance samples are lumaAvg and chromaAvg respectively, then β is derived by β = chromaAvg - α·lumaAvg. The average value of neighboring luminance samples (lumaAvg) can be calculated by the average of all selected luminance samples, the luminance DC mode value of the current luminance CB, or the average of the maximum and minimum luminance samples (e.g., or calculated. Similarly, the average value of neighboring chrominance samples (chromaAvg) can be calculated by the average of all selected chrominance samples, the chrominance DC mode value of the current chrominance CB, or the average of the maximum and minimum chrominance samples (e.g., or calculated.

[0324] In another sub - embodiment, the default candidates include, but are not limited to, the candidates described below. The default candidate is α·G + β, where G is the luminance sample gradient rather than the down - sampled luminance sample L. Sixteen GLM filters described in the "Gradient Linear Model (GLM)" section are applied. The final scaling parameter α comes from the set {0, 1 / 8, -1 / 8, +2 / 8, -2 / 8, +3 / 8, -3 / 8, +4 / 8, -4 / 8}. The offset parameter β = 1 / (1 << bit_depth) or is derived based on neighboring luminance and chrominance samples.

[0325] In another embodiment, the default candidate can be an early candidate with incremental scaling parameter optimization. For example, if the scaling parameter of the early candidate is α, the scaling parameter of the default candidate is (α + Δα), where Δα can come from the set {1 / 8, -1 / 8, +2 / 8, -2 / 8, +3 / 8, -3 / 8, +4 / 8, -4 / 8}. The offset parameter of the default candidate will be derived by (α + Δα) and the average of neighboring luminance and chrominance samples of the current block.

[0326] In another embodiment, the default candidate can be a shortcut indicating a cross-component mode (i.e., using the current neighboring luminance / chrominance reconstruction samples to derive a cross-component model), rather than inheriting parameters from neighbors. For example, the default candidate can be CCLM_LA, CCLM_L, CCLM_A, MMLM_LA, MMLM_L, MMLM_A, single-model CCCM, multi-model CCCM, or a cross-component model with a specified GLM mode.

[0327] In another embodiment, the default candidate can be a cross-component mode (i.e., using the current neighboring luminance / chrominance reconstruction samples to derive a cross-component model), rather than inheriting parameters from neighbors, and also has a scaling parameter update (Δα). Then, the scaling parameter of the default candidate is (α + Δα). For example, the default candidate can be CCLM_LA, CCLM_L, CCLM_A, MMLM_LA, MMLM_L, or MMLM_A. For another example, Δα can come from the set {1 / 8, -1 / 8, +2 / 8, -2 / 8, +3 / 8, -3 / 8, +4 / 8, -4 / 8}. The offset parameter of the default candidate will be derived by the average of (α + Δα) and the neighboring luminance and chrominance samples of the current block. For another example, Δα can be different for each color component.

[0328] In another embodiment, the default candidate can be an early candidate with some selected model parameters. For example, assuming the early candidate has m parameters, it can select k out of the m parameters from the early candidate as the default candidate, where 0 < k < m and m > 1.

[0329] In another embodiment, the default candidate can be the first model of the early MMLM candidate (i.e., the model used when the sample value is less than or equal to the classification threshold). In another embodiment, the default candidate can be the second model of the early MMLM candidate (i.e., the model used when the sample value is greater than or equal to the classification threshold). In another embodiment, the default candidate can be a combination of the two models of the early MMLM candidate. For example, if the models of the early MMLM candidate are and The model parameters of the default candidate can be where α is a weighting factor, which can be predefined or implicitly derived based on the neighboring template cost, is the x-th parameter of the y-th model.

[0330] In another embodiment, the default candidate can be derived from the reconstructed samples of the non-adjacent neighboring area. Assume that the position of the current block is (x, y) and the block size is w×h. If the reconstructed samples in the MxN area of ​​(x+dx, y+dy) are "available", the default candidate can be derived using the reconstructed luminance and chrominance samples in the area. For example, MxN can be 8x8. As another example, MxN can be 16x8. As another example, MxN can be 16x16. As another example, MxN can be w×h. "Available" can mean that the reconstructed samples within the current block are available, or that the reconstructed samples within k rows of neighboring samples are available. k can be defined by the IBC neighboring search area, or by the neighboring buffer area of ​​other intra-frame codec tools (for example, multi-reference line intra prediction, CCLM or CCCM).

[0331] In another embodiment, let the current block be at (x, y) and the block size be w×h. mid +dx,y mid If the reconstructed samples in the MxN region of (x + dy) are available, the reconstructed samples in this region can be used to derive the default candidate, where (x mid ,y mid )=(x+w / 2,y+h / 2).

[0332] In another embodiment, the default candidate derived from the reconstructed samples of the non-adjacent neighboring regions can be any type of cross-component model or certain specific types of cross-component models. For example, the derived model can be a CCLM, MMLM, CCCM, CCCM multi-model, or other cross-component model. In another example, the derived model is a CCCM model. In another example, the derived model is a CCLM model. In another example, the derived model is a CCCM or CCCM multi-model.

[0333] In another embodiment, assume that two value sets α are defined x and α y :

[0334] α x ={α x1 ,α x2 ,α x3 ,…,α xn},α xi <α xj if i <j

[0335] α y ={α y1 ,α y2 ,α y3 ,…,α yn},α yi <αyj if i <j。

[0336] α x and α y All values ​​in are positive. (dx,dy) can be (α xi ×w,-α yi ×h),(-α xi ×w,α yi ×h),(-α xi ×w,-α yi ×h),(α xi ×w,0),(-α xi ×w,0),(0,α yi ×h),(0,y mid -α yi ×h).

[0337] In another embodiment, the position of the current block is (x, y) and the block size is w×h. Let δx and δy be two fixed positive numbers, and (dx, dy) can be (α xi ×δx,-α yi ×δy),(-α xi ×δx,+α yi ×δy),(-α xi ×δx,-α yi ×δy),(α xi ×δx,0),(-α xi ×δx,0),(0,α yi ×δy),or(0,-α yi ×δy).

[0338] When constructing the candidate list, the candidates are inserted into the list according to a predefined order. For example, the predefined order may be spatially adjacent candidates, temporal candidates, spatially non-adjacent candidates, historical candidates, and then default candidates. In one embodiment, if a cross-element model is derived for a non-LM coding block (e.g., as mentioned in the section entitled "Inheriting neighboring model parameters to optimize cross-element model parameters"), the candidate models for the non-LM coding block are included in the list after the candidate models for the LM coding block are included. In another embodiment, if a cross-element model is derived for a non-LM coding block, the candidate models for the non-LM coding block are included in the list before the default candidates are included. In yet another embodiment, if a cross-element model is derived for a non-LM coding block, the candidate models for the non-LM coding block have a lower priority than the candidate models from the LM coding block to be included in the list.

[0339] When constructing a candidate list, only candidates with a specific prediction mode can be added to the list. For example, it is possible to restrict that only candidates derived through CCLM or MMLM mode are allowed to be added to the list. Another example is that it is possible to restrict that only candidates derived through a single model mode (e.g., CCLM, or CCLM with a single model) are allowed to be added to the list. Another example is that it is possible to restrict that only candidates derived through a multi-model mode (e.g., MMLM, or CCCM with multiple models) are allowed to be added to the list. Another example is that it is possible to restrict that only candidates derived through a GLM mode are allowed to be added to the list. Another example is that it is possible to restrict that only candidates derived through a specific mode (e.g., CCLM, MMLM, CCCM, CCCM with multiple models, or GLM) are allowed to be added to the list. In one embodiment, if only candidates with a specific prediction mode can be added to the list, when prediction mode signaling is performed, the prediction mode can be signaled first, and then whether the proposed cross-element merging mode is used can be signaled. If the proposed cross-element merging mode is used, the candidate index is signaled.

[0340] In chroma intra-frame fusion mode, non-CCLM-coded intra-frame predictions and CCLM-coded intra-frame predictions are fused together to obtain the final intra-frame prediction. In one embodiment, when cross-component model parameters are inherited from a block / position encoded in chroma intra-frame fusion mode, the model parameters used to obtain CCLM-coded intra-frame predictions are inherited and further optimized. In another embodiment, the fusion weights, the codec mode of the non-CCLM-coded intra-frame predictions, and the model parameters used to obtain CCLM-coded intra-frame predictions are inherited and further optimized. In yet another embodiment, the codec mode of the non-CCLM-coded intra-frame predictions is implicitly derived (e.g., derived as DM or planar mode), and the fusion weights and model parameters used to obtain CCLM-coded intra-frame predictions are inherited and further optimized. In yet another embodiment, if the non-CCLM-coded intra-frame predictions for the block / position encoded in chroma intra-frame fusion mode can be implicitly derived (e.g., the non-CCLM-coded intra-frame predictions are DM or planar mode), the fusion weights and model parameters used to obtain CCLM-coded intra-frame predictions are inherited and further optimized.

[0341] Remove or modify similar neighbor model parameters

[0342] When inheriting cross-element model parameters from other blocks, the similarity between the inherited model and the existing models in the candidate list or those model candidates derived from the neighboring reconstructed samples of the current block (for example, models derived using the neighboring reconstructed samples of the current block, such as CCLM, MMLM or CCCM). If the model of the candidate parameters is similar to the existing model, the model will not be included in the candidate list. In one embodiment, the similarity of (α×lumaAvg+β) or α between the existing candidates can be compared to decide whether to include the model of the candidate. For example, if the (α×lumaAvg+β) or α of the candidate is the same as one of the existing candidates, the model of the candidate is not included. Another example is that if the difference in (α×lumaAvg+β) or α between the candidate and one of the existing candidates is less than a threshold, the model of the candidate is not included. In addition, the threshold can be adaptively adjusted based on codec information (for example, the size or area of ​​the current block). As another example, when comparing similarities, if both the candidate and the existing model use CCCM, the decision to include the candidate's model can be made by checking the value of (c0C+c1N+c2S+c3E+c4W+c5P+c6B). In another embodiment, if the candidate position points to the same CU as the existing candidate, the model of the candidate parameters is not included. In yet another embodiment, if the candidate's model is similar to one of the existing candidate models, the inherited model parameters can be adjusted so that the inherited model is different from the existing candidate model. For example, if the inherited scaling parameters are similar to one of the existing candidate models, a predefined offset (e.g., 1>>S or -(1>>S), where S is a shift parameter) can be added so that the inherited parameters are different from the existing candidate model.

[0343] In another embodiment, only some of the model parameters of the existing models in the candidate list are compared. For example, a CCLM candidate has scale and offset parameters, and only the scale or offset parameters may be compared to see if they are the same or similar to the existing candidates. If the scale or offset parameters are the same or similar, the model will not be included in the candidate list. Another example is that a CCCM candidate has c0 to c6 parameters, and only n parameters (n<7) may be compared to see if they are the same or similar to the existing candidates. If the scale or offset parameters are the same or similar, the model will not be included in the candidate list.

[0344] In another embodiment, the candidate model can be applied to the neighboring reconstructed samples of the current block and the difference can be compared with the existing candidate models. If the difference value is less than or equal to the threshold, the model will not be included in the candidate list. For example, if the application result is The corresponding results of the existing models in the candidate list are arrive if or Then the model will not be included in the candidate list. For the selection of the neighboring reconstructed sample, the neighboring reconstructed sample with the maximum value, the neighboring reconstructed sample with the minimum value, the average / median / mode of the neighboring reconstructed samples, the left neighboring reconstructed sample, the upper neighboring reconstructed sample, or the upper left neighboring reconstructed sample can be selected.

[0345] In another embodiment, when adding candidates to a list, the number of candidates of the same type (e.g., MMLM, CCCM, or GLM) is limited. For example, if there are k MMLM candidates in the current list, no further MMLM candidates are allowed to be added to the list. Another example is, if there are k CCCM candidates in the current list, no further CCCM candidates are allowed to be added to the list. Another example is, if there are k GLM candidates in the current list, no further GLM candidates are allowed to be added to the list.

[0346] In another embodiment, the default candidate will not be compared with existing models in the candidate list and will be included in the candidate list.

[0347] Reorder the candidates in the list

[0348] The candidates in the list can be reordered to reduce the syntax overhead when signaling the selected candidate index. The reordering rules can depend on the codec information or model errors of the neighboring blocks. For example, if the neighboring upper or left block is coded or decoded by MMLM, the MMLM candidates in the list can be moved to the beginning of the current list. Similarly, if the neighboring upper or left block is coded or decoded by single model LM or CCCM, the single model LM or CCCM candidates in the list can be moved to the beginning of the current list. Similarly, if the neighboring upper or left block uses GLM, the GLM-related candidates in the list can be moved to the beginning of the current list.

[0349] In another embodiment, the reordering rule is based on applying the candidate model to the neighboring templates of the current block and then comparing the error with the reconstructed samples of the neighboring templates. Figure 27 As shown, the size of the upper adjacent template 2720 of the current block is w a ×h a , the size of the left adjacent template 2730 of the current block 2710 is w b ×h b Assume that there are K models in the current candidate list, α k and β k is the final scale and offset parameter after inheriting candidate k. The model error of candidate k corresponding to the upper neighboring template is:

[0350]

[0351] in, and is the luminance reconstructed sample (e.g., after a downsampling process or application of the GLM mode) and the chrominance reconstructed sample at position (i, j) in the upper template, 0≤i <w a and 0≤j <h a .

[0352] Similarly, the model error of candidate k through the left neighboring template is:

[0353]

[0354] in and is the luminance reconstruction sample (e.g., after downsampling or Gradient Linear Model (GLM) processing) and chrominance reconstruction sample at position (m,n) in the left template, 0≤m <w b and 0≤n <h b . Then the model error of candidate k is: After calculating all candidate model errors, we can get a model error list E = {e 0 ,e 1 ,e 2 ,…,e k ,…,e K The candidate indices in the inheritance candidate list can then be rearranged by sorting the model error list in ascending order.

[0355] In another embodiment, if the candidate k is predicted using a Convolutional Cross-Component Mode (CCCM), and Defined as:

[0356]

[0357] where c0 k ,c1 k ,C2 k ,c3 k ,c4 k ,c5 k and c6 k is the final filter coefficient after inheriting candidate k. P and B are the nonlinear term and bias term.

[0358] In another embodiment, if the above neighboring template is not available, then Similarly, if the left neighbor template is not available, then If both templates are unavailable, the candidate index re-ranking method using model error is not applied.

[0359] In another embodiment, not all positions in the upper and left adjacent templates are used to calculate the model error. Some positions in the upper and left adjacent templates can be selected to calculate the model error. For example, a starting position and a sampling interval can be defined, depending on the width of the current block, to partially select positions in the upper adjacent template. Similarly, a second starting position and a second sampling interval can be defined, depending on the height of the current block, to partially select positions in the left adjacent template. For another example, h a or w b Can be a constant value (for example, h a or w b can be 1, 2, 3, 4, 5 or 6). For another example, h a or w b It can depend on the size of the block. If the size of the current block is greater than or equal to a threshold, h a or w b is equal to the first value. Otherwise, h a or w b Equal to the second value.

[0360] In another embodiment, different types of candidates are reordered separately before being added to the final candidate list. For each type of candidate, the candidate is added to a primary candidate list of predefined size N1. The candidates in the primary list are reordered. The candidate with the smallest cost (N2)) is then added to the final candidate list, where N2≤N1. In another embodiment, the candidates are classified into different types according to their sources, including but not limited to spatial proximity models, temporal proximity models, non-adjacent spatial proximity models, and historical candidates. In another embodiment, the candidates are classified into different types according to cross-element model patterns. For example, the types can be CCLM, MMLM, CCCM, and CCCM multi-model. For another example, the type can be GLM inactive or GLM active.

[0361] In another embodiment, after the candidates are reordered based on template cost, the candidates can be further checked for redundancy. If the difference in template cost between a candidate and its previous candidate in the list is less than a threshold, the candidate is considered redundant. If a candidate is considered redundant, it can be removed from the list or moved to the end of the list.

[0362] Inherit candidates from the candidate list of neighboring blocks

[0363] The candidates in the current inherited candidate list can come from the neighboring blocks. For example, the first k candidates in the inherited candidate list of the neighboring blocks can be inherited. Figure 28 As shown, the current block can inherit the first two candidates in the inheritance candidate list of the upper neighboring block and the first two candidates in the inheritance candidate list of the left neighboring block. In one embodiment, after adding the adjacent space candidates and the non-adjacent space candidates, if the current inheritance candidate list is not full, the candidates in the candidate list of the adjacent block are included in the current inheritance candidate list. In another embodiment, when including the candidates in the candidate list of the adjacent block, the candidates in the candidate list of the left neighboring block are included before the candidates in the candidate list of the upper neighboring block. In another embodiment, when including the candidates in the candidate list of the adjacent block, the candidates in the candidate list of the upper neighboring block are included before the candidates in the candidate list of the left neighboring block.

[0364] Signal inheritance candidate index in the list

[0365] An on / off flag may be signaled to indicate whether the current block inherits cross-element model parameters from neighboring blocks. This flag may be signaled per CU / CB, per PU, per TU / TB, per color element, or per chroma color element. A high-level syntax may be signaled in an SPS, PPS (Picture Parameter Set), PH (Picture Header), or SH (Slice Header) to indicate whether the proposed method is allowed for the current sequence, picture, or slice.

[0366] The maximum allowed number of candidates is signaled to indicate the maximum size of the merge candidate list. This number can be signaled per CU / CB, per PU, per TU / TB, per color element, or per chroma color element. High-level syntax can be signaled in the SPS, PPS, PH, or SH to indicate whether the proposed method is allowed for the current sequence, picture, or slice. The maximum allowed number of candidates for the proposed method (i.e., CCP merge mode) can be shared with the maximum allowed number of candidates for inter merge mode.

[0367] If the current block inherits cross-element model parameters from a neighboring block, the inherited candidate index is signaled. This index can be signaled (e.g., using a truncated unary code, an Exp-Golomb code, or a fixed-length code) and shared between the current Cb and Cr blocks. For another example, the index can be signaled on each color element. For example, one inheritance index is signaled for the Cb element and another inheritance index is signaled for the Cr element. For another example, the inheritance index can be stored using chroma intra prediction syntax (e.g., IntraPredModeC[xCb][yCb]).

[0368] If the current block inherits cross-element model parameters from a neighboring block, the current chroma intra prediction mode (e.g., IntraPredModeC[xCb][yCb] defined in the VVC standard) is temporarily set to a cross-element mode (e.g., CCLM_LA) during the bitstream syntax parsing phase. Subsequently, in the prediction phase or the reconstruction phase, the candidate list is derived and the inherited candidate model is determined by the inherited candidate index. After obtaining the inherited model, the codec information of the current block is updated according to the inherited candidate model. The codec information of the current block includes, but is not limited to, a prediction mode (e.g., CCLM_LA or MMLM_LA), a related sub-mode flag (e.g., CCCM mode flag), a prediction mode (e.g., GLM mode index), and the current model parameters. Then, a prediction of the current block is generated based on the updated codec information.

[0369] Remove or modify similarity model parameters when adding candidates to the history table

[0370] When adding a cross-element model to the history table, the similarity between the model to be added and the existing models in the history table can be further checked. If the model to be added is similar to the existing model, the model to be added will not be included in the history table. In one embodiment, the similarity of (α×lumaAvg+β) or α with the existing candidates can be compared to determine whether to include the model to be added. For example, if the (α×lumaAvg+β) or α of the model to be added is the same as one of the existing candidates, the model to be added is not included. Another example is that if the difference in (α×lumaAvg+β) or α between the model to be added and the existing model is less than a threshold, the model to be added is not included. In addition, the threshold can be adaptively adjusted based on codec information (for example, the size or area of ​​the current block). Another example is that when comparing similarities, if both the model to be added and the existing model use CCCM, the value of (c0C+c1N+c2S+c3E+c4W+c5P+c6B can be checked to determine whether to include the model to be added. In another embodiment, if the CU position of the current model to be added is the same as the CU position of the existing candidate, the parameters of the model to be added are not included. In another embodiment, if the model to be added is similar to one of the existing candidate models, the inherited model parameters can be adjusted to make the model to be added different from the existing candidate models. For example, if the scaling parameters to be added are similar to one of the existing candidate models, the scaling parameters to be added can be added with a predefined offset (e.g., 1>>S or -(1>>S), where S is a shift parameter) to make the model to be added different from the existing candidate models.

[0371] In another embodiment, only some of the model parameters are compared with existing models in the history table. For example, a CCLM candidate has scale and offset parameters, and only the scale or offset parameters can be compared to see if they are the same or similar to existing candidates. If the scale or offset parameters are the same or similar, the model to be added will not be included in the history table. Another example is that a CCCM candidate has c0 to c6 parameters, and only n parameters (n<7) can be compared to see if they are the same or similar to existing candidates. If the scale or offset parameters are the same or similar, the model to be added will not be included in the history table.

[0372] In another embodiment, the model to be added can be applied to the neighboring reconstructed samples of the current block and the difference can be compared with the existing candidate models. If the difference value is less than or equal to a threshold, the model to be added will not be included in the history table. For example, if the result of the application is The corresponding results of the existing models in the history table are arrive if or The model to be added will not be included in the history table. For the selection of neighboring reconstruction samples, you can choose the neighboring reconstruction sample with the maximum value, the neighboring reconstruction sample with the minimum value, the average / median / mode of the neighboring reconstruction samples, the neighboring reconstruction sample on the left, the neighboring reconstruction sample on the top, or the neighboring reconstruction sample on the top left.

[0373] In another embodiment, when adding candidates to the history table, the number of candidates of the same type (e.g., MMLM, CCCM, or GLM) is limited. For example, if the current history table contains k MMLM-type candidates, no further MMLM-type candidates are allowed to be added to the history table. Another example is if the current history table contains k CCCM-type candidates, no further CCCM-type candidates are allowed to be added to the history table. Another example is if the current history table contains k GLM-type candidates, no further GLM-type candidates are allowed to be added to the history table.

[0374] In another embodiment, the constraints or rules for preventing redundant candidates from being added to the history table are the same as the constraints or rules for preventing redundant candidates from being added to the candidate list (e.g., the constraints or rules mentioned in the section entitled "Removing or Modifying Similar Neighbor Model Parameters").

[0375] Inheriting multiple cross-component models

[0376] The final prediction for the current block can be a combination of multiple cross-element models, or a fusion of the predictions of the selected cross-element model and non-cross-element codec tools (e.g., intra-frame angular prediction mode, intra-frame planar / DC mode, or inter-frame prediction mode). In one embodiment, if the size of the current candidate list is N, k candidates (where k≤N) can be selected from a total of N candidates. Then, k predictions are generated respectively by applying the cross-element models of the selected k candidates to the corresponding luma reconstructed samples. The final prediction for the current block is the combination result of these k predictions. For example, if two candidate predictions (denoted as p cand1 and p cand2 ) are combined, and the final prediction of the current block at the (x, y) position is p final (x,y)=(1-α)×p cand1 (x,y)+α×p cand2 (x,y), where α is a weighting factor. In addition, the weighting factor α can be predefined or implicitly derived from the neighboring template costs. For example, by using the template costs defined in the section titled "Inheriting the Non-Adjacent Spatial Neighborhood Model", the corresponding template costs of the two candidates are e cand1 and e cand2 , then α is e cand1 / (e cand1 +e cand2 In another embodiment, if two candidate models are combined, the selected model is from the first two candidates in the list. In yet another embodiment, if i candidate models are combined, the selected model is from the first i candidates in the list.

[0377] In another embodiment, if the size of the current candidate list is N, k candidates (where k ≤ N) may be selected from a total of N candidates. These k cross-element models may be combined into a final cross-element model by weighted averaging of the corresponding model parameters. For example, if a cross-element model has M parameters, the jth parameter of the final cross-element model is the weighted average of the jth parameters of the selected k candidates, where j is 1…M. The final prediction is then generated by applying the final cross-element model to the corresponding luminance reconstruction sample. For example, if two candidate models are and The final cross-component model is where α is a weighting factor that can be predefined or implicitly derived from the neighboring template cost, is the xth model parameter of the yth candidate. For example, by using the template cost defined in the section titled “Inheriting the Non-Adjacent Spatial Neighborhood Model”, the corresponding template cost of the two candidates is e cand1 and e cand2 , then α is e cand1 / (e cand1+e cand2 ). For another example, one of the two candidate models is from a spatially adjacent neighboring candidate, and the other is from a non-adjacent spatial candidate or a historical candidate. If a spatially adjacent neighboring candidate is not available, both candidate models are from non-adjacent spatial candidates or historical candidates. In another embodiment, if two candidate models are combined, the selected model is from the first two candidates in the list. In yet another embodiment, if i candidate models are combined, the selected model is from the first i candidates in the list.

[0378] In another embodiment, two cross-element models are combined into a final model by weighted averaging the corresponding model parameters, wherein the two cross-element models are respectively from the above-mentioned spatial neighboring candidates and the left spatial neighboring candidates. The above-mentioned spatial neighboring candidates refer to neighboring candidates whose vertical positions are less than or equal to the top boundary position of the current block. The left spatial neighboring candidates refer to neighboring candidates whose horizontal positions are less than or equal to the left boundary position of the current block. The weighting factor α is determined according to the horizontal and vertical spatial positions within the current block. For example, if two candidate predictions (denoted as p above and p left ) are combined, the final prediction of the current block at the (x, y) position is p final (x,y)=(1-α)×p above (x,y)+α×p left (x, y), where α = y / (x + y). In another embodiment, the spatially adjacent candidate is the first candidate in the list whose vertical position is less than or equal to the top boundary position of the current block. The left spatially adjacent candidate is the first candidate in the list whose horizontal position is less than or equal to the left boundary position of the current block.

[0379] In another embodiment, the cross-component model candidate can be combined with the prediction of the non-cross-component codec tool. For example, a cross-component model candidate is selected from the list, and its prediction is represented as p ccm Another prediction can come from chroma DM, chroma DIMD or intra angle mode, denoted as p non-ccm The final prediction of the current block at position (x, y) is p final (x,y)=(1-α)×p ccm (x,y)+α×p non-ccm(x,y), where α is a weighting factor that can be predefined or implicitly derived from the neighboring template cost. For the same example, the prediction of the non-cross-element codec tool can be predefined or transmitted by signal. The prediction of the non-cross-element codec tool is chroma DM or chroma DIMD. For another example, the prediction of the non-cross-element codec tool is transmitted by signal, but the index of the cross-element model candidate is predefined or determined by the codec mode of the neighboring block. For the same example, if at least one neighboring spatial block is encoded using CCCM mode, the first candidate with CCCM model parameters is selected. If at least one neighboring spatial block is encoded using GLM mode, the first candidate with GLM mode parameters is selected. Similarly, if at least one neighboring spatial block is encoded using MMLM mode, the first candidate with MMLM parameters is selected.

[0380] In another embodiment, the cross-component model candidate can be combined with the prediction of the current cross-component model. For example, a cross-component model candidate is selected from the list, and its prediction is represented as p ccm Another prediction can come from the cross-element prediction mode of the current neighboring reconstructed samples, denoted as p curr-ccm The final prediction of the current block at position (x, y) is p final (x,y)=(1-α)×p ccm (x,y)+α×p curr-ccm (x,y), where α is a weighting factor that can be predefined or implicitly derived from the neighboring template cost. For the same example, the prediction of the current cross-element model can be predefined or transmitted by signal. The prediction of the non-cross-element codec tool is chroma DM or chroma DIMD. For another example, the prediction of the non-cross-element codec tool is transmitted by signal, but the index of the cross-element model candidate is predefined or determined by the codec mode of the neighboring block. For the same example, if at least one neighboring spatial block is encoded using CCCM mode, the first candidate with CCCM model parameters is selected. If at least one neighboring spatial block is encoded using GLM mode, the first candidate with GLM mode parameters is selected. Similarly, if at least one neighboring spatial block is encoded using MMLM mode, the first candidate with MMLM parameters is selected.

[0381] In another embodiment, multiple cross-element models can be combined into a final cross-element model. For example, a model can be selected from one candidate and a second model can be selected from another candidate as a multi-model mode. The selected candidate can be a CCLM / MMLM / GLM / CCCM encoding candidate. The multi-model classification threshold can be the average of the offset parameters of the two selected modes (for example, offset / β in CCLM, or c6×B or c6 in CCCM). In one embodiment, if two candidate models are combined, the selected models are the first two candidates in the list. In another embodiment, the classification threshold is set to the average of the neighboring luminance and chrominance samples of the current block.

[0382] Refining inheritance candidate positions

[0383] In one embodiment, the final inherited model of the current block is derived from the cross-element model of the indicated candidate position with a position increment. For example, if the currently selected candidate position is Then you can further signal a position increment, To indicate the location of the final inherited model. That is, the final inherited model of the current block comes from In one embodiment, the signal position increment can only have horizontal position increment or vertical position increment, that is, or Furthermore, the signal position increment can be shared between multiple color components or specific to each color component signal. For example, the signal position increment is shared for the current Cb and Cr blocks, or the signal position increment is used only for the current Cb block or the current Cr block. or There may be a sign bit to indicate positive or negative position increments. or When the size is , the signal can be indexed through the lookup table. For example, if the lookup table is {1,2,4,8,16,…}, If equal to 8, then the signal table index is 3 (the first table index is 0).

[0384] In one embodiment, when a candidate is selected from the candidate list, the model of the neighboring position of the selected candidate is further searched. The final inherited model can come from the neighboring position of the selected candidate. The position of the predefined search pattern in the area around the selected candidate is searched. In one embodiment, the neighboring position searched is different from the selected candidate in the horizontal or vertical direction, that is, the position increment is or In another embodiment, the neighboring position to be searched is different from the selected candidate on the diagonal line, that is, the position increment is in Note that position increments can be positive or negative.

[0385] In another embodiment, only when the selected candidate is a non-neighboring candidate, the model of the candidate's neighboring position is further searched. Positions with a predefined search pattern are searched within the area around the selected candidate. For example, assuming that the distance between non-neighboring candidates is the width and height of the current codec block. After selecting a non-neighboring candidate, positions with both horizontal and vertical distances less than the width and height of the current codec block are further searched, i.e., In the range of ±width, In one embodiment, the searched neighboring position is different from the selected candidate in the horizontal or vertical direction, that is, the position difference is or In another embodiment, the neighboring position searched differs from the selected candidate on the diagonal, i.e., the position difference is in

[0386] Inheriting from a shared cross-component model

[0387] In one embodiment, the current picture is divided into multiple non-overlapping regions, each of which is of size M×N. A shared cross-element model is derived for each region. The neighboring available luminance / chrominance reconstruction samples of the current region are used to derive the shared cross-element model of the current region. Then, for the blocks within the current region, it can be determined whether to inherit the shared cross-element model or derive the cross-element model by the neighboring available luminance / chrominance reconstruction samples of the block. In one embodiment, M×N can be a predefined value (e.g., 32x32 with respect to the chroma format), a signaled value (e.g., signaled at the sequence / picture / slice / tile level), a derived value (e.g., depending on the CTU size), or the maximum allowed transform block size.

[0388] In another embodiment, multiple shared cross-element models may be used for each region. For example, multiple shared cross-element models may be derived using various proximity templates (e.g., top and left neighboring samples, top neighboring samples only, left neighboring samples only). Furthermore, the shared cross-element model for the current region may be inherited from a previously used cross-element model. For example, a shared model may be inherited from adjacent spatial neighbors, non-adjacent spatial neighbors, temporal neighbors, or a model in a history list.

[0389] When signaling, the first flag can be used to determine whether the current cross-component model inherits from a shared cross-component model. If the current cross-component model inherits from a shared cross-component model, the second syntax indicates the inheritance index of the shared cross-component model (e.g., signaled using truncated unary, Exp-Golomb, or fixed-length codes).

[0390] Share buffer resources with existing codec tools

[0391] To store cross-element model (CCM) information (e.g., prediction mode, related sub-mode flags, prediction mode or model parameters) for further model inheritance, the buffer used to store inter-frame codec information (e.g., motion vector buffer) is shared with the cross-element merge mode to store CCM information. By sharing the buffer between different codec tools, the buffer size can be reduced. Otherwise, buffer space must be allocated separately to store CCM information and inter-frame codec information. The key idea of ​​sharing the buffer is that a block is encoded using only one selected codec mode from multiple candidates. Therefore, the codec information of various codec modes can share a common buffer. Assume that the minimum allowed block size is m×n, the current CTU size is p×q, and the current picture size is r×s. The CTU-level buffer and the picture-level buffer are used to store the inter-frame codec and CCM information of the current CTU and each picture, respectively. Create a CTU-level buffer to store the final inter-frame codec or CCM information. The CTU-level buffer size is because corresponds to the number of blocks in the horizontal direction, corresponds to the number of blocks in the vertical direction, Corresponding to the total number of blocks in the CTU. Create a picture-level buffer to store the final inter-frame codec or CCM information of the current picture. The size of the picture-level buffer is Where i≥m and j≥n. In other words, the codec information is stored in the picture buffer in units of i×j. Corresponding to the second block number in the horizontal direction, Corresponding to the second block number in the vertical direction, Corresponding to the total number of the second block in the picture. After encoding or decoding the current block, the inter-frame codec or CCM information of the current block is first saved to the corresponding position of the CTU-level buffer in units of m×n, where the corresponding position is the position covered by the current block in units of m×n. Later, after encoding or decoding the current CTU, the inter-frame codec or CCM information in the current CTU-level buffer is saved to the corresponding position of the picture-level buffer in units of i×j.

[0392] However, if the units of the CTU-level buffer and the picture-level buffer are different (e.g., i>m or j>n), the inter-frame codec or CCM information in the CTU-level buffer should be subsampled to be saved in the picture-level buffer. Assuming i / m=g and j / n=h, select a corresponding position from each g×h grid in the CTU-level buffer to save the inter-frame codec or CCM information to the picture-level buffer. For example, Figure 29As shown, if g=2 and h=2, a position is selected from each 2x2 grid to save the inter-frame codec or CCM information to the corresponding position of the picture level buffer. In one embodiment, the selected position can be the upper left, lower left, upper right or lower right position of each 2x2 grid. Figure 29 As shown, the inter-frame codec or CCM information at the upper left position marked with a diagonal line in each 2x2 grid is saved to the picture-level buffer. In another embodiment, when the CCM information in the CTU-level buffer is subsampled to save it to the picture-level buffer, the prediction mode within the g×h grid can be conditionally checked. For example, if more than a certain percentage of the positions within the g×h grid are intra-frame modes (for example, more than 50% or 75%), the data selected and saved is CCM information. Otherwise (that is, most of the positions within the g×h grid are inter-frame modes), the data selected and saved is inter-frame codec information. When selecting candidates to save to the picture-level buffer, the first allowed candidate can be selected according to a predefined scanning order. For example, if the data selected and saved is CCM information, the first grid with CCM information within the g×h grid can be selected using the predefined scanning order. For another example, if the data selected and saved is inter-frame codec information, the first grid with inter-frame codec information within the g×h grid can be selected using the predefined scanning order.

[0393] Since the buffer for storing inter-frame codec information is shared with the cross-element merge mode, the CU prediction mode (e.g., intra-frame prediction or inter-frame prediction) can be checked to identify whether the information stored at a certain buffer location is inter-frame codec or CCM information. In one embodiment, if the CU prediction mode is intra-frame prediction, the stored information is CCM information. Otherwise (i.e., the CU prediction mode is non-intra-frame prediction), the stored information is inter-frame codec information. In another embodiment, an invalid inter-frame prediction reference index or an invalid MV value (e.g., a horizontal or vertical MV value) can be set to identify that the stored information is CCM information. Otherwise (i.e., a valid inter-frame prediction index), the stored information is inter-frame codec information. For example, in the VVC standard specification, an inter-frame prediction reference index greater than 2 is invalid, then the inter-frame prediction reference index can be set to a value greater than 2 to identify that the stored information is CCM information (e.g., the inter-frame prediction reference index is 3).

[0394] According to this method, the current block is divided into two or more prediction regions / sub-blocks, each of which can be predicted using inter-frame or intra-frame codec tools. In addition, at least one prediction region is encoded using CC merge mode, where the cross-element model of at least one region is inherited from a neighboring block / location in space, history, or time. In one embodiment, the current block is partitioned using quadtree, binary tree, or ternary tree partitioning. The partitioning can be symmetric or asymmetric.

[0395] In another embodiment, the current block is divided into two regions, one of which is predicted by an inter-frame or intra-frame codec tool and the other is predicted by a CC merge mode. The inherited candidate index of the region predicted by the CC merge mode can be indicated explicitly or implicitly. For example, the candidate index can be explicitly signaled by the method in the section entitled "Signaling the inherited candidate index in the list". As another example, the first candidate in the list can be implicitly selected as the candidate index. The candidates in the list can be reordered by the method mentioned in the section entitled "Reordering candidates in the list".

[0396] In another embodiment, the current block is divided into two regions, both of which are predicted using CC merge mode, and the first two candidates in the list are the candidate indices of the two regions. The candidate index of the first region (e.g., the region with the top-left corner sample of the current block) can be implicitly set to the first candidate, and the candidate index of the second region can be set to the second candidate. In addition, the list can be reordered by the method mentioned in the section entitled "Reordering Candidates in the List". As another example, if both regions are predicted using CC merge mode, an index is explicitly signaled to indicate the candidate index of the first region, and the candidate index of the second region is the signaled index + k or the signaled index - k, where k can be 1, 2, 3, 4, or 5. For the same example, the candidate index of the first region is implicitly derived from the cross-element model stored in the top-left corner position of the current block relative to the previously coded slice / image, as mentioned in the section entitled "Inheriting Temporal Neighborhood Model Parameters". In another embodiment, if both regions are predicted using CC merge mode, the first candidate in the list is the candidate index of both regions.

[0397] Skip mode across components

[0398] With the help of many cross-component models, intra chroma prediction becomes more accurate and the entropy of the residual becomes smaller. In addition, the probability of all CBFs (Codec Block Flags) or coding flags of chroma components being equal to 0 in intra codec mode increases compared to previous video codec standards. Based on this observation, several methods related to skip mode shortcuts are proposed to improve the codec performance of intra chroma prediction.

[0399] In one embodiment, a skip flag may be signaled before the merge flag, which indicates whether the current coding block inherits cross-element model parameters from neighboring blocks.

[0400] In one embodiment, if the skip flag is true (i.e., skip mode is used), the merge flag will be inferred to be 1, the index of the inheritance candidate will be directly signaled, and the CBF of the Cb component and the CBF of the Cr component are both inferred to be 0, as shown in FIG. Figure 30 As shown. Figure 30 In FIG, when the skip flag (ie, skipFlag) is true (ie, has a value of 1), the index of the inheritance candidate (ie, mergeIdx) is signaled (3010). The merge flag is inferred to be 1, and the CBFs of both the Cb and Cr components are inferred to be 0, as indicated by the dashed box 3012.

[0401] In another embodiment, if the skip flag is true, the merge flag will be inferred to be 0, the CCP mode index will be signaled, and the CBF of the Cb component and the CBF of the Cr component will both be inferred to be 0, as shown in FIG. Figure 31 As shown. Figure 31 In FIG, when the skip flag (i.e., skipFlag) is true (i.e., has a value of 1), the index of the inheritance candidate (i.e., LMModeIdx) is signaled (3110). The merge flag is inferred to be 0, and the CBFs of both the Cb and Cr components are inferred to be 0, as indicated by the dashed box 3112.

[0402] In one embodiment, high-level syntax may be signaled in the SPS, PPS, PH, or SH to indicate whether the current sequence, picture, or slice is allowed to use the proposed skip flag.

[0403] The improved syntax coding and decoding of cross-element prediction can be implemented at the encoder side or the decoder side. For example, any of the proposed syntax coding and decoding methods for cross-element prediction can be implemented in the intra / inter coding and decoding module (e.g., Figure 1B 150 / MC 152) in the Intra Pred. 150 / MC 152), or the intra / inter encoding / decoding module is a module in the encoder (for example, Figure 1A Any proposed shared buffer for storing codec information between multiple codecs, including CCM mode, can also be implemented as a circuit coupled to the intra / inter codec module of a decoder or encoder. However, the decoder or encoder can also use additional processing units to implement the required cross-element prediction processing. Although the intra prediction unit (e.g., Figure 1A Units 110 / 112 and Figure 1BThe units 150 / 152 in the figures are shown as separate processing units, but they may correspond to executable software or firmware code stored on a medium (such as a hard disk or flash memory) for a central processing unit (CPU) or a programmable device (for example, a digital signal processor (DSP) or a field programmable gate array (FPGA)).

[0404] Figure 32 A flowchart of an exemplary video decoding system is shown, which uses information related to cross-component prediction to decode syntax data using one or more context models, according to one embodiment of the present invention. The steps shown in the flowchart can be implemented as program code executable by one or more processors (e.g., one or more central processing units) at the encoder end. The steps shown in the flowchart can also be implemented in hardware, for example, with one or more electronic devices or processors configured to perform the steps in the flowchart. According to the decoder method, in step 3210, encoded data related to a current block is received, including a first color block and a second color block, where the current block is encoded using cross-component prediction (CCP). In step 3220, encoded syntax data is parsed for one or more syntaxes related to a transform codec or a residual codec applied to the current block. In step 3230, the encoded syntax data is decoded using one or more context models based on information related to the CCP to generate the one or more syntaxes. In step 3240, the current block is decoded using the CCP, where decoding includes performing transform coding or residual coding on the current block using the one or more syntaxes.

[0405] Figure 33 A flowchart of an exemplary video encoding system is shown, which, according to one embodiment of the present invention, uses information related to cross-component prediction to encode syntax data using one or more context models. According to the decoder-side method, in step 3310, input data related to a current block is received, including a first color block and a second color block. In step 3320, the current block is encoded using cross-component prediction (CCP), wherein the encoding process of the current block includes transform coding and decoding of the current block to generate transform data or residual coding and decoding of the current block to generate residual data. In step 3330, based on information related to CCP, one or more syntax related to transform coding and decoding or residual coding and decoding are encoded using one or more context models to generate encoded syntax data. In step 3340, the encoded syntax data is transmitted by signal.

[0406] Figure 34 A flow chart of an exemplary video codec system is shown, which, according to one embodiment of the present invention, encodes or decodes a current block using information including chroma syntax corresponding to one or more codec block flags. In step 3410, input data associated with the current block is received, including a luma block and two or more chroma blocks, wherein the input data includes pixel data to be encoded at the encoder end or encoded data associated with the current block for decoding at the decoder end, and the current block uses cross-component prediction (CCP) for intra-frame encoding and decoding. In step 3420, the current block is encoded or decoded using CCP, wherein the encoding or decoding uses information including chroma syntax corresponding to one or more codec block flags, wherein the one or more codec block flags include a root chroma codec block flag.

[0407] Figure 35 A flow chart of an exemplary video encoding and decoding system is presented, which, according to one embodiment of the present invention, uses a merge flag to indicate whether a current block inherits one or more cross-component model parameters from an inherited CCP candidate. According to the method, in step 3510, input data associated with a current block is received, including a luma block and two or more chroma blocks. The input data includes pixel data to be encoded at the encoder or previously encoded data associated with the current block for decoding at the decoder. The current block is intra-encoded or decoded using cross-component prediction (CCP). In step 3520, a merge candidate list is generated for the current block, where the merge candidate list includes CCP candidates inherited from neighboring blocks. In step 3530, a merge flag is signaled or parsed, indicating whether the current block inherits one or more cross-component model parameters from the inherited CCP candidate. In step 3540, a skip flag is signaled or parsed before the merge flag, where the merge flag is inferred when the skip flag is true. In step 3550, the current block is encoded or decoded using information including the merge candidate list.

[0408] The flowchart shown is intended to illustrate an example of video encoding and decoding according to the present invention. Those skilled in the art may modify each step, rearrange the steps, split the steps, or combine the steps to practice the present invention without departing from the spirit of the present invention. In this disclosure, specific syntax and semantics are used to illustrate examples of implementing embodiments of the present invention. Those skilled in the art may practice the present invention by replacing the steps with equivalent syntax and semantics without departing from the spirit of the present invention.

[0409] The above description is intended to enable those skilled in the art to practice the present invention in the context of specific applications and their requirements. For professionals in the technical field, various modifications to the described embodiments will be apparent, and the general principles defined herein can be applied to other embodiments. Therefore, the present invention is not intended to be limited to the specific embodiments shown and described, but should be given the maximum scope consistent with the principles and novel features disclosed herein. In the above detailed description, various specific details are shown to provide a thorough understanding of the present invention. However, professionals in the technical field will understand that the present invention can be put into practice.

[0410] Based on the above claims in Chinese and English and the rules, the following is the translation of Spec:

[0411] As described above, embodiments of the present invention can be implemented in various hardware, software code, or a combination of both. For example, one embodiment of the present invention may be one or more circuits integrated into a video compression chip, or program code integrated into video compression software, to perform the processing described herein. Another embodiment of the present invention may be program code to be executed on a digital signal processor (DSP) to perform the processing described herein. The invention may also involve multiple functions performed by a computer processor, digital signal processor, microprocessor, or field programmable gate array (FPGA). These processors can be configured to perform specific tasks embodied in accordance with the present invention by executing machine-readable software code or firmware code. The software code or firmware code can be developed in different programming languages ​​and in different formats or styles. The software code can also be compiled for different target platforms. However, different code formats, styles, and languages ​​for the software code, as well as other methods for configuring the code to perform tasks consistent with the present invention, do not depart from the spirit and scope of the present invention.

[0412] The present invention may be embodied in other specific forms without departing from its spirit or essential characteristics. The examples described are to be considered in all respects as illustrative and not restrictive. The scope of the present invention is therefore indicated by the appended claims rather than by the foregoing description. All changes that come within the meaning and range of equivalence of the claims are intended to be included within their scope.

Claims

1. A method for decoding a color image using a codec tool, comprising one or more cross-component model related modes (CCM-Related Modes), the method comprising: Receiving codec data associated with a current block, including a first color block and a second color block, wherein the current block is coded and decoded using Cross-Component Prediction (CCP); Parsing codec syntax data of one or more syntaxes related to transform coding or residual coding applied to the current block; Decoding the codec syntax data using one or more context models based on information associated with the CCP to generate the one or more syntaxes; and The current block is decoded using the CCP, wherein the decoding includes transform coding or residual coding of the current block using the one or more syntaxes.

2. The method of claim 1 , wherein the one or more syntax elements are selected from the group consisting of root_cbf, cb_cbf, cr_cbf, transform_skip_flag, lfnst_idx, mts_idx, tu_joint_cbcr_residual_flag, last_significant_position, cu_qp_delta_abs, cu_qp_delta_sign_flag, cu_chroma_qp_offset_flag, cu_chroma_qp_offset_idx, or a combination thereof. 3 . The method of claim 1 , wherein the one or more context models correspond to a plurality of context models, and selection among the plurality of context models depends on a CCP type of the current block. 4 . The method of claim 1 , wherein the one or more context models correspond to a plurality of context models, and selection among the plurality of context models depends on prediction modes of neighboring blocks of the current block. The method of claim 4 , wherein the prediction mode of the neighboring block corresponds to a CCP mode.

6. The method of claim 1, wherein the one or more context models correspond to a plurality of context models, and selection between the plurality of context models depends on a number of neighboring blocks encoded or decoded in CCP mode.

7. The method of claim 1, wherein the one or more context models correspond to a CCP single model, a CCP multi-model, a convolutional cross-component model (CCCM), a gradient linear model (GLM), or a combination thereof.

8. An apparatus for video decoding, the apparatus comprising one or more electronic devices or processors arranged to: Receiving codec data associated with a current block, including a first color block and a second color block, wherein the current block is coded and decoded using Cross-Component Prediction (CCP); Parsing codec syntax data of one or more syntaxes related to transform coding or residual coding applied to the current block; Decoding the codec syntax data using one or more context models based on information associated with the CCP to generate the one or more syntaxes; and The current block is decoded using the CCP, wherein a decoding process of the current block includes transform coding or residual coding of the current block using the one or more syntaxes.

9. A method for encoding a color image using a codec tool comprising one or more Cross Component Models Related Modes, the method comprising: receiving input data associated with a current block, including a first color block and a second color block; encoding the current block using cross-component prediction (CCP), wherein the encoding includes transform coding the current block to generate transform data or residual coding the current block to generate residual data; and Encoding one or more syntaxes related to the transform codec or the residual codec using one or more context models according to the information related to the CCP to generate codec syntax data; and The signal transmits the codec syntax data.

10. An apparatus for video encoding, the apparatus comprising one or more electronic devices or processors arranged to: receiving input data associated with a current block, including a first color block and a second color block; encoding the current block using cross-component prediction (CCP), wherein the encoding process of the current block includes transform coding (transform coding) of the current block to generate transform data or residual coding (residual coding) of the current block to generate residual data; and Encoding one or more syntaxes related to the transform codec or the residual codec using one or more context models according to the information related to the CCP to generate codec syntax data; and The signal transmits the codec syntax data.

11. A method for encoding and decoding a color image using a codec tool comprising one or more cross component model related modes, the method comprising: receiving input data associated with a current block, including a luma block and two or more chroma blocks, wherein the input data includes pixel data to be encoded at an encoder or encoded data associated with the current block for decoding at a decoder, and the current block is intra-coded and decoded using Cross-Component Prediction (CCP); and The current block is encoded or decoded using the CCP, wherein the encoding or decoding uses information including a chroma syntax corresponding to one or more coded block flags, the one or more coded block flags including a root chroma coded block flag.

12. The method of claim 11, wherein CCP candidates inherited from neighboring blocks are included in a merge candidate list (Merge Candidate List) of the current block, and a merge flag indicates whether the current block inherits one or more cross-component model parameters from the inherited CCP candidates.

13. The method of claim 11, wherein the one or more codec block flags further include two or more component codec block flags corresponding to the two or more chroma blocks.

14. The method of claim 13 , wherein if the root chroma codec block flag is false, each element codec block flag corresponding to the two or more chroma blocks is equal to 0; or if the root chroma codec block flag is true, at least one of the two or more element codec block flags corresponding to the two or more chroma blocks is equal to 1.

15. A device for video encoding and decoding, the device comprising one or more electronic devices or processors arranged to: receiving input data associated with a current block, including a luma block and two or more chroma blocks, wherein the input data includes pixel data to be encoded at an encoder or encoded data associated with the current block for decoding at a decoder, and the current block is intra-coded and decoded using Cross-Component Prediction (CCP); and The current block is encoded or decoded using the CCP, wherein the encoding or decoding process of the current block uses information including syntax of one or more Coded Block Flags, and the one or more Coded Block Flags include a Root Chroma Coded Block Flag.

16. The apparatus of claim 15, wherein the one or more codec block flags further include two or more element codec block flags corresponding to the two or more chroma blocks.

17. The apparatus of claim 16 , wherein if the root chroma codec block flag is false, each of the two or more component codec block flags corresponding to the two or more chroma blocks is equal to 0; or if the root chroma codec block flag is true, at least one of the two or more component codec block flags corresponding to the two or more chroma blocks is equal to 1.

18. A method for encoding and decoding a color image using a codec tool comprising one or more Cross Component Models Related Modes, the method comprising: receiving input data associated with a current block, including a luma block and two or more chroma blocks, wherein the input data includes pixel data to be encoded at an encoder or encoded data associated with the current block for decoding at a decoder, and the current block is intra-encoded and decoded using Cross-Component Prediction (CCP); generating a merge candidate list (Merge Candidate List) for the current block, wherein the merge candidate list includes CCP candidates inherited from neighboring blocks; Signaling or parsing a Merge Flag indicating whether the current block inherits one or more cross-component model parameters from the inherited CCP candidate; signaling or parsing a Skip Flag before the Merge Flag, wherein the Merge Flag is inferred when the Skip Flag is true; as well as The current block is encoded or decoded using information including the merge candidate list.

19. The method of claim 18, wherein when the skip flag is true, an index associated with the inherited CCP candidate is signaled or resolved. 20 . The method of claim 18 , wherein when the merge flag is inferred to be 1, the current block inherits one or more cross-component model parameters (Cross-Component Model Parameters) from the inherited CCP candidate.

21. The method of claim 18, wherein when the merge flag is inferred to be 0, a CCP mode index is signaled or parsed.

22. The method of claim 18, wherein a high-level syntax is signaled or parsed in a Sequence Parameter Set (SPS), a Picture Parameter Set (PPS), a Picture Header (PH), or a Slice Header (SH) to indicate whether the skip flag is allowed to be used for the current sequence, picture, or slice.

23. An apparatus for video encoding and decoding, the apparatus comprising one or more electronic devices or processors configured to: receiving input data associated with a current block, including a luma block and two or more chroma blocks, wherein the input data includes pixel data to be encoded at an encoder or codec data associated with the current block to be decoded at a decoder, and the current block is intra-coded and decoded using Cross-Component Prediction (CCP); generating a merge candidate list (Merge Candidate List) of the current block, wherein the merge candidate list includes inherited CCP candidates from neighboring blocks; Signaling or parsing a Merge Flag indicating whether the current block inherits one or more cross-element model parameters from the inheriting CCP candidate; signaling or parsing a Skip Flag before the Merge Flag, wherein the Merge Flag is inferred when the Skip Flag is true; The current block is encoded or decoded using information including the merge candidate list.

Images