Method and device for improving performance of convolutional cross-component model in video coding and decoding system

Abstract

A CCCM method and apparatus for video coding are provided. According to one method, an autocorrelation matrix associated with a coverage area of a convolution filter for reference luminance samples in a reference region is derived. A cross-correlation vector associated with the coverage area between reference luma samples and reference chroma samples in the reference area is derived. The coefficients of the convolution filter are derived by using a Gaussian elimination method based on the autocorrelation matrix and the cross-correlation vector. For a target chroma sample in the chroma block, a convolution filter with derived coefficients is applied to corresponding locations of the luma block to form a convolutional cross-component model predictor for the target chroma sample. A final predictor for the target chroma sample is generated from a set of prediction candidates including a convolutional cross-component model predictor. The target chroma sample is encoded or decoded using the final predictor.

Description

[0001] Related references

[0002] This application claims priority to U.S. Provisional Application No. 63 / 369,524, filed on July 27, 2022. The entire contents of the above are incorporated herein by reference. Technical Field

[0003] The present invention relates to a video coding and decoding system, and more particularly to a scheme for improving the performance or reducing the complexity of a cross-component linear model (CCLM) related mode in a video coding and decoding system. Background Art

[0004] Versatile video coding (VVC) is the latest international video coding standard developed by the Joint Video Experts Team (JVET) of the ITU-T Video Coding Experts Group (VCEG) and the ISO / IEC Moving Picture Experts Group (MPEG). The standard was published as an ISO standard in February 2021: ISO / IEC23090-3:2021, Information technology - Codec representation of immersive media - Part 3: Versatile video coding. VVC is based on its previous generation High Efficiency Video Coding (HEVC) by adding more codec tools to improve codec efficiency and handle various types of video sources including three-dimensional (3D) video signals.

[0005] Figure 1AAn example adaptive inter / intra video codec system incorporating loop processing is shown. For intra prediction, prediction data is derived based on previously encoded video data in the current picture. For inter prediction 112, motion estimation (ME) is performed at the encoder end and motion compensation (MC) is performed based on the results of ME to provide prediction data derived from other pictures and motion data. Switch 114 selects intra prediction 110 or inter prediction 112, and the selected prediction data is provided to adder 116 to form a prediction error, also known as a residual. The prediction error is then processed by a transform (T) 118 followed by a quantization (Q) 120. The transformed and quantized residual is then encoded by an entropy encoder 122 to be included in a video bitstream corresponding to the compressed video data. The bitstream associated with the transform coefficients is then packaged with auxiliary information (such as auxiliary information such as motion and coding modes associated with intra prediction and inter prediction) and other information (parameters associated with loop filters applied to underlying image areas, etc.). As Figure 1A As shown, auxiliary information associated with intra prediction 110, inter prediction 112 and loop filter 130 is provided to entropy encoder 122. When inter prediction mode is used, one or more reference pictures must also be reconstructed at the encoder end. Therefore, the transformed and quantized residual is processed by inverse quantization (Inverse Quantization, referred to as IQ) 124 and inverse transformation (Inverse Transformation, referred to as IT) 126 to restore the residual. The residual is then added back to the prediction data 136 in reconstruction (REC) 128 to reconstruct the video data. The reconstructed video data can be stored in the reference picture buffer 134 and used for prediction of other frames.

[0006] like Figure 1A As shown, the input video data undergoes a series of processes in the encoding system. Due to the series of processes, the reconstructed video data from REC128 may be subject to various damages. Therefore, before the reconstructed video data is stored in the reference picture buffer 134, a loop filter 130 is generally applied to the reconstructed video data to improve the video quality. For example, a deblocking filter (DF for short), a sample adaptive offset (SAO for short), and an adaptive loop filter (ALF for short) may be used. The loop filter information may need to be incorporated into the bitstream so that the decoder can correctly recover the required information. Therefore, the loop filter information is also provided to the entropy encoder 122 to be incorporated into the bitstream. In Figure 1A In , a loop filter 130 is applied to the reconstructed video before the reconstructed samples are stored in a reference picture buffer 134 . Figure 1A The system in is intended to illustrate an example structure of a typical video encoder. It may correspond to a High Efficiency Video Coding (HEVC) system, VP8, VP9, ​​H.264, or VVC.

[0007] like Figure 1B The decoder shown may use similar or partially identical functional blocks as the encoder, except for transform 118 and quantization 120, since the decoder only needs inverse quantization 124 and inverse transform 126. The decoder uses an entropy decoder 140 instead of an entropy encoder 122 to decode the video bitstream into quantized transform coefficients and required codec information (e.g., ILPF information, intra-frame prediction information, and inter-frame prediction information). The intra-frame prediction 150 at the decoder end does not need to perform a pattern search. Instead, the decoder only needs to generate an intra-frame prediction based on the intra-frame prediction information received from the entropy decoder 140. In addition, for inter-frame prediction, the decoder only needs to perform motion compensation (MC 152) based on the intra-frame prediction information received from the entropy decoder 140 without motion estimation.

[0008] According to VVC, the input picture is divided into non-overlapping square block areas called coding tree units (CTUs), similar to HEVC. Each CTU can be divided into one or more smaller-sized coding units (CUs). The generated CU partitions can be square or rectangular. In addition, VVC divides CTUs into prediction units (PUs) as a unit to apply prediction processing such as inter-frame prediction, intra-frame prediction, etc.

[0009] Partition CTU using tree structure

[0010] In HEVC, CTUs are divided into CUs to accommodate various local characteristics using a quadtree (QT) structure represented as a codec tree. The decision to encode and decode a picture area using inter-picture (temporal) or intra-picture (spatial) prediction is made at the leaf-CU level. Each leaf-CU can be further partitioned into one, two, or four PUs according to the PU partition type. Within a PU, the same prediction process is applied, and related information is transmitted to the decoder on a PU basis. After obtaining the residual block by applying a prediction process based on the PU partition type, the leaf-CU can be divided into transform units (TUs) according to another quadtree structure similar to the codec tree of the CU. One of the key features of the HEVC structure is that it has multiple partition concepts, including CU, PU, ​​and TU.

[0011] In VVC, the concept of multiple partition unit types is replaced by a quadtree with a nested multi-type tree (MTT) with a binary and ternary partition structure, that is, it eliminates the independence of the concepts of CU, PU, ​​and TU, unless the size of the CU is too large for the maximum transform length, and supports more flexible CU partition shapes. In the codec tree structure, the CU can be a square or a rectangle. The coding tree unit (CTU) is first divided into a quadtree (also known as a quadtree) structure. Then the quadtree leaf nodes can be further divided into a multi-type tree structure. Figure 2 As shown, there are four types of splits in the multi-type tree structure, vertical binary split (SPLIT_BT_VER 210), horizontal binary split (SPLIT_BT_HOR220), vertical ternary split (SPLIT_TT_VER 230), and horizontal ternary split (SPLIT_TT_HOR 240). The multi-type tree leaf node is called a coding unit (CU), and this segmentation will be used for prediction and transform processing without further division unless the CU is too large for the maximum transform length. This means that in most cases, CU, PU, ​​and TU have the same block size in a quadtree with a nested multi-type tree coding block structure. An exception occurs when the maximum supported transform length is smaller than the width or height of the CU color component.

[0012] Figure 3The transmission mechanism of partitioning information in a quadtree with a nested multi-type tree codec tree structure is shown. The coding tree unit (CTU) is regarded as the root of the quadtree and is first divided by the quadtree structure. Then, each quadtree leaf node (when large enough) is further divided by the multi-type tree structure. In a quadtree with a nested multi-type tree codec tree structure, for each CU node, a first flag (split_cu_flag) is sent to indicate whether the node is further divided. If the current CU node is a quadtree CU node, the second flag (split_qt_flag) is sent to indicate whether it is a QT partition or an MTT partition mode. When the node is partitioned using the MTT partition mode, the third flag (mtt_split_cu_vertical_flag) is sent to indicate the partition direction, and then the fourth flag (mtt_split_cu_binary_flag) is sent to indicate whether the partition is a binary partition or a ternary partition. According to the values ​​of mtt_split_cu_vertical_flag and mtt_split_cu_binary_flag, the multi-type tree split mode (MttSplitMode) of the CU is derived, as shown in Table 1.

[0013] Table 1 - Derivation of MttSplitMode based on multi-type tree syntax elements

[0014] MttSplitMode mtt_split_cu_vertical_flag mtt_split_cu_binary_flag SPLIT_TT_HOR 0 0 SPLIT_BT_HOR 0 1 SPLIT_TT_VER 1 0 SPLIT_BT_VER 1 1

[0015] Figure 4 The CTU is shown to be divided into multiple CUs with quadtree and nested multi-type tree codec block structures, where the bold block edges represent quadtree partitions and the remaining edges represent multi-type tree partitions. The quadtree with nested multi-type tree partitions provides a content-adaptive codec tree structure composed of CUs. The size of the CU can be as large as the CTU or as small as 4×4 in units of luma samples. For the 4:2:0 chroma format, the maximum chroma CB size is 64×64 and the minimum size chroma CB consists of 16 chroma samples.

[0016] In VVC, the maximum supported luma transform size is 64 × 64 and the maximum supported chroma transform size is 32 × 32. When the width or height of the CB is larger than the maximum transform width or height, the CB is automatically split in the horizontal and / or vertical direction to meet the transform size limit in that direction.

[0017] The following parameters may be defined for quadtree definition with nested multi-type tree coding tree scheme. These parameters are specified by SPS syntax elements and may be further refined by picture header syntax elements.

[0018] -CTU size: the root node size of the quadtree

[0019] -MinQTSize: Minimum allowed quadtree leaf node size

[0020] -MaxBtSize: Maximum allowed binary tree root node size

[0021] -MaxTtSize: Maximum allowed ternary tree root node size

[0022] -MaxMttDepth: The maximum allowed hierarchical depth of the multi-type tree split from the leaves of the quad tree

[0023] -MinCbSize: The minimum allowed size of the decoded block node

[0024] In one example of a quadtree with a nested multi-type tree codec tree structure, the CTU size is set to 128×128 luma samples and two corresponding 64×64 blocks of 4:2:0 chroma samples, MinQTSize is set to 16×16, MaxBtSize is set to 128×128, MaxTtSize is set to 64×64, MinCbsize (width and height) is set to 4×4, and MaxMttDepth is set to 4. Quadtree partitioning is first applied to the CTU to generate quadtree leaf nodes. The size of the quadtree leaf node can range from 16×16 (i.e., MinQTSize) to 128×128 (i.e., CTU size). If the leaf QT node is 128×128, the binary tree will not be further split because the size exceeds MaxBtSize and MaxTtSize (i.e., 64×64). Otherwise, the quadtree leaf node may be further split by the multi-type tree. Therefore, the quadtree leaf node is also the root node of the multitree, and its multitree depth (mttDepth) is 0. When the multitree depth reaches MaxMttDepth (i.e. 4), no further splitting is considered. When the width of the multitype tree node is equal to MinCbsize, no further vertical splitting is considered. Similarly, when the height of the multitype tree node is equal to MinCbsize, no further horizontal splitting is considered.

[0025] In VVC, the codec tree scheme supports the ability to have separate block tree structures for luma and chroma. For P and B slices, the luma and chroma CTBs in one CTU must share the same codec tree structure. However, for I slices, luma and chroma can have separate block tree structures. When separate block tree mode is applied, the luma CTB is partitioned into CUs by one codec tree structure, and the chroma CTB is partitioned into chroma CUs by another codec tree structure. This means that a CU in an I slice may consist of codec blocks for the luma component or codec blocks for two chroma components, while a CU in a P or B slice always consists of codec blocks for all three color components unless the video is monochrome.

[0026] Virtual Pipeline Data Unit (VPDU)

[0027] A virtual pipeline data unit (VPDU) is defined as a non-overlapping unit in a picture. In a hardware decoder, consecutive VPDUs are processed simultaneously by multiple pipeline stages. In most pipeline levels, the VPDU size is roughly proportional to the buffer size, so it is important to keep the VPDU size small. In most hardware decoders, the VPDU size can be set to the maximum transform block (TB) size. However, in VVC, ternary tree (TT) and binary tree (BT) segmentation may cause the VPDU size to increase.

[0028] To keep the VPDU size to 64x64 luma samples, the following specification segmentation restrictions (with syntax signaling modifications) are applied to the VTM, as Figure 5 As shown:

[0029] - For CUs with width or height, or width and height equal to 128, TT partitioning (such as Figure 5 ) are not allowed.

[0030] - For 128xN CUs with N≤64 (i.e., width equal to 128 and height less than 128), horizontal BT is not allowed.

[0031] - For Nx128 CUs with N≤64 (i.e., height equal to 128 and width less than 128), vertical BT is not allowed.

[0032] exist Figure 5 In the example above, the luma block size is 128x128. The dashed line indicates a block size of 64x64. According to the above restrictions, the segmentation examples that are not allowed are indicated by "X", such as Figure 5 As shown in various examples (510-580).

[0033] Intra-frame chroma partitioning and prediction constraints

[0034] In typical hardware video encoders and decoders, the processing throughput drops when a picture has more small intra blocks due to the sample processing data dependency between adjacent intra blocks. The predictor generation for an intra block requires reconstructed samples from the top and left boundaries of the adjacent blocks. Therefore, intra prediction must be processed continuously block by block.

[0035] In HEVC, the smallest intra CU is 8x8 luma samples. The luma component of the smallest intra CU can be further split into 4 4x4 luma intra prediction units (Pus), but the chroma component of the smallest intra CU cannot be further split. Therefore, the worst-case hardware processing throughput occurs when a 4x4 chroma intra block or a 4x4 luma intra block is processed. In VVC, in order to improve the worst-case throughput, by constraining the division of chroma intra CBs, chroma intra CBs with less than 16 chroma samples (size 2x2, 4x2, and 2x4) and chroma intra CBs with a width less than 4 chroma samples (size 2xN) are not allowed.

[0036] In a single codec tree, the smallest chroma intraprediction unit (SCIPU) is defined as a codec tree node whose chroma block size is greater than or equal to 16 chroma samples and has at least one sub-luminance block less than 64 luminance samples, or is defined as a codec tree node whose chroma block size is not 2xN and has at least one sub-luminance block (with 4xN luminance samples). It is required that in each SCIPU, all CBs are inter, or all CBs are non-inter, that is, intra or intra block copy (IBC). In the case of non-inter SCIPU, it is also required that the chroma of the non-inter SCIPU should not be further split and the luminance of the SCIPU is allowed to be further split. In this way, small chroma intra CBs with a size less than 16 chroma samples or a size of 2xN are removed. In addition, in the case of non-inter SCIPU, chroma scaling is not applied. Here, no additional syntax is sent, and whether the SCIPU is non-inter can be derived from the prediction mode of the first luma CB in the SCIPU. If the current segment is an I-segment, or the current SCIPU has a 4x4 luma partition after further splitting once, the type of the SCIPU is inferred to be non-inter (because inter 4x4 is not allowed in VVC); otherwise, the type of the SCIPU (inter or non-inter) is indicated by a flag before parsing the CU in the SCIPU.

[0037] For dual trees in intra pictures, 2xN intra chroma blocks are removed by disabling vertical binary and vertical ternary partitioning for 4xN and 8xN chroma partitions, respectively. Small chroma blocks of size 2x2, 4x2, and 2x4 are also removed by partition restriction.

[0038] In addition, by considering the picture width and height to be multiples of max(8, MinCbSizeY), the limitation on the picture size is taken into account to avoid 2x2 / 2x4 / 4x2 / 2xN intra chroma blocks at the picture corners.

[0039] Intra-mode codec with 67 intra-prediction modes

[0040] To capture arbitrary edge directions that occur in natural videos, the number of directional intra modes in VVC is expanded from 33 used in HEVC to 65. New directional modes not found in HEVC are Figure 6 Indicated by dashed arrows in . Planar and DC modes remain unchanged. These denser directional intra prediction modes apply to all block sizes and for luma and chroma intra prediction.

[0041] In VVC, for non-square blocks, several traditional angular intra prediction modes are adaptively replaced with wide-angle intra prediction modes.

[0042] In HEVC, each intra-coded block has a square shape and the length of each of its sides is a power of 2. Therefore, no division operation is required to generate intra predictors using DC mode. In VVC, blocks can have a rectangular shape, which in general requires the use of division operations for each block. To avoid division operations for DC prediction, only the longer sides are used to calculate the average for non-square blocks.

[0043] In order to keep the complexity of most probable mode (MPM) list generation low, the intra mode codec with 6 MPMs is used by considering two available adjacent intra modes. The following three aspects are considered when constructing the MPM list:

[0044] - Default intra mode

[0045] - Adjacent Intra Mode

[0046] - Intra-mode for export

[0047] Regardless of whether the MRL and ISP codecs are applied, a unified 6-MPM list is used for intra blocks. The MPM list is constructed based on the intra modes of the neighboring blocks on the left and above. Assuming that the mode on the left is marked as Left and the mode of the block above is marked as Above, the unified MPM list is constructed as follows:

[0048] - When a neighboring block is not available, its intra mode is set to planar by default.

[0049] -If both Left and Above modes are non-angle modes:

[0050] -MPM List → {plane, DC, V, H, V-4, V+4}

[0051] - If one of Left and Above modes is an angle mode and the other is a non-angle mode:

[0052] - Set mode Max to the larger mode between Left and Above

[0053] - MPM list → {Flat, Max, Max-1, Max+1, Max-2, Max+2}

[0054] -If Left and Above are both angled and they are different:

[0055] - Set mode Max to the larger mode between Left and Above

[0056] -If Max-Min is equal to 1:

[0057] MPM list → {Plane, Left, Above, Min-1, Max+1, Min-2}

[0058] - Otherwise, if Max-Min is greater than or equal to 62:

[0059] MPM list → {Plane, Left, Above, Min+1, Max-1, Min+2}

[0060] - Otherwise, if Max-Min is equal to 2:

[0061] MPM list → {Plane, Left, Above, Min+1, Min-1, Max+1}

[0062] -otherwise:

[0063] MPM list → {Plane, Left, Above, Min-1, Min+1, Max-1}

[0064] -If Left and Above are both angled and they are the same:

[0065] -MPM list → {Plane, Left, Left-1, Left+1, Left-2, Left+2}

[0066] In addition, the first bin of the MPM index codeword is decoded by the CABAC context. A total of three contexts are used, corresponding to whether the current intra block is MRL-enabled, ISP-enabled, or a normal intra block.

[0067] In the 6 MPM list generation processes, pruning is used to remove repeated patterns so that only unique patterns can be included in the MPM list. For the entropy coding and decoding of the 61 non-MPM modes, Truncated Binary Code (TBC) is used.

[0068] Wide-Angle Intra Prediction for Non-SquareBlocks

[0069] The conventional angular intra prediction direction is defined as clockwise from 45 degrees to -135 degrees. In VVC, several traditional angular intra prediction modes are adaptively replaced with wide-angle intra prediction modes for non-square blocks. The replaced modes are sent using the original mode index, which is remapped to the index of the wide-angle mode after parsing. The total number of intra prediction modes remains unchanged, i.e. 67, and the intra mode encoding and decoding method remains unchanged.

[0070] To support these prediction directions, a top reference of length 2W+1 and a left reference of length 2H+1 are respectively Fig. 7A and Figure 7B Definition shown. Fig. 7A and Figure 7B The Dia.mode in is the diagonal mode, that is, mode 34.

[0071] The number of alternative modes in the wide-angle direction mode depends on the aspect ratio of the block. The alternative intra prediction modes are shown in Table 2.

[0072] Table 2 - Intra prediction modes replaced by wide angle mode

[0073] Aspect Ratio Alternative intra prediction modes W / H==16 Mode 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15 W / H==8 Mode 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13 W / H==4 Mode 2,3,4,5,6,7,8,9,10,11 W / H==2 Mode 2,3,4,5,6,7, W / H==1 none W / H==1 / 2 Mode 61, 62, 63, 64, 65, 66 W / H==1 / 4 Mode 57,58,59,60,61,62,63,64,65,66 W / H==1 / 8 Mode 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66 W / H==1 / 16 Mode 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66

[0074] In VVC, both 4:2:2 and 4:4:4 as well as 4:2:0 chroma formats are supported. The chroma derived mode (DM) derivation table for the 4:2:2 chroma format was originally ported from HEVC, expanding the number of entries from 35 to 67 to be consistent with the expansion of intra prediction modes. Since the HEVC specification does not support prediction angles below -135 degrees and above 45 degrees, the luma intra prediction modes from 2 to 5 are mapped to 2. Therefore, the chroma DM derivation table for the 4:2:2: chroma format is updated by replacing some values ​​of the mapping table entries to more accurately convert the prediction angles of chroma blocks.

[0075] Cross-Component Linear Model (CCLM) prediction

[0076] In order to reduce cross-component redundancy, the cross-component linear model (CCLM) prediction mode is adopted in VVC. The chroma samples are predicted based on the reconstructed luminance samples of the same CU using a linear model as follows:

[0077] pred c (i, j) = α·rec L′ (i, j)+β (1)

[0078] Where predC(i,j) represents the predicted chrominance samples in the CU, and recL(i,j) represents the downsampled reconstructed luminance samples of the same CU.

[0079] The CCLM parameters (α and β) are derived from up to four adjacent chroma samples and their corresponding downsampled luma samples. Assuming the current chroma block size is W×H, W′ and H′ are set to:

[0080] - When LM_LA mode is used, W′=W, H′=H;

[0081] - When LM_A mode is used, W′=W+H;

[0082] - When LM_L mode is used, H'=H+W.

[0083] The upper neighboring position is denoted as S[0, -1] ... S[W'-1, -1] and the left neighboring position is denoted as S[-1, 0] ... S[-1, H'-1]. Then four samples are selected as:

[0084] - S[W′ / 4, -1], S[3*W′ / 4, -1], S[-1, H′ / 4], S[-1, 3*H′ / 4], when LM mode is used and both upper and left neighboring samples are available;

[0085] -S[W′ / 8, -1], S[3*W′ / 8, -1], S[5*W′ / 8, -1], S[7*W′ / 8, -1], when LM-A mode is used or only upper adjacent samples are available;

[0086] -S[-1, H′ / 8], S[-1, 3*H′ / 8], S[-1, 5*H′ / 8], S[-1, 7*H′ / 8], when LM-L mode is used or only left adjacent samples are available.

[0087] The four adjacent luma samples at the selected position are downsampled and compared four times to find the two larger values: x 0 A and x 1 A , and two smaller values: x 0 B and x 1 B Their corresponding chrominance sample values ​​are represented by y 0 A ,y 1 A ,y 0 B and 1 B Then x A 、x B ,y A and B is exported as:

[0088] xA=(x0A+x1A+1)>>1;

[0089] xB=(x0B+x1B+1)>>1;

[0090] yA=(y0A+y1A+1)>>1;

[0091] yB=(y0B+y1B+1)>>1 (2)

[0092] Finally, the linear model parameters α and β are obtained according to the following equations.

[0093]

[0094] β=y B -α.x B (4)

[0095] Figure 8 An example of the positions of the left and upper samples and samples of the current block involved in the LM_LA mode is shown. Figure 8The relative sample positions of an NxN chroma block 810, a corresponding 2Nx2N luma block 820, and their neighboring samples (shown as solid circles) are shown.

[0096] The division operation to calculate the parameter α is implemented by a lookup table. To reduce the memory required to store the table, the diff value (the difference between the maximum and minimum values) and the parameter α are expressed in exponential notation. For example, diff is approximated using a 4-bit significant part and an exponent. Therefore, the table for 1 / diff is reduced to 16 elements with 16 significant values, as shown below:

[0097] DivTable[]={0,7,6,5,5,4,4,3,3,2,2,1,1,1,1,0} (5)

[0098] This will have the advantage of reducing computational complexity as well as the memory size required to store the required tables.

[0099] In addition to the upper template and the left template being used together to calculate the linear model coefficients, they can also be used alternately in 2 other LM modes, called LM_A and LM_L modes.

[0100] In LM_A mode, only the upper template is used to calculate the linear model coefficients. To obtain more samples, the upper template is expanded to (W+H) samples. In LM_L mode, only the left template is used to calculate the linear model coefficients. To obtain more samples, the left template is expanded to (H+W) samples.

[0101] In LM_LA mode, the left and upper templates are used to calculate the linear model coefficients.

[0102] To match the chroma sample positions of 4:2:0 video sequences, two types of downsampling filters are applied to luma samples to achieve a 2 to 1 downsampling ratio in both horizontal and vertical directions. The choice of downsampling filter is specified by the SPS level flag. The two downsampling filters are as follows, corresponding to "type-0" and "type-2" content, respectively.

[0103] Rec L ′(i, j)=[rec L (2i-1, 2j-1)+2·rec L (2i-1, 2j-1)+rec L (2i+1,2j-1)+rec L (2i-1, 2j)+2·rec L (2i,2j)+rec L (2i+1,2j)+4]>>3 (6)

[0105] Rec L ′(i, j)=rec L (2i, 2j-1)+rec L (2i-1, 2j)+4·rec L (2i,2j)+rec L (2i+1,2j)+rec L (2i, 2j+1)+4]>>3 (7)

[0107] Note that when the upper reference line is at a CTU boundary, only one luma line (common line buffer in intra prediction) is used to generate the downsampled luma samples.

[0108] This parameter calculation is performed as part of the decoding process, not just as an encoder search operation. Therefore, no syntax is used to convey the α and β values ​​to the decoder.

[0109] For chroma intra mode coding and decoding, a total of 8 intra modes are allowed for chroma intra mode coding and decoding. These modes include five traditional intra modes and three cross-component linear model modes (LM_LA, LM_A and LM_L). The chroma mode transmission and derivation process is shown in Table 3. The chroma mode coding and decoding depends directly on the intra prediction mode of the corresponding luminance block. Since separate block partitioning structures for luminance and chrominance components are enabled in the I segment, one chroma block can correspond to multiple luminance blocks. Therefore, for the chroma DM mode, the intra prediction mode of the corresponding luminance block covering the center position of the current chroma block is directly inherited.

[0110] Table 3 - Chroma prediction modes derived from luma mode when CCLM is enabled

[0111]

[0112] Regardless of the value of sps_cclm_enabled_flag, a single binarization table is used, as shown in Table 4.

[0113] Table 4 - Unified binarization table of chrominance prediction mode

[0114]

[0115]

[0116] In Table 4, the first bin indicates whether it is normal mode (0) or CCLM mode (1). If it is LM mode, the next bin indicates whether it is LM_LA (0). If it is not LM_LA, the next bin indicates whether it is LM_L (0) or LM_A (1). For this case, when sps_cclm_enabled_flag is 0, the first bin of the binarization table corresponding to intra_chroma_pred_mode can be discarded before entropy coding and decoding. Or, in other words, the first bin is inferred to be 0, so no coding and decoding is performed. This single binarization table is used for cases where sps_cclm_enabled_flag is equal to 0 and 1. The first two bins in Table 4 use their own context model for context coding and decoding, and the remaining bins are bypass coded and decoded.

[0117] Additionally, to reduce luma-chroma latency in dual trees, when the 64x64 luma codec tree node is partitioned as Not Split (and ISP is not used for 64x64 CU) or QT, the chroma CUs in the 32x32 / 32x16 chroma codec tree nodes are allowed to use CCLM as follows:

[0118] - If a 32x32 chroma node is not split or is split using QT splitting, all chroma CUs in the 32x32 node can use CCLM.

[0119] - If the 32x32 chroma node uses horizontal BT partitioning and the 32x16 child node is not partitioned or uses vertical BT partitioning, all chroma CUs in the 32x16 chroma node can use CCLM. Under all other luma and chroma codec tree partitioning conditions, chroma CUs are not allowed to use CCLM.

[0120] Multiple Model CCLM (Multiple Model, MMLM for short)

[0121] In JEM (J. Chen, E. Alshina, G J. Sullivan, J.-R. Ohm, and J. Boyce, Algorithm Description of Joint Exploration Test Model 7, document JVET-G1001, ITU-T / ISO / IEC Joint Video Exploration Team (JVET), Jul. 2017), a multiple model CCLM mode (Multiple model CCLM mode, referred to as MMLM) is proposed for predicting chrominance samples from the luma samples of the entire CU using two models. In MMLM, the adjacent luma samples and the adjacent chroma samples of the current block are divided into two groups, each group is used as a training set to derive a linear model (ie, a specific α and β are derived for a specific group). In addition, the samples of the current luma block are also classified based on the same rules as the classification of the adjacent luma samples.

[0122] Fig. 9 An example of classifying neighboring samples into two groups is shown. Threshold is calculated as the average of neighboring reconstructed brightness samples. Neighboring samples with Rec'L[x,y]<=Threshold are classified into group 1; and neighboring samples with Rec'L[x,y]>Threshold are classified into group 2.

[0123]

[0124] Local illumination compensation (LIC)

[0125] Local illumination compensation (LIC) is an inter-frame prediction method that uses neighboring samples of the current block and the reference block. It is based on a linear model using a scaling factor a and an offset b. It derives the scaling factor a and the offset b by referring to the neighboring samples of the current block and the reference block. In addition, the codec tool is adaptively enabled or disabled for each CU.

[0126] For more details of LIC, please refer to document JVET-C1001 (J. Chen, et al., "Algorithm Description of Joint Exploration Test Model 3", Joint Video Exploration Team (JVET) of ITU-T SG 16WP 3and ISO / IEC JTC 1 / SC 29 / WG 11, 3rd Meeting: Geneva, CH, 26 May-1 June 2016, document JVET-C1001).

[0127] Convolutional cross-component model (CCCM)

[0128] In CCCM, a convolutional model is used to improve the chrominance prediction performance. The convolutional model uses a 7-tap filter consisting of a 5-tap plus-shaped spatial component, a nonlinear term, and a bias term. The input to the spatial 5-tap component of the filter consists of the center (C) luminance sample (which is paired with the chrominance sample to be predicted) and its upper / north (N), lower / south (S), left / west (W), and right / east (E) neighboring samples, such as Fig.10 shown.

[0129] The non-linear term (denoted as P) is expressed as a power of two of the center luminance sample C and scaled to the sample value range of the content:

[0130] P=(C*C+midVal)>>bitDepth (9)

[0131] Therefore, for 10 bits of content, the calculation is as follows:

[0132] P=(C*C+512)>>10 (10)

[0133] The bias term (denoted as B) represents a scalar offset between input and output (similar to the offset term in CCLM) and is set to the intermediate chrominance value (eg, 512 for 10-bit content).

[0134] The current pixel position (i.e., Fig.10 The output of the filter at "C" in ) is computed as the convolution between the filter coefficients ci and the input value, and clipped to the range of valid chroma samples:

[0135] predChromaVal=c0C+c1N+c2S+c3E+c4W+c5P+c6B (11)

[0136] Filter coefficient c i It is computed by minimizing the MSE between the predicted chroma samples and the reconstructed chroma samples in the reference region. Fig.11 A reference region consisting of 6 rows of chroma samples above and to the left of the PU is shown. The reference region extends 1 PU width to the right, and 1 PU height below the PU boundary. The region has been adjusted to contain only available samples. The region shown in gray needs to be extended to support the "side samples" of the plus-shaped spatial filter, and is filled if not available.

[0137] MSE minimization is performed by computing the autocorrelation matrix of the luma input and the cross-correlation vector between the luma input and the chroma output. The autocorrelation matrix is ​​subjected to an LDL decomposition, and the final filter coefficients are calculated using backward substitution. This process roughly follows the calculation of the ALF filter coefficients in ECM, but the LDL decomposition is chosen instead of the Cholesky decomposition to avoid the use of square root operations.

[0138] The convolutional cross component model (CCCM) has been disclosed for consideration of next generation video codecs beyond VVC and has shown performance improvements. The present invention discloses the desire to further improve the performance of CCCM or reduce its complexity. Accordingly, the present invention discloses some solutions to further improve the performance of CCCM. In addition, some solutions to reduce the complexity of CCCM are also disclosed. Summary of the invention

[0139] A method and apparatus for a convolutional cross-colour prediction model (CCCM) for video coding and decoding are disclosed. According to the method, input data associated with a current block including a luminance block and a chrominance block is received, wherein the input data includes pixel data to be encoded on the encoder side or encoded data associated with the current block to be decoded on the decoder side. An autocorrelation matrix of a reference luminance sample in a reference region is derived, wherein the autocorrelation matrix is ​​associated with a footprint of a convolution filter. A cross-correlation vector between a reference luminance sample and a reference chrominance sample in the reference region is derived, wherein the cross-correlation vector is associated with a footprint of the convolution filter. Coefficients of the convolution filter are derived using Gaussian elimination based on the autocorrelation matrix and the cross-correlation vector. For a target chrominance sample in the chrominance block, a convolutional cross-component model predictor of the target chrominance sample is generated by applying a convolutional filter having coefficients derived to corresponding positions of the luminance block. A final predictor of the target chrominance sample is generated from a prediction candidate set including the convolutional cross-component model predictor. The target chrominance sample is encoded or decoded using the final predictor.

[0140] According to another method, the number of filter taps of the target convolution filter is determined based on one or more conditions. For target chroma samples in the chroma block, a convolution cross-component model predictor of the target chroma samples is implemented by applying the target convolution filter to the corresponding position of the luma block. A final predictor of the target chroma samples including the convolution cross-component model predictor is generated. The target chroma samples are encoded or decoded using the final predictor.

[0141] In one embodiment, the one or more conditions correspond to one or more predetermined implicit rules. In one embodiment, the target convolution filter is implicitly determined based on the current block size.

[0142] In one embodiment, the target convolution filter is derived by setting the derived coefficients of the reference convolution filter to zero if the derived coefficients are less than a predetermined threshold. In another embodiment, the target convolution filter is derived by setting the derived coefficients of the reference convolution filter to zero if the derived coefficients are less than the sum of the coefficients of a plurality of reference convolution filters multiplied by a predetermined threshold.

[0143] In one embodiment, the target convolution filter is selected from a plurality of convolution filters having different numbers of filter taps.

[0144] In one embodiment, the target convolution filter is selected from a plurality of convolution filters generated by separating the reference convolution filters. In one embodiment, one of the plurality of convolution filters achieving the best performance is explicitly sent as the target convolution filter.

[0145] In one embodiment, a target convolution filter is selected from a plurality of convolution filters having different shapes according to a block-level syntax.

[0146] In one embodiment, the target convolution filter includes optional non-linear taps and syntax is used to indicate whether the optional non-linear taps are used.

[0147] According to another method, one or more syntax elements for indicating a target convolutional cross-component filter selected from a plurality of convolutional cross-component filters are sent or parsed, wherein each convolutional cross-component filter uses at least two different luma samples from two different locations. For a target chroma sample in a chroma block, a convolutional cross-component model predictor for the target chroma sample is generated by applying the target convolutional cross-component filter to a corresponding location of the luma block. A final predictor for the target chroma sample is generated from a set of prediction candidates including the convolutional cross-component model predictor. The target chroma sample is encoded or decoded using the final predictor.

[0148] In one embodiment, the one or more syntax elements are sent or parsed at the sequence parameter set (SPS), picture parameter set (PPS), picture header (PH), slice header (SH) or coding tree unit (CTU) level. BRIEF DESCRIPTION OF THE DRAWINGS

[0149] Figure 1A An example adaptive inter / intra video codec system incorporating loop processing is shown.

[0150] Figure 1B Shown with Figure 1A The decoder corresponding to the encoder in .

[0151] Figure 2 Examples of multi-type tree structures corresponding to vertical binary segmentation (SPLIT_BT_VER), horizontal binary segmentation (SPLIT_BT_HOR), vertical ternary segmentation (SPLIT_TT_VER), and horizontal ternary segmentation (SPLIT_TT_HOR) are shown.

[0152] Figure 3 An example of a signaling mechanism for partition split information in a quadtree with a nested multi-type tree codec tree structure is shown.

[0153] Figure 4 An example of partitioning a CTU into multiple CUs using a quadtree and a nested multi-type tree coding block structure is shown, where bold block edges represent quadtree partitions and the remaining edges represent multi-type tree partitions.

[0154] Figure 5 Some examples of disabling TT splitting when the width or height of the luma codec block is greater than 64 are shown.

[0155] Figure 6 The intra-frame prediction mode adopted by the VVC video coding standard is shown.

[0156] Fig. 7A -B shows an example of wide-angle intra prediction, where the width of the block is greater than the height ( Fig. 7A ) and blocks that are taller than they are wide ( Figure 7B ).

[0157] Figure 8 An example of the positions of the left and above samples involved in the LM_LA mode and the samples of the current block is shown.

[0158] Fig. 9Shows an example of classifying adjacent samples into two groups according to the multimodal CCLM.

[0159] Fig.10 An example showing the spatial part of a CCCM convolution filter.

[0160] Fig.11 An example of a reference region (and its filling) used to derive CCCM filter coefficients is shown.

[0161] Fig.12 A 3x2 downsampling filter is shown for downsampling luma samples in the YUV420 color format.

[0162] Fig.13A -B shows two filter shapes with multiple filter taps according to one embodiment of the present invention.

[0163] Fig.14A -D shows four filter shapes with multiple filter taps according to one embodiment of the present invention.

[0164] Fig.15A -D shows four filter shapes with multiple filter taps according to one embodiment of the present invention.

[0165] Fig.16 A flow chart illustrating an exemplary video coding system incorporating a convolutional cross-component model (CCCM) correlation mode using Gaussian elimination according to an embodiment of the present invention.

[0166] Fig.17 A flow chart of an exemplary video coding system utilizing multiple CCCM models with multiple filter taps according to an embodiment of the present invention is shown.

[0167] Fig.18 A flow chart of an exemplary video coding system incorporating signaling for selecting a target filter from multiple CCCM models according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0168] It is readily understood that the components of the present invention as generally described and illustrated in the figures herein may be arranged and designed in a variety of different configurations. Therefore, the following more detailed description of embodiments of the systems and methods of the present invention, as shown in the figures, is not intended to limit the scope of the claimed invention, but is merely representative of selected embodiments of the present invention. References in this specification to "an embodiment," "some embodiments," or similar language mean that specific features, structures, or characteristics described in conjunction with the embodiment may be included in at least one embodiment of the present invention. Therefore, the phrases "in an embodiment" or "in some embodiments" appearing in various places throughout this specification do not necessarily all refer to the same embodiment.

[0169] In addition, the described features, structures or characteristics may be combined in any suitable manner in one or more embodiments. However, those skilled in the relevant art will recognize that the present invention may be implemented without one or more specific details or using other methods, components, etc. In other cases, well-known structures or operations are not shown or described in detail to avoid obscuring various aspects of the present invention. The illustrated embodiments of the present invention will be best understood by reference to the accompanying drawings, in which the same parts are represented by the same numbers throughout. The following description is by way of example only, and simply describes some selected embodiments of the apparatus and method consistent with the present invention as claimed herein.

[0170] The following method is proposed to reduce the complexity of CCCM.

[0171] In the current CCCM, LDL decomposition followed by forward and backward substitution is used to derive the optimal filter coefficients. However, most of the processes in this method are sequential, which may cause some latency issues or difficulty in acceleration. We propose to use an elimination-based method to derive the optimal coefficients of the CCCM model, which has lower latency and is easier to parallelize.

[0172] In one embodiment, Gaussian elimination can be used to derive the optimal coefficients of the CCCM model. In mathematics, Gaussian elimination, also known as row reduction, is an algorithm for solving a system of linear equations. It consists of a series of operations performed on the corresponding coefficient matrix. In order to perform row reduction on a matrix, a series of basic row operations are used to modify the matrix until the lower left corner of the matrix is ​​filled with zeros as much as possible. The computational complexity of Gaussian elimination is lower than that of LDL decomposition and Cholesky decomposition.

[0173] According to the present invention, the LDL or Cholesky decomposition used in the conventional CCCM for MSE minimization is replaced by Gaussian elimination to reduce computational complexity. Therefore, first, the autocorrelation matrix of the luminance input and the cross-correlation vector between the luminance input and the chrominance output are calculated. The autocorrelation matrix of the luminance input can be calculated using a reference region (e.g. Fig.11 The cross-correlation vector between the luma input and chroma output can be calculated using the reference region (e.g. Fig.11 The luminance data and chrominance data in the reference area in the image (the reference area in the image) are calculated based on the coverage area of ​​the reference convolution filter (e.g., a 7-tap CCCM filter).

[0174] CCCM uses a 7-tap filter to formulate the chroma prediction model, which has been shown to achieve significant codec gains. However, solving for the optimal parameters of the 7-tap filter can be quite complex, especially on tiny chroma blocks. Overfitting of the reference samples can also be a problem. Therefore, several methods have been proposed to implicitly or explicitly reduce the number of filter taps of the initial or reference convolution filter (e.g., the 7-tap filter used in traditional CCCM).

[0175] In one embodiment, a CCCM model with adaptive filter shape is proposed to reduce the number of filter taps. It will adaptively reduce the number of filter taps through one or more predetermined implicit rules.

[0176] In one embodiment, the implicit selection of the CCCM model with adaptive filter shape depends on the CU size.In one embodiment, one or more predetermined values ​​are used for the filter taps instead of being derived at the decoder side.

[0177] In one embodiment, if the derived coefficients of the CCCM model are less than a predetermined threshold, the coefficients are set to zero to reduce the number of filter taps.

[0178] In one embodiment, first the sum of all derived coefficients of the reference convolution filter is calculated. If a coefficient is less than the sum of all coefficients of the reference convolution filter multiplied by a predetermined threshold, we set the coefficient to zero to reduce the number of filter taps.

[0179] In one embodiment, there are multiple CCCM models with different numbers of filter taps. When the model with more filter taps performs better than the model with fewer filter taps, the model with more filter taps will be selected. Otherwise, we would rather choose the model with fewer filter taps, even if it is slightly worse than the model with more filter taps.

[0180] Another method is proposed to reduce the number of filter taps. The original 7-tap CCCM filter can be divided into multiple filter shapes, and then the best filter is explicitly emitted through the RDO process.

[0181] In one embodiment, there are two filter shapes. The first filter consists of a center term, a left term, a right term, a nonlinear term, and a bias term ( Fig.13A ); The second filter consists of a center term, an upper term, a lower term, a nonlinear term, and a bias term ( Fig. 13B ).

[0182] predChromaVall=c0C+c1E+c2W+c3P+c4B, (12)

[0183] predChromaVal2=c0C+c1N+c2S+c3P+c4B. (13)

[0184] In one embodiment, there are four filter shapes. The first filter consists of a center term, a right term, a nonlinear term, and a bias term ( Fig.14A ); The second filter consists of a center term, a left-side term, a nonlinear term, and a bias term ( Fig. 14B ); The third filter consists of a center term, an upper term, a nonlinear term, and a bias term ( Fig. 14C ); The fourth filter consists of a center term, a lower term, a nonlinear term, and a bias term ( Fig.14D ).

[0185] predChromaVal1=c0C+c1E+c2P+c3B, (14)

[0186] predChromaVal2=c0C+c1W+c2P+c3B, (15)

[0187] predChromaVa13=c0C+c1N+c2P+c3B, (16)

[0188] predChromaVal4=c0C+c1S+c2P+c3B.(17)

[0189] In one embodiment, there are four filter shapes. The first filter consists of a center term, a right term, an upper term, a nonlinear term, and a bias term ( Fig.15A ); The second filter consists of a center term, a left term, an upper term, a nonlinear term, and a bias term ( Fig. 15B ); The third filter consists of a center term, a right term, a bottom term, a nonlinear term, and a bias term ( Fig. 15C ); The fourth filter consists of a center term, a left term, a bottom term, a nonlinear term, and a bias term ( Fig.15D ).

[0190] predChromaVall=c0C+c1E+c2N+c3P+c4B, (18)

[0191] predChromaVal2=c0C+c1W+c2N+c3P+c4B, (19)

[0192] predChromaVal3=c0C+c1W+c2S+c3P+c4B, (20)

[0193] predChromaVal4=c0C+c1E+c2S+c3P+c4B. (21)

[0194] In one embodiment, nonlinear terms are removed to further reduce model complexity.

[0195] In one embodiment, the non-linear term is optional and is determined by the encoder. An additional flag is signaled to enable the non-linear term.

[0196] In one embodiment, there is another mode consisting of a center term, a nonlinear term, and a bias term, while the nonlinear term in the other modes can be removed.

[0197] In one embodiment, the CCCM model that allows multiple filter shapes may depend on high-level syntax. For example, the flag may be sent at the Sequence Parameter Set (SPS), Picture Parameter Set (PPS), Picture Header (PH), Slice Header (SH), or Coding Tree Unit (CTU) level.

[0198] In one embodiment, allowing CCCM models with multiple filter shapes may depend on block-level syntax. For example, a flag may be sent for each CU, or a flag may be sent for each chroma block.

[0199] In one embodiment, indicating which filter shape to use in CCCM may depend on syntax, and the syntax may be encoded by truncated unary encoding or truncated binary encoding.

[0200] Instead of using sample values ​​directly, a method of constructing a CCCM model using a differential form is proposed. All or part of the adjacent spatial terms are replaced by differential terms, which are the differences between each adjacent spatial term and the center term. In addition, a clipping operation can be applied to the differential terms to reduce noise.

[0201] In one embodiment, a 6-tap CCCM model is used, which consists of four spatial difference terms, a nonlinear term, and a bias term. The difference term is the difference between each adjacent sample value and the center sample value.

[0202] predChromaVal=c0(NC)+c1(SC)+c2(EC)+c3(WC)+c4P+c5B (22)

[0203] In one embodiment, a 6-tap CCCM model is used, which consists of four spatial difference terms, a nonlinear term and a bias term. The difference terms correspond to the intercepted differences between each adjacent sample value and the center sample value, where a clipping operation is performed after the difference is calculated. A threshold is predefined for the clipping operation.

[0204] predChromaVal=c0*Clip(NC)+c1*Clip(SC)+c2*Clip(EC)+c3*Clip(WC)+c4P+c5B (23)

[0205] In one embodiment, a clipped difference term is used in the CCCM model. The predetermined clipping threshold of each difference term may be different or partially different.

[0206] In one embodiment, a clipped difference term is used in the CCCM model.There are multiple predetermined clipping thresholds, and syntax may be sent to indicate the selected threshold.

[0207] In one embodiment, we have two 6-tap CCCM models. The first model uses differential terms and the second model uses clipped differential terms. A flag may be sent to indicate the selected model.

[0208] The convolutional cross-component model (CCCM) with Gaussian elimination or with multiple CCCM models as described above can be implemented at the encoder side and / or the decoder side. For example, any of the proposed CCCM methods can be implemented in the intra-frame codec module (e.g. Figure 1B Intra-frame prediction 150) and / or inter-frame prediction module (e.g. Figure 1B MC 152 in the encoder) or the intra-frame coding module in the encoder (e.g. Figure 1A Intra-frame prediction 110 in the example) and / or inter-frame prediction module (e.g. Figure 1A Any proposed CCCM method can also be implemented as a circuit coupled to an intra / inter codec module at a decoder or encoder. However, the decoder or encoder can also use additional processing units to implement the required CCCM processing. The intra prediction unit (e.g. Figure 1A Unit 110 and Figure 1B Unit 150 in ) and / or an inter-frame prediction module (e.g. Figure 1A Inter-frame prediction 112 and Figure 1BMC 152 in the figure is shown as a separate processing unit, which can correspond to executable software or firmware code stored on a medium such as a hard disk or flash memory, for a central processing unit (CPU) or a programmable device (such as a digital signal processor (DSP) or a field programmable gate array (FPGA)).

[0209] Fig.16 A flowchart of an exemplary video coding system for a convolution cross component model (CCCM) correlation mode in combination with Gaussian elimination according to an embodiment of the present invention is shown. The steps shown in the flowchart can be implemented as executable program codes on one or more processors (e.g., one or more CPUs) on the encoder side. The steps shown in the flowchart can also be implemented based on hardware, such as one or more electronic devices or processors arranged to perform the steps in the flowchart. According to the method, in step 1610, input data associated with a current block including a luminance block and a chrominance block is received, wherein the input data includes pixel data to be encoded on the encoder side or encoded data associated with the current block to be decoded on the decoder side. In step 1620, an autocorrelation matrix of reference luminance samples in a reference region is derived, wherein the autocorrelation matrix is ​​associated with a footprint of a convolution filter. In step 1630, a cross-correlation vector between reference luminance samples and reference chrominance samples in a reference region is derived, wherein the cross-correlation vector is associated with a footprint of a convolution filter. In step 1640, coefficients of the convolution filter are derived based on the autocorrelation matrix and the cross-correlation vector using Gaussian elimination. In step 1650, for a target chroma sample in the chroma block, a convolutional cross-component model predictor for the target chroma sample is generated by applying a convolution filter having coefficients derived to corresponding positions of the luma block. In step 1660, a final predictor for the target chroma sample is generated from a set of prediction candidates including the convolutional cross-component model predictor. In step 1670, the target chroma sample is encoded or decoded using the final predictor.

[0210] Fig.17A flow chart of an exemplary video codec system using multiple CCCM models with multiple filter taps according to an embodiment of the present invention is shown. According to the method, in step 1710, input data associated with a current block including a luminance block and a chrominance block is received, wherein the input data includes pixel data to be encoded on the encoder side or encoded data associated with the current block to be decoded on the decoder side. In step 1720, the number of filter taps of the target convolution filter is determined according to one or more conditions. In step 1730, for a target chrominance sample in the chrominance block, a convolution cross-component model predictor of the target chrominance sample is generated by applying the target convolution filter to the corresponding position of the luminance block. In step 1740, a final predictor of the target chrominance sample including the convolution cross-component model predictor is generated. In step 1750, the target chrominance sample is encoded or decoded using the final predictor.

[0211] Fig.18 A flowchart of an exemplary video codec system incorporating signaling for selecting a target filter from a plurality of CCCM models according to an embodiment of the present invention is shown. According to the method, in step 1810, input data associated with a current block including a luminance block and a chrominance block is received, wherein the input data includes pixel data to be encoded on the encoder side or encoded data associated with the current block to be decoded on the decoder side. In step 1820, one or more syntax elements for indicating a target convolution cross-component filter selected from a plurality of convolution cross-component filters are sent or parsed, wherein each convolution cross-component filter uses at least two different luminance samples from two different positions. In step 1830, for a target chrominance sample in a chrominance block, a convolution cross-component model predictor of the target chrominance sample is generated by applying the target convolution filter to the corresponding position of the luminance block. In step 1840, a final predictor of the target chrominance sample including the convolution cross-component model predictor is generated. In step 1850, the target chrominance sample is encoded or decoded using the final predictor.

[0212] The flowchart shown is intended to illustrate an example of video encoding and decoding according to the present invention. Without departing from the spirit of the present invention, those skilled in the art may modify each step, rearrange the steps, split the steps or combine the steps to implement the present invention. In the present disclosure, specific syntax and semantics are used to illustrate examples to implement embodiments of the present invention. A technician may implement the present invention by replacing the above syntax and semantics with equivalent syntax and semantics without departing from the spirit of the present invention.

[0213] The above description is presented to enable one of ordinary skill in the art to implement the present invention provided in the context of a specific application and its requirements. Various modifications to the described embodiments will be apparent to those skilled in the art, and the general principles defined herein may be applied to other embodiments. Therefore, the present invention is not intended to be limited to the specific embodiments shown and described, but to conform to the widest scope consistent with the principles and novel features disclosed herein. In the above detailed description, various specific details are described in order to provide a thorough understanding of the present invention. However, it will be understood by those skilled in the art that the present invention may be implemented.

[0214] The embodiments of the present invention as described above can be implemented in various hardware, software codes or a combination of the two. For example, one embodiment of the present invention can be a program code integrated into one or more circuits in a video compression chip or integrated into video compression software to perform the processing described herein. An embodiment of the present invention can also be a program code to be executed on a digital signal processor (Digital Signal Processor, referred to as DSP) to perform the processing described herein. The present invention can also be related to many functions performed by a computer processor, a digital signal processor, a microprocessor or a field programmable gate array (field programmable gate array, referred to as FPGA). These processors can be configured to perform specific tasks according to the present invention by executing machine-readable software code or firmware code that defines the specific method embodied in the present invention. The software code or firmware code can be developed in different programming languages ​​and different formats or styles. The software code can also be compiled for different target platforms. However, the different code formats, styles and languages ​​of the software code and other ways of configuring the code to perform tasks according to the present invention will not depart from the spirit and scope of the present invention.

[0215] The present invention may be embodied in other specific forms without departing from its spirit or essential characteristics. The described examples are to be considered in all respects as illustrative only and not restrictive. Therefore, the scope of the present invention is indicated by the appended claims rather than by the foregoing description. All changes within the equivalent meaning and scope of the claims should be included within their scope.

Claims

1. A method for video encoding and decoding of a color picture using cross-component prediction, the method comprising: Receiving input data associated with a current block including a luma block and a chroma block, wherein the input data includes pixel data to be encoded at an encoder side or encoded data associated with the current block to be decoded at a decoder side; deriving an autocorrelation matrix for a plurality of reference luminance samples in a reference region, wherein the autocorrelation matrix is ​​associated with a footprint of a convolution filter; deriving a cross-correlation vector between the plurality of reference luma samples and a plurality of reference chroma samples in the reference region, wherein the cross-correlation vector is associated with the footprint of the convolution filter; deriving a plurality of coefficients of the convolution filter based on the autocorrelation matrix and the cross-correlation vector using Gaussian elimination; generating, for a target chroma sample in the chroma block, a convolved cross-component model predictor of the target chroma sample by applying the convolution filter having the plurality of coefficients derived to corresponding positions of the luma block; generating a final predictor for the target chroma samples from a set of prediction candidates including the convolutional cross-component model predictor; as well as The target chroma sample is encoded or decoded using the final predictor.

2. An apparatus for video encoding and decoding of color pictures using cross-component prediction, the apparatus comprising one or more electronic devices or processors arranged to: Receiving input data associated with a current block including a luma block and a chroma block, wherein the input data includes pixel data to be encoded at an encoder side or encoded data associated with the current block to be decoded at a decoder side; deriving an autocorrelation matrix for a plurality of reference luminance samples in a reference region, wherein the autocorrelation matrix is ​​associated with a footprint of a convolution filter; deriving a cross-correlation vector between the plurality of reference luma samples and a plurality of reference chroma samples in the reference region, wherein the cross-correlation vector is associated with the footprint of the convolution filter; deriving a plurality of coefficients of the convolution filter based on the autocorrelation matrix and the cross-correlation vector using Gaussian elimination; generating, for a target chroma sample in the chroma block, a convolved cross-component model predictor of the target chroma sample by applying the convolution filter having the plurality of coefficients derived to corresponding positions of the luma block; generating a final predictor for the target chroma samples from a set of prediction candidates including the convolutional cross-component model predictor; as well as The target chroma sample is encoded or decoded using the final predictor.

3. A method for video encoding and decoding of a color picture using cross-component prediction, the method comprising: Receiving input data associated with a current block including a luma block and a chroma block, wherein the input data includes pixel data to be encoded at an encoder side or encoded data associated with the current block to be decoded at a decoder side; Determining the number of multiple filter taps of the target convolution filter according to one or more conditions; For a target chroma sample in the chroma block, generating a convolved cross-component model predictor of the target chroma sample by applying the target convolution filter to a corresponding position of the luma block; generating a final predictor for the target chroma samples from a set of prediction candidates including the convolutional cross-component model predictor; as well as The target chroma sample is encoded or decoded using the final predictor.

4. The method for video encoding and decoding of a color picture using cross-component prediction as claimed in claim 3, characterized in that: The one or more conditions correspond to one or more predetermined implicit rules.

5. The method for video encoding and decoding of a color picture using cross-component prediction according to claim 4, characterized in that: The target convolution filter is implicitly determined according to the current block size.

6. The method for video encoding and decoding of a color picture using cross-component prediction according to claim 3, characterized in that: If the derived coefficients are less than a predetermined threshold, the target convolution filter is derived by setting the derived coefficients of the reference convolution filter to zero.

7. The method for video encoding and decoding of a color picture using cross-component prediction according to claim 3, characterized in that: If the derived coefficient is less than a sum of a plurality of coefficients of the reference convolution filter multiplied by a predetermined threshold, the target convolution filter is derived by setting the derived coefficient of the reference convolution filter to zero.

8. The method for video encoding and decoding of a color picture using cross-component prediction as claimed in claim 3, characterized in that: The target convolution filter is selected from a plurality of convolution filters having a plurality of different numbers of filter taps.

9. The method for video encoding and decoding of a color picture using cross-component prediction according to claim 3, characterized in that: The target convolution filter is selected from a plurality of convolution filters generated by separating the reference convolution filters.

10. The method for video encoding and decoding of a color picture using cross-component prediction according to claim 9, characterized in that: One of the plurality of convolution filters achieving the best performance is explicitly sent as the target convolution filter.

11. The method for video encoding and decoding of a color picture using cross-component prediction according to claim 3, characterized in that: The target convolution filter includes optional non-linear taps, and syntax is used to indicate whether the optional non-linear taps are used.

12. An apparatus for video encoding and decoding of color pictures using cross-component prediction, the apparatus comprising one or more electronic devices or processors arranged to: Receiving input data associated with a current block including a luma block and a chroma block, wherein the input data includes pixel data to be encoded at an encoder side or encoded data associated with the current block to be decoded at a decoder side; Determining the number of multiple filter taps of the target convolution filter according to one or more conditions; For a target chroma sample in the chroma block, generating a convolved cross-component model predictor of the target chroma sample by applying the target convolution filter to a corresponding position of the luma block; generating a final predictor for the target chroma sample from a set of prediction candidates including the convolutional cross-component model predictor; as well as The target chroma sample is encoded or decoded using the final predictor.

13. A method for video encoding and decoding of a color picture using cross-component prediction, the method comprising: Receiving input data associated with a current block including a luma block and a chroma block, wherein the input data includes pixel data to be encoded at an encoder side or encoded data associated with the current block to be decoded at a decoder side; Sending or parsing one or more syntax elements to indicate a target convolutional cross-component filter selected from a plurality of convolutional cross-component filters, wherein each convolutional cross-component filter uses at least two different luma samples from two different locations; For a target chroma sample in the chroma block, generating a convolution cross-component model predictor of the target chroma sample by applying the target convolution cross-component filter to a corresponding position of the luma block; generating a final predictor for the target chroma samples from a set of prediction candidates including the convolutional cross-component model predictor; as well as The target chroma sample is encoded or decoded using the final predictor.

14. The method for video encoding and decoding of a color picture using cross-component prediction according to claim 13, characterized in that: The target convolution filter is selected from a plurality of convolution filters of different shapes according to a block-level syntax.

15. The method for video encoding and decoding of a color picture using cross-component prediction according to claim 13, characterized in that: The one or more syntax elements are sent or parsed at a sequence parameter set, a picture parameter set, a picture header, a slice header or a codec tree unit level.

16. An apparatus for video encoding and decoding of color pictures using cross-component prediction, the apparatus comprising one or more electronic devices or processors arranged to: Receiving input data associated with a current block including a luma block and a chroma block, wherein the input data includes pixel data to be encoded at an encoder side or encoded data associated with the current block to be decoded at a decoder side; Sending or parsing one or more syntax elements to indicate a target convolutional cross-component filter selected from a plurality of convolutional cross-component filters, wherein each convolutional cross-component filter uses at least two different luma samples from two different locations; For a target chroma sample in the chroma block, generating a convolution cross-component model predictor of the target chroma sample by applying the target convolution cross-component filter to a corresponding position of the luma block; generating a final predictor for the target chroma samples from a set of prediction candidates including the convolutional cross-component model predictor; as well as The target chroma sample is encoded or decoded using the final predictor.

Images