Novel intra prediction method and apparatus coding reference line and intra prediction mode in video coding system

Abstract

A method and apparatus for video coding using joint prediction by combining intra prediction modes with one or more reference lines. According to the method, a set of joint prediction candidates is determined, where each joint prediction candidate corresponds to a combination of an intra prediction mode from a set of intra prediction mode candidates and one or more reference samples in one or more reference lines from a set. And determining a target joint prediction candidate from the joint prediction candidate set, wherein the one or more reference samples are used for assisting in determining the target joint prediction candidate. And encoding or decoding the current block by using one or more predictors including the target joint prediction candidate.

Description

[0001] Cross-references

[0002] This application is a non-provisional application of and claims priority to U.S. Provisional Patent Application No. 63 / 369,093, filed on July 22, 2022. The entire contents of this U.S. Provisional Patent Application are incorporated herein by reference. Technical Field

[0003] The present invention relates to video coding systems, and more particularly to chroma intra prediction using a joint prediction mode by combining an intra prediction mode and one or more reference lines in a video coding system. Background Art

[0004] Versatile Video Coding (VVC) is the latest international video coding standard jointly developed by the Joint Video Experts Group (JVET) of the ITU-T Video Coding Experts Group (VCEG) and the ISO / IEC Moving Picture Experts Group (MPEG). The standard has been published as an ISO standard: ISO / IEC 23090-3:2021, Information technology - Coded representation of immersive media - Part 3: Versatile video coding, released in February 2021. VVC is an improvement on its predecessor HEVC (High Efficiency Video Coding) by adding more coding tools to improve coding efficiency, and can also handle various types of video sources, including 3D video signals.

[0005] Figure 1A An exemplary adaptive inter / intra video coding system including a loop process is described. For intra prediction, prediction data is derived from previously encoded video data in the current picture. For inter prediction 112, motion estimation (ME) is performed on the encoder side and motion compensation (MC) is performed based on the results of ME to provide prediction data derived from other pictures and motion data. Switch 114 selects intra prediction 110 or inter prediction 112 and the selected prediction data is provided to adder 116 to form a prediction error, also known as a residual. The prediction error is then processed by transform (T) 118 and subsequent quantization (Q) 120. The transformed and quantized residual is then encoded by entropy encoder 122 to be included in a video bitstream corresponding to the compressed video data. The bitstream associated with the transform coefficients is then packaged with auxiliary information (such as motion and coding modes associated with intra prediction and inter prediction) and other information (such as parameters associated with loop filters applied to underlying image areas). Auxiliary information associated with intra prediction 110, inter prediction 112, and in-loop filter 130 is provided to entropy encoder 122, such as Figure 1AAs shown. When using inter-frame prediction mode, one or more reference pictures must also be reconstructed at the encoder end. Therefore, the transformed and quantized residual is processed by inverse quantization (IQ) 124 and inverse transform (IT) 126 to recover the residual. The residual is then added back to the prediction data 136 at reconstruction (REC) 128 to reconstruct the video data. The reconstructed video data can be stored in a reference picture buffer 134 and used to predict other frames.

[0006] like Figure 1A As shown, the input video data undergoes a series of processes in the coding system. Due to a series of processes, the reconstructed video data from REC128 may be subject to various impairments. Therefore, loop filter 130 is often applied to reconstructed video data before the reconstructed video data is stored in reference picture buffer 134 to improve video quality. For example, deblocking filter (DF), sample adaptive offset (SAO) and adaptive loop filter (ALF) can be used. It may be necessary to merge loop filter information into the bitstream so that the decoder can correctly restore the required information. Therefore, loop filter information is also provided to entropy encoder 122 to be merged into the bitstream. Figure 1A In , a loop filter 130 is applied to the reconstructed video before the reconstructed samples are stored in a reference picture buffer 134 . Figure 1A The system in is intended to illustrate an exemplary structure of a typical video encoder. It may correspond to the High Efficiency Video Coding (HEVC) system, VP8, VP9, ​​H.264, or VVC.

[0007] like Figure 1B As shown, the decoder can use similar or identical functional blocks as the encoder, except for transform 118 and quantization 120, because the decoder only needs inverse quantization 124 and inverse transform 126. Instead of entropy encoder 122, the decoder uses entropy decoder 140 to decode the video bitstream into quantized transform coefficients and required encoding information (such as ILPF information, intra-frame prediction information, and inter-frame prediction information). Intra-frame prediction 150 on the decoder side does not need to perform mode search. Instead, the decoder only needs to generate intra-frame prediction based on the intra-frame prediction information received from entropy decoder 140. In addition, for inter-frame prediction, the decoder only needs to perform motion compensation (MC152) based on the inter-frame prediction information received from entropy decoder 140 without motion estimation.

[0008] According to VVC, similar to HEVC, the input picture is divided into non-overlapping square block areas called CTUs (Coding Tree Units). Each CTU can be divided into one or more coding units (CUs) of smaller size. The generated CU partitions can be square or rectangular. In addition, VVC divides CTUs into prediction units (PUs) as units for applying prediction processes such as inter-frame prediction, intra-frame prediction, etc.

[0009] The VVC standard incorporates various new coding tools to further improve the coding efficiency beyond the HEVC standard. The coding and decoding tools related to the present invention are described as follows.

[0010] Partition CTU using tree structure

[0011] In HEVC, a CTU is partitioned into CUs using a quadtree (QT) structure represented as a coding tree to accommodate various local features. The decision whether to use inter (temporal) prediction or intra (spatial) prediction to encode a picture area is made at the leaf-CU level. Depending on the PU split type, each leaf-CU can be further split into 1, 2, or 4 PUs. Within a PU, the same prediction process is applied, and relevant information is transmitted to the decoder on a PU basis. After applying the prediction process based on the PU partition type to obtain the residual block, the leaf-CU can be divided into transform units (TUs) according to another quadtree structure similar to the coding tree of the CU. One of the key features of the HEVC structure is that it has multiple partition concepts including CU, PU, ​​and TU.

[0012] Figure 2 The partitioning of a CTU into multiple CUs using a quadtree and nested multi-type tree coding block structure is shown, where the bold block edges represent quadtree partitions and the remaining edges represent multi-type tree partitions. The quadtree with nested multi-type tree partitions provides a content-adaptive coding tree structure consisting of CUs. The size of a CU can be as large as a CTU or as small as 4×4 in units of luma samples. For the case of 4:2:0 chroma format, the maximum chroma CB size is 64×64 and the minimum size chroma CB consists of 16 chroma samples.

[0013] In VVC, the maximum supported luma transform size is 64 × 64, and the maximum supported chroma transform size is 32 × 32. When the width or height of the CB is larger than the maximum transform width or height, the CB is automatically split horizontally and / or vertically to meet the transform size limit in that direction.

[0014] The following parameters are defined for the quadtree with nested multi-type tree coding tree scheme. These parameters are specified by the SPS (sequence parameter set) syntax element and can be further refined by the picture header syntax element.

[0015] –CTU size: the root node size of the quadtree

[0016] –MinQTSize: Minimum allowed quad tree leaf node size

[0017] –MaxBtSize: Maximum allowed binary tree root node size

[0018] –MaxTtSize: Maximum allowed ternary tree root node size

[0019] –MaxMttDepth: The maximum level depth allowed for a quad-tree leaf splitting multi-type tree

[0020] –MinCbSize: The minimum allowed encoding block node size

[0021] In an example of a quadtree with nested multi-type tree coding tree structure, the CTU size is set to 128×128 luma samples with two corresponding 64×64 blocks of 4:2:0 chroma samples, MinQTSize is set to 16×16, MaxBtSize is set to 128×128, MaxTtSize is set to 64×64, MinCbsize (width and height) is set to 4×4, and MaxMttDepth is set to 4. Quadtree partitioning is applied to the CTU to first generate the leaf nodes of the quadtree. The size of the leaf node of the quadtree can range from 16×16 (i.e., MinQTSize) to 128×128 (i.e., CTU size). If the leaf QT node is 128×128, it will not be further split by the binary tree because the size exceeds MaxBtSize and MaxTtSize (i.e., 64×64). Otherwise, the leaf qdtree node can be further divided through the multi-type tree. Therefore, the quadtree leaf node is also the root node of the multi-type tree, and its multi-type tree depth (mttDepth) is 0. When the multi-type tree depth reaches MaxMttDepth (i.e. 4), it is considered that no further splitting will be performed. When the width of the multi-type tree node is equal to MinCbsize, no further horizontal splitting is considered. Similarly, when the height of the multi-type tree node is equal to MinCbsize, no further vertical splitting is considered.

[0022] In VVC, the coding tree scheme supports the ability for luma and chroma to have separate block tree structures. For P and B slices, the luma and chroma CTBs in one CTU must share the same coding tree structure. However, for I slices, luma and chroma can have separate block tree structures. When the separated block tree mode is applied, the luma CTB is divided into CUs through one coding tree structure, and the chroma CTB is divided into chroma CUs through another coding tree structure. This means that a CU in an I slice can consist of coding blocks of the luma component or coding blocks of two chroma components, while a CU in a P or B slice always consists of coding blocks of all three color components unless the picture is monochrome.

[0023] Intra-frame chroma partitioning and prediction constraints

[0024] In typical hardware video encoders and decoders, when a picture has small intra blocks, the processing throughput decreases due to the sample processing data dependency between adjacent intra blocks. The predictor generation of an intra block requires the top and left boundary reconstructed samples from adjacent blocks. Therefore, intra prediction must be processed sequentially block by block.

[0025] In HEVC, the smallest intra CU is 8x8 luma samples. The luma component of this smallest intra CU can be further split into 4 4x4 luma intra prediction units (PUs), but the chroma component of the smallest intra CU cannot be split any further. Therefore, the worst-case hardware processing throughput occurs when processing 4x4 chroma intra blocks or 4x4 luma intra blocks. In VVC, to improve the worst-case throughput, chroma intra CBs with less than 16 chroma samples (sizes of 2x2, 4x2, and 2x4) and intra chroma CBs with widths less than 4 chroma samples (size of 2xN) are not allowed by restricting the partitioning of chroma samples.

[0026] In a single coding tree, the smallest chroma intra prediction unit (SCIPU) is defined as a coding tree node with a chroma block size greater than or equal to 16 chroma samples and with at least one sub-luminance block less than 64 luma samples, or a coding tree node with a chroma block size not 2xN and at least one sub-luminance block 4xN luma samples. It is required that in each SCIPU, all CBs are inter-frame or all CBs are non-inter-frame, i.e., intra-frame or intra-block copy (IBC). In the case of non-inter-frame SCIPU, it is also required that the chroma of the non-inter-frame SCIPU should not be further split and the luma of the SCIPU is allowed to be further split. In this way, small chroma intra-frame CBs with a size less than 16 chroma samples or a size of 2xN are removed. In addition, chroma scaling is not applied in the case of non-inter-frame SCIPU. Here, no additional syntax is signaled, and whether the SCIPU is non-inter-frame can be derived from the prediction mode of the first luma CB in the SCIPU. If the current slice is an I-slice or the current SCIPU is further split once and has a 4x4 brightness partition, the type of SCIPU is inferred to be non-inter (because inter 4x4 is not allowed in VVC); otherwise, before parsing the CU in the SCIPU, a flag is used to indicate the type of SCIPU (inter or non-inter).

[0027] For double trees in intra pictures, 2xN intra chroma blocks are removed by disabling vertical binary and vertical ternary partitioning for 4xN and 8xN chroma partitions, respectively. Small chroma blocks of size 2x2, 4x2, and 2x4 are also removed by partitioning restrictions.

[0028] Furthermore, restrictions on picture size are considered by considering picture width and height to be multiples of max(8, MinCbSizeY) to avoid 2x2 / 2x4 / 4x2 / 2xN intra chroma blocks in the picture corners.

[0029] Intra-mode coding with 67 intra-prediction modes

[0030] To capture arbitrary edge directions present in natural videos, the number of directional intra modes in VVC is expanded from 33 used in HEVC to 65. New directional modes not present in HEVC are introduced in Figure 3 Depicted as dashed arrows in , planar and DC modes remain unchanged. These more dense directional intra prediction modes apply to all block sizes and luma and chroma intra prediction.

[0031] In VVC, for non-square blocks, several traditional angular intra prediction modes are adaptively replaced with wide-angle intra prediction modes.

[0032] In HEVC, each intra-coded block has a square shape and the length of each of its sides is a power of 2. Therefore, no division operation is required to generate intra predictors using DC mode. In VVC, blocks can have a rectangular shape, which in general requires the use of division operations for each block. To avoid division operations for DC prediction, only the longer sides are used to calculate the average for non-square blocks.

[0033] In order to keep the complexity of the most probable mode (MPM) list generation low, an intra mode coding method with 6 MPMs is used by considering two available adjacent intra modes. The following three aspects are considered to construct the MPM list:

[0034] – Default intra mode

[0035] – Adjacent Intra Mode

[0036] – Intra mode for export.

[0037] A unified 6-MPM list is used for intra blocks, regardless of whether the MRL and ISP coding tools are applied. The MPM list is constructed based on the intra modes of the left and upper neighboring blocks. Assuming that the mode on the left is denoted as Left and the mode of the upper block is denoted as Above, the unified MPM list is constructed as follows:

[0038] – When a neighboring block is not available, its intra mode is set to planar by default.

[0039] – If both Left and Above modes are non-angle modes:

[0040] –MPM list → {plane,DC,V,H,V-4,V+4}

[0041] – If one of the Left and Above modes is an angle mode and the other is a non-angle mode:

[0042] – Set mode Max to the larger mode between Left and Above

[0043] – MPM list → {plane, Max, Max-1, Max+1, Max–2, Max+2}

[0044] – If Left and Above are both angled and they are different:

[0045] – Set mode Max to the larger mode between Left and Above

[0046] – If Max-Min = 1,

[0047] MPM list → {Plane,Left,Above,Min-1,Max+1,Min-2}

[0048] - Otherwise, if Max-Min is greater than or equal to 62:

[0049] MPM list → {Plane,Left,Above,Min+1,Max-1,Min+2}

[0050] – Otherwise, if Max-Min=2:

[0051] MPM list → {Plane,Left,Above,Min+1,Min-1,Max+1}

[0052] -otherwise:

[0053] MPM list → {Plane,Left,Above,Min-1,-Min+1,Max-1}

[0054] – If Left and Above are both in angle mode and are the same:

[0055] – MPM list → {plane, Left, Left-1, Left+1, Left-2, Left+2}

[0056] In addition, the first bin of the MPM index codeword is CABAC context coded. A total of three contexts are used, corresponding to whether the current intra block is MRL enabled, ISP enabled, or a normal intra block.

[0057] In the 6MPM list generation process, pruning is used to remove repeated patterns so that only unique patterns can be included in the MPM list. For the entropy coding of the 61 non-MPM patterns, truncated binary code (TBC) is used.

[0058] Wide-angle intra prediction for non-square blocks

[0059] Conventional angular intra prediction directions are defined as clockwise from 45 degrees to -135 degrees. In VVC, several traditional angular intra prediction modes are adaptively replaced with wide-angle intra prediction modes for non-square blocks. The replaced modes are signaled using the original mode index, which is remapped to the index of the wide-angle mode after parsing. The total number of intra prediction modes remains unchanged, i.e. 67, and the intra mode encoding method remains unchanged.

[0060] To support these predictions, Figure 4A and Figure 4B As shown, a top reference with a length of 2W+1 and a left reference with a length of 2H+1 are defined.

[0061] The number of alternative modes in wide-angle direction mode depends on the aspect ratio of the block. The alternative intra prediction modes are shown in Table 1.

[0062] Table 1 – Intra prediction modes replaced by Wide mode

[0063]

[0064] In VVC, 4:2:2, 4:4:4, and 4:2:0 chroma formats are supported. The chroma derived mode (DM) derivation table for the 4:2:2 chroma format was originally ported from HEVC, expanding the number of entries from 35 to 67 to keep pace with the expansion of intra prediction modes. Since the HEVC specification does not support prediction angles below -135° and above 45°, the luma intra prediction modes are mapped from 2 to 5 to 2. Therefore, the chroma DM derivation table for the 4:2:2 chroma format is updated by replacing some values ​​of the mapping table entries to more accurately convert the prediction angles of the chroma blocks.

[0065] Chroma DM mode

[0066] For the chroma DM mode, the intra prediction mode of the corresponding (collocated) luma block covering the center position of the current chroma block is directly inherited.

[0067] 4-tap interpolation filter and reference sample smoothing

[0068] A four-tap intra-frame interpolation filter is used to improve the accuracy of directional intra-frame prediction. In HEVC, a two-tap linear interpolation filter is used to generate intra-frame prediction blocks in directional prediction mode (i.e., excluding planar and DC predictors). In VVC, two sets of 4-tap IFs replace the low-precision linear interpolation in HEVC, one set is a DCT-based interpolation filter (DCTIF) and the other is a 4-tap smoothing interpolation filter (SIF). The DCTIF is constructed in the same way as used for chroma component motion compensation in HEVC and VVC. The SIF is obtained by convolving a 2-tap linear interpolation filter with a [1 2 1] / 4 filter.

[0069] Depending on the intra prediction mode, the following reference sample processing is performed:

[0070] – Directional intra prediction modes fall into one of the following groups:

[0071] Group A: vertical or horizontal mode (HOR_IDX, VER_IDX),

[0072] Group B: Directional modes representing non-fractional angles (-14, -12, -10, -6, 2, 34, 66, 72, 76, 78, 80) and plane modes,

[0073] Group C: Remaining direction mode;

[0074] – If the directional intra prediction mode is classified as belonging to group A, no filter is applied to the reference samples to generate the prediction samples;

[0075] – Otherwise, if the mode belongs to group B and the mode is a directional mode and all of the following conditions are true, then the [1,2,1] reference sample filter (depending on the MDIS condition) may be applied to the reference samples to further copy these filtered values ​​into the intra predictor according to the selected direction, but no interpolation filter is applied:

[0076] refIdx is equal to 0 (no MRL)

[0077] TU size greater than 32

[0078] ·brightness

[0079] No ISP blocking

[0080] – Otherwise, if the mode is classified as belonging to group C, the MRL (multi-reference line) index is equal to 0, and the current block is not an ISP block, only the internal reference sample interpolation filter is applied to the reference samples to generate prediction samples that fall into fractional or integer positions between the reference samples according to the selected direction (no reference sample filtering is performed). The interpolation filter type is determined as follows:

[0081] Set minDistVerHor equal to Min(Abs(predModeIntra-50),Abs(predModeIntra-18))

[0082] Set nTbS equal to (Log2(W)+Log2(H))>>1

[0083] Set intraHorVerDistThres[nTbS] as shown in Table 2 below:

[0084] Table 2 – intraHorVerDistThres[nTbs] table

[0085]

[0086] – If minDistVerHor is greater than intraHorVerDistThres[nTbS], use SIF for interpolation

[0087] – Otherwise, use DCTIF for interpolation

[0088] Decoder-side Intra Mode Derivation (DIMD) chroma mode

[0089] The DIMD chroma mode uses the DIMD derivation method to derive the chroma intra prediction mode of the current block based on the adjacent reconstructed Y, Cb and Cr samples in the second adjacent rows and columns, such as Figure 5 As shown. Figure 5 In FIG. 5 , regions 510, 520, and 530 correspond to the juxtaposed Y block, the current Cb block, and the current Cr block. The circles outside regions 510, 520, and 530 correspond to the corresponding adjacent reconstructed samples. The gray circles represent the sample positions for determining the DIMD gradient. Specifically, the horizontal gradient and the vertical gradient are calculated for each juxtaposed reconstructed luminance sample of the current chrominance block and the reconstructed Cb and Cr samples to construct the HoG. Then, the intra-frame prediction mode with the largest histogram amplitude value is used to perform chrominance intra-frame prediction on the current chrominance block.

[0090] When the intra prediction mode derived from the DIMD chroma mode is the same as the intra prediction mode derived from the DM mode, the intra prediction mode having the second largest histogram magnitude value is used as the DIMD chroma mode.

[0091] The CU level flag is signaled to indicate whether the recommended DIMD chroma mode is applied, as shown in Table 3.

[0092] Table 3. Binarization process of intra_chroma_pred_mode in the proposed method

[0093]

[0094] According to this method, the DM mode and the four default modes can be fused with the MMLM_LT mode as follows:

[0095] pred=(w0*pred0+w1*pred1+(1<<(shift-1)))>>shift,(1)

[0096] Where pred0 is the predictor obtained by applying the non-LM mode, pred1 is the predictor obtained by applying the MMLM_LT mode, and pred is the final predictor of the current chroma block. The two weights w0 and w1 are determined by the intra-frame prediction mode of the adjacent chroma blocks, and the shift is set equal to 2. Specifically, when the upper and left adjacent blocks are both encoded in the LM mode, {w0,w1}={1,3}; when the upper and left adjacent blocks are both encoded in the non-LM mode, {w0,w1}={3,1}; otherwise, {w0,w1}={2,2}.

[0097] For syntax design, if non-LM mode is selected, a flag is signaled to indicate whether fusion is applied. And the proposed fusion is only applicable to I slices.

[0098] DIMD Chroma Mode with Fusion

[0099] DIMD chroma mode and chroma intra prediction fusion mode are combined. Specifically, the described DIMD chroma mode is applied, and for I slices, DM mode, four default modes and DIMD chroma mode can be fused with MMLM_LT mode using the described weights, while for non-I slices, only DIMD chroma mode can be fused with MMLM_LT mode using the same weights.

[0100] Cross-Component Linear Model (CCLM) Forecasting

[0101] In order to reduce cross-component redundancy, VVC uses the cross-component linear model (CCLM) prediction mode, and uses a linear model to predict the chrominance samples based on the reconstructed luminance samples of the same CU, as follows:

[0102] pred C (i,j)=α·rec L ′(i,j)+β (2)

[0103] where pred_ C (i,j) represents the predicted chroma sample in CU, rec_ L '(i,j) represents the downsampled reconstructed luminance samples of the same CU.

[0104] The CCLM parameters (α and β) are derived from up to four adjacent chroma samples and their corresponding downsampled luma samples. Assuming the current chroma block size is W×H, W' and H' are set to

[0105] When LM_LA mode is applied (i.e., using both left and upper templates), W'=W, H'=H;

[0106] When the LM_A mode (i.e., only the upper template is used) is adopted, W'=W+H;

[0107] When the LM_L mode (ie, only the left template is used) is applied, H'=H+W.

[0108] The upper template is also called the top template, and LM_LA and LM_A can also be called LM_LT and LM_T modes. Sometimes, these modes are also called CCLM_LT, CCLM_L, and CCLM_T modes. They can also be referred to as LM modes for short. The upper adjacent positions are recorded as S[0,-1]…S[W'-1,-1], and the left adjacent positions are recorded as S[-1,0]…S[-1,H'-1]. Then four samples are selected as

[0109] S[W' / 4,-1],S[3*W' / 4,-1],S[-1,H' / 4],S[-1,3*H' / 4], when LM mode is applied and the neighboring samples above and to the left are available;

[0110] S[W' / 8,-1],S[3*W' / 8,-1],S[5*W' / 8,-1],S[7*W' / 8,-1], when LM-A mode is applied or only upper adjacent samples are available;

[0111] S[-1,H' / 8],S[-1,3*H' / 8],S[-1,5*H' / 8],S[-1,7*H' / 8] when LM-L mode is applied or only left neighbor samples are available.

[0112] The four adjacent brightness samples at the selected position are downsampled and compared four times to find the two larger values: x 0 A and x 1 A , and two smaller values: x 0 B and x 1 B Their corresponding chrominance sample values ​​are represented by y 0 A ,y 1 A ,y 0 B and 1B Then x A ,x B ,y A and B Export as:

[0113] Xa=(x0A+x1A+1)>>1;

[0114] Xb=(x0B+x1B+1)>>1;

[0115] Ya=(y0A+y1A+1)>>1;

[0116] Yb=(y0B+y1B+1)>>1 (3)

[0118] Finally, the linear model parameters α and β are obtained according to the following equations.

[0119]

[0120] β=Y b -α·X b (5)

[0122] Figure 6 An example of the positions of the upper left sample and the current block sample involved in the LM_LA mode (Linear Mode_Left and Above) is given. Figure 6 The relative sample positions of an NxN chroma block 610, a corresponding 2Nx2N luma block 620, and their neighboring samples (shown as solid circles) are shown.

[0123] The division operation to calculate the parameter α is implemented by a lookup table. To reduce the memory required to store the table, the diff value (the difference between the maximum and minimum values) and the parameter α are expressed in exponential notation. For example, diff is approximated by a 4-bit significant part and an exponent. Therefore, for 16 significant values, the table of 1 / diff is reduced to 16 elements, as shown below:

[0124] DivTable [ 16] = { 0, 7, 6, 5, 5, 4, 4, 3, 3, 2, 2, 1, 1, 1, 1, 0} (6)

[0125] This will help reduce the complexity of the calculations and the amount of memory required to store the required tables.

[0126] In addition to the upper template and the left template being used together to calculate the linear model coefficients, they can also be used alternately in 2 other LM modes, called LM_A and LM_L modes.

[0127] In LM_A mode, only the upper template is used to calculate the linear model coefficients. To obtain more samples, the upper template is expanded to (W+H) samples. In LM_L mode, only the left template is used to calculate the linear model coefficients. To obtain more samples, the left template is expanded to (H+W) samples.

[0128] In LM_LA mode, the left and upper templates are used to calculate the linear model coefficients.

[0129] To match the chroma sample positions of a 4:2:0 video sequence, two types of downsampling filters are applied to the luma samples to achieve a 2:1 downsampling ratio in both horizontal and vertical directions. The choice of downsampling filter is specified by the SPS level flag. The two downsampling filters are as follows, corresponding to "type-0" and "type-2" content, respectively.

[0130] Rec L ′(i,j)=[rec L (2i-1,2j-1)+2·rec L (2i-1,2j-1)+rec L (2i+1,2j-1)+rec L (2i-1,2j)+2·rec L (2i,2j)+rec L (2i+1,2j)+4]>>3 (7)

[0131] Rec L ′(i,j)=rec L (2i,2j-1)+rec L (2i-1,2j)+4·rec L (2i,2j)+rec L (2i+1,2j)+rec L (2i,2j+1)+4]>>3 (8)

[0132] Note that when the upper reference line is at a CTU boundary, only one luma line (the usual line buffer in intra prediction) is used for downsampling.

[0133] This parameter calculation is performed as part of the decoding process, not just as an encoder search operation. Therefore, there is no syntax for conveying the α and β values ​​to the decoder.

[0134] For chroma intra mode coding, a total of 8 intra modes are allowed for chroma intra mode coding. These modes include five traditional intra modes and three cross-component linear model modes (LM_LA, LM_L, and LM_A). The chroma mode signaling and derivation process are shown in Table 4. Chroma mode coding depends directly on the intra prediction mode of the corresponding luminance block. Since separate block partitioning structures for luminance and chrominance components are enabled in the I slice, a chroma block can correspond to multiple luminance blocks. Therefore, for the chroma derivation (Chroma DM) mode, the intra prediction mode of the corresponding luminance block at the center position of the current chroma block is directly inherited and overwritten.

[0135] Table 4 – Chroma prediction modes derived from luma mode when CCLM is enabled

[0136]

[0137]

[0138] As shown in Table 5, a single binary table is used regardless of the value of sps_cclm_enabled_flag.

[0139] Table 5 – Unified binary table for chroma prediction modes

[0140]

[0141] In Table 5, the first binary code indicates whether it is normal (0) or CCLM mode (1). If it is LM mode, the next binary code indicates whether it is LM_LA (0). If it is not LM_LA, the next binary code indicates whether it is LM_L (0) or LM_A (1). For this case, when sps_cclm_enabled_flag is 0, the first binary code of the binarization table of the corresponding intra_chroma_pred_mode can be discarded before entropy coding. Or, in other words, the first binary code is inferred to be 0 and therefore not encoded. This single binarization table is used for the cases where sps_cclm_enabled_flag is equal to 0 and 1. The first two binary codes in Table 5 are context encoded using their own context model, and the remaining binary codes are bypass encoded.

[0142] Additionally, to reduce luma-chroma latency in dual trees, when the 64x64 luma coding tree node is partitioned using NotSplit (and ISP is not used for 64x64 CU) or QT, the chroma CU tree nodes in 32x32 / 32x16 chroma coding can use CCLM in the following ways:

[0143] – If a 32x32 chroma node is not partitioned or partitioned QT partitioned, all chroma CUs in the 32x32 node can use CCLM;

[0144] – If a 32x32 chroma node adopts horizontal BT partitioning and the 32x16 child node is not partitioned or uses vertical BT partitioning, all chroma CUs in the 32x16 chroma node can use CCLM.

[0145] Under all other luma and chroma coding tree partitioning conditions, CCLM is not allowed for chroma CUs.

[0146] Multi-model CCLM (MMLM)

[0147] In JEM (J. Chen, E. Alshina, G. J. Sullivan, J.-R. Ohm, and J. Boyce, Algorithm Description of Joint Exploration Test Model 7, document JVET-G1001, ITU-T / ISO / IEC Joint Video Development Team (JVET), July 2017), a multiple model CCLM mode (MMLM) is proposed for predicting chrominance samples from luma samples of the entire CU using two models. In MMLM, the adjacent luma samples and adjacent chroma samples of the current block are divided into two groups, and each group is used as a training set to derive a linear model (i.e., a specific α and β are derived for a specific group). In addition, the samples of the current luma block are also classified based on the same rules as the classification of the adjacent luma samples.

[0148] Figure 7 An example of dividing neighboring samples into two groups is shown. The threshold is calculated as the average of neighboring reconstructed luminance samples. Neighboring samples with Rec′L[x,y]<=threshold are classified into group 1; and neighboring samples with Rec′L[x,y]>threshold are classified into group 2.

[0149]

[0150] Self-aware Filter Estimation CCLM

[0151] A self-perceptual filter estimation CCLM (SAFE-CCLM) method is disclosed to explore the advantages of multiple downsampling filters without signaling the filter index. Using SAFE-CCLM, N candidate luminance downsampling filters are predefined. When SAFE-CCLM is applied, a linear model between luminance and chrominance components is first derived for each candidate filter in the same way as ECM. Secondly, a prediction value is calculated using each linear model in a test area, which includes a column of left adjacent samples and a row of top adjacent samples. Thirdly, the SAD (sum of absolute differences) cost between the reconstructed sample and its corresponding predicted value in the test area is calculated for each candidate filter. Finally, the candidate filter with the smallest SAD cost is selected as the downsampling filter, and CCLM prediction is performed on the current block. When CCLM is indicated for a block, a SAFE-CCLM flag is signaled to further indicate whether SAFE-CCLM is applied.

[0152] Using SAFE-CCLM, N candidate brightness downsampling filters are predefined. Figure 8 As shown, the downsampling filtering process for the 4:2:0 color format is performed as follows

[0153] Rec' L (x,y)={c0Rec L (2x-1,2y)+c1Rec L (2x,2y)+c2Rec L (2x+1,2y)+

[0154] c3Rec L (2x-1,2y+1)+c4Rec L (2x,2y+1)+c5Rec L (2x+1,2y+1)+4}>>3,

[0155] Where RecL(x,y) and Rec'L(x,y) are the reconstructed luma samples before and after downsampling filtering at position (x,y), and {c0,c1,c2,c3,c4,c5} are the filter coefficients. It should be noted that {c0,c1,c2,c3,c4,c5} are set to {1,2,1,1,2,1} in the current ECM. SAFE-CCLM predefines N candidate luma sampling filters with different coefficients.

[0156] Fig. 9A flowchart of SAFE-CCLM is shown. First, when SAFE-CCLM is applied, a linear model between the luminance and chrominance components of the current block 910 is derived in the same manner as in the ECM of each candidate filter. Second, a prediction value is calculated using each linear model in a test area, which includes a column of left-adjacent samples and a row of top-adjacent samples. Third, the SAD cost between the reconstructed chrominance samples and their corresponding prediction values ​​in the test area (areas 912 and 914) is calculated for each filter candidate. Finally, the filter candidate (920) with the smallest SAD cost is selected as the downsampling filter to perform CCLM prediction on the current block.

[0157] When CCLM is indicated for a block, a SAFE-CCLM flag is signaled to further indicate whether SAFE-CCLM is applied.

[0158] According to SAFE-CCLM, for N=32, the filter candidate set is listed in Table 6.

[0159] Table 6. Filter candidate set when N = 32

[0160]

[0161]

[0162] Although CCLM mode can improve coding efficiency, techniques need to be developed to further improve the efficiency of chroma intra prediction. Summary of the invention

[0163] A method and apparatus for video encoding and decoding are disclosed. According to the method, input data associated with a current block is received, wherein the input data includes pixel data of the current block to be encoded on the encoder side or encoded data associated with the current block to be decoded on the decoder side. A set of joint prediction candidates is determined, wherein each joint prediction candidate corresponds to a combination of an intra-frame prediction mode from a set of intra-frame prediction mode candidates and one or more reference samples in one or more reference lines from a set of reference line candidates. A target joint prediction candidate is determined from the set of joint prediction candidates. The current block is encoded or decoded by using one or more predictors including the target joint prediction candidate.

[0164] In one embodiment, the one or more reference samples include one or more prediction samples or one or more reconstructed samples in one or more non-adjacent neighboring lines of the current block. In one embodiment, the one or more reference samples further include one or more prediction samples or one or more reconstructed samples in neighboring lines of the current block.

[0165] In one embodiment, an index indicating a target joint prediction candidate from a set of joint prediction candidates is signaled at the encoder side or parsed at the decoder side.

[0166] In one embodiment, a set of indexes are mapped to the joint prediction candidate set according to a cost measurement associated with the members of the joint prediction candidate set. In one embodiment, the cost measurement corresponds to a boundary matching cost or a template matching cost of the members of the joint prediction candidate set. In one embodiment, the joint prediction candidate set is reordered according to the cost measurement, and the shortest codeword is assigned to the target index with the minimum cost measurement. In one embodiment, the target index of the target joint prediction candidate in the joint prediction candidate set indicating the current block is signaled on the encoder side or parsed on the decoder side. In one embodiment, the first K joint prediction candidates with the minimum cost measurement are mapped to the index set, and K is a positive integer. In one embodiment, when K is equal to 1, the target index of the target joint prediction candidate in the joint prediction candidate set indicating the current block is not signaled on the encoder side, or the target index is inferred on the decoder side.

[0167] In one embodiment, the set of intra prediction mode candidates includes all available intra prediction modes or a subset of all available intra prediction modes.

[0168] In one embodiment, the intra prediction mode candidate group, the reference line candidate group, or both are determined according to the block size of the current block.

[0169] In one embodiment, the target joint prediction candidate is used to replace the original intra prediction. In one embodiment, whether the target joint prediction candidate is used to replace the original intra prediction is enabled or disabled according to the syntax sent by signaling, or is implicitly determined.

[0170] In another embodiment, the final intra prediction is selected between the target joint prediction candidate and the original intra prediction. In one embodiment, whether the target joint prediction candidate is used to replace the original intra prediction is enabled or disabled according to a signaled syntax, or is implicitly determined.

[0171] In one embodiment, the target joint prediction candidate is used to encode or decode the chroma block of the current block, and the intra-frame prediction mode candidate group for chroma MRL (multi-reference line) prediction includes planar, vertical, horizontal, DC, one or more LM modes, chroma DM, DIMD chroma mode, diagonal (DIA), VDIA (mode 66 of 67 intra-frame prediction modes) or any combination thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0172] Figure 1AAn exemplary adaptive inter / intra video coding system incorporating loop processing is shown.

[0173] Figure 1B Shows Figure 1A The corresponding decoder of the encoder in .

[0174] Figure 2 An example of using quadtree and nested multi-type tree coding block structures to partition a CTU into multiple CUs is shown, where bold block edges represent quadtree partitions and the remaining edges represent multi-type tree partitions.

[0175] Figure 3 The intra-frame prediction mode adopted by the VVC video coding standard is marked.

[0176] Figure 4A -B shows blocks that are wider than they are tall ( Figure 4A ) and blocks that are taller than they are wide ( Figure 4B ) is an example of wide-angle intra prediction.

[0177] Figure 5 Adjacent reconstructed Y, Cb, and Cr samples used to derive the DIMD gradient are illustrated.

[0178] Figure 6 Examples of the positions of the left and upper samples and samples of the current block involved in the LM_LA mode are marked.

[0179] Figure 7 An example of dividing adjacent samples into two groups according to the multimodal CCLM is marked.

[0180] Figure 8 An example of downsampling of luma samples for Self-Perceptual Filter Estimation (SAFE)-CCLM for 4:2:0 color format is shown.

[0181] Fig. 9 A flow chart of Self-Perception Filter Estimation (SAFE)-CCLM is shown.

[0182] Fig. 10A An example of non-neighboring reference samples of the current block is shown, where line 0 is used for conventional prediction, and line 1 and line 2 are non-neighboring reference samples of the current block.

[0183] Fig. 10B An example of extended non-adjacent reference lines is shown.

[0184] Fig.11 An example of mixing is marked, where, when the intra prediction mode is angular mode, the position (x) of the sample to be mixed (r1) in one reference line and the corresponding position (x') of the mixed sample (r2) in another reference line depend on the intra prediction mode.

[0185] Fig.12 An example of a linear model based on multiple reference lines is shown for deriving model parameters in CCLM / MMLM.

[0186] Fig.13 An example of using luma reference line samples to derive model parameters without a luma downsampling process is shown.

[0187] Fig.14A An example of combining multiple adjacent reference lines to derive cross-component model parameters is shown.

[0188] Fig. 14B An example of combining reference lines using a 3×3 window is shown.

[0189] Fig. 14C An example of combining reference lines using a 3×2 window is shown.

[0190] Fig.15 An example of implicitly deciding reference line selection based on boundary matching cost is shown.

[0191] Fig.16 A flow chart of an exemplary video encoding system for incorporating joint prediction by combining intra prediction modes and one or more reference lines according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0192] It will be readily understood that the components of the present invention as generally described and illustrated in the accompanying drawings herein may be arranged and designed in a variety of different configurations. Therefore, the following more detailed description of the embodiments of the system and method of the present invention as shown in the figures is not intended to limit the scope of the claimed invention, but only represents selected embodiments of the present invention. References throughout this specification to "an embodiment", "one embodiment" or similar language mean that a particular feature, structure or characteristic described in conjunction with the embodiment may be included in at least one embodiment of the present invention. Therefore, the phrases "in one embodiment" or "in one embodiment" appearing throughout this specification do not necessarily all refer to the same embodiment.

[0193] In addition, the described features, structures or characteristics may be combined in one or more embodiments in any suitable manner. However, those skilled in the relevant art will recognize that the present invention may be practiced without one or more specific details, or using other methods, components, etc. In other cases, well-known structures or detailed descriptions of operations are not shown or are not shown to avoid obscuring aspects of the present invention. The illustrated embodiments of the present invention will be best understood with reference to the accompanying drawings, in which like parts are represented by like numbers throughout. The following description is intended to be exemplary only and simply illustrates certain selected embodiments of the apparatus and methods consistent with the present invention as claimed herein.

[0194] The following method is proposed to improve the accuracy of cross-component prediction and / or intra / inter prediction. In the conventional prediction scheme, the reference samples of the current block (e.g., the neighboring prediction samples and / or neighboring reconstruction samples of the current block) are adjacent to the left boundary and / or the upper boundary of the current block. In the present invention, it is proposed to use "non-adjacent neighboring prediction samples and / or neighboring reconstruction samples" (or non-adjacent reference samples) as the reference samples of the current block. The following is an example of non-adjacent reference samples of the current block, such as Fig. 10A As shown in Figure 1. Line 0 is used for traditional prediction, and lines 1 and 2 are non-adjacent reference samples of the current block. Fig. 10A As shown, line 0 refers to the pixels in the row immediately above the upper boundary of the current block and the pixels in the column immediately above the left boundary of the current block. The segments of the row or column of line 0 can be Fig. 10A Similarly, Line 1 refers to the pixels in the row immediately above the top of Line 0 and the pixels in the column immediately to the left of Line 0. Likewise, the segments of the row or column of Line 1 can be longer or shorter than the segments of the row or column. Fig. 10A Example shown. Similar conditions apply Fig. 10A Line 2 and Fig. 10B In one embodiment, the non-adjacent reference samples in the present invention may be one or more non-adjacent reference lines (e.g., line 1 and / or line 2) shown in this embodiment. In another embodiment, the non-adjacent reference samples in the present invention are not limited to line 1 and line 2. For another example, the non-adjacent reference samples in the present invention may be any extended non-adjacent reference line (e.g., line n) such as Fig. 10B In another embodiment, the non-adjacent reference samples in the proposal may be any subset of the samples in each of the selected one or more non-adjacent reference lines.

[0195] In another embodiment, a flag (eg, an SPS flag) is signaled to indicate whether one or more non-neighboring reference lines of the current block are allowed as candidate reference lines in addition to neighboring reference lines (used in conventional prediction).

[0196] In another embodiment, an implicit rule is designed to indicate whether one or more non-adjacent reference lines are allowed as candidate reference lines for the current block in addition to the adjacent reference lines (used in conventional prediction). For example, the implicit rule depends on the block width, height, area, pattern information from other color components, or pattern information of adjacent blocks. For another example, when the current block area is less than a predefined threshold, only adjacent reference lines can be used to generate intra-frame predictions for the current block. For another example, when most of the adjacent blocks (e.g., top and left adjacent blocks) are used, non-adjacent reference lines can be used as candidate reference lines for the current block in addition to the adjacent reference lines. For another example, when most of the adjacent blocks (e.g., top and left adjacent blocks) are used, only non-adjacent reference lines can be used as candidate reference lines for the current block. For another example, the reference line selection for the current color component depends on the reference line selection for other color components.

[0197] In another embodiment, the candidate reference lines of the current block include an adjacent reference line (eg, line 0) and one or more non-adjacent reference lines.

[0198] In another embodiment, the candidate reference lines of the current block only include one or more non-adjacent reference lines.

[0199] In another embodiment, when the current encoding / decoding component is a chroma component, the non-adjacent reference line refers to "only line 1 and line 2" or "only any subset of line 1 and line 2". That is, for chroma components (e.g., Cb, Cr), in addition to line 0, "only line 1 and line 2" or "only line 1" or "only line 2" are allowed as candidate reference lines for the current block.

[0200] Below, the current block is encoded using one or more LM modes. For example, the LM mode includes one or more CCLM modes and / or one or more MMLM modes. For another example, the LM mode in the present invention can be "changed to" or "further includes" any mode that uses cross-component information to predict the current component. For another example, the LM mode can refer to any extension / variant of the CCLM and / or MMLM mode. The change of CCLM here means that when the block indication refers to the use of one of the cross-component modes (e.g., CCLM_LT, MMLM_LT, CCLM_L, CCLM_T, MMLM_L, MMLM_T and / or intra-frame prediction modes, which are not one of the traditional DC, plane and angle modes). The following shows an example of using the convolutional cross-component mode (CCCM) as an optional mode. When this optional mode is applied to the current block, the cross-component information of the model (including nonlinear terms) is used to generate chrominance prediction. Model parameters are generated using adjacent or current luminance / chrominance reconstruction or prediction blocks and a predefined regression-based method. The optional modes may follow the template selection of CCLM, so the CCCM family contains CCCM_LT, CCCM_L and / or CCCM_T.

[0201] In another embodiment, the proposed method is applied to chroma blocks.

[0202] In one sub-embodiment, the chroma block refers to a chroma CB (including luma and chroma CB) belonging to a CU. For example, the chroma block is located in an inter-frame slice / tile. For another example, the chroma block is split from a single tree partition.

[0203] In another sub-embodiment, the chroma block refers to a chroma CB (including only chroma CB) belonging to a CU. For example, the chroma block is located in an intra slice / tile. For another example, the chroma block is split from a binary tree partition.

[0204] In another embodiment, in addition to allowing the LM mode to have multiple candidate reference lines, when the current block is encoded using an intra-frame prediction mode (e.g., an intra-frame chroma mode in a candidate list of DIMD chroma mode, chroma DM mode, chroma MRL, DC, planar or angular mode, or a mode selected from 67 intra-frame prediction modes, or a mode from an extended 67 intra-frame prediction modes (e.g., 131 intra-frame prediction modes)), one or more reference lines are selected from multiple candidate reference lines.

[0205] In another embodiment, the candidate list of chroma MRL includes plane, vertical, horizontal, DC, LM mode, chroma DM, DIMD chroma mode, diagonal (DIA), VDIA (mode 66 of 67 intra-frame prediction modes), or any subset of the above. For example, the candidate list of chroma MRL includes plane (changed to VDIA if chroma DM replication is used), vertical (changed to VDIA if chroma DM replication is used), horizontal (changed to VDIA if chroma DM replication is used), DC (changed to VDIA if chroma DM replication is used), 6 LM modes (i.e., CCLM_TL, CCLM_T, CCLM_L, MMLM_TL, MMLM_T and MMLM_L), chroma DM. For another example, the candidate list of chroma MRL includes plane (changed to VDIA if replicated with chroma DM), vertical (changed to VDIA if replicated with chroma DM), horizontal (changed to VDIA if replicated with chroma DM), DC (changed to VDIA if replicated with chroma DM), chroma DM. For another example, the candidate list of chroma MRL includes 6 LM modes and chroma DM.

[0206] In another embodiment, when making reference to neighboring samples to generate intra prediction, only one reference line is used.

[0207] In one sub-embodiment, the reference line is selected from a plurality of candidate reference lines.

[0208] In another embodiment, when making reference to neighboring samples to generate intra prediction, only one or more reference lines are used.

[0209] In one sub-embodiment, whether only one reference line is used or multiple reference lines are used depends on an implicit rule.

[0210] - For example, the implicit rule depends on the block width, block height or block area.

[0211] In one possible approach, when the block area, block width or block height is smaller than a predefined threshold, only one reference line is used to generate intra prediction.

[0212] ■The threshold can be any positive integer, such as 2, 4, 8, 16, 32, 64, 128, 256, 512, 1024... or the maximum transform size.

[0213] - For another example, the implicit rule depends on the mode information of the current block, such as the intra prediction mode of the current block.

[0214] One possible approach is to use more than one reference line to generate intra prediction when the current intra prediction mode may have non-integer prediction samples (referring to samples located at non-integer positions).

[0215] In another way, when the current intra prediction mode may require an intra interpolation filter to generate the intra prediction, because one or more prediction samples may fall into fractional or integer positions between reference samples depending on the direction selected (by the current intra prediction mode).

[0216] - For another example, the implicit rule depends on the mode information of the previous coded block, such as the intra prediction mode of the neighboring block, the selected reference line of the neighboring block.

[0217] In another sub-embodiment, using only one reference line or using multiple reference lines depends on the unambiguous syntax.

[0218] In another embodiment, more than one reference line is used when making intra predictions with reference to neighboring samples.

[0219] In another embodiment, a hybrid MRL is designed when more than one reference line is used to generate intra predictions for the current block.

[0220] -The first version of hybrid MRL is as follows. Forecasts are generated using each reference line, and then a hybrid process is applied to mix multiple forecast hypotheses from each used reference line.

[0221] - The second version of hybrid MRL is as follows: A hybrid process is applied to hybridize each used reference line, and the resulting hybrid reference line is used to generate intra prediction.

[0222] In one way, when blending reference lines, the intra prediction mode is considered. When the intra prediction mode of the current block 1110 is an angular mode (e.g. Fig.11 When the position (x) of the sample to be mixed (r1) in one reference line and the corresponding position (x') of the sample to be mixed (r2) in another reference line depend on the intra prediction mode. When the intra prediction mode is not an angular mode, such as DC or planar, the position (x) of the sample to be mixed (r1) in one reference line and the corresponding position (x') of the sample to be mixed (r2) in another reference line are the same or follow the direction of the intra prediction mode.

[0223] In other words, when blending reference lines, the intra prediction mode is not considered. When the intra prediction mode is angular mode, the position (x) of the sample to be blended (r1) in one reference line is the same as the corresponding position (x') of the sample to be blended (r2) in another reference line.

[0224] In the following examples / embodiments, the first version of hybrid MRL and the two reference lines used are taken as an example. The final prediction of the current block is formed by weighted averaging the prediction from the first reference line and the prediction from the second reference line. (Note that the proposed method can be applied to the second version of hybrid MRL. When the second version is applied and two reference lines are used to generate the prediction, the final prediction of the current block is formed by the weighted average prediction of the following items: the first reference line and the second reference line. And the following examples / embodiments may need to be modified accordingly.)

[0225] In one sub-embodiment, when the proposed hybrid process is applied, fusion of chroma intra prediction modes is disabled. (Inferred as disabled)

[0226] In another sub-embodiment, when fusion of chroma intra prediction modes is applied, the proposed blending process is disabled. (Inferred as disabled)

[0227] In another sub-embodiment, implicit rules are used to predefine the weights.

[0228] - For example, the weights are fixed to any of (w1,w2) = (3,1), (3,1), (2,2), where w1 is the weight of the first reference line and w2 is the weight of the first reference line as the second reference line. Since the sum of w1 and w2 is 4, a right shift of 2 is applied after adding the predicted samples of the first and second reference lines. (If the second version of hybrid MRL is applied, a right shift of 2 is applied after adding the first and second reference lines.)

[0229] - For another example, the implicit rule depends on the width, height, area, mode information of the current block or the width or height of neighboring blocks.

[0230] In another sub-embodiment, the weights are indicated using an explicit syntax.

[0231] In another sub-embodiment, the combination of signaling an index to indicate the selected current block refers to an intra prediction mode, a first reference line, and a second reference line.

[0232] - In one possible approach, the index is signaled using a truncated unary codeword.

[0233] - In another possible way, the index is represented by the context.

[0234] - In another possible approach, the correspondence from the index to the selected combination depends on boundary / template matching. An exemplary flow is as follows,

[0235] Step 0: Calculate the boundary / template matching cost for each candidate combination.

[0236] ■If boundary matching is used, for each candidate combination, the prediction of the current block is a hybrid prediction of multiple hypothesis predictions of the first reference line (based on intra-frame prediction mode) and the second reference line (based on intra-frame prediction mode). (If the second version of hybrid MRL is applied, for each candidate combination, the prediction of the current block is a hybrid prediction of the first reference line (based on intra-frame prediction mode) and the second reference line (based on intra-frame prediction mode).

[0237] ■ If template matching is used, for each candidate combination, the prediction on the template is a hybrid prediction mode of multi-hypothesis predictions of the first reference line (based on intra prediction mode) and the second reference line (based on intra prediction mode). If a pair equal to line1 and line2 is a candidate pair, and the template width and height are equal to 1, then line1 will be the reference line adjacent to the template, and line2 will be the reference line adjacent to line1. (If the second version of hybrid MRL is applied, for each candidate combination, the prediction of the template is a hybrid prediction of the first reference line (based on intra prediction mode) and the second reference line (based on intra prediction mode).

[0238] Step 1: Each combination is signaled following the cost order from step 0. The index equal to 0 signals the shortest or most efficient codeword and corresponds to the pair with the smallest boundary / template matching cost.

[0239] The encoder and decoder perform steps 0 and 1 to obtain the same correspondence from signal index to combination.

[0240] The number of candidate combinations for signaling can be reduced from the original total candidate combinations to the top K candidate combinations with the minimum cost, and the codewords of the combinations selected for signaling can be reduced. When K is set to 1, the selected combination can be inferred as the combination with the minimum cost without signaling the index. The following takes chroma MRL as an example.

[0241] ■If the candidate intra prediction mode includes planar (if repeated with chroma DM, it is changed to VDIA), vertical (if repeated with chroma DM, it is changed to VDIA), horizontal (if repeated with chroma DM, it is changed to VDIA), DC (if repeated with chroma DM, it is changed to VDIA), chroma DM and 6 LM modes, the candidate reference lines include lines 0, 1, 2 (the first reference line is line n, and the second reference line is line n+1),

[0242] ■Initially, the total number of candidate combinations is 11*3.

[0243] ■ Using the proposed method, only the top K combinations with the smallest costs are candidate combinations of the signal, where K can be a positive integer, such as 1, 2, 3, or 32.

[0244] ■ In one possible approach, when the boundary / template of the current block is not available, a default combination (eg, any one of the candidate intra prediction modes, any pair from the candidate reference lines) is defined and used.

[0245] ■ In another possible approach, chroma MRL is inferred to be disabled when the boundary / template of the current block is not available.

[0246] In another sub-embodiment, the proposed signaling / method is applied when multiple reference lines (which may consist of adjacent reference lines and / or one or more non-adjacent reference lines) are used to generate intra-frame prediction of the current block. For example, the proposed signaling / method (similar to the replacement method) will replace the original signaling / method for intra-frame prediction and / or reference lines. For another example, the proposed signaling / method (similar to the replacement method) will be enabled or disabled depending on the syntax or implicit rules sent by the signal, and when the proposed signaling method is disabled, the original signaling / method and / or reference lines for intra-frame prediction are followed.

[0247] In another sub-embodiment, the proposed signaling / method is applied when only non-adjacent reference lines are used to generate intra prediction of the current block. For example, the proposed signaling / method (similar to the replacement method) will replace the original signaling / method for intra prediction and / or reference lines. For another example, the proposed signaling / method (similar to the replacement method) will be enabled or disabled depending on the syntax or implicit rules sent by the signal, and when the proposed signaling method is disabled, the original signaling / method and / or reference lines for intra prediction are followed.

[0248] In another sub-embodiment, an index is signaled to indicate a selected combination of the current block, and the combination refers to an intra prediction mode, a first reference line, a second reference line, and a weight.

[0249] - In one possible approach, the index is signaled using a truncated unary codeword.

[0250] - In another possible way, the index is represented by the context.

[0251] - In another possible approach, the correspondence from the index to the selected combination depends on boundary / template matching. An exemplary flow is as follows,

[0252] Step 0: Calculate the boundary / template matching cost for each candidate combination.

[0253] ■If boundary matching is used, for each candidate combination, the prediction of the current block is a hybrid prediction and weight of "a hybrid prediction of multiple hypothesis predictions of the first reference line (based on intra-frame prediction mode) and the second reference line (based on intra-frame prediction mode)".

[0254] ■If template matching is used, for each candidate combination, the prediction on the template is a hybrid prediction and weights of multiple hypothesis predictions of the first reference line (based on intra prediction mode) and the second reference line (based on intra prediction mode). If a pair equal to line1 and line2 is a candidate pair, and the template width and height are equal to 1, then line1 will be the reference line adjacent to the template, and line2 will be the reference line adjacent to line1.

[0255] Step 1: Each combination is signaled following the cost order from step 0. The index equal to 0 signals the shortest or most efficient codeword and corresponds to the pair with the smallest boundary / template matching cost.

[0256] The encoder and decoder perform steps 0 and 1 to obtain the same correspondence from signal index to combination.

[0257] The number of candidate combinations for signaling can be reduced from the original total candidate combinations to the top K candidate combinations with the minimum cost, and the codewords for the combination selected for signaling can be reduced. When K is set to 1, the selected combination can be inferred as the combination with the minimum cost without signaling the index.

[0258] ◆If the candidate intra prediction mode includes planar (if repeated with chroma DM, it is changed to VDIA), vertical (if repeated with chroma DM, it is changed to VDIA), horizontal (if repeated with chroma DM, it is changed to VDIA), DC (if repeated with chroma DM, it is changed to VDIA), chroma DM and 6LM mode repetition, the candidate reference lines include lines 0, 1, 2, and the candidate weights (w1, w2) include (1, 3), (3, 1), (2, 2),

[0259] ◆Using the proposed method, only the top K combinations with the smallest cost are candidate combinations for signaling, where K can be a positive integer, such as 1, 2, 3,...

[0260] ◆In one possible approach, when the boundary / template of the current block is not available, a default combination (eg, any one of the candidate intra prediction modes, any pair of the candidate reference lines, any one of the candidate weights) is defined and used.

[0261] ◆ In another possible approach, chroma MRL is inferred to be disabled when the boundary / template of the current block is not available.

[0262] In another sub-embodiment, a first index is signaled to indicate a first reference line, and a second index is signaled to indicate a second reference line. The signal of the second index depends on the first reference line. For example, the total available candidate reference lines include line 0, line 1, and line 2. In one possible manner,

[0263] First reference line = first index (range from 0 to 2)

[0264] Second reference line = second index (ranging from 0 to 1) + ((second index >= first reference line)? 1:0)

[0265] In another sub-embodiment, the first reference line cannot be the same as the second reference line.

[0266] In another sub-embodiment, the first index is signaled to indicate the first reference line. The second reference line is inferred from the first reference line. In other words, the index is signaled to determine a pair of the first reference line and the second reference line. For another example, the second reference line is a reference line adjacent to the first reference line.

[0267] - In one possible approach, the first reference line is line n and the second reference line is line n+1.

[0268] If line n+1 exceeds the "farthest reference line allowed for the current block" (e.g. line 2, if only lines 0, 1, and 2 are candidate reference lines for the current block), the second reference line cannot be used.

[0269] In another possible manner, the first reference line is line n and the second reference line is line n-1.

[0270] If n is equal to 0, the second reference line cannot be used.

[0271] - In another possible approach, the correspondence from the index to the selected reference line pair depends on boundary / template matching. An exemplary flow is as follows,

[0272] Step 0: Calculate the boundary / template matching cost for each candidate reference line pair.

[0273] ■ If boundary matching is used, then for each candidate pair, the prediction of the current block is a hybrid prediction of multiple hypotheses from the first reference line and the second reference line prediction.

[0274] ■ If template matching is used, for each candidate pair, the prediction on the template is a mixture of multiple hypotheses of the predictions for the first reference line and the second reference line. If a pair equal to line1 and line2 is a candidate pair, and the template width and height are equal to 1, then line1 will be the reference line adjacent to the template, and line2 will be the reference line adjacent to line1.

[0275] Step 1: The signals for each pair follow the cost order from step 0. The index equal to 0 signals the shortest or most efficient codeword and corresponds to the pair with the smallest boundary / template matching cost.

[0276] The encoder and decoder perform steps 0 and 1 to obtain the same correspondence from the signal index to the pair.

[0277] The number of candidate pairs for signaling can be reduced from the original total candidate pairs to the top K candidate pairs with minimum costs, and the codewords of the pairs selected for signaling can be reduced.

[0278] In one possible approach, when the boundary / template of the current block is not available, preset reference pairs (eg, line 0 and line 2, line 0 and line 1, line 1 and line 2) are defined and used as the first pair of reference lines.

[0279] In another possible manner, when the boundary / template of the current block is not available, a preset reference line (eg, line 0) is defined and used as the first reference line, and only the first reference line is used to generate intra prediction.

[0280] In another sub-embodiment, the first reference line is implicitly derived and the second reference line depends on the explicit signal and the first reference line. For example, the first reference line is inferred to be line 0. For another example, the first reference line is the reference line with the minimum boundary matching cost. For another example, the first reference line is the reference line with the minimum template matching cost.

[0281] - In one possible way, when the border / template of the current block is not available, a preset reference line (eg, line 0) is defined and used as the first reference line.

[0282] - In another possible manner, when the boundary / template of the current block is not available, a preset reference line is defined and used as the first reference line, and only the first reference line is used to generate intra prediction.

[0283] In another sub-embodiment, the first reference line and the second reference line are both implicitly derived.

[0284] - In one possible approach, the selected reference line pair depends on boundary / template matching. An exemplary process is shown in the following figure,

[0285] ■Step 0: Calculate the boundary / template matching cost for each candidate reference line pair.

[0286] If boundary matching is used, then for each candidate pair, the prediction of the current block is a hybrid prediction of multiple hypotheses from the first reference line and the second reference line prediction.

[0287] If template matching is used, for each candidate pair, the prediction on the template is a mixture of multiple hypotheses of the predictions for the first reference line and the second reference line. If a pair equal to line1 and line2 is a candidate pair, and the template width and height are equal to 1, then line1 will be the reference line adjacent to the template, and line2 will be the reference line adjacent to line1.

[0288] ■Step 1: Infer the selected reference line pairs as the ones with the minimum boundary / template matching cost.

[0289] ■The encoder and decoder perform steps 0 and 1 to obtain the selected pair.

[0290] ■ Another possible way is to define preset reference line pairs (eg, line 0 and line 2, line 0 and line 1, line 1 and line 2) and use them as the first and second reference lines when the boundary / template of the current block is not available.

[0291] ■ In another possible way, when the boundary / template of the current block is not available, a preset reference line (eg, line 0) is defined and used as the first reference line, and only the first reference line is used to generate intra prediction.

[0292] ■ Another possible approach is that for a candidate reference line pair, if the first reference line is line n, the second reference line should be line n+1.

[0293] If line n+1 is beyond the "farthest reference line allowed by the current block" (e.g. line 2), the second reference line cannot be used.

[0294] ■Another possible way is that for a candidate reference line pair, if the first reference line is line n, then the second reference line is line n-1.

[0295] If n is equal to 0, the second reference line cannot be used.

[0296] In another embodiment, when only one reference line is used to generate intra prediction of the current block, the selection of the reference line depends on an implicit rule. For example, the selected reference line (among the candidate reference lines) has the smallest boundary matching cost. For another example, the selected reference line (among the candidate reference lines) has the smallest template matching cost.

[0297] - In one possible way, when the border / template of the current block is not available, a preset reference line (eg, line 0) is defined and used as the first reference line.

[0298] In another embodiment, when only one reference line is used to generate an intra prediction of the current block, the selection of the reference line depends on an explicit rule.

[0299] In one sub-embodiment, an index is signaled to indicate the selected reference line for the current block.

[0300] In another sub-embodiment, an index is signaled to indicate a selected combination for the current block, and the combination refers to an intra prediction mode and one or more reference lines.

[0301] In another possible approach, the index is signaled using a truncated unary codeword.

[0302] - In another possible way, the index is represented by the context.

[0303] - In another possible approach, the correspondence from the index to the selected combination depends on boundary / template matching. An exemplary process is shown in the following figure,

[0304] Step 0: Calculate the boundary / template matching cost for each candidate combination.

[0305] ■ If boundary matching is used, then for each candidate combination, the prediction of the current block is the prediction from the reference line (based on intra prediction mode).

[0306] ■ If template matching is used, for each candidate combination, the prediction on the template is the prediction from the reference line (based on the intra prediction mode). If the reference line equal to line1 is the candidate reference line, and the template width and height are equal to 1, then line1 will be the reference line adjacent to the template, and line2 will be the reference line adjacent to line1.

[0307] Step 1: Each combination is signaled following the cost order from step 0. The index equal to 0 signals the shortest or most efficient codeword and corresponds to the pair with the smallest boundary / template matching cost.

[0308] The encoder and decoder perform steps 0 and 1 to obtain the same correspondence from signal index to combination.

[0309] The number of candidate combinations for signaling can be reduced from the original total candidate combinations to the top K candidate combinations with the minimum cost, and the codewords for the combination selected for signaling can be reduced. When K is set to 1, the selected combination can be inferred as the combination with the minimum cost without signaling the index. The following takes chroma MRL as an example:

[0310] ■If the candidate intra prediction modes include planar (if repeated with chroma DM, changed to VDIA), vertical (if repeated with chroma DM, changed to VDIA), horizontal (if repeated with chroma DM, changed to VDIA), DC (if repeated with chroma DM, changed to VDIA), chroma DM and 6LM modes, the candidate reference lines include lines 0, 1, 2 or only lines 1, 2.

[0311] ■Initially, the total number of candidate combinations is 11*(3 or 2).

[0312] ■ Using the proposed method, only the top K combinations with the smallest cost are candidate combinations of the signal, where K can be a positive integer, such as 1, 2, 3 or (32 or 21).

[0313] ■ In one possible approach, when the boundary / template of the current block is not available, a default combination (eg, any one of the candidate intra prediction modes, any one of the candidate reference lines) is defined and used.

[0314] ■ In another possible approach, chroma MRL is inferred to be disabled when the boundary / template of the current block is not available.

[0315] In another sub-embodiment, the proposed signaling / method is applied when the adjacent reference line or the non-adjacent reference line is used to generate the intra prediction of the current block. For example, the proposed signaling / method (similar to the replacement method) will replace the original signaling / method for intra prediction and / or reference lines. For another example, the proposed signaling / method (similar to the replacement method) will depend on the syntax sent by the signal or the implicit rule to be enabled or disabled, and when the proposed signaling / method is disabled, the original signaling / method for intra prediction and / or the reference line is followed.

[0316] In another sub-embodiment, the proposed signaling / method is applied when only non-adjacent reference lines are used to generate intra predictions for the current block. For example, the proposed signaling / method (similar to the alternative method) will replace the original signaling / method for intra prediction and / or reference lines. For another example, the proposed signaling / method (similar to the alternative method) will depend on the signaled syntax or implicit rules to be enabled or disabled, and when the proposed signaling / method is disabled, the original signaling / method for intra prediction and / or follows the reference lines.

[0317] In another embodiment, the candidate boundary matching cost is calculated as follows. The boundary matching cost of the candidate mode refers to the discontinuity measure (including top boundary matching and / or left boundary matching) between the current prediction (prediction sample in the current block) and the neighboring reconstruction (reconstruction sample in one or more neighboring blocks) generated from the candidate mode. The top boundary matching refers to the comparison between the current top prediction sample and the neighboring top reconstruction sample, and the left boundary matching refers to the comparison between the current left prediction sample and the neighboring left reconstruction sample.

[0318] In one sub-embodiment, a predefined subset of the current prediction is used to calculate the boundary matching cost. The line n of the top boundary within the current block and / or the line m of the left boundary within the current block are used. (In addition, the top adjacent reconstructed line n2 and / or the left adjacent reconstructed line m2 are used.)

[0319] -This is an example of calculating the cost of boundary matching. (n=2, m=2, n2=2, m2=2)

[0320]

[0321] The weights (a, b, c, d, e, f, g, h, i, j, k, l) can be any positive integers, for example, a = 2, b = 1, c = 1, d = 2, e = 1, f = 1, g = 2, h = 1, i = 1, j = 2, k = 1, l = 1.

[0322] This is another example of calculating the boundary matching cost. (n=2,m=2,n2=1,m2=1)

[0323]

[0324] The weights (a, b, c, g, h, i) can be any positive integers, for example, a=2, b=1, c=1, g=2, h=1, i=1.

[0325] This is another example of calculating the boundary matching cost. (n=1,m=1,n2=2,m2=2)

[0326]

[0327] The weights (d, e, f, j, k, l) can be any positive integers, for example, d=2, e=1, f=1, j=2, k=1, l=1.

[0328] This is another example of calculating the boundary matching cost. (n=1,m=1,n2=1,m2=1)

[0329]

[0330] The weights (a, c, g, i) can be any positive integers, for example, a = 1, c = 1, g = 1, i = 1.

[0331] This is another example of calculating the boundary matching cost. (n=2,m=1,n2=2,m2=1)

[0332]

[0333] The weights (a, b, c, d, e, f, g, i) can be any positive integers, for example, a = 2, b = 1, c = 1, d = 2, e = 1, f = 1, g = 1, i = 1.

[0334] This is another example of calculating the boundary matching cost. (n=1,m=2,n2=1,m2=2)

[0335]

[0336] The weights (a, c, g, h, i, j, k, l) can be any positive integers, for example, a = 1, c = 1, g = 2, h = 1, i = 1, j = 2, k = 1, l = 1.

[0337] The following examples of n and m can also be applied to n2 and m2. For another example, n can be any positive integer, such as 1, 2, 3, 4, etc. For another example, m can be any positive integer, such as 1, 2, 3, 4, etc. For another example, n and / or m vary with block width, height, or area.

[0338] - One possible approach is that for larger blocks (e.g. area > threshold 2), m becomes larger.

[0339] oThreshold 2 = 64, 128 or 256.

[0340] oWhen area > threshold 2, m increases to 2. (Initially, m is 1.)

[0341] o When area > threshold 2, m increases to 4. (Originally m is 1 or 2.)

[0342] - Another possible approach is that for taller blocks (e.g. height > threshold 2*width), m becomes larger and / or n becomes smaller.

[0343] oThreshold 2 = 1, 2 or 4.

[0344] o When height > threshold 2*width, m increases to 2. (Initially, m is 1.)

[0345] o When height > threshold 2*width, m increases to 4. (Initially, m is 1 or 2.)

[0346] - Another possible way is that for larger blocks (e.g. area > threshold 2), n becomes larger.

[0347] oThreshold 2 = 64, 128 or 256.

[0348] oWhen area > threshold 2, n increases to 2. (Initially, n is 1.)

[0349] o When area > threshold 2, n increases to 4. (Originally n is 1 or 2.)

[0350] - Another possible approach is that for wider blocks (e.g. width > threshold 2*height), n becomes larger and / or m becomes smaller.

[0351] oThreshold 2 = 1, 2 or 4.

[0352] o When width > threshold 2*height, n increases to 2. (Initially, n is 1.)

[0353] o When width > threshold 2*height, n increases to 4. (Initially, n is 1 or 2.)

[0354] In another embodiment, the candidate template matching cost is calculated as follows. The candidate template matching cost refers to the distortion between the template prediction (prediction samples within the template) and the template reconstruction (reconstruction samples within the template) generated from the candidate (including top template matching and / or left template matching). Top template matching refers to the distortion between the top template prediction samples and the top template reconstruction samples, and left template matching refers to the distortion between the left template prediction samples and the left template reconstruction samples. The distortion can be SAD, SATD or any difference measurement matrix / method.

[0355] Method 1: Linear model based on multiple reference lines

[0356] If the chrominance component or the luminance component of the current block has multiple adjacent reference lines (including adjacent reference lines and / or non-adjacent reference lines), such as Fig.12 As shown, for the current block 1210, the adjacent samples CCLM / MMLM used to derive model parameters can be adapted through reference line selection. In one embodiment, if the current block has N adjacent reference lines, the i-th adjacent reference line is selected to derive the model parameters in CCLM / MMLM, where N>1 and N≥i≥1. If the block has N adjacent reference lines, more than one reference line will be selected in CCLM / MMLM to derive model parameters. For example, if the current block has N adjacent reference lines, 2 of N, 3 of N, ..., or 3 of N adjacent reference lines can be selected to derive model parameters. The selected adjacent reference lines can be non-adjacent adjacent reference lines. For example, if 2 of N adjacent reference lines are selected, the 2 lines can be the first and third reference lines, the second and fourth reference lines, the first and fourth reference lines ... and so on.

[0357] In addition, if a chroma neighboring reference line is selected, another luma reference line can be selected, which does not need to be the corresponding luma reference line. For example, if the i-th chroma neighboring reference line is selected, the j-th luma neighboring reference line can be selected, where i and j can be different or the same. In addition, it is also possible to use luma reference line samples without the luma downsampling process to derive model parameters. For example, Fig.13 As shown, if the corresponding luma sample associated with a chroma sample is Y0, Y1, Y2, Y3, then each Y0, Y1, (Y0+Y2+1)>>1, (Y'2+(Y0<<1)+Y2+2)>>2, (Y0+(Y2<<1)+Y'0+2)>>2, or (Y0+Y2-Y'2) samples can be selected to specify adjacent luma lines to derive model parameters. For another example, it can select each Y1, Y3, (Y1+Y3+1)>>1, (Y'3+(Y1<<1)+Y3+2)>>2, (Y1+(Y3<<1)+Y'1+2)>>2, or (Y1+Y3-Y'3) sample at the specified adjacent luma line to derive model parameters.

[0358] In another embodiment, if more than one adjacent reference line CCLM / MMLM is used, the luminance downsampling filter between each reference line may be different. For example, if two reference lines are used in CCLM / MMLM, the downsampling filter for outputting the selected line closest to the current block boundary may be an m-tap high-pass or low-pass filter. The downsampling filter for outputting the other reference line may be an n-tap high-pass or low-pass filter. In another embodiment, m is 6 and n is 4. In another embodiment, m is greater than n. In yet another embodiment, m is less than n.

[0359] In addition, if one of the multiple reference lines is invalid due to unavailable neighboring samples or due to CTU line buffer size limitations, another valid reference line is used to replace the invalid reference line. For example, as shown in FIG. 14, if the third reference line is invalid, and the first and second reference lines are valid, the third reference line can be replaced by the first or second reference line. In another embodiment, only valid reference lines are used in cross-component model derivation. In other words, invalid reference lines are not used in cross-component model derivation.

[0360] Method 2: Linear model derivation based on multiple reference lines

[0361] If the chrominance component or the luminance component of the current block has multiple adjacent reference lines, the multiple adjacent reference lines can be combined or integrated into one line to derive the model parameters in CCLM / MMLM. Fig.14A As shown, if three adjacent reference lines are available for the current block 1410, a 3x3 window ( Fig. 14B ) combines three adjacent reference lines into one line and uses the combined line to derive the cross-component model parameters. The combined result of the 3x3 window is expressed as where w i can be positive, negative or 0, and b is the offset value. Similarly, in the same example, you can first use Fig. 14C The 3x2 window in is used to combine three adjacent reference lines. The combination result of the 3x2 window is expressed as where w i Can be positive or negative, b is the offset value. Note that C i can be adjacent luminance or chrominance samples. In another embodiment, the generalized formula is Where L i and C i are adjacent luminance and chrominance samples, S is the window size applied, and w i Can be positive, negative or 0, b is the offset value.

[0362] Method 3: CCLM / MMLM model derivation based on multiple reference lines

[0363] If the model derivation of CCLM / MMLM is based on different adjacent reference line selections, the indication of the selected line of CCLM / MMLM can be explicitly determined or implicitly derived. For example, if the current block allows one or two reference lines and the selected line of CCLM / MMLM is explicitly determined, the first codeword (bin) is used to indicate whether one line or two lines are used. Then, the second codeword or multiple codewords (encoded by truncated unary or fixed length code) are used to indicate which reference line or which line combination is selected. For example, if one reference line is used, it can be selected from {first line, second line, third line...}. If two reference lines are used, it can be selected from {first line+second line, second line+third line, first line+third line...}.

[0364] The selected lines of CCLM / MMLM can be exported implicitly by using decoder-side tools, such as matching by template cost or boundary. For example, on the decoder side, the final line selection for the current block is the CCLM / MMLM whose line minimizes the difference between the boundary samples of the current block and the neighboring samples along the boundary, as shown below Fig.15 .

[0365] At the decoder side, the final line selection for the current block is the CCLM / MMLM whose line can minimize the distortion of the neighboring templates (e.g., Fig.15 ). For example, after deriving the model parameters of CCLM / MMLM through one line, the model is applied to the luminance samples of the adjacent templates and the cost is calculated. Then, the model parameters of CCLM / MMLM are derived through another line, the model is applied to the luminance samples of the adjacent templates, and the current cost is compared with the cost of the earlier model. This process is continuously applied to other models, and the final chrominance prediction of the current block is selected based on the model with the smallest cost.

[0366] In addition, the use of more than one reference line may depend on the current block size or CCLM / MMLM mode. In one embodiment, if the current block width is less than a threshold, more than one reference line is used in CCLM_A or MMLM_A. Similarly, if the current block height is less than a threshold, more than one reference line is used in CCLM_L or MMLM_L. If the (width+height) of the current block is less than a threshold, more than one reference line is used in CCLM_LA or MMLM_LA. For another example, if the area of ​​the current block is less than a threshold, more than two reference lines are used in CCLM or MMLM. In another embodiment, more than one reference line is used in CCLM_A, CCLM_L, MMLM_A or MMLM_L. In another embodiment, syntax is signaled at the SPS (sequence parameter set), PPS (picture parameter set), PH (picture header), SH (slice header), CTU, CU or PU level to indicate whether there is more than one reference current block that allows the use of the line.

[0367] The method proposed in the present invention can be enabled and / or disabled according to implicit rules (such as block width, height or area) or according to explicit rules (such as syntax on block, tile, slice, picture, SPS or PPS level). For example, when the block area is less than a threshold, the proposed reordering is applied.

[0368] The term "block" in the present invention may refer to a TU / TB, a CU / CB, a PU / PB, a predefined region, or a CTU / CTB. A CU with the proposed method may refer to an intra block encoded in an intra coding mode, a CU with the proposed method may refer to an inter block encoded in an inter coding mode, and / or an IBC block encoded in an IBC coding mode using a reference to a block vector of a reference block in the current picture (determined by a syntax element or an implicit search algorithm of template matching at the encoder / decoder). For example, the inter coding mode is an inter merge mode, where the prediction is generated by a combination of inter prediction and intra prediction using the proposed method, or the inter coding mode is an inter merge mode, where the chroma prediction is generated by intra prediction using the proposed method. For another example, the IBC coding mode is an IBC sub-mode, where the prediction is generated by combining a prediction from a block vector and a prediction from an intra using the proposed method, or the IBC coding mode is an IBC sub-mode, where the chroma prediction is generated by intra prediction using the proposed method.

[0369] Any combination of the methods proposed in the present invention may be applied.

[0370] The joint prediction by combining intra prediction modes and one or more reference lines as described above can be implemented at the encoder side or the decoder side. For example, any proposed method for evaluating the cost associated with a set of intra prediction candidates using one or more reference lines can be implemented at the decoder or the intra coding module (e.g., Figure 1B Intra-frame prediction 150 or Figure 1A Any of the proposed methods may also be implemented as a circuit coupled to an intra-coding module at a decoder or encoder. However, the decoder or encoder may also use additional processing units to implement the required processing. The intra-prediction unit (e.g., Figure 1A Unit 110 and Figure 1B The units 150 in the figure are shown as separate processing units, which may correspond to executable software or firmware program code or programmable devices stored on a CPU (central processing unit), such as a DSP (digital signal processor) or FPGA (field programmable gate array) on a medium (such as a hard disk or flash memory).

[0371] Fig.16 A flowchart of an exemplary video encoding system for merging joint prediction by combining intra-prediction modes and one or more reference lines according to an embodiment of the present invention is shown. The steps shown in the flowchart can be implemented as executable program codes on one or more processors (e.g., one or more CPUs) on the encoder side. The steps shown in the flowchart can also be implemented based on hardware, such as one or more electronic devices or processors arranged to perform the steps in the flowchart. According to the method, input data associated with a current block is received in step 1610, wherein the input data includes pixel data of the current block to be encoded on the encoder side or encoded data associated with the current block to be decoded on the decoder side. A set of joint prediction candidates is determined in step 1620, wherein each joint prediction candidate corresponds to a combination of an intra-prediction mode from a set of intra-prediction mode candidates and one or more reference samples in one or more reference lines from a set of reference line candidates. In step 1630, a target joint prediction candidate is determined from the set of joint prediction candidates. In step 1640, the current block is encoded or decoded by using one or more predictors including the target joint prediction candidate.

[0372] The flowchart shown is intended to illustrate an example of video encoding according to the present invention. Without departing from the spirit of the present invention, those skilled in the art may modify each step, rearrange the steps, split the steps, or combine the steps to implement the present invention. In the present disclosure, specific syntax and semantics have been used to illustrate examples to implement embodiments of the present invention. Without departing from the spirit of the present invention, those skilled in the art may implement the present invention by replacing syntax and semantics with equivalent syntax and semantics.

[0373] The above description is provided to enable those of ordinary skill in the art to practice the present invention provided in the context of a specific application and its requirements. Various modifications to the described embodiments will be apparent to those skilled in the art, and the general principles defined herein may be applied to other embodiments. Therefore, the present invention is not intended to be limited to the specific embodiments shown and described, but to conform to the widest scope consistent with the principles and novel features disclosed herein. In the above detailed description, various specific details are illustrated to provide a thorough understanding of the present invention. However, those skilled in the art will appreciate that the present invention may be implemented.

[0374] The embodiments of the present invention as described above can be implemented in various hardware, software codes or a combination of the two. For example, one embodiment of the present invention can be one or more circuits integrated into a video compression chip or a program code integrated into the video compression software to perform the processing described herein. The embodiments of the present invention can also be a program code to be executed on a digital signal processor (DSP) to perform the processing described herein. The present invention can also be related to many functions performed by a computer processor, a digital signal processor, a microprocessor or a field programmable gate array (FPGA). These processors can be configured to perform specific tasks according to the present invention by executing machine-readable software codes or firmware codes that define the specific methods embodied by the present invention. The software code or firmware code can be developed in different programming languages ​​and different formats or styles. The software code can also be compiled for different target platforms. However, the different code formats, styles and languages ​​of the software code and other ways of configuring the code to perform tasks according to the present invention will not depart from the spirit and scope of the present invention.

[0375] The present invention may be embodied in other specific forms without departing from its spirit or essential characteristics. The described examples should be considered in all respects as illustrative and not restrictive. Therefore, the scope of the present invention is indicated by the appended claims rather than by the foregoing description. All changes that fall within the meaning and scope of equivalents to the scope of the claims should be included within their scope.

Claims

1. A video encoding and decoding method, the method comprising: Receiving input data related to a current block, wherein the input data includes pixel data of the current block to be encoded at an encoder side or encoded data related to the current block to be decoded at a decoder side; determining a set of joint prediction candidates, wherein each joint prediction candidate corresponds to a combination of an intra prediction mode from a set of intra prediction mode candidates and one or more reference samples in one or more reference lines from a set of reference line candidates; Determine a target joint prediction candidate from the joint prediction candidate set; and The current block is encoded or decoded by using one or more predictors including the target joint prediction candidate.

2. The method according to claim 1, characterized in that The one or more reference samples include one or more prediction samples or one or more reconstructed samples in one or more non-adjacent neighboring lines of the current block.

3. The method according to claim 2, characterized in that The one or more reference samples further include one or more prediction samples or one or more reconstructed samples in a neighboring line of the current block.

4. The method according to claim 1, characterized in that: An index indicating a target joint prediction candidate from the set of joint prediction candidates is signaled at the encoder side or parsed at the decoder side.

5. The method according to claim 1, characterized in that: A set of indices is mapped to the joint prediction candidate set according to cost measures associated with members of the joint prediction candidate set.

6. The method according to claim 5, characterized in that The cost measure corresponds to a boundary matching cost or a template matching cost of members of the joint prediction candidate set.

7. The method according to claim 5, characterized in that The joint prediction candidate set is reordered according to the cost measure, and the shortest codeword is assigned to the target index with the smallest cost measure.

8. The method according to claim 5, characterized in that A target index indicating a target joint prediction candidate in a joint prediction candidate set for a current block is signaled at the encoder side or parsed at the decoder side.

9. The method according to claim 5, characterized in that The top K joint prediction candidates with the smallest cost measure are mapped to the index set, and K is a positive integer.

10. The method according to claim 9, characterized in that When K is equal to 1, a target index indicating a target joint prediction candidate in a joint prediction candidate set for the current block is not signaled at the encoder side, or is inferred at the decoder side.

11. The method according to claim 1, characterized in that: The intra prediction mode candidate group includes all available intra prediction modes or a subset of all available intra prediction modes.

12. The method according to claim 1, characterized in that The intra prediction mode candidate group, the reference line candidate group, or both are determined according to a block size of a current block.

13. The method according to claim 1, characterized in that The target joint prediction candidate is used to replace the original intra prediction.

14. The method according to claim 13, characterized in that Whether the target joint prediction candidate is used to replace the original intra prediction is enabled or disabled depending on the signaled syntax, or is determined implicitly.

15. The method according to claim 1, characterized in that The final intra prediction is selected between the target joint prediction candidate and the original intra prediction.

16. The method according to claim 15, characterized in that The target joint prediction candidate is enabled or disabled depending on the signaled syntax to replace the original intra prediction, or is determined implicitly.

17. The method according to claim 1, characterized in that The target joint prediction candidate is used to encode or decode the chrominance block of the current block, and the intra-frame prediction mode candidate group for chrominance multi-reference line (MRL) prediction includes planar, vertical, horizontal, DC, one or more LM modes, chrominance DM, DIMD chrominance mode, diagonal (DIA), VDIA (mode 66 of 67 intra-frame prediction modes) or any combination thereof.

18. The method according to claim 1, characterized in that The one or more reference samples are used to assist in determining the target joint prediction candidate.

19. An apparatus for video encoding and decoding, the apparatus comprising one or more electronic devices or processors arranged to: Receive input data related to the current block, where The input data includes pixel data to be encoded of the current block at the encoder side or encoded data to be decoded of the current block at the decoder side; Determine a set of joint prediction candidates, wherein each joint prediction candidate corresponds to a combination of an intra prediction mode in a set of intra prediction mode candidates and one or more reference samples in one or more reference lines in a set of reference line candidates; Determining a target joint prediction candidate from the joint prediction candidate set; as well as The current block is encoded or decoded by using one or more predictors including the target joint prediction candidate.

Images