Inter chroma across type reference and constrained reference regions

Abstract

A cross-component prediction method for video coding is provided. A video codec receives data to be encoded or decoded as a current block of pixels of a current picture of a video. The video codec identifies a predefined region suitable for the current block. The video codec derives a candidate list that may be inherited by the current block, the list including cross-component prediction models from locations outside the predefined region. The video codec selects one or more cross-component models from the candidate list. A video codec encodes or decodes the current block by applying the selected one or more cross-component models to the reconstructed or predicted first component of the current block to generate a predictor for the second component of the current block.

Description

[0001] Cross-referencing

[0002] This disclosure is part of a non-provisional application that claims priority to U.S. Provisional Patent Application No. 63 / 513,366, filed July 13, 2023. The contents of the above application are incorporated herein by reference. Technical Field

[0003] This disclosure generally relates to video encoding and decoding. In particular, this disclosure relates to methods for encoding and decoding pixel blocks via cross-component prediction. Background Technology

[0004] Unless otherwise stated herein, the methods described in this section are not prior art to the claims listed below, and are not recognized as prior art by virtue of being included in this section.

[0005] High-Efficiency Video Coding (HEVC) is an international video coding standard developed by the Joint Collaborative Team on Video Coding (JCT-VC). HEVC is based on a hybrid block-based motion-compensated DCT transform coding architecture. The basic unit of compression, called a coding unit (CU), is a 2Nx2N pixel block. Each CU can be recursively divided into four smaller CUs until a predefined minimum size is reached. Each CU contains one or more prediction units (PUs).

[0006] Versatile Video Coding (VVC) is the latest international video coding standard developed by the Joint Video Experts Team (JVET) of ITU-T SG16 WP3 and ISO / IEC JTC1 / SC29 / WG11. The input video signal is predicted from the reconstructed signal derived from the coded image region. The prediction residual signal is processed through block transform. The transform coefficients are quantized and entropy-coded in the bitstream along with other additional information. The reconstructed signal is generated from the prediction signal and the reconstructed residual signal by performing an inverse transform on the dequantized transform coefficients. The reconstructed signal is further processed by in-loop filtering to remove coding artifacts. The decoded image is stored in the frame buffer and used to predict future images in the input video signal.

[0007] In VVC, the encoded image is divided into non-overlapping block regions represented by relevant coding tree units (CTUs). The leaf nodes of the coding tree correspond to coding units (CUs). The encoded image can consist of multiple slices, each containing an integer number of CTUs. The CTUs within a slice are processed in raster scan order. Bi-predictive (B) slices can be decoded using either intra-frame or inter-frame prediction, using at most two motion vectors (MVs) and a reference index to predict the sample values ​​for each block. Predictive (P) slices are decoded using either intra-frame or inter-frame prediction, using at most one motion vector and a reference index to predict the sample values ​​for each block. Intra (I) slices are decoded using only intra-frame prediction.

[0008] CTUs can be partitioned into one or more non-overlapping coding units (CUs) using quadtree (QT) and nested multi-type-tree (MTT) structures to accommodate various local motion and texture characteristics. CUs can be further partitioned into smaller CUs using one of five partitioning types: quadtree partitioning, vertical binary tree partitioning, horizontal binary tree partitioning, vertical center-side ternary tree partitioning, and horizontal center-side ternary tree partitioning.

[0009] Each CU contains one or more prediction units (PUs). A prediction unit, along with its associated CU syntax, serves as the basic unit of signal predictor information. The specified prediction process is used to predict the values ​​of associated pixel samples within a PU. Each CU may contain one or more transform units (TUs) to represent prediction residual blocks. A transform unit (TU) consists of a transform block (TB) for the luminance sample and two corresponding chrominance sample transform blocks, each TB corresponding to a residual sample block from one color component. Integer transforms are applied to the transform blocks. The level values ​​of the quantization coefficients, along with other additional information, are entropy-encoded in the bitstream. The terms coding tree block (CTB), coding block (CB), prediction block (PB), and transform block (TB) are defined as two-dimensional sample arrays specifying the monochromatic components associated with the CTU, CU, PU, ​​and TU. Therefore, a CTU consists of one luminance CTB, two chrominance CTBs, and associated syntax elements. CUs, PUs, and TUs have similar relationships. (In this disclosure, the term “block” may refer to TU / TB, CU / CB, PU / PB, or CTU / CTB.)

[0010] For each inter-frame predicted CU, motion parameters include motion vectors, reference image indices, reference image list usage indices, and additional information used for inter-frame predicted sample generation. Motion parameters can be signaled explicitly or implicitly. When a CU is coded in skip mode, the CU is associated with a PU and has no significant residual coefficients, no encoded motion vector increments, and no reference image indices. A merging mode is specified where the motion parameters of the current CU are obtained from neighboring CUs, including spatial and temporal candidates, and additional plans introduced in the VVC. The merging mode can be applied to any inter-frame predicted CU. An alternative to the merging mode is explicit transmission of motion parameters, where motion vectors, corresponding reference image indices for each reference image list, reference image list usage flags, and other necessary information are explicitly transmitted in each CU. Summary of the Invention

[0011] The following summary is for illustrative purposes only and is not intended to limit the scope of the invention in any way. That is, the following summary aims to introduce the concepts, highlights, benefits, and advantages of the novel and non-obvious techniques described herein. Selected, but not all, embodiments will be further described in the detailed description below. Therefore, the following summary is not intended to identify the essential features of the claimed subject matter, nor is it intended to determine the scope of the claimed subject matter.

[0012] Some embodiments of this disclosure provide a cross-component prediction method for video encoding and decoding. A video codec receives data of a current pixel block to be encoded or decoded into a current image of a video. The video codec identifies a predefined region applicable to the current block. The video codec derives a candidate list that can be inherited by the current block, including cross-component prediction models from locations outside the predefined region. The video codec selects one or more cross-component models from the candidate list. The video codec encodes or decodes the current block by applying the selected one or more cross-component models to a first component of the reconstructed or predicted current block to generate a predictor for a second component of the current block. The video codec can generate cross-component prediction by applying the selected one or more cross-component models to a first component (e.g., luminance Y or chrominance Cb) of the reconstructed or predicted current block to generate a predictor for a second component (e.g., chrominance Cb or chrominance Cr) of the current block. The current block can be inter-frame prediction or intra-block copy (IBC) prediction.

[0013] The predefined region may encompass the current block and another block that also uses the same candidate list (or a portion thereof) for cross-component prediction. This other block may use at least one of spatially adjacent candidates, spatially non-adjacent candidates, and historical candidates from the candidate list of the current block. The predefined region may encompass first and second blocks, wherein a first candidate list derived for the first block excludes cross-component prediction information or models from the second block, and a second candidate list derived for the second block excludes cross-component prediction information or models from the first block. Therefore, the first and second candidate lists are derived independently. In some embodiments, the historical model candidate list is updated only when the size of the current block is greater than a threshold determined based on the size of the predefined region.

[0014] In some embodiments, subsequent codec blocks encoded with a prediction mode different from the current block are allowed to inherit the selected cross-component model, such that subsequent intra-coded blocks can inherit the cross-component model from inter-coded blocks. In some embodiments, subsequent codec blocks encoded with a prediction mode different from the current block are not allowed to inherit the selected cross-component model.

[0015] In some embodiments, the video codec identifies a representative prediction mode for the current block and uses this representative prediction mode to select at least one transform set (major and minor) and a transpose flag for transforming or inverse transforming the prediction residuals of the current block, where the current block is a chroma block. In some embodiments, the representative prediction mode is identified by performing texture gradient analysis on the predicted chroma samples (or subsamples of the predicted chroma samples) of the current block, where the current block is a chroma block. In some embodiments, the representative prediction mode is used if the current chroma block is referenced when deriving the most probable mode (MPM) set for intra-frame encoding and decoding of subsequent blocks. Attached Figure Description

[0016] The accompanying drawings are included to provide a further understanding of this disclosure and form part of this disclosure. The drawings illustrate embodiments of this disclosure and, together with the description, serve to explain the principles of this disclosure. It will be understood that the drawings are not necessarily drawn to scale, as some components may be shown out of proportion to their actual dimensions in order to clearly illustrate the concepts of this disclosure.

[0017] Figure 1 Shows the template and reference sample used for Template-based intra modederivation (TIMD).

[0018] Figure 2 Explain the positions of the candidates for spatial merging.

[0019] Figure 3 Displays spatially adjacent blocks used to export spatial merge candidates.

[0020] Figure 4 Explain the scaling of motion vectors in the time-merging candidates.

[0021] Figure 5 Display the candidate positions for time merging candidates.

[0022] Figure 6 This conceptually describes a list of merged blocks within the same merge estimation region (MER).

[0023] Figure 7A -B delineates the region of interest (ROI) for the Low Frequency Non-Separable Transform (LFNST).

[0024] Figure 8 This displays the mapping from intra-frame prediction modes to LFNST set indices.

[0025] Figure 9 This demonstrates the use of a cross-component residual model (CCRM) to reconstruct component samples.

[0026] Figure 10 This section provides a conceptual explanation of the parameters for the inheritance time-adjacent model.

[0027] Figure 11A -B indicates that non-adjacent spatial adjacent blocks can inherit model parameters from it.

[0028] Figure 12 This conceptually describes a list of models for blocks within the same cross-component mergeestimation region (CCMER).

[0029] Figure 13 This section describes an example video encoder that may enable cross-component prediction.

[0030] Figure 14 This section describes the part of the video encoder that enables cross-component prediction of various features.

[0031] Figure 15 This conceptually illustrates the process of using cross-component prediction to encode pixel blocks.

[0032] Figure 16 This section describes an example video decoder that may enable cross-component prediction.

[0033] Figure 17 This describes the part of the video decoder that enables cross-component prediction of various features.

[0034] Figure 18 This conceptually illustrates the process of decoding pixel blocks using cross-component prediction.

[0035] Figure 19 This invention provides a conceptual illustration of an electronic system that implements certain embodiments of the present disclosure. Detailed Implementation

[0036] In the following detailed description, numerous specific details are provided by way of example to provide a thorough understanding of the related techniques. Any variations, derivatives, and / or extensions of the techniques described herein are within the scope of this disclosure. In some cases, well-known methods, procedures, components, and / or circuits relating to one or more example embodiments may be described at a relatively high level without going into detail to avoid unnecessarily obscuring various aspects of the techniques disclosed herein.

[0037] I. Intra-frame prediction mode

[0038] a. Directional intra-frame mode

[0039] Intra-prediction methods utilize one or more reference lines adjacent to the current prediction unit (PU) and one of the intra-prediction modes to generate a predictor for the current PU. The intra-prediction direction can be selected from a mode set containing multiple prediction directions, DC modes, and planar modes. An intra-prediction mode can also refer to any intra-prediction mode that uses spatial reconstruction samples to determine the predictor for the current block. The number of directional intra-prediction modes can be 33 or extended to 65 directional modes. By including DC and planar modes, the number of intra-prediction modes is 35 (or 67). Some intra-prediction modes (e.g., 3 or 5) are identified as the set of most probable modes (MPMs) for intra-prediction of the current prediction block, and thus one of these MPMs can be selected via signaling.

[0040] Traditional angular intra-prediction directions are defined as clockwise from 45 degrees to -135 degrees. In VVC, several traditional angular intra-prediction modes are adaptively replaced with wide-angle intra-prediction modes that are not square blocks. The replaced modes are signaled using the original mode index, parsed, and then remapped to the wide-angle mode index.

[0041] When referring to intra-frame prediction, the terms "angle" and "directivity" are used interchangeably in this disclosure. For example, the terms {angle mode, angle intra-frame prediction, angle intra-frame mode} and {directivity mode, directional intra-frame prediction, directional intra-frame mode} are used interchangeably.

[0042] b. Decoder-side intra-mode derivation (DIMD)

[0043] To implicitly derive the intra-prediction mode of the block, texture gradient analysis is performed on both the encoder and decoder sides. This process begins with an empty Histogram of Gradients (HoG) with 65 entries, corresponding to 65 angular modes. The magnitudes of these entries are determined during texture gradient analysis.

[0044] c. Template-based intra-mode derivation (TIMD)

[0045] Template-based intra-predictive mode (TIMD) mode implicitly derives the intra-predictive mode of the CU at the encoder and decoder using a neighboring template, instead of signaling the exact intra-predictive mode bits to the decoder. Figure 1The template and reference samples used for TIMD are shown. As shown, the predicted samples for template 110 are generated for each candidate mode using reference sample 120 of the template. The SATD between the predicted samples and the reconstructed samples of the template is calculated as the cost. The intra-prediction mode with the lowest cost is selected as the TIMD mode and used for intra-prediction of CU 100. Candidate modes can be the 67 intra-prediction modes in VVC or expanded to 131 intra-prediction modes. Typically, the MPM can provide cues to indicate the orientation information of the CU. Therefore, in order to reduce the intra-mode search space and utilize the characteristics of the CU, the intra-prediction modes are implicitly derived from the MPM list.

[0046] d. Intra-template matching (IntraTMP)

[0047] Intra-Template Matching Prediction (IntraTMP) is a special intra-prediction mode that copies the best prediction block from the reconstructed portion of the current frame, whose L-shaped template matches the current template. For a predefined search range, the encoder searches for the template most similar to the current template in the reconstructed portion of the current frame and uses the corresponding block as the prediction block. The encoder then signals this mode and performs the same prediction operation on the decoder side.

[0048] II. Inter-frame prediction

[0049] For each inter-frame predicted CU, motion parameters include motion vectors, reference image indices, and reference image list usage indices, as well as additional information for inter-frame predicted sample generation. Motion parameters can be signaled explicitly or implicitly. When a CU is encoded in skip mode, the CU is associated with a PU and has no significant residual coefficients, no encoded motion vector increments, and no reference image indices. A merging mode is specified where the motion parameters of the current CU are obtained from neighboring CUs, including spatial and temporal candidates, as well as additional plans introduced in the VVC. The merging mode can be applied to any inter-frame predicted CU, not just skip mode. An alternative to the merging mode is explicit transmission of motion parameters, where the motion vectors for each reference image list, the corresponding reference image indices, and the reference image list usage flags, along with other necessary information, are explicitly signaled on each CU.

[0050] The following are some inter-frame predictive coding tools:

[0051] • Extended merge forecast

[0052] Merge mode with MVD (MVD)

[0053] Symmetric MVD (SMVD) signaling

[0054] • Affine motion compensation prediction

[0055] • Subblock-based temporal motion vector prediction (SbTMVP)

[0056] • Adaptive motion vector resolution (AMVR)

[0057] • Sports field storage: 1 / 16 th Luminance sample MV storage and 8x8 motion field compression

[0058] • Bi-prediction with CU-level weight (BCW)

[0059] • Bidirectional optical flow (BDOF)

[0060] • Decoder-side motion vector refinement (DMVR)

[0061] • Geometric partitioning mode (GPM)

[0062] • Combined inter and intra prediction (CIIP)

[0063] a. Extended merge forecast

[0064] • The merged candidate list can be constructed by including the following five types of candidates in order:

[0065] • Spatial MVP (space merge candidate) from a spatially adjacent CU

[0066] • Time MVP (time merging candidate) from the same CU

[0067] • Historical MVP (HMVP merge candidate) from the FIFO table

[0068] • Paired average MVP (paired merged candidate)

[0069] ·Zero MV.

[0070] Figure 2 The locations of the spatial merging candidates are indicated. A maximum of four merging candidates can be selected from those at the locations shown in the figure. The derivation order is B. 0, A 0, B 1, A1 and B2. Position B2 is only considered if one or more coding units (CUs) at positions B0, A0, B1, and A1 are unavailable (e.g., because it belongs to another segment or tile) or if it is intra-frame coded. After adding a candidate for position A0, the addition of the remaining candidates undergoes a redundancy check to ensure that candidates with the same motion information are excluded from the list, thereby improving coding efficiency.

[0071] In addition to the spatial candidates mentioned above, non-adjacent spatial merging candidates are inserted after the temporal motion vector prediction (TMVP) in the regular merging candidate list. Figure 3 The spatial neighbor blocks used to derive spatial merge candidates are displayed. The distance between non-adjacent spatial candidates and the current coding block is based on the width and height of the current coding block. Line buffer limits do not apply.

[0072] For temporal merging candidates, only one candidate is added to the list. Specifically, when deriving this temporal merging candidate, a scaled motion vector is derived based on the corresponding coding unit (CU) belonging to the corresponding reference image. The list of reference images and reference indices used to derive the corresponding CU are explicitly signaled in the fragment header. Figure 4 The scaling of motion vectors for temporal merging candidates is explained. The scaled motion vectors are derived from the motion vectors of the corresponding CUs using the Proof-of-Cost (POC) distances tb and td, where tb is defined as the POC difference between the current image and the reference image, and td is defined as the POC difference between the corresponding image and the reference image. The reference image index for the temporal merging candidates is set to zero.

[0073] Figure 5 The candidate positions for time merge candidates are displayed. As shown, the position of a time candidate is selected between candidate C0 and C1. If the CU at position C0 is unavailable, is intra-coded, or is outside the current CTU line, position C1 is used. Otherwise, position C0 is used when deriving the time merge candidate.

[0074] Historical motion vector prediction (HMVP) merging candidates are added to the merging list after spatial motion vector prediction (MVP) and temporal motion vector prediction (TMVP). In this method, motion information from previous coded blocks is stored in a table and used as the motion vector prediction (MVP) for the current CU. A table containing multiple HMVP candidates is maintained during encoding / decoding. The table is reset (cleared) when a new CTU row is encountered. Whenever a non-sub-block inter-coded CU is encountered, the associated motion information is added as a new HMVP candidate to the last entry of the table.

[0075] Pairwise averaging candidates are generated by averaging predefined candidate pairs from an existing list of merged candidates, using the first two merged candidates. The first merged candidate is defined as p0Cand, and the second merged candidate is defined as p1Cand. The average motion vector is calculated for each reference list based on the availability of motion vectors for p0Cand and p1Cand. If both motion vectors in a list are available, even if they point to different reference images, they are averaged, with the reference image set to the reference image of p0Cand; if only one motion vector is available, that motion vector is used directly; if no motion vector is available, the list remains invalid. Furthermore, if the half-pixel interpolation filter indices of p0Cand and p1Cand are different, they are set to 0.

[0076] When the merge list is not full after adding pairwise average merge candidates, zero motion vector prediction (MVP) is inserted at the end until the maximum number of merge candidates is reached.

[0077] b. Merged Estimated Region (MER)

[0078] Merge estimation region (MER) allows CUs within the same MER to independently derive merge candidate lists. Candidate blocks located in the same MER as the current CU will not be included in the current CU's merge candidate list generation. Figure 6 This conceptually illustrates the merge lists of blocks within the same merge estimation region (MER). The figure illustrates MER 605 in image 600. Block 610 is located within MER 605 and uses merge candidate list 615. Block 620 is also located within MER 605 and uses merge candidate list 625. Both merge lists 615 and 625 are derived based on candidate locations outside of MER 605.

[0079] Furthermore, the update process for the historical motion vector prediction candidate list is only performed when the following conditions are met: (xCb + cbWidth) >> Log2ParMrgLevel is greater than xCb >> Log2ParMrgLevel, and (yCb + cbHeight) >> Log2ParMrgLevel is greater than (yCb >> Log2ParMrgLevel), where (xCb, yCb) is the top-left brightness sample position of the current CU in the image, and (cbWidth, cbHeight) is the CU size. The MER size is selected at the encoder end and signaled in the sequence parameter set (SPS) with log2_parallel_merge_level_minus2 (i.e., Log2ParMrgLevel -2).

[0080] III. Multi Transform Selection (MTS)

[0081] Multiple Transform Selection (MTS) is a new cosine transform variant introduced in VVC. In its predecessor, HEVC, DCT-II was the only such variant. Compared to HEVC, MTS adds several sine / cosine transform types: DST-7 and DCT-8. In the current VVC design, only the DST7 and DCT8 transform kernels are used for MTS, and these kernels are used for intra-frame and inter-frame coding. Additional major transforms include DCT5, DST4, DST1, and the identity transform (IDT).

[0082] a. Intra-coded MTS

[0083] The MTS transform set depends on the TU size and intra-frame mode information. For blocks predicted via IntraTMP, the DIMD process is used to predict the blocks to derive the intra-frame mode for transform selection. Specifically, the horizontal and vertical gradients are computed for each predicted sample to construct the HoG. The intra-frame predicted mode with the largest histogram magnitude is then used to select the MTS transform set.

[0084] In general, 16 different TU sizes were considered, and for each TU size, 5 different categories were considered based on intra-frame mode information. For each category, 1, 4, or 6 different transform pairs were considered. The number of intra-MTS candidates was adaptively selected based on the sum of the absolute values ​​of the transform coefficients (between 1, 4, and 6 MTS candidates). The sum was compared with two fixed thresholds to determine the total number of allowed MTS candidates.

[0085] 1 candidate: sum <= th0

[0086] Four candidates: th0 < sum <= th1

[0087] 6 candidates: sum > th1

[0088] Note that while a total of 80 different categories were considered, some of these different categories often share the exact same set of transformations. Therefore, there are 58 unique entries in the resulting LUT (fewer than 80).

[0089] For angular intra-prediction modes, the joint symmetry of the TU shape and intra-prediction is considered. Therefore, mode i (i > 34) with a TU shape A×B will be mapped to the same category as mode j = (68 – i) with a TU shape B×A. However, for each transform pair, the order of the horizontal and vertical transform kernels is swapped. For example, a 16x4 block with mode 18 (horizontal prediction) and a 4x16 block with mode 50 (vertical prediction) are mapped to the same category. However, the vertical and horizontal transform kernels are swapped. For wide-angle modes, the nearest regular angular mode is used to determine the transform set. For example, mode 2 is used for all modes between -2 and -14. Similarly, mode 66 is used for modes 67 through 80.

[0090] b. MTS used for inter-frame coding

[0091] For the MTS of inter-frame coded CUs, four candidates are used: {(DST7, DST7), (DST7, DCT8), (DCT8, DST7), (DCT8, DCT8)}. For larger resolution sequences (width > 1080), the maximum CU size used in the inter-frame MTS is set to 32 (i.e., the inter-frame MTS is used for CUs with width <= 32 and height <= 32), and for the remaining sequences (smaller resolutions), it is set to 16. For 4-pt, 8-pt, and 16-pt transforms, the current AMT transform cores, namely DST-7 and DCT-8, are replaced by separable KLTs.

[0092] c. Low Frequency Non-Separable Transform (LFNST)

[0093] Low-frequency non-separable transform (LFNST) is a secondary transform applied between the forward master transform and quantization (at the encoder) and between dequantization and inverse master transform (at the decoder).

[0094] LFNST is a direct matrix multiplication that transforms coefficients from the DCT-II transform. The primary goal of LFNST is to further compress redundancy among low-frequency master transform coefficients, which are transform coefficients derived from intra-frame predictions in the normal direction. In some embodiments, LFNST is applied only to intra-frame blocks of both luma and chroma components.

[0095] For some embodiments, the LFNST design is extended as follows. The number of LFNST sets (S) and candidates (C) is expanded to S=35 and C=3, and the LFNST set (lfnstTrSetIdx) for a given intra-mode (predModeIntra) is derived according to the following formula:

[0096] • For predModeIntra < 2, lfnstTrSetIdx equals 2

[0097] ·lfnstTrSetIdx = predModeIntra, where predModeIntra is in [0,34].

[0098] ·lfnstTrSetIdx = 68 – predModeIntra, for predModeIntra in [35,66]

[0099] Three different cores, LFNST4, LFNST8, and LFNST16, are defined to indicate the LFNST core set, respectively applied to 4xN / Nx4 (N 4) 8xN / Nx8 (N 8) and MxN (M, N) 16). The kernel dimension is specified as follows:

[0100] (LFSNT4, LFNST8*, LFNST16*) = (16x16, 32x64, 32x96)

[0101] Forward LFNST is applied to the upper left low-frequency region, known as the region of interest (ROI). When LFNST is applied, the principal transform coefficients in the region outside the ROI are cleared to zero. Figure 7AThe region of interest (ROI) of block 710 in LFNST16 was depicted. The ROI consists of six 4x4 sub-blocks that are contiguous in the scan order. Since the number of input samples is 96, the transformation matrix of the forward LFNST16 can be Rx96. In this contribution, R is chosen to be 32, and the forward LFNST16 generates 32 coefficients (two 4x4 sub-blocks) accordingly, which are placed in the coefficient scan order. Figure 7B The region of interest (ROI) of block 720 LFNST8 was depicted. The forward LFNST8 matrix can be Rx64, with R chosen to be 32. The generated coefficients are placed in the same way as those of LFNST16. Figure 8 This shows the mapping from intra-frame prediction modes to LFNST set indices.

[0102] d. Non-separable Primary Transform (NSPT) for intra-frame coding

[0103] The separable DCT-II plus LFNST transform combination can be replaced by NSPT, suitable for block shapes of 4x4, 4x8, 8x4 and 8x8, 4x16, 16x4, 8x16 and 16x8. The affected block sizes are summarized in Table 1.

[0104]

[0105] Table 1

[0106] All NSPTs comprise 35 groups and 3 candidates (similar to the current LFNST). The kernel of an NSPT has the following shape:

[0107] ·NSPT4x4: 16x16

[0108] ·NSPT4x8 / NSPT8x4: 32x20

[0109] ·NSPT8x8: 64x32

[0110] ·NSPT4x16 / NSPT16x4: 64x24

[0111] ·NSPT8x16 / NSPT16x8: 128x40

[0112] In some embodiments, NSPT4x8 / NSPT8x4, NSPT8x8, NSPT4x16 / NSPT16x4, and NSPT8x16 / NSPT16x8 are used to set 12, 32, 40, and 88 coefficients to zero, respectively.

[0113] IV. Cross-component prediction

[0114] a. Cross Component Linear Model (CCLM)

[0115] To reduce cross-component redundancy, the VVC uses a cross-component linear model (CCLM) prediction mode, where chroma samples are predicted based on reconstructed luminance samples from the same CU using a linear model, as shown below:

[0116]

[0117] in Represents the predicted chromaticity samples in CU. This indicates downsampling of the reconstructed brightness samples within the same CU. CCLM parameters ( and It is derived from at most four adjacent chroma samples and their corresponding downsampled luminance samples. Assuming the current chroma block size is W×H, then W' and H' are set to...

[0118] • When W' = W, H' = H is applied in LM_LA mode

[0119] ·W' = W + H when applying LM_A mode

[0120] ·H' = W + H when applying LM_L mode

[0121] In this disclosure, the terms {LM_LA, LM_L, LM_A} and {CCLM_LT, CCLM_L, CCLM_T} are used interchangeably.

[0122] b. Multiple model CCLM (MMLM)

[0123] The Multi-Model CCLM mode (MMLM) uses two models to predict chromaticity samples for the entire CU from luminance samples. Similar to CCLM, three Multi-Model CCLM modes (MMLM_LA, MMLM_A, and MMLM_L) are used to indicate whether top and left neighbor samples, top neighbor samples only, or left neighbor samples only were used in the model parameter derivation. In this disclosure, the terms {MMLM_LA, MMLM_A} and {MMLM_LT, MMLM_T} are used interchangeably.

[0124] In MMLM, the neighboring luminance samples and neighboring chrominance samples of the current block are classified into two groups, each used as a training set to derive a linear model (i.e., derive specific α and β for a specific group). Furthermore, samples of the current luminance block are also classified according to the same rules used to classify neighboring luminance samples.

[0125] The threshold is calculated as the average value of the neighboring reconstructed brightness samples. For neighboring samples located in [x,y], if Rec′... L If [x,y] <= threshold, then classify as group 1; while for neighboring samples located in [x,y], if Rec′ L If [x,y] > the threshold, then it is classified as group 2. Therefore, the multi-model CCLM prediction for the chromaticity samples is:

[0126] If Rec′ L [x,y] ≤ threshold

[0127] If Rec′ L [x,y] > threshold

[0128] c. Convolutional Cross-Component Model (CCCM)

[0129] In some embodiments, a convolutional cross-component model (CCCM) is applied to improve cross-component prediction performance. In some embodiments, the convolutional model has a 7-tap filter with a 5-tap plus shape space component, a nonlinear term, and a bias term.

[0130] The output of the filter is calculated as the filter coefficients c. iThe input values ​​are convolved and cropped to the range of valid chroma samples. Filter coefficients are computed by minimizing the MSE between predicted and reconstructed chroma samples in the reference region. MSE minimization is performed by computing the autocorrelation matrix of the luminance input and the cross-correlation vector between the luminance input and chroma output. The autocorrelation matrix is ​​decomposed using LDL, and the final filter coefficients are computed by back-substitution. This process roughly follows the computation of ALF filter coefficients in ECM; however, LDL decomposition is chosen instead of Cholesky decomposition to avoid the use of square root operations.

[0131] The CCCM mode can use a 3x2 filter to process unsampled luminance samples. This filter has a 6-tap spatial term, four nonlinear terms, and one bias term, as shown below:

[0132]

[0133] The 6-tap spatial term corresponds to the 6 adjacent luminance samples (i.e., L0, L1, …, L5) surrounding a chrominance sample (i.e., C) to be predicted, and the four nonlinear terms are derived from samples L0, L1, L2 and L3.

[0134] d. Gradient Linear Model (GLM)

[0135] Compared to CCLM, GLM uses the gradient of luminance samples to derive a linear model, rather than downsampled luminance values. Specifically, when applying GLM, the input to the CCLM process is the downsampled luminance samples... The gradient of the brightness sample The rest of CCLM (e.g., parameter derivation, linear transformation of predicted samples) remains unchanged.

[0136]

[0137] For signaling, when CCLM mode is enabled for the current coding unit (CU), flags are issued separately for the Cb and Cr components to indicate whether GLM is enabled for each component; if GLM is enabled for a component, a further syntax element is issued to select one of 16 gradient filters for gradient calculation. GLM can be combined with an existing CCLM by issuing an additional flag in the bitstream. When this combination is applied, the filter coefficients used to derive the linear model input luminance samples are calculated as a combination of the selected gradient filter of GLM and the downsampling filter of CCLM.

[0138] e. Cross-Component Residual Model (CCRM)

[0139] Cross-component residual model (CCRM) can be used to predict chrominance samples from reconstructed luminance samples when using inter-frame prediction or intra-blockcopy (IBC) for blocks. Figure 9 This illustrates the use of a cross-component residual model (CCRM) to reconstruct component samples. The figure shows the decoder side of the method. The cross-component filter is derived using the predicted signals for both luma and chroma. The derived filter is applied to the reconstructed luma signal to generate the final chroma prediction.

[0140] In this invention, the term "LM" can be considered as a CCLM / MMLM mode or any other extension / variation of CCLM (such as the CCLM extension / variation proposed in this invention). One variant is MMLM, which uses a threshold to determine different models for different samples in the current chroma component. Another variant, for Cb (or Cr), derives model parameters from multiple co-localized luma blocks. More possible variants are shown below. CCLM variants here mean that when a block indicates the use of a cross-component mode (e.g., CCLM_LT, MMLM_LT, CCLM_L, CCLM_T, MMLM_L, MMLM_T and / or an intra-prediction mode, not the traditional DC, planar, and angular modes), several optional modes can be selected. An example of a convolutional cross-component mode (CCCM) as an optional mode is shown below. When this optional mode is applied to the current block, cross-component information with the model, including nonlinear terms, is used to generate chroma predictions. Optional modes can follow the template selection of CCLM, so the CCCM family includes CCCM_LT, CCCM_L, and / or CCCM_T. The method proposed in this invention (for CCLM) can be used for any other cross-component mode.

[0141] V. Intra-block copying

[0142] Intra-Block Copy (IBC) is implemented as a block-level coding mode, performing block matching (BM) on the encoder to find the optimal block vector (or motion vector) for each coding unit (CU). Here, the block vector (BV) indicates the displacement from the current block to a reference block that has already been reconstructed in the current image. The luma block vector of an IBC-coded CU is integer precision. The chroma block vector is also rounded to integer precision. When combined with AMVR, the IBC mode can switch between 1-pixel and 4-pixel motion vector precision. IBC-coded CUs are considered a third prediction mode in addition to intra-frame or inter-frame prediction modes. The IBC mode is suitable for CUs with a width and height of 64 luma samples or less.

[0143] VI. Exporting and using cross-component models

[0144] a. Cross-component prediction of inter-coded blocks

[0145] Some embodiments of the present invention provide a method for improving inter-frame block prediction accuracy using cross-component information. To improve the chroma component prediction accuracy of inter-frame blocks, luminance information from the corresponding luminance component of the current chroma block, and / or chroma information from the current chroma block, and / or chroma information from previously encoded chroma blocks can be used.

[0146] In some embodiments, for coding units comprising luminance (Y) and chrominance (Cb and / or Cr) components (under single-tree splitting), the prediction of Cb and / or Cr is improved by applying a cross-component model to information from Y (current reconstruction or prediction). In some embodiments, for coding units comprising luminance (Y) and chrominance (Cb and / or Cr) components (under single-tree splitting) or for coding units comprising chrominance (Cb and / or Cr) components (under chrominance dual-tree splitting), the prediction of Cr is improved by applying a cross-component model to information from Cb (current reconstruction or prediction).

[0147] b. CCLM of inter-frame blocks

[0148] The CCLM of inter-frame blocks can also be called inter-frame CCLM, and "CCLM" can be extended to any LM mode, or any cross-component mode, or replaced by any LM mode. When using a convolutional cross-component model derived using a regression-based method, the CCLM of inter-frame blocks can also be called inter-frame CCCM. In some embodiments, for the chroma component, in addition to the original inter-frame prediction (generated by motion compensation, which can be a single prediction and / or a dual prediction, multiple prediction hypotheses from multiple motion candidates that may reference one or more merge candidates and / or one or more AMVP candidates, and / or any combination of the above, or may be just a single prediction), one or more prediction hypotheses (generated by CCLM and / or any other LM mode) are used to output the current prediction.

[0149] In some embodiments, the current prediction is a weighted sum of inter-frame prediction and CCLM prediction. In some embodiments, the inter-frame prediction can be generated by any inter-frame mode mentioned in the above description / documentation. For example, the inter-frame mode can be a regular merging mode. Another example is that the inter-frame mode can be a CIIP mode. Yet another example is that the inter-frame mode can be GPM or any GPM variant (e.g., GPM intra-frame references a prediction cell that uses intra-frame prediction).

[0150] In some embodiments, inter-frame CCLM is supported only when the current block uses any one (or more) predefined inter-frame modes, or when any one (or more) enable flags of the predefined inter-frame modes are indicated as enabled. The significance of supporting inter-frame CCLM is that it allows selection of whether to apply inter-frame CCLM to the current block.

[0151] Another example is that if chroma prediction samples are generated using a CCLM mode and luma prediction comes from an inter-frame coding tool, a flag is used to indicate whether the CCLM model used for chroma prediction is inherited from a CCLM model used in a previous coding block or from a predefined CCLM mode. If the CCLM model is inherited from a CCLM model used in a previous coding block, an index is used to indicate which model in the list was inherited or modified. Otherwise, the predefined CCLM mode is used to implicitly derive the CCLM model for the current chroma prediction.

[0152] c. Cross-type references

[0153] In some embodiments, when the current inter-frame block uses any cross-component model (e.g., inherited cross-component model, self-guided cross-component model, cross-component model for chroma fusion, meaning chroma prediction is based on adding one or more cross-component prediction hypotheses to one or more existing non-cross-component prediction hypotheses), or any combination thereof, the model parameters used can be saved and / or referenced by subsequent coding blocks. For an example of an inherited CCCM, all or any subset of model parameters can be saved. In some embodiments, the saved model parameters are allowed if the subsequent coding block is intra-frame. The saved model parameters are allowed if the subsequent coding block is inter-frame or another mode type (e.g., IBC). In some embodiments, the buffer used to store the model parameters can be different if the subsequent block and the reference block containing the saved model parameters belong to different mode types (one intra-frame, one inter-frame). In some embodiments, the saved model parameters are not allowed if the mode type of the subsequent coding block is different from the current block (e.g., not an inter-frame block).

[0154] d. Generate prediction hypotheses for the chromaticity components.

[0155] In some embodiments, a prediction- or reconstruction-based model is used to generate a predictive hypothesis for the current chromaticity component. In some embodiments of the prediction-based linear model, the derived model parameters are applied to a predicted sample of the first component (Y) to obtain a predicted sample of the second or third component.

[0156]

[0157] The predicted samples of the first component are downsampled using a downsampling filter (which can be fixed to a predefined filter or selected from some candidate filters). In some embodiments based on the reconstruction linear model, the derived model parameters are applied to the reconstructed samples of the first component (Y) to obtain the predicted samples of the second or third component.

[0158]

[0159] The reconstructed samples of the first component can be downsampled using a downsampling filter (which can be fixed to a predefined filter or selected from some candidate filters). The proposed approach for prediction- or reconstruction-based convolutional models is similar to that for linear models. The main difference is that the model coefficient pattern follows CCCM (instead of CCLM), and luminance samples may or may not be downsampled first. If luminance samples are not downsampled, more taps (model coefficients) can be used to access the unsampled luminance samples.

[0160] In some embodiments, multiple hypotheses (MH) for cross-component prediction are blended, or multiple models are used to generate prediction hypotheses for the current block. Each cross-component linear model (CCLM) method is suitable for different scenarios. For some complex features, combined predictions may yield better performance. Therefore, the Multiple Hypothesis Cross-Component Linear Model (CCLM) is proposed to blend predictions from multiple CCLM methods. The CCLM methods to be blended can be from (but are not limited to) the CCLM methods mentioned above and / or the models defined in Sections IV, VI, and VII. In some embodiments, a weighted scheme is used for blending.

[0161] e. Self-guided cross-component model

[0162] For some embodiments, an example of a self-guided cross-component model is CCRM. When performing self-guided modeling, the model (filter shape / mode, parameter terms) is unified with a regular intra-frame model. For example, the CCRM model can be unified with any predefined existing intra-frame cross-component model (e.g., CCCM, GLM, MMLM using non-downsampled luma samples), and / or self-guided modeling simply means that the inputs to derive the model parameters are from the current chroma and the corresponding luma sample (e.g., motion compensation results if the current block is inter-frame).

[0163] In some embodiments, a self-guided cross-component candidate refers to one or more models used to generate cross-component predictions for the current block, as shown below. The cross-component predictions for the current block (used to generate target prediction samples) are formed by combining one or more proposed source terms and models (referring to the weighted settings of the proposals) according to the following formula:

[0164] pred(i,j) = (sourceTermSet0(i,j) + sourceTermSet1(i,j) + … +biasTermSet)

[0165] In the proposed weighted settings, (i,j) is a sample position in the current block, pred(i,j) is a target (predicted) sample in the current block, which can be obtained after our proposed mechanism, sourceTermSet0 contains one or more source terms from the luma component, sourceTermSet1 contains one or more source terms from the chroma component, and biasTermSet contains one or more bias terms.

[0166] `sourceTermSet0(i,j)` contains one or more luminance source terms, denoted as `sourceTerm00`, `sourceTerm01`, ..., and / or `sourceTerm0`. n-1 The value of n represents the number of taps in the source itemset. In some embodiments, the source items can be linear and / or nonlinear, linear only, and / or nonlinear only. In another embodiment, n is a predefined value, such as 1, 2, ... or any positive integer.

[0167] `sourceTermSet1(i,j)` contains one or more chromaticity (Cb or Cr) source terms, denoted as `sourceTerm00`, `sourceTerm01`, ..., and / or `sourceTerm0`. m-1 The value of m represents the number of taps in the source itemset. In one embodiment, the source items can be linear and / or nonlinear, linear only, and / or nonlinear only. In another embodiment, m is a predefined value, such as 1, 2, ... or any positive integer.

[0168] The biasTermSet contains one or more bias terms. A bias term is a predefined value. In one embodiment, the bias term is a midValue based on the bit depth specified in the standard. For example, the bias term is set to... In another embodiment, the bias term is the same for every sample in the current block. That is, the bias term is independent of position (i, j).

[0169] In some embodiments, for a sourceTermSet0(i,j) (sourceTerm00, sourceTerm01, ..., and / or sourceTerm0) containing n taps... n-1The pattern of n taps refers to any subset of the window region M x N around / including position (iL, jL). If the target sample is luminance, (iL, jL) is (i, j). If the target sample is chrominance (e.g., Cb or Cr), (iL, jL) is the luminance position corresponding to (i, j).

[0170] In some embodiments, for a sourceTermSet1(i,j) (sourceTerm00, sourceTerm01, ..., and / or sourceTerm0) containing m taps... m-1 The pattern of m taps refers to a pattern defined around / includes the position (i). C j C The pattern of any subset of the window region M2 x N2. If the target sample is chromaticity (Cb or Cr), (i C j C (i,j) is the target sample. If the target sample is brightness, (i) is the target sample. C j C ) is the chromaticity position corresponding to (i, j).

[0171] In some embodiments, a weighted setting is used to estimate the relationship between "predicted and / or reconstructed samples on the reference region of the current (chroma) block" and "predicted and / or reconstructed samples on the reference region of the corresponding luma block" using a predefined regression method (minimizing distortion). Weights (referring to model parameters) are generated according to the regression method, and then the weights are applied to the source terms to obtain the target (predicted) samples in the current block. In one embodiment, the predefined regression method may be a linear minimum mean square error (LMMSE) method as CCLM, or any method consistent with the regression method used for CCLM. In some embodiments, the predefined regression method may be an LDL decomposition method as CCCM, or any method consistent with the regression method used for CCCM. In another embodiment, the predefined regression method may be Gaussian elimination.

[0172] In some embodiments, the spatial neighborhood of the current block may include the aforementioned reference region, left reference region, top-left reference region, and / or any subset thereof. The reference region of the corresponding luminance block is the spatial neighborhood of the corresponding luminance block.

[0173] The size of the above reference area is A w x A H The size of the reference region on the left is L. w x L H The size of the upper left reference region is AL.W x AL H Let W represent the width of the current block and H represent the height of the current block. A w = Current block width (W), k*W (k can be a positive value), W + current block height (i.e. W+H), any predefined value, or any adaptive value that depends on the current block's block position, block width, block height, and / or block area.

[0174] ·A H or AL H = H, any predefined value (1, 2, 4, …), or any adaptive value that depends on the current block's position, width, height, and / or area.

[0175] ·L W or AL W = W, any predefined value (1, 2, 4, …), or any adaptive value that depends on the current block's position, width, height, and / or area.

[0176] ·L H = H, k*H, H + W, any predefined value, or any adaptive value that depends on the current block's position, width, height, and / or area.

[0177] In some embodiments, the reference region of the current block is the vector co-addressable region of the current block, and the reference region of the corresponding luma block is the vector co-addressable region of the corresponding luma block. For inter-frame coding units containing luma and chroma blocks, the vector co-addressable region of the current block refers to the motion compensation result obtained using the motion information (motion vector and reference image) of the current block, and the vector co-addressable region of the corresponding luma block refers to the motion compensation result obtained using the motion information (motion vector and reference image) of the corresponding luma block. For intra-block copying (IBC) or intra-TMP, the vector co-addressable region of the current block refers to the motion compensation result obtained using the motion information (block vector and current image) of the current block, and the vector co-addressable region of the corresponding luma block refers to the motion compensation result obtained using the motion information (block vector and current image) of the corresponding luma block.

[0178] In some embodiments, the two types of reference regions for the current block described above can be used together. For example, typically, samples in the vector co-location region of the current block are used as input samples when deriving model parameters; however, for smaller blocks, samples in spatially adjacent reference regions are used as additional input samples when deriving model parameters.

[0179] f. Representative prediction patterns used for transformation selection

[0180] In some embodiments, the operations of the encoding or decoding phase are determined based on the prediction mode used by the current block. For example, in some embodiments, the type of transform applied to the prediction residual of the current block can be selected based on the intra-prediction mode (e.g., DC, planar, directional mode) associated with the current block. In some embodiments, for blocks coded not by conventional intra-component prediction but by cross-component prediction, representative intra-prediction modes can be identified or derived to determine the operations of the encoding or decoding phase, such as selecting primary and / or secondary transform types.

[0181] In another embodiment, when cross-component prediction is used for the current chroma block, a representative prediction mode is derived. The motivation for deriving a representative prediction mode is that the intra-prediction mode (e.g., DC, planar, directional mode) of the current block will be used in the encoding / decoding phases after / before / in parallel with the prediction process. However, the cross-component model is not one of the aforementioned intra-prediction modes. Therefore, a representative prediction mode is required.

[0182] In some embodiments, during the transform / inverse transform phase, the primary transform (e.g., a separable primary transform such as MTS, a non-separable primary transform such as NSPT) and / or secondary transform (e.g., LFNST) use representative prediction modes to select the transform set and / or transpose flag. The secondary transform is not limited to LFNST and can be any predefined separable / non-separable and / or low-frequency / non-low-frequency transform performed in the encoder (or decoder) after (or before) the primary transform, and applied to the coefficients of the primary transform (or received dequantization) in the encoder (or decoder).

[0183] In some embodiments, for a subsequent coded block, its MPM list can be derived by referencing the intra-prediction mode information of its neighboring blocks. The neighboring blocks referenced by the subsequent block may be coded in a cross-component mode (therefore, there is no non-cross-component intra-prediction mode information, particularly no directional intra-prediction mode information). For that subsequent block, the MPM list for the chroma components of that subsequent coded block is derived using the intra-prediction mode information of one or more neighboring coded blocks. If the neighboring coded blocks use a cross-component mode, the representative prediction mode (of the neighboring coded blocks) is used to derive the MPM list.

[0184] In some embodiments, the DIMD procedure (as described in Section Ib above) is used to derive a representative prediction mode as the intra-prediction mode for the current block. For example, DIMD is performed on a predicted sample or a subsample of a predicted sample within the current chroma block (e.g., cross-component prediction from Y to Cr / Cb or Cr to Cb). In another example, DIMD is performed on reconstructed samples or subsamples of reconstructed samples on adjacent templates of the current chroma block. When using DIMD, the horizontal and vertical gradients are computed for each sample to construct a HoG, and then the intra-prediction mode with the largest histogram magnitude is used to determine the representative prediction mode for selecting the transform set and transpose flag.

[0185] VII. Inheritance of cross-component models

[0186] In some embodiments, previously derived cross-component models and / or corresponding information can be inherited and used for the current chroma block by: (1) constructing a candidate list (modelList) for the current block, which includes cross-component models; (2) selecting one or more model information from the list; and (3) using the selected model information (similar to intra-frame chroma cross-component modes) to generate prediction hypotheses for one or more current chroma components (Cb or Cr) by applying and / or modifying the selected model information to reconstructed or predicted samples of the corresponding luma components.

[0187] When the selected model information involves a traditional cross-component linear model, this method is called the inter-cross-component linear model (inter CCLM) mode. When the selected model information involves a convolutional cross-component model derived through a regression-based method (e.g., CCCM), this method is called the inter-cross-component convolution model (inter CCCM) mode. Furthermore, in some embodiments, self-derived (or “re-derived”) cross-component modes are added to the candidate list. In some embodiments, self-derived cross-component modes are not added to the list, and a selection is made using the proposed inherited mode and / or the proposed self-derived mode. In some embodiments, the selection of using the proposed inherited mode and / or the proposed self-derived mode is determined according to explicit rules, implicit rules, or both. This candidate list selection method will be described in more detail in Section VIII below.

[0188] a. Selection of inheritance mode and / or re-export mode

[0189] In some embodiments, a flag can be used to indicate / select whether to use a re-exported model (or a "self-exported model" as described in Section VI.e). If the flag is 0, the cross-component model used to encode neighbor merging candidates is inherited. If the flag is 1, the re-exported approach is used. In some embodiments, implicit rules (without additional flags) are used to determine whether to use a re-exported model. In some embodiments, a re-exported model can be used if no model can be inherited during candidate list construction, or if spatially adjacent and / or non-adjacent candidates and / or historical candidates and / or temporal candidates and / or all or any subset of those mentioned in this document (e.g., candidates preceding the default candidate) are unavailable.

[0190] In some embodiments, when using an inheritance approach, the candidate with the lowest cost (e.g., the first candidate in the candidate list) is implicitly selected to generate cross-component predictions. In some embodiments, one or more candidates are selected from the candidate list by signaling an index. More details on selecting candidates from the candidate list can be found in Section VIII.

[0191] b. Model parameter storage for inheritance

[0192] In some embodiments, when the current inter-frame block uses model parameters of a self-derived cross-component mode, the model parameters used can be saved and / or referenced by subsequent or subsequent coded blocks. For example, for the case where the self-derived cross-component mode is CCRM, all or any subset of the model parameters can be saved. In some embodiments, the saved model parameters are allowed if the subsequent coded block is intra-coded. The saved model parameters are allowed if the subsequent coded block is inter-coded or another mode type (e.g., IBC). In some embodiments, the buffer used to store the model parameters may be different if the subsequent block and the reference block containing the saved model parameters belong to different mode types (one intra-, one inter-). In some embodiments, the saved model parameters are not allowed if the mode type of the subsequent coded block is different from the mode type of the reference block (e.g., not an inter-frame block).

[0193] In some embodiments, when the current inter-frame block uses an inherited cross-component mode, the model parameters used can be saved and / or referenced by subsequent or following coded blocks. For example, for an inherited CCCM, all or any subset of model parameters can be saved. In some embodiments, the saved model parameters are allowed if the subsequent coded block is intra-coded. The saved model parameters are allowed if the subsequent coded block is inter-coded or any mode type (e.g., IBC). In some embodiments, the buffer used to store the model parameters may be different if the subsequent block and the reference block containing the saved model parameters belong to different mode types (e.g., one intra-, one inter-coded). In some embodiments, the saved model parameters are not allowed if the mode type of the subsequent coded block is different from the mode type of the reference block (e.g., not an inter-frame block).

[0194] c. Inherit CCM information

[0195] In some embodiments, inherited cross-component model (CCM) information can be stored along with inherited model parameters. CCM information can be inherited along with inherited model parameters. Prediction for the current block can be generated based on the inherited CCM information and inherited model parameters. CCM information may include, but is not limited to, prediction patterns (e.g., CCLM, MMLM, CCCM, 2-parameter GLM, 3-parameter GLM (GLM model with a brightness term), model index indicating the model shape used in the convolutional model, multi-model classification threshold, information indicating the use of non-downsampled samples in the convolutional model, downsampling filter flag (whether downsampling is performed), downsampling filter index when multiple downsampling filters are used, neighboring rows for deriving the model, template type for deriving the model, post-filter flag, and model parameters).

[0196] In some embodiments, a hybrid cross-component prediction model (CCCM) with various terms (e.g., spatial, gradient, positional, nonlinear, and bias terms) can be inherited. In addition to storing model parameters, prediction patterns can be stored in the cross-component prediction information to indicate that the inherited model is a hybrid cross-component prediction model composed of various terms. If multiple types of hybrid cross-component prediction models exist, model indexes can also be stored in the cross-component prediction information to indicate which type of hybrid cross-component prediction model was inherited. For example, the gradient and location-based CCCM (GL-CCCM) proposed in JVET-AB0119 is a hybrid cross-component prediction model that includes a spatial term for a center position, two gradient terms for horizontal and vertical directions, two X and Y positional terms for relative horizontal and relative vertical positions, a nonlinear term, and a bias term. Prediction patterns can be stored in the cross-component prediction information to indicate that the inherited model is a GL-CCCM model. In some embodiments, the inherited model parameters can be optimized and then used in the current block and / or saved for reference in subsequent encoded blocks.

[0197] d. Inherited spatial proximity model parameters

[0198] In some embodiments, inherited model parameters may come from a neighboring block. Models from blocks at predefined locations are added to the candidate list in a predefined order. The predefined order can be any possible order of spatially neighboring blocks. In some embodiments, the predefined locations and predefined order can be the same as the spatial candidates for the inter-frame merge mode. In some embodiments, the predefined locations can be... Figure 2 The diagram depicts the predefined locations of spatially adjacent blocks. The predefined order can be B0, A0, B1, A1, and B2.

[0199] In some embodiments, assuming the current block's position, width, and height are (x, y), W, and H, respectively, the predefined position can include the position directly above the current block, such as (x + W >> 1, y - 1) or (x + (W + 1) >> 1, y - 1), if W is greater than or equal to a threshold TH. The predefined position can also include the position to the left of the current block, such as (x - 1, y + H >> 1) or (x - 1, y + (H + 1) >> 1), if H is greater than or equal to the threshold TH. TH can be 2, 4, 8, 16, 32, or 64. The predefined position includes the position directly above (W >> 1) or ((W >> 1) – 1) if W is greater than or equal to TH, and the position to the left (H >> 1) or ((H >> 1) – 1) if H is greater than or equal to TH.

[0200] In some embodiments, a maximum number of models inherited from spatial neighbors can be added to the candidate list, and the maximum number is less than the number of predefined locations.

[0201] e. Inheritance temporal proximity model parameters

[0202] In some embodiments, if the current slice / image is a non-intra-frame slice / image, the inherited model parameters may come from blocks in previously encoded slices / images (i.e., model parameters from temporal neighbors).

[0203] Figure 10 This conceptually illustrates the parameters of the inheritance temporal proximity model. As shown in the figure, the current position is (x, y), and the block size is... The inherited model parameters can be derived from previously encoded slices / images at positions... , , , , , or The blocks, in which , In some embodiments, Δx and Δy are set to 0. In some embodiments, if the prediction mode of the current block is inter-frame, Δx and Δy are set to the horizontal and vertical motion vectors of the current block, as in the first case of Section VIII.b, “Temporal Model Information from Corresponding Blocks.” In some embodiments, if the current block is an inter-frame unidirectional prediction, Δx and Δy are set to the horizontal and vertical motion vectors in reference image list 0 or 1, depending on the list used by the current block. In some embodiments, if the current block is an inter-frame bidirectional prediction, Δx and Δy are set to the horizontal and vertical motion vectors in reference image list 0. In some embodiments, if the current block is an inter-frame bidirectional prediction, Δx and Δy are set to the horizontal and vertical motion vectors in reference image list 1.

[0204] In some embodiments, if the current slice / image is a non-intra-frame slice / image, the inherited model parameters can be derived from blocks in previously encoded slices / images. In some embodiments, the current block position is (x, y), and the block size is... The inherited model parameters can come from blocks at certain predefined locations in previously encoded slices / images.

[0205] In some embodiments, the predefined location may be or ,in Two value sets and Defined as:

[0206] like

[0207] like

[0208] All in and The values ​​in the array are all positive. For example, It can be .For example, It can be ,in and They are two fixed positive numbers. For example, .For example, .For example, .For example, and .

[0209] In some embodiments, a predefined location Within the corresponding region of the current coded block, that is, and Predefined positions can be .

[0210] In some embodiments, a predefined location Outside the corresponding region of the current coding block, that is, and Predefined positions can be .

[0211] In some embodiments, from closer The model of the position is first added to the final merge candidate list. The previously encoded image from which the inherited parameter model is obtained is subsequently referred to as the corresponding image. In some embodiments, the previously encoded image from which the inherited parameter model comes, i.e., the corresponding image, is one of the images in the reference list. In some embodiments, the corresponding image is marked in the image / title. The reference list and reference index are marked in the image / title. For example, the corresponding image is selected as L0[0]. In another example, the corresponding image is selected as L1[0].

[0212] f. Inheriting the non-adjacent spatial proximity model

[0213] In some embodiments, the inherited model parameters may come from non-adjacent spatial neighbor blocks. Models from predefined locations are added to the candidate list in a predefined order. In some embodiments, the predefined locations and predefined order are the same as the non-adjacent spatial neighbor candidates in the inter-frame merge mode.

[0214] Figure 11A-B shows the current block 1100 and its non-adjacent spatial neighboring blocks, from which model parameters can be inherited. The figure also shows predefined locations and their predefined order. Figure 11A It displays non-adjacent spatial locations arranged in a predefined order according to the first mode (mode 1). Figure 11B This displays non-adjacent spatial locations arranged according to a predefined order of Mode 2. The positions of the numbered squares are predefined positions. The number within each square indicates the predefined order. Positions in Mode 1 are added to the list before positions in Mode 2. The distance between each predefined position is proportional to the width and height of the current block.

[0215] In some embodiments, the maximum number of inherited models from non-adjacent spatial neighbors that can be added to the candidate list is less than the number of predefined locations.

[0216] g. Inheriting model parameters from history tables

[0217] In some embodiments, the inherited model parameters may come from a cross-component model history table. The history table stores CCM information for valid previously encoded blocks. A valid previously encoded block refers to any block containing valid CCM information. Cross-component models in the history table can be added to the candidate list in a predefined order. In some embodiments, the order in which historical candidates are added can be from the beginning to the end of the table. In some embodiments, the order in which historical candidates are added can be from the end to the beginning of the table.

[0218] In some embodiments, a cross-component model history table can be maintained to store previous cross-component models (i.e., CCM information), and the cross-component model history table can be reset at the start of the current image, the current slice, the current tile, every M CTU rows, or every N CTU, where N and M can be any value greater than 0. In another embodiment, the cross-component model history table can be reset at the end of the current image, the current slice, the current tile, the current CTU row, or the current CTU.

[0219] In some embodiments, multiple history tables are used to store different types of cross-component models. For example, the first history table is used to store a single model, and the second history table is used to store multiple models. Another example is that the first history table is used to store gradient models, and the second history table is used to store non-gradient models. Yet another example is that the first history table is used to store simple linear models (e.g., y = ax + b), and the second history table is used to store complex models (e.g., CCCM).

[0220] In some embodiments, when adding historical candidates to a candidate list from multiple historical tables, the addition order can be from the beginning to the end of a table, and then the next historical table can be added in the same or reverse order.

[0221] VIII. Candidate List with Cross-Component Models

[0222] a. Cross-Component Model Estimation Region (CCMER)

[0223] In some embodiments, blocks within the same predefined region (e.g., CCMER) may share the same candidate list (i.e., modelList) or partially identical candidate lists (e.g., adjacent / non-adjacent and / or historical candidates are the same). In some embodiments, cross-component model estimation region (CCMER) allows blocks within the same cross-component model estimation region (CCMER) (e.g., CUs, TUs, and / or any predefined region) to independently derive candidate lists. It also allows for the independent deriving (i.e., merging) of candidate lists for different blocks (e.g., CUs, TUs, and / or any predefined region) within the same cross-component model estimation region (CCMER). That is, if two blocks are in the same CCMER, the model of one block will not be inserted into the merged list of the other block. Candidate blocks in the same CCMER as the current block (e.g., CUs, TUs, and / or any predefined region) are not included in the candidate list generation for the current block.

[0224] Figure 12 This conceptually illustrates candidate lists for blocks within the same Cross-Component Merge Estimation Region (CCMER). The figure shows CCMER 1205 in image 1200. Block 1210 is located within CCMER 1205 and uses candidate list 1215. Block 1220 is also located within CCMER 1205 and uses candidate list 1225. The candidate models or information for candidate lists 1215 and 1225 originate from candidate locations outside CCMER 1205. Because the blocks are located within CCMER 1205, information for blocks 1210 and 1220 is not included as candidates in candidate lists 1215 and 1225.

[0225] In some embodiments, the update process of the historical model candidate list (a FIFO table for historical model information) is only updated when the block size (block width and / or block height) is greater than a threshold, and this threshold can be determined based on the size of the CCMER. For example, the threshold (i.e., the CCMER size) is fixed as a positive integer, such as 4, 8, 16, 32, 64, 128, 256, or any positive integer value, the threshold is specified in a standard, and / or the threshold depends on the signaling at the block, slice, image, SPS, or PPS level. The following illustrates some implementations of these embodiments:

[0226] The historical model candidate list update process (the FIFO table for historical model information) is only updated if (blockPostitionX + blockWidth) >> Log2ParCCMrgLevel is greater than blockPostitionX >> Log2ParCCMrgLevel, and (blockPostitionY + blockHeight) >> Log2ParCCMrgLevel is greater than (blockPostitionY >> Log2ParCCMrgLevel). If the block references a CU, (blockPostitionX, blockPostitionY) is the top-left brightness sample position of the current CU in the image, and (blockWidth, blockHeight) is the size of the CU. The CCMER size is selected at the encoder end and signaled as log2_parallel_ccmerge_level_minus2 in the sequence parameter set, where Log2ParCCMrgLevel = sps_log2_parallel_ccmerge_level_minus2 + 2. Another example is that the CCMER size is unified and / or derived with the size of the merged estimated region (MER) used for regular merging.

[0227] b. A candidate list containing cross-component models

[0228] In some embodiments, for each candidate in the list, it refers to a candidate model information. The definition of model information can be found in Section VII.c, “Inheriting CCM Information,” above. When constructing a merged candidate list (modelList), it contains one or more of the following candidate model information:

[0229] • Spatial model information from spatial neighbor blocks (corresponding to the "spatial MVP from spatial neighbor CUs" between frames)

[0230] • Timing model information from co-location blocks (corresponding to the "Timing MVP from co-location CU" between frames)

[0231] • Historical model information from the FIFO table (corresponding to "historical MVP from the FIFO table" between frames)

[0232] • Pairwise average model information (corresponding to the "pairwise average MVP" between frames)

[0233] • Default model information (corresponding to "zero MV" between frames)

[0234] In some embodiments, when the candidate type is “spatial model information from spatial neighbor blocks”, a valid spatial neighbor block can come from one of spatially adjacent and non-adjacent neighbors (or any subset of the current block’s neighbor search region) and satisfy a predefined condition. For an example of a non-adjacent neighbor, the predefined condition (e.g., a validity / availability check) refers to the non-adjacent neighbor being in the available region of the non-adjacent spatial candidate. For example, the predefined condition is that the neighbor is encoded via cross-component mode encoding (e.g., CCLM, MMLM, CCCM, GLM, modes that inherit mode information from a similar merged candidate list, MH CCLM, and / or any cross-component mode that belongs to a cross-component branch (containing many cross-component modes) but not to a conventional intra-prediction mode) or via a mode combined with a cross-component mode (e.g., chroma fusion (or LM-assisted angle / plane mode), inter-frame CCLM, inter-frame CCCM, and / or any conventional mode whose syntax does not belong to a cross-component branch but uses cross-component information to generate predictions).

[0235] In some embodiments, when checking the validity of neighboring coded blocks, a second round of validity checks is further used when the aforementioned validity checks (e.g., the neighboring block is not a cross-component mode or the neighboring block does not use / combine cross-component modes) are performed. The motion vectors and / or block vectors of the neighboring blocks can be used to find the cross-component model. Variations on how to find the model using motion vectors and / or block vectors can be found in the description of "Temporal Model Information from Co-located Blocks". If found, the second round of validity checks of the neighboring block is satisfied, and the found model can be inserted into the list; otherwise, the neighboring block is not suitable for insertion. When scanning spatial neighboring blocks, if a candidate is valid, the candidate is added to the list. The following shows some scanning orders for adding spatial model information from spatial neighboring blocks to the list. More details can be found in Section VII.d "Inheriting Spatial Neighbor Model Parameters" and Section VII.f "Inheriting Non-Adjacent Spatial Neighbor Models".

[0236] In some embodiments, for the temporal model information from co-localized blocks, in a first case, the co-localized block is derived from a block located in a reference image or a predefined co-localized image using the current block position and / or the current block motion as an inter-frame pattern; and / or in a second case, the co-localized block is derived from a block located in a reference image or a predefined co-localized image using the current block position and / or the motion of neighboring blocks as an inter-frame pattern. In the first case (current block motion), for example, when the current block is encoded using an inter-frame prediction pattern, the co-localized block is referenced by the motion information of the current block (including motion vectors and a reference image indicated by a reference index).

[0237] If the current block is in a sub-block motion mode (e.g., affine mode), each sub-block in the current block has its own co-localization temporal model information, and / or the co-localization temporal model information of all or any subset referenced by different sub-block motions is added to the list. Another example is that when the reference image indicated by the reference index differs from a predefined co-localization image—which could be a co-localization image used for temporal motion vector prediction in inter-frame modes or any co-localization image specified in the standard—to maintain motion or cross-component model information stored and available for the current block, temporal information from the reference image is prohibited. Another example is that when the reference image indicated by the reference index differs from a predefined co-localization image—which could be a co-localization image used for temporal motion vector prediction in inter-frame modes or any co-localization image specified in the standard—to maintain motion or cross-component model information stored and available for the current block, the motion vector is scaled to reference the predefined co-localization image, and the scaled motion vector is used to find the co-localization block in the co-localization image to obtain the cross-component model in the co-localization block. The scaling process is shown in Section VII.e, “Inheriting Temporal Neighbor Model Parameters.”

[0238] For the second case described above (neighboring block motion), for example, the temporal model information can come from the co-located blocks referenced by the motion information of the current block's neighboring blocks. Similar to the first case described above (current block motion), the prohibition method or scaling method can be used in the second case described above (neighboring block motion). If the proposed method is applied to IBC blocks or any pattern that uses block vectors (in the first case, the current block is IBC; in the second case, neighboring blocks are IBCs), the block vector information is used as the motion vector, where the block vector information is determined by signaling and / or template matching within a predefined search range, such as intraTMP and / or any implicit or explicit predefined rules. In some embodiments, the prohibition method is applied when the temporal information does not come from the available area of ​​the temporal candidate. More details can be found in Section VII.e, “Inheriting Temporal Neighborhood Model Parameters”.

[0239] In some embodiments, for history-based model information, a history-based table (FIFO table) is created to store model information from previous encoded blocks. This table can be reset at the beginning and / or end of a CTU, slice, image, block, and / or sequence. One or more history-based candidates can be added to the candidate list in a head-to-tail or tail-to-head order. Further variations can be found in Section VII.g, “Inheriting Model Parameters from a History Table.”

[0240] In some embodiments, for pairwise averaging of model information, the candidate model information is derived based on model information from multiple previous candidates in a list. For example, a video codec may average and / or modify the model parameters of multiple candidates as the model parameters to be applied. In another example, a video codec may combine multiple predictions as the final prediction, where each prediction is generated by applying a model from a candidate list.

[0241] In some embodiments, if the list is not full after inserting all predefined candidates, default model information is added. Examples of default CCLM model information are shown below.

[0242] For example, the default alpha (or called...) , a (or scaling parameter) is {0, 1 / 8, -1 / 8, 2 / 8, -2 / 8, 3 / 8, -3 / 8, …}, beta (or called (b, or offset parameter) is based on the selected default alpha, average neighboring reconstructed luminance sample value, and average neighboring reconstructed chromaticity (Cb / Cr) sample value.

[0243] In some embodiments, when a candidate is selected from a list and its model information is used for the current block, or when model information is inherited from a previous encoded block (when a candidate is added to the list), only a subset of the model information is inherited. For example, only alpha is inherited. The beta of the current block is obtained through the inherited alpha, average neighbor reconstructed luminance sample values, and / or average neighbor reconstructed chrominance (Cb / Cr) sample values. For example, when inheriting MMLM model information, scaling parameters and / or classification thresholds are inherited. Offset parameters in each class are derived based on the inherited classification threshold and the average neighbor reconstructed luminance sample values ​​and / or average neighbor reconstructed chrominance (Cb / Cr) sample values ​​in each class. If no neighbor reconstructed samples are available in a class, the offset parameters are inherited directly from the candidate. For example, when inheriting CCCM model information, all convolution parameters, offsets, and / or classification thresholds are inherited. For example, when inheriting GLM model information, if the GLM candidate is a 3-parameter GLM mode, all gradient mode indices and model parameters are inherited; otherwise, if the GLM candidate is a 2-parameter GLM mode, offset parameters are derived using the inherited scaling parameters, average neighbor reconstructed luminance sample values, and / or average neighbor reconstructed chrominance (Cb / Cr) sample values. For example, when inheriting chrominance fusion model information, the derived MMLM parameters are inherited and used for the MMLM candidate of the current block.

[0244] In some embodiments, when constructing the candidate list, one or more self-derived cross-component candidates (described in Section VI.e) are included. In some embodiments, self-derived cross-component candidates are added only if the list contains insufficient inherited candidates. For example, a self-derived candidate precedes or is treated as the default candidate. In some embodiments, self-derived cross-component candidates are added to the candidate list at any predefined position. For example, the position is after spatially adjacent candidates. In another example, the position is after spatially non-adjacent candidates. In yet another example, the position is after all or any subset of temporal candidates. More details / variations regarding candidate list construction can be found in Sections VII and VIII.

[0245] c. The order in which candidates are inserted into the list

[0246] In some embodiments, a candidate list is constructed by adding candidates in a predefined order until a maximum number of candidates is reached. The added candidates may include, but are not limited to, all of the candidates described above. For example, the predefined order may be spatially adjacent candidates, temporally adjacent candidates, spatially non-adjacent candidates, historical candidates, and then a default candidate.

[0247] In some embodiments, if all predefined neighboring and historical candidates have been added but the maximum number of candidates has not been reached, some default candidates are added to the candidate list until the maximum number of candidates is reached.

[0248] In some embodiments, the default candidate can be a CCLM model. Scaling parameters From collection .

[0249] Where N is a positive integer. Offset parameter It can be Alternatively, it can be derived based on neighboring luminance and chrominance samples. For example, if the average values ​​of neighboring luminance and chrominance samples are lumaAvg and chromaAvg, In one sub-implementation, the order in which the default candidates are included can depend on the scaling parameter. The absolute value and sign. For example, default candidates are added to the list in the following order:

[0250]

[0251] In some embodiments, the default candidate can be an earlier candidate refined with an incremental scaling parameter. The earlier candidate is a CCLM model. If the scaling parameter of the earlier candidate is... The default candidate scaling parameter is .For example, It can be

[0252] ,

[0253] Where N is a positive integer. Offset parameter Based on The average values ​​of the luminance and chrominance samples of the current block are derived. In some embodiments, earlier candidates are the CCLM candidates that are added to the list first. In one sub-implementation, the inclusion order of the default candidates may depend on refinement. The absolute value and sign. For example, default candidates are added to the list in the following order:

[0254]

[0255] d. Enable / disable / select model information from the candidate list

[0256] In some embodiments, when the proposed inter-frame CCLM (or inter-frame CCCM) is not applied, the prediction for the current block is derived from the original inter-frame prediction. In some embodiments, the choice to apply inter-frame CCLM (or inter-frame CCCM) depends on signaling. In some embodiments, when signaling indicates that inter-frame CCLM (or inter-frame CCCM) should be applied, additional signaling is used to select one or more models from the total candidates (e.g., CCLM_LT, CCLM_L, CCLM_T, MMLM_LT, MMLM_T, MMLM_L, or any subset / extension of the above modes).

[0257] For example, if one LM mode is selected, the LM prediction is generated from the selected LM. In another example, if multiple LM modes are selected, the LM prediction is generated by a mixture of prediction assumptions from the multiple LM modes. In yet another example, additional signaling refers to indices in the bitstream, which can be truncated unary codes with and / or without context. In yet another sub-implementation, no additional signaling is required, and one or more models are selected according to implicit rules.

[0258] In some embodiments, the selected one or more models are implicitly determined, or the one or more models used for the current block are determined without signaling `modeIdx`. For example, the first candidate in the list is used. If the list is reordered by template cost, the first candidate is the one with the lowest template cost. In some embodiments, the order in the reordering may not be necessary, as each candidate in the list can be checked one by one, and the candidate with the lowest cost so far can be retained. For another example, the candidate / model used is implicitly selected from the list based on the block's encoding information using predefined rules.

[0259] In some embodiments, when signaling instructs the application of inter-frame CCLM (or inter-frame CCCM), one or more models (e.g., modelList may include candidates as described in Section VIII.b, CCLM_LT, CCLM_L, CCLM_T, MMLM_L, MMLM_T, MMLM_L, or any subset / extension of the above modes) are implicitly selected (or predefined) from the total candidates for inter-frame CCLM (or inter-frame CCCM). For example, CCLM_LT is used to generate the LM prediction for inter-frame CCLM. For example, MMLM_LT is used to generate the LM prediction for inter-frame CCLM. Similar rules can be applied to inter-frame CCCM, using convolutional models instead. For another example of inter-frame CCLM, predefined rules depend on block width, block height, or block area. Similar rules can be applied to inter-frame CCCM, using convolutional models instead. Examples of such rules include:

[0260] • Boundary matching settings (used as predefined rules) are applied only when the block width, block height, or block area is greater than a threshold.

[0261] • Boundary matching settings (used as predefined rules) are applied only when the block width, block height, or block area is less than a threshold.

[0262] • When the block width, block height, or block area is less than a threshold, the selected LM mode is inferred to be any one (or more) of the total candidate LM modes. For example, the selected LM mode may be fixed as CCLM_LT, or the selected LM mode may be fixed as MMLM_LT.

[0263] For example, predefined rules depend on boundary matching settings. (The candidate modes used in the boundary matching settings refer to each candidate LM mode used for inter-frame CCLM. The predictions of the candidate modes used in the boundary matching settings refer to the predictions generated by each candidate LM mode or to the mixed predictions of each candidate LM mode and the original inter-frame predictions.) Similar rules can be applied to inter-frame CCLM, instead using convolutional models.

[0264] In some embodiments, inter-frame CCLM (or inter-frame CCCM) is supported only when the current block size condition is met. In some embodiments, the size condition is that the block width, block height, or block area is greater than a predefined threshold. The predefined threshold can be a positive integer, such as 8, 16, 32, 64, 128, 256, ... In some embodiments, the size condition is that the block width, block height, or block area is less than a predefined threshold. The predefined threshold can be a positive integer, such as 8, 16, 32, 64, 128, 256, 512, 1024, 2048, 4096, ...

[0265] In some embodiments, the original inter-frame prediction (generated by motion compensation) is used for the luma component, and the prediction for the chroma component is generated by CCLM and / or any other LM mode. In some embodiments, the current CU is considered an inter-frame CU, an intra-frame CU, or a new type of prediction mode (neither intra-frame nor inter-frame). In some embodiments, one or more LM modes (or cross-component modes) used to generate one or more prediction assumptions for LM auxiliary angle / plane mode / inter-frame CCLM / inter-frame CCCM / MH CCLM are selected from a predefined merged candidate list (called modelList). In some embodiments, a modelIdx (model index) is signaled to select a candidate from the candidate list (modelList) and the selected candidate is used for the current block. The candidate list contains one or more candidates, where each candidate refers to model (or cross-component mode) information. If there is only one candidate in the list (the list size is only 1), then modelIdx is not signaled and / or can be inferred to be 0 or a default value. In one embodiment, modelIdx is implicitly determined, or one or more models used for the current block are determined without transmitting modelIdx. For example, use the first candidate in the list (i.e., modelIdx = 0).

[0266] In some embodiments, candidates in the list can be reordered to reduce the syntactic overhead of signaling selected candidate indices, or the syntax of signaling selected candidate indices can be bypassed by selecting one or more candidates using implicit rules. Therefore, if all or any subset of candidates contained in the candidate list are reordered, the additional signaling specifies the reordering of candidate indices in the list. For example, if the list is reordered by template cost, the first candidate is the one with the lowest template cost. Another example is implicitly selecting candidates / models to be used from the list based on the block's encoding information using predefined rules.

[0267] In some embodiments, when constructing the candidate list, one or more predefined candidates are added. Predefined candidates may include any subset / expansion and / or more of the following candidates:

[0268] CCLM family: CCLM_LT, CCLM_L, CCLM_T

[0269] MMLM family: MMLM_LT, MMLM_L, MMLM_T

[0270] CCCM family: CCCM_LT, CCCM_L, CCCM_T

[0271] The methods described above can also be applied to IBC blocks or blocks with any IBC submode (e.g., IBC merging or IBC AMVP or any IBC mode under the IBC syntax). ("Inter-frame coding" in this document can be changed to "IBC coding".) That is, for chroma components, block vector prediction can be combined with or replaced by cross-component prediction.

[0272] Any combination of the methods proposed in this invention can be applied. Any of the methods proposed above can be implemented in the encoder and / or decoder. For example, any of the proposed methods can be implemented in the inter-frame / intra-frame / prediction / IBC module of the encoder, and / or in the inter-frame / intra-frame / prediction / IBC module of the decoder. Alternatively, any of the proposed methods can be implemented as circuitry coupled to the inter-frame / intra-frame / prediction module of the encoder and / or the inter-frame / intra-frame / prediction / IBC module of the decoder to provide the information required by the inter-frame / intra-frame / prediction / IBC / quantization module.

[0273] IX. Example Video Encoder

[0274] Figure 13 An example video encoder 1300 is shown, which can perform cross-component prediction. As shown, the video encoder 1300 receives an input video signal from a video source 1305 and encodes the signal into a bitstream 1395. The video encoder 1300 has multiple components or modules for encoding the signal from the video source 1305, including at least some components selected from transform module 1310, quantization module 1311, inverse quantization module 1314, inverse transform module 1315, intra-frame image estimation module 1320, intra-frame prediction module 1325, motion compensation module 1330, motion estimation module 1335, in-loop filter 1345, reconstructed image buffer 1350, MV buffer 1365, MV prediction module 1375, and entropy encoder 1390. Motion compensation module 1330 and motion estimation module 1335 are part of inter-frame prediction module 1340.

[0275] In some embodiments, modules 1310-1390 are software instruction modules executed by one or more processing units (e.g., processors) of a computing device or electronic device. In some embodiments, modules 1310-1390 are hardware circuit modules implemented by one or more integrated circuits (ICs) of an electronic device. Although modules 1310-1390 are shown as independent modules, some modules may be combined into a single module.

[0276] Video source 1305 provides an uncompressed raw video signal, which represents the pixel data for each video frame. Subtractor 1308 calculates the difference between the raw video pixel data from video source 1305 and the predicted pixel data 1313 from motion compensation module 1330 or intra-frame prediction module 1325, as the prediction residual 1309. Transform module 1310 converts the difference (or residual pixel data or residual signal 1308) into transform coefficients (e.g., by performing a discrete cosine transform, or DCT for short). Quantization module 1311 quantizes the transform coefficients into quantized data (or quantization coefficients) 1312, which is then encoded into a bitstream 1395 by entropy encoder 1390.

[0277] The inverse quantization module 1314 inverse-quantizes the quantized data (or quantization coefficients) 1312 to obtain transform coefficients, and the inverse transform module 1315 performs an inverse transform on the transform coefficients to generate a reconstruction residual 1319. The reconstruction residual 1319 is added to the predicted pixel data 1313 to generate reconstructed pixel data 1317. In some embodiments, the reconstructed pixel data 1317 is temporarily stored in the line buffer 1327 (or intra-frame prediction buffer) for intra-frame image prediction and spatial MV prediction. The reconstructed pixels are filtered by the in-loop filter 1345 and stored in the reconstructed image buffer 1350. In some embodiments, the reconstructed image buffer 1350 is external memory to the video encoder 1300. In some embodiments, the reconstructed image buffer 1350 is internal memory to the video encoder 1300.

[0278] Intra-frame image estimation module 1320 performs intra-frame prediction based on reconstructed pixel data 1317 to generate intra-frame prediction data. The intra-frame prediction data is provided to entropy encoder 1390 for encoding into a bitstream 1395. The intra-frame prediction data is also used by intra-frame prediction module 1325 to generate predicted pixel data 1313.

[0279] The motion estimation module 1335 performs inter-frame prediction by generating MVs (Motion Rendering Models) that reference pixel data from previously decoded frames stored in the reconstructed image buffer 1350. These MVs are then provided to the motion compensation module 1330 to generate predicted pixel data.

[0280] The video encoder 1300 uses MV prediction to generate a predicted MV instead of encoding the complete actual MV in the bitstream, and encodes the difference between the MV used for motion compensation and the predicted MV as residual motion data and stores it in the bitstream 1395.

[0281] The MV prediction module 1375 generates a predicted MV based on a reference MV generated for encoding a previous video frame, i.e., a motion-compensated MV used to perform motion compensation. The MV prediction module 1375 retrieves the reference MV of the previous video frame from the MV buffer 1365. The video encoder 1300 stores the MV generated for the current video frame in the MV buffer 1365 as a reference MV for generating the predicted MV.

[0282] The MV prediction module 1375 creates a predicted MV using a reference MV. The predicted MV can be calculated through spatial MV prediction or temporal MV prediction. The difference (residual motion data) between the predicted MV of the current frame and the motion-compensated MV (MC MV) is encoded into a bitstream 1395 by the entropy encoder 1390.

[0283] The entropy encoder 1390 encodes various parameters and data into a bitstream 1395 using entropy coding techniques such as context-adaptive binary arithmetic coding (CABAC) or Huffman coding. The entropy encoder 1390 encodes various header elements, flags, quantization transform coefficients 1312, and residual motion data as syntax elements into the bitstream 1395. The bitstream 1395 is then stored in a storage device or transmitted to the decoder via a communication medium such as a network.

[0284] The in-loop filter 1345 performs filtering or smoothing operations on the reconstructed pixel data 1317 to reduce coded artifacts, particularly at the boundaries of pixel blocks. In some embodiments, the filtering or smoothing operations performed by the in-loop filter 1345 include a deblocking filter (DBF), a sample adaptive offset (SAO), and / or an adaptive loop filter (ALF). In some embodiments, luminance-mapped chrominance scaling (LMCS) is performed before the loop filter.

[0285] Figure 14 This demonstrates how the video encoder 1300 partially implements various features of cross-component prediction. The current block can be initially encoded via inter-frame prediction, intra-frame prediction, or IBC prediction (motion compensation module 1330 or intra-frame prediction module 1325) to generate an initial predictor 1415. The initial predictor 1415 can be the predicted or reconstructed luma component or a chroma component (e.g., Cb) of the current block. A cross-component model 1410 is applied to the initial predictor 1415 to generate a cross-component predictor 1425 for the (residual) chroma component (e.g., Cr). Samples of the cross-component predictor 1425 (e.g., predicted chroma) can be included in the predicted pixel data 1313.

[0286] The cross-component model 1410 can be generated by the model builder 1405 based on samples stored in the reconstructed image buffer 1350 or the intra-frame prediction buffer 1327, which store samples of the reconstructed pixel data 1317 acquired before the in-loop filter 1345.

[0287] Section IV above describes several cross-component models that can be used as cross-component model 1410. Cross-component model 1410 can also be selected from a candidate list (ModelList) provided by the candidate list construction module 1430. The candidate list is described in Section VIII.

[0288] CCM memory 1435 stores cross-component model (CCM) information for different images, including cross-component models generated by model builder 1405. The stored CCM information can be inherited by subsequent blocks as candidates for the candidate list.

[0289] For each currently encoded block, the candidate list construction module 1430 generates a corresponding candidate list from which the encoder can select a candidate to obtain a cross-component model applied to the current block. In some embodiments, the candidate list builder 1430 uses a predefined CCMER to restrict the candidate list to exclude candidates from within the CCMER and ensure that blocks within the same CCMER share the same candidate list (or partially the same candidate list), or independently derive their respective candidate lists. In some embodiments, the candidate list builder 1430 allows the current block to inherit the cross-component model from blocks encoded with different prediction modes (e.g., inter-coding and intra-coding). In some embodiments, the candidate list builder 1430 does not allow such inheritance.

[0290] In some embodiments, the DIMD module 1440 performs texture analysis on the CC predictor 1425 to identify a representative pattern 1445 for the current block and uses the identified representative pattern as an intra-frame prediction mode to select the primary and / or secondary transform types for the transform module 1310 and the inverse transform module 1315 (as described in Section 3).

[0291] Figure 15 A process 1500 using cross-component prediction to encode pixel blocks is conceptually illustrated. In some embodiments, one or more processing units (e.g., processors) of a computing device implementing encoder 1300 execute process 1500 by executing instructions stored in a computer-readable medium. In some embodiments, an electronic device implementing encoder 1300 executes process 1500.

[0292] The encoder receives (at block 1510) data to be encoded as the current pixel block in the current image. The current block can be inter-frame predicted using the MV with a reference block in the reference image. The current block can be inter-frame predicted using the BV with a reference block in the current image.

[0293] The encoder identifies (at block 1520) a predefined region (e.g., CCMER) applicable to the current block. The encoder derives (at block 1530) a candidate list that can be inherited by the current block, which includes cross-component prediction models from locations outside the predefined region. The predefined region may encompass the current block and another block that also uses the candidate list (or a portion of the candidate list) for cross-component prediction. In some embodiments, the other block uses at least one of spatially adjacent candidates, spatially non-adjacent candidates, and historical candidates from the candidate list of the current block for cross-component prediction.

[0294] The predefined region can encompass both the first and second blocks, where a first candidate list derived for the first block excludes cross-component prediction information or models from the second block, and a second candidate list derived for the second block excludes cross-component prediction information or models from the first block. Therefore, the first and second candidate lists are derived independently.

[0295] In some embodiments, the history-based model candidate list is updated only when the size of the current block is greater than a threshold determined based on a predefined region size.

[0296] In some embodiments, subsequent codec blocks encoded with a prediction mode different from the current block are allowed to inherit the selected cross-component model, such that subsequent intra-coded or IBC-coded blocks can inherit the cross-component model from inter-coded blocks. In some embodiments, subsequent codec blocks encoded with a prediction mode different from the current block are not allowed to inherit the selected cross-component model.

[0297] The encoder selects one or more cross-component models from the candidate list (at block 1540). The selection can be communicated by the index based on the reordering of the list.

[0298] The encoder encodes (at block 1550) the current block by generating a prediction of the current block using one or more selected cross-component models and producing prediction residuals. The encoder can generate predictions by applying one or more selected cross-component models to the reconstruction or prediction of the first component (e.g., luminance Y or chrominance Cb) of the current block to generate a predictor of the second component (e.g., chrominance Cb or chrominance Cr) of the current block.

[0299] In some embodiments, the encoder identifies a representative prediction mode for the current block and uses the representative prediction mode to select at least one transform set (major and minor) and a transpose flag for transforming or inverse transforming the prediction residuals of the current block, where the current block is a chroma block. In some embodiments, the representative prediction mode is identified by performing texture gradient analysis (e.g., DIMD) on prediction samples (or subsamples of prediction samples) of the current block. In some embodiments, the representative prediction mode is identified by performing texture gradient analysis (e.g., DIMD) on reconstructed samples (or subsamples of reconstructed samples) of the current block. In some embodiments, the representative prediction mode is used if the current chroma block is referenced when deriving the most probable mode (MPM) set for intra-frame encoding and decoding of subsequent blocks.

[0300] X. Example Video Decoder

[0301] In some embodiments, the encoder may convey (or generate) one or more syntax elements in the bitstream, such that the decoder may parse the one or more syntax elements from the bitstream.

[0302] Figure 16 An example video decoder 1600 capable of cross-component prediction is shown. As illustrated, the video decoder 1600 is an image or video decoding circuit that receives a bitstream 1695 and decodes its contents into pixel data for video frames used for display. The video decoder 1600 has several components or modules for decoding the bitstream 1695, including some selected from inverse quantization module 1611, inverse transform module 1610, intra-frame prediction module 1625, motion compensation module 1630, in-loop filter 1645, decoded image buffer 1650, MV buffer 1665, MV prediction module 1675, and resolver 1690. Motion compensation module 1630 is part of inter-frame prediction module 1640.

[0303] In some embodiments, modules 1610-1690 are software instruction modules executed by one or more processing units (e.g., processors) of a computing device. In some embodiments, modules 1610-1690 are hardware circuit modules implemented by one or more integrated circuits (ICs) of an electronic device. Although modules 1610-1690 are illustrated as independent modules, some of these modules may be combined into a single module.

[0304] Parser 1690 (or entropy decoder) receives bitstream 1695 and performs preliminary parsing according to the syntax defined by the video coding or image coding standard. The parsed syntax elements include various header elements, flags, and quantization data (or quantization coefficients) 1612. Parser 1690 then uses entropy coding techniques, such as context-adaptive binary arithmetic coding (CABAC) or Huffman coding, to further parse the syntax elements.

[0305] The inverse quantization module 1611 inverse-quantizes the quantized data (or quantization coefficients) 1612 to obtain transform coefficients, and the inverse transform module 1610 inverse-transforms the transform coefficients 1616 to generate the reconstructed residual signal 1619. The reconstructed residual signal 1619 is added to the predicted pixel data 1613 from the intra-frame prediction module 1625 or the motion compensation module 1630 to generate the decoded pixel data 1617. The decoded pixel data is filtered by the in-loop filter 1645 and stored in the decoded image buffer 1650. In some embodiments, the decoded image buffer 1650 is an external memory of the video decoder 1600. In some embodiments, the decoded image buffer 1650 is an internal memory of the video decoder 1600.

[0306] Intra-prediction module 1625 receives intra-prediction data from bitstream 1695 and generates predicted pixel data 1613 from decoded pixel data 1617 stored in decoded image buffer 1650. In some embodiments, decoded pixel data 1617 is also stored in line buffer 1627 (or intra-prediction buffer) for intra-image prediction and spatial motion vector prediction.

[0307] In some embodiments, the contents of the decoded image buffer 1650 are used for display. The display device 1605 can directly retrieve the contents of the decoded image buffer 1650 for display, or retrieve the contents of the decoded image buffer into a display buffer. In some embodiments, the display device receives pixel values ​​from the decoded image buffer 1650 via pixel transfer.

[0308] The motion compensation module 1630 generates predicted pixel data 1613 from the decoded pixel data 1617 stored in the decoded image buffer 1650 based on the motion compensated motion vectors (MC MVs). These motion compensated motion vectors are decoded by adding the residual motion data received from the bitstream 1695 to the predicted motion vectors received from the motion vector prediction module 1675.

[0309] The motion vector prediction module 1675 generates predicted motion vectors based on reference motion vectors generated for decoding previous video frames, such as motion-compensated motion vectors used for performing motion compensation. The motion vector prediction module 1675 retrieves the reference motion vectors of the previous video frames from the motion vector buffer 1665. The video decoder 1600 stores the motion-compensated motion vectors generated for decoding the current video frame in the motion vector buffer 1665 as reference motion vectors for generating the predicted motion vectors.

[0310] The in-loop filter 1645 filters or smooths the decoded pixel data 1617 to reduce coding artifacts, particularly at pixel block boundaries. In some embodiments, the filtering or smoothing operations performed by the in-loop filter 1645 include a deblocking filter (DBF), sample adaptive offset (SAO), and / or adaptive loop filter (ALF). In some embodiments, luminance-mapped chrominance scaling (LMCS) is performed before the loop filter.

[0311] Figure 17 A portion of a video decoder 1600 implementing various features of cross-component prediction is shown. The current block can be initially encoded via inter-frame prediction, intra-frame prediction, or intra-block copy (IBC) prediction (motion compensation module 1630 or intra-frame prediction module 1625), which generates an initial predictor 1715. The initial predictor 1715 can be one of the luma or chroma components (e.g., Cb) of the predicted or reconstructed current block. A cross-component model 1710 is applied to the initial predictor 1715 to generate a cross-component predictor 1725 for the (residual) chroma component (e.g., Cr). Samples of the cross-component predictor 1725 can be included in the predicted pixel data 1613.

[0312] The cross-component model 1710 can be generated by the model builder 1705 based on samples stored in the decoded image buffer 1650 and / or the intra-frame prediction buffer 1627, which are samples of the decoded pixel data 1617 acquired before the in-loop filter 1645.

[0313] Section IV above describes several cross-component models that can be used as cross-component model 1710. Cross-component model 1710 can also be selected from the candidate list (ModelList) provided by the candidate list construction module 1730. The candidate list is described in Section VIII.

[0314] CCM memory 1735 stores cross-component model (CCM) information for different images, including cross-component models generated by model builder 1705. The stored CCM information can be inherited by subsequent blocks as candidates for the candidate list.

[0315] For each currently decoded block, the candidate list construction module 1730 generates a corresponding candidate list from which the decoder can select a candidate to obtain a cross-component model to be applied to the current block. In some embodiments, the candidate list construction module 1730 uses a predefined cross-component model exclusion region (CCMER) to restrict the candidate list, excluding candidates from within the CCMER and ensuring that blocks within the same CCMER share the same candidate list (or a portion of the same candidate list), or independently derive their own candidate lists. In some embodiments, the candidate list construction module 1730 allows the current block to inherit a cross-component model from a block encoded by different prediction modes (e.g., inter-frame coding and intra-frame coding). In some embodiments, the candidate list construction module 1730 does not allow such inheritance.

[0316] In some embodiments, the DIMD module 1740 performs texture analysis on the cross-component predictor (CC predictor) 1725 to identify a representative pattern 1745 for the current block, and uses the identified representative pattern as an intra-frame prediction mode to select the primary transform and / or secondary transform type (as described in Section III) for use by the inverse transform module 1610.

[0317] Figure 18 A process 1800 for decoding pixel blocks using cross-component prediction is conceptually illustrated. In some embodiments, one or more processing units (e.g., processors) of a computing device implementing decoder 1600 execute process 1800 by executing instructions stored in a computer-readable medium. In some embodiments, an electronic device implementing decoder 1600 executes process 1800.

[0318] The decoder receives (in block 1810) data to be decoded into the current pixel block in the current image. The current block can be used for inter-frame prediction by referencing a reference block in the reference image with a motion vector (MV). The current block can be used for intra-frame block copy (IBC) prediction by referencing a reference block in the current image with a block vector (BV).

[0319] The decoder identifies (in block 1820) a predefined region (e.g., CCMER) applicable to the current block. The decoder derives (in block 1830) a candidate list that can be inherited by the current block, which includes cross-component prediction models from locations outside the predefined region. The predefined region may encompass the current block and another block that also uses the candidate list (or a portion of the candidate list) for cross-component prediction. In some embodiments, the other block uses at least one of spatially adjacent candidates, spatially non-adjacent candidates, and historical candidates from the current block's candidate list.

[0320] The predefined region can encompass both the first and second blocks, where a first candidate list derived for the first block excludes cross-component prediction information or models from the second block, and a second candidate list derived for the second block excludes cross-component prediction information or models from the first block. Therefore, the first and second candidate lists are derived independently.

[0321] In some embodiments, the history-based model candidate list is updated only when the size of the current block is greater than a threshold determined based on a predefined region size.

[0322] In some embodiments, subsequent coded blocks encoded with a prediction mode different from the current block are allowed to inherit the selected cross-component model, such that subsequent blocks encoded with intra-frame coding or intra-block copy (IBC) can inherit the cross-component model from inter-frame coded blocks. In some embodiments, subsequent coded blocks encoded with a prediction mode different from the current block are not allowed to inherit the selected cross-component model.

[0323] The decoder selects one or more cross-component models from the candidate list (in block 1840). Selection can be signaled by reordering the index of the list.

[0324] The decoder reconstructs the current block (in block 1850) by generating a prediction of the current block using one or more selected cross-component models. The decoder can then provide the reconstructed current block as part of the reconstructed current image for display. The decoder can generate a predictor for the second component (e.g., chromaticity Cb or chromaticity Cr) of the current block by applying one or more selected cross-component models to the reconstruction or prediction of the first component (e.g., luminance Y or chromaticity Cb) of the current block.

[0325] In some embodiments, the decoder identifies a representative prediction pattern for the current block and uses the representative prediction pattern to select at least one set of transforms (major and minor transforms) and a transpose flag for inverse transforming the prediction residual of the current block, where the current block is the current chroma block. In some embodiments, the representative prediction pattern is identified by performing texture gradient analysis (e.g., DIMD) on prediction samples (or subsamples of prediction samples) of the current block. In some embodiments, the representative prediction pattern is identified by performing texture gradient analysis (e.g., DIMD) on reconstructed samples (or subsamples of reconstructed samples) of the current block. In some embodiments, the representative prediction pattern is used if the current chroma block is referenced when deriving the most probable mode (MPM) set for intra-frame coding of subsequent blocks.

[0326] XI. Example Electronic System

[0327] Many of the features and applications described above are implemented as software processes, which are specified as a set of instructions recorded on a computer-readable storage medium (also known as a computer-readable medium). When these instructions are executed by one or more computing or processing units (e.g., one or more processors, processor cores, or other processing units), they cause the processing unit to perform the operations indicated in the instructions. Examples of computer-readable media include, but are not limited to, CD-ROMs, flash drives, random-access memory (RAM) chips, hard disks, erasable programmable read-only memories (EPROMs), electrically erasable programmable read-only memories (EEPROMs), and the like. Computer-readable media do not include carrier waves and electronic signals transmitted via wireless or wired connections.

[0328] In this specification, the term "software" includes firmware residing in read-only memory or an application stored in magnetic memory that can be read into memory and processed by a processor. Furthermore, in some embodiments, multiple software inventions may be implemented as sub-parts of a larger program while remaining independent software inventions. In some embodiments, multiple software inventions may also be implemented as independent programs. Finally, any combination of independent programs that collectively implement the software inventions described herein is within the scope of this disclosure. In some embodiments, when a software program is installed on one or more electronic systems and runs, one or more specific machine implementations are defined that execute and perform the operations of the software program.

[0329] Figure 19 An electronic system 1900 is conceptually illustrated, in which certain embodiments of the present disclosure are implemented. The electronic system 1900 may be a computer (e.g., a desktop computer, personal computer, tablet computer, etc.), a telephone, a PDA, or any other type of electronic device. Such an electronic system includes various types of computer-readable media and interfaces for various other types of computer-readable media. The electronic system 1900 includes a bus 1905, a processing unit 1910, a graphics processing unit (GPU) 1915, system memory 1920, a network 1925, a read-only memory 1930, a permanent storage device 1935, an input device 1940, and an output device 1945.

[0330] Bus 1905 collectively represents all system, peripheral, and chipset buses that communicate with the numerous internal devices of electronic system 1900. For example, bus 1905 communicates with processing unit 1910, GPU 1915, read-only memory 1930, system memory 1920, and permanent storage device 1935.

[0331] From these various memory units, processing unit 1910 retrieves instructions to execute and processes data to perform the processes of this disclosure. The processing unit may be a single processor or a multi-core processor in different embodiments. Some instructions are passed to and executed by GPU 1915. GPU 1915 may offload various computational or supplemental image processing provided by processing unit 1910.

[0332] Read-only memory (ROM) 1930 stores static data and instructions used by processing unit 1910 and other modules of the electronic system. On the other hand, permanent storage device 1935 is a read-write storage device. This device is a non-volatile memory unit that stores instructions and data even when the electronic system 1900 is powered off. Some embodiments of this disclosure use mass storage devices (e.g., magnetic disks or optical disks and their corresponding disk drives) as permanent storage device 1935.

[0333] Other embodiments use removable storage devices (e.g., floppy disks, flash memory devices, etc., and their corresponding disk drives) as permanent storage devices. Similar to permanent storage device 1935, system memory 1920 is a read-write storage device. However, unlike storage device 1935, system memory 1920 is volatile read-write memory, such as random access memory. System memory 1920 stores some instructions and data used by the processor during runtime. In some embodiments, processes according to this disclosure are stored in system memory 1920, permanent storage device 1935, and / or read-only memory 1930. For example, various memory units include instructions for processing multimedia clips according to some embodiments. From these various memory units, processing unit 1910 retrieves instructions to execute and processes data to perform the processes of some embodiments.

[0334] Bus 1905 is also connected to input and output devices 1940 and 1945. Input device 1940 enables a user to pass information to the electronic system and select commands. Input device 1940 includes an alphanumeric keypad and pointing device (also known as a "cursor control device"), a camera (e.g., a webcam), a microphone, or similar devices to receive voice commands, etc. Output device 1945 displays images generated by the electronic system or otherwise outputs data. Output device 1945 includes printers and display devices, such as cathode ray tube (CRT) or liquid crystal display (LCD), and speakers or similar audio output devices. Some embodiments include devices such as touchscreens that function as both input and output devices.

[0335] Finally, as Figure 19 As shown, bus 1905 also connects electronic system 1900 to network 1925 via a network adapter (not shown). In this way, the computer can become part of a computer network (e.g., a local area network (“LAN”), a wide area network (“WAN”), or an intranet, or a network of networks, such as the Internet). Any or all components of electronic system 1900 can be used in conjunction with this disclosure.

[0336] Some embodiments include electronic components, such as microprocessors, storage, and memory, that store computer program instructions in a machine-readable or computer-readable medium (also referred to as a computer-readable storage medium, machine-readable medium, or machine-readable storage medium). Examples of such computer-readable media include RAM, ROM, read-only compact discs (CD-ROM), recordable compact discs (CD-R), rewritable compact discs (CD-RW), read-only digital versatile discs (e.g., DVD-ROM, dual-layer DVD-ROM), various recordable / rewritable DVDs (e.g., DVD-RAM, DVD-RW, DVD+RW, etc.), flash memory (e.g., SD cards, mini-SD cards, micro-SD cards, etc.), magnetic and / or solid-state drives, read-only and recordable Blu-ray® discs, ultra-density discs, any other optical or magnetic media, and floppy disks. The computer-readable medium may store a computer program executable by at least one processing unit and include a set of instructions for performing various operations. Examples of computer programs or computer code include machine code, such as code generated by a compiler, and files containing high-level code that are executed by a computer, electronic component, or microprocessor using an interpreter.

[0337] While the above discussion primarily concerns microprocessors or multi-core processors that execute software, many of the functions and applications described above are executed by one or more integrated circuits, such as application-specific integrated circuits (ASICs) or field-programmable gate arrays (FPGAs). In some embodiments, these integrated circuits execute instructions stored on the circuit itself. Furthermore, some embodiments execute software stored in programmable logic devices (PLDs), ROM, or RAM devices.

[0338] The terms “computer,” “server,” “processor,” and “memory” as used in this specification and any claim of this application refer to electronic or other technical devices. These terms do not include individuals or groups. For the purposes of this specification, the term “display” or “show” means “displayed on an electronic device.” The terms “computer-readable medium,” “computer-readable media,” and “machine-readable medium” as used in this specification and any claim of this application are entirely limited to tangible physical objects that store information in a computer-readable form. These terms do not include any wireless signals, wired download signals, or any other transient signals.

[0339] Although this disclosure has been described in many specific details, those skilled in the art will recognize that this disclosure may be embodied in other specific forms without departing from its spirit. Furthermore, many figures (including...) Figure 15 and Figure 18 The process is conceptually described. The specific operations of these processes may not be performed in the exact order shown and described. Specific operations may not be performed in a continuous series of operations, and different specific operations may be performed in different embodiments. Furthermore, the process may be implemented using several sub-processes or as part of a larger macro-process. Therefore, those skilled in the art will understand that this disclosure should not be limited to the foregoing illustrative details, but should be defined by the appended claims.

[0340] Additional notes

[0341] The topics described herein sometimes illustrate different components contained within or connected to different other components. It should be understood that the architectures shown are merely examples, and many other architectures can actually be implemented to achieve the same functionality. Conceptually, any arrangement of components to achieve the same functionality is effectively “associated” to achieve the desired function. Therefore, any two components combined in this document to achieve a specific function can be considered “associated” together to achieve the desired function, regardless of the architecture or intermediate components. Similarly, any two such associated components can also be considered “operably connected” or “operably coupled” together to achieve the desired function, and any two components that can be suchly associated can also be considered “operably coupled” together to achieve the desired function. Specific examples of operational coupling include, but are not limited to, physically connectable and / or physically interactive components and / or wirelessly interactive components and / or logically interactive and / or logically interactive components.

[0342] Furthermore, regarding the use of virtually any plural and / or singular terms in this document, those skilled in the art can appropriately translate from plural to singular and / or from singular to plural depending on the context and / or application. For clarity, various singular / plural permutations may be explicitly listed herein.

[0343] Furthermore, those skilled in the art will understand that, generally, the terms used herein, particularly in appended claims, such as the body of an appended claim, are generally considered "open" terms; for example, the word "comprising" should be interpreted as "including but not limited to," the word "having" should be interpreted as "having at least," and the word "including" should be interpreted as "including but not limited to," etc. Those skilled in the art will also further understand that if there is a specific number of intents in the introduced claim statement, then that intent will be explicitly stated in the claim, and where there is no such statement, then such intent does not exist. For example, to aid understanding, the following appended claims may contain the use of the introductory phrases "at least one" and "one or more" to introduce the claim statement. However, the use of these phrases should not be construed as implying that introducing the claim statement with the indefinite article "a" or "an" would limit any particular claim containing such an introductory claim statement to containing only one such statement, even if the same claim includes the introductory phrase "one or more" or "at least one" and indefinite articles such as "a" or "an," for example, "a" and / or "an" should be interpreted as "at least one" or "one or more"; the same applies to definite articles used to introduce the claim statement. Furthermore, even if a specific number of claims is explicitly stated, those skilled in the art will recognize that such a statement should be interpreted as at least the stated number. For example, the simple statement "two statements" without any other modifiers means at least two statements, or two or more statements. Additionally, when using conventions such as "at least one A, B, and C," this structure is generally interpreted in the manner understood by those skilled in the art. For example, "a system having at least one A, B, and C" includes, but is not limited to, a system having only A, a system having only B, a system having only C, a system having both A and B, a system having both A and C, a system having both B and C, and / or a system having both A, B, and C simultaneously. Similarly, when using conventions such as "at least one A, B, or C," this structure is generally interpreted in the manner understood by those skilled in the art. For example, "a system having at least one A, B, or C" includes, but is not limited to, a system having only A, a system having only B, a system having only C, a system having both A and B, a system having both A and C, a system having both B and C, and / or a system having both A, B, and C simultaneously. Those skilled in the art will further understand that virtually any extractive word and / or phrase presenting two or more alternative terms, whether in the specification, claims, or drawings, should be understood to include the possibility of including one, any, or both terms. For example, the phrase “A or B” would be understood to include the possibility of including “A” or “B” or “A and B”.

[0344] As can be understood from the foregoing, various embodiments of this disclosure have been described herein for illustrative purposes, and various modifications may be made without departing from the scope and spirit of this disclosure. Therefore, the various embodiments disclosed herein are not intended to be limiting, and the true scope and spirit are indicated by the following claims.

Claims

1. A video encoding / decoding method, comprising: Receive data of the current pixel block of the current image to be encoded or decoded into video; Identify the predefined regions applicable to the current block; Export a candidate list that can be inherited by the current block, which includes cross-component prediction models from locations outside the predefined region; Select one or more cross-component models from the candidate list; and The current block is encoded or decoded using the selected one or more cross-component models.

2. The video encoding / decoding method of claim 1, wherein encoding or decoding the current block includes applying the selected cross-component model to a first component reconstructed or predicted in the current block to generate a predictor for a second component of the current block.

3. The video encoding and decoding method as described in claim 2, wherein the first component is a luminance component and the second component is a chrominance component.

4. The video encoding / decoding method as described in claim 2, wherein the first and second components are two different chroma components.

5. The video encoding / decoding method as described in claim 1, wherein the current block is predicted by inter-frame prediction or intra-block copying (IBC).

6. The video encoding / decoding method of claim 1, wherein subsequent encoding / decoding blocks encoded / decoded by a prediction mode different from the current block are allowed to inherit the selected cross-component model.

7. The video encoding / decoding method of claim 1, wherein subsequent encoding / decoding blocks encoded / decoded by a prediction mode different from the current block are not allowed to inherit the selected cross-component model.

8. The video encoding / decoding method of claim 1, wherein the predefined region covers the first and second blocks, wherein a first candidate list derived for the first block excludes cross-component prediction information from the second block, and a second candidate list derived for the second block excludes cross-component prediction information from the first block.

9. The video encoding / decoding method as described in claim 8, wherein the first candidate list and the second candidate list are independently derived.

10. The video encoding / decoding method of claim 1, wherein the predefined region encompasses the current block and another block, which also uses all or part of the candidate list for cross-component prediction.

11. The video encoding / decoding method of claim 10, wherein the other block uses at least one of spatially adjacent candidates, spatially non-adjacent candidates, and historical candidates from the candidate list of the current block.

12. The video encoding / decoding method of claim 1, wherein the historical model candidate list is updated only when the size of the current block is greater than a threshold determined based on the predefined region size.

13. A video encoding / decoding method, comprising: Receive data for the current pixel block to be encoded or decoded into the current video image; Based on the prediction residuals of the current block, derive a candidate list that can be inherited by the current block; Select one or more cross-component models from the candidate list; Identify a representative prediction pattern for the current block, and use the representative prediction pattern to select at least one transformation set and transpose flag for transforming or inversely transforming the prediction residuals of the current block, wherein the current block is the current chroma block; as well as The current block is encoded or decoded using the selected one or more cross-component models.

14. The video encoding / decoding method of claim 13, wherein the representative prediction pattern is identified by performing texture gradient analysis of the predicted chroma samples of the current block.

15. The video encoding / decoding method of claim 13, wherein if the current chroma block is referenced when deriving the most probable mode (MPM) set for intra-frame encoding / decoding of subsequent blocks, the representative prediction mode is used.

16. An electronic device comprising: The video encoding / decoding circuit is configured to perform the following operations: Receive data for the current pixel block to be encoded or decoded into the current video image; Identify the predefined regions applicable to the current block; Export a candidate list that can be inherited by the current block, which includes cross-component prediction models from locations outside the predefined region; Select one or more cross-component models from the candidate list; and The current block is encoded or decoded using the selected one or more cross-component models.

17. A video decoding method, comprising: Receive the data of the current pixel block to be decoded into the current image of the video; Identify the predefined regions applicable to the current block; Export a candidate list that can be inherited by the current block, which includes cross-component prediction models from locations outside the predefined region; Select one or more cross-component models from the candidate list; and Reconstruct the current block using the selected one or more cross-component models.

18. A video coding method, comprising: Receive data of the current pixel block as the current image of the video for encoding; Identify the predefined regions applicable to the current block; Export a candidate list that can be inherited by the current block, which includes cross-component prediction models from locations outside the predefined region; Select one or more cross-component models from the candidate list; and Encode the current block using the selected one or more cross-component models.

Images