Hybrid candidate for cross-component merge mode

Abstract

A video codec constructs one or more cross-component prediction (CCP) merge candidate lists for a current block of a current picture of a video, the current block including a first color block and a second color block. The candidate lists include one or more regular candidates and one or more mixed candidates. Each regular candidate is associated with a set of CCP parameters, and each mixed candidate corresponds to two or more regular candidates. The video codec selects a mixed candidate from the one or more CCP merge candidate lists. The video codec generates a mixed prediction for the second color block based on two or more cross-component predictions (CCPs) generated by applying the two or more regular candidates corresponding to the selected mixed candidate to the first color block reconstructed by applying intra prediction to samples of the current block neighboring or adjacent to the current block. The video codec encodes or decodes the current block using the generated mixed prediction.

Description

[0001] Cross-referencing This disclosure is part of a non-provisional application and claims priority to U.S. Provisional Patent Application No. 63 / 623,353, filed January 22, 2024. The contents of the above application are incorporated herein by reference. [Technical Field] This disclosure generally relates to video encoding and decoding. Specifically, this disclosure relates to a method for encoding and decoding pixel blocks using cross-component prediction (CCP). [Background Technology] Unless otherwise stated herein, the methods described in this section are not prior art to the following claims and are not considered prior art by virtue of being incorporated herein.

[0004] High-Efficiency Video Coding (HEVC) is an international video coding standard developed by the Joint Collaborative Team on Video Coding (JCT-VC). HEVC is based on a hybrid block-based motion-compensated DCT transform coding architecture. The basic unit of compression is called a coding unit (CU), which is a 2Nx2N pixel block. Each CU can be recursively divided into four smaller CUs until a preset minimum size is reached. Each CU contains one or more prediction units (PUs).

[0005] Versatile Video Coding (VVC) is the latest international video coding standard developed by the Joint Video Experts Team (JVET) of ITU-T SG16 WP3 and ISO / IEC JTC1 / SC29 / WG11. The input video signal is predicted from a reconstructed signal, which originates from a previously encoded image region. The prediction residual signal is processed through block transform. The transform coefficients, along with other auxiliary information, are quantized and entropy-coded in the bitstream. The reconstructed signal is generated by adding the predicted signal to the reconstructed residual signal obtained after inverse transforming the inverse quantization transform coefficients. The reconstructed signal is further filtered within the loop to remove coding artifacts. The decoded images are stored in the frame buffer and used to predict subsequent images in the input video signal.

[0006] In VVC, the encoded image is divided into non-overlapping block regions represented by relevant coding tree units (CTUs). Leaf nodes of the coding tree correspond to coding units (CUs). The encoded image can consist of several slices, each containing an integer number of CTUs. The CTUs within a slice are processed in raster scan order. Double-predictive (B) slices can predict the sample value of each block using at most two motion vectors (MVs) and a reference index, either through intra-frame or inter-frame prediction. Predictive (P) slices can predict the sample value of each block using at most one motion vector and a reference image index, either through intra-frame or inter-frame prediction. Intra-frame (I) slices are decoded solely through intra-frame prediction.

[0007] CTUs can be divided into one or more non-overlapping coding units (CUs) using a quadtree (QT) or nested multi-type-tree (MTT) structure to accommodate different local motion and texture characteristics. CUs can be further divided into smaller CUs using one of five segmentation types: quadtree segmentation, vertical binary tree segmentation, horizontal binary tree segmentation, vertical center-side ternary tree segmentation, and horizontal center-side ternary tree segmentation.

[0008] Each CU contains one or more prediction units (PUs). The prediction unit, along with the associated CU syntax, serves as the basic unit for signaling predictor information. The specified prediction process is used to predict the values ​​of associated pixel samples within the PU. Each CU may contain one or more transform units (TUs) to represent prediction residual blocks. A transform unit (TU) consists of a transform block (TB) for one luma sample and two corresponding chroma sample transform blocks; each TB corresponds to a residual sample block for one color component. Integer transforms are applied to the transform blocks. The level values ​​of the quantization coefficients, along with other auxiliary information, are entropy-coded in the bitstream. Terms such as coding tree block (CTB), coding block (CB), prediction block (PB), and transform block (TB) are used to specify two-dimensional sample arrays of a single color component associated with the CTU, CU, PU, ​​and TU, respectively. Therefore, a CTU consists of one luma CTB, two chroma CTBs, and associated syntax elements. CUs, PUs, and TUs have similar correspondences.

[0009] For each CU in inter-frame prediction, motion parameters include motion vectors, reference picture indices, reference picture list usage indices, and other additional information used to generate inter-frame prediction samples. Motion parameters can be signaled explicitly or implicitly. When a CU is coded in skip mode, it is associated with a PU and has no significant residual coefficients, nor are there any encoded motion vector differences or reference picture indices. Merging mode specifies that the motion parameters of the current CU can be obtained from neighboring CUs, including spatial and temporal candidates, as well as additional schemes introduced in VVC. Merging mode can be applied to any CU in inter-frame prediction. An alternative to merging mode is explicit transmission of motion parameters, i.e., each CU explicitly signales its motion vectors, the corresponding reference picture index for each reference picture list, the reference picture list usage flag, and other necessary information.

[0010] Intra-block copy (IBC) or current picture referencing (CPR) refers to encoding pixel blocks by referencing the pixel positions within the current image that are related to the current block using block vectors. [Summary of the Invention] The following abstract is for illustrative purposes only and is not intended to be limiting in any way. That is, the following abstract aims to introduce the concepts, key points, benefits, and advantages of the novel and non-obvious techniques described herein. Some, not all, embodiments will be further elaborated in the detailed description below. Therefore, the following abstract is not intended to identify the essential features of the claimed subject matter, nor is it intended to determine the scope of the claimed subject matter.

[0012] Some embodiments of this disclosure provide a method for encoding or decoding a video block using hybrid cross-component prediction (CCP). A video encoder receives input data relating to a current block of a current image of a video to be encoded or decoded. The current block includes a first color patch and a second color patch. The video encoder constructs one or more CCP merging candidate lists for the current block, the candidate lists including one or more regular candidates and one or more hybrid candidates. Each regular candidate is associated with a set of CCP parameters, and each hybrid candidate corresponds to two or more regular candidates. The video encoder selects a candidate from the one or more CCP merging candidate lists. The video encoder generates a prediction for the second color patch based on the selected candidate, wherein if the selected candidate is a hybrid candidate, the prediction is generated based on two or more cross-component predictions (CCPs) generated by applying the two or more regular candidates corresponding to the selected hybrid candidate to the first color patch reconstructed by intra-frame prediction referencing samples neighboring or adjacent to the current block. The video encoder uses the generated prediction to encode or decode the current block.

[0013] In some implementations, two or more CCP merge candidate lists are constructed for the current block. These CCP merge candidate lists include one or more regular candidates and one or more mixed candidates, and the selected candidate is chosen from one of the two or more CCP merge candidate lists. In some implementations, a flag is used for signaling to indicate which of the two or more CCP merge candidate lists the selected candidate is selected from.

[0014] In some implementations, the hybrid candidate indicates the two or more regular candidates by providing an index assigned to the one or more CCP merge candidate lists. In some implementations, the two or more regular candidates indicated by the hybrid candidate are limited to not being linear models. In some implementations, the two or more regular candidates indicated by the hybrid candidate are limited to using non-downsampled luminance samples as input, or limited to not being linear models, unless at least one of the indicated regular candidates uses non-downsampled luminance samples as input.

[0015] In some implementations, the two or more regular candidates indicated by the hybrid candidate are limited to candidates inherited from the position adjacent to the current block or from the history table. In some implementations, the two or more regular candidates indicated by the hybrid candidate are limited to candidates from one of the one or more CCP merge candidate lists. In some implementations, the two or more regular candidates indicated by the hybrid candidate are limited to those excluding the default candidate with default parameters.

[0016] In some implementations, the hybrid candidate is inserted into one of the one or more CCP merge candidate lists, after the non-default regular candidate and before the default candidate. In some implementations, the regular candidates in the one or more CCP merge candidate lists are reordered before the hybrid candidate is inserted into the list. In some implementations, the one or more CCP merge candidate lists are reordered entirely after the hybrid candidate is inserted into the list. In some implementations, each hybrid candidate is inserted into one of the one or more CCP merge candidate lists in an order determined based on the cost of the hybrid candidate.

[0017] In some implementations, two CCP merging candidate lists are constructed, organized as a first list (containing only regular candidates) and a second list (containing only hybrid candidates), with the second list containing only the hybrid candidates. The second list is constructed after the first list has been built and reordered according to the (template) cost of the candidates in the first list. The first and second lists may be the same size or different. The two or more regular candidates indicated by each hybrid candidate in the second list may be limited to candidates in the first list. The hybrid candidates in the second list can indicate their two or more regular candidates using the index of the first list. In some implementations, the hybrid candidates in the second list are limited to indicating only regular candidates that are not default candidates in the first list. In some implementations, the hybrid candidates in the second list are reordered according to the cost of the candidate in the second list, and the hybrid candidate with the lowest cost in the second list may be implicitly selected without signaling transmission.

[0018] In some implementations, each hybrid candidate is inserted into one of the one or more CCP merge candidate lists in an order determined based on the indices of the two or more regular candidates indicated by the hybrid candidate. The index of the regular candidate can be determined for the CCP merge candidate list based on the cost of the candidate.

[0019] In some implementations, a flag is used for signaling to indicate whether the first list or the second list is used to predict the current block. (If hybrid prediction is not possible, this flag can be skipped and no signaling is transmitted.) In some implementations, the video encoder selects a hybrid candidate by signaling a candidate index to indicate the position of the hybrid candidate in the second list.

[0020] In some implementations, the two or more conventional candidates (or their CCP parameters) are applied to the reconstruction of the first color patch via intra-frame prediction (referencing samples that are neighboring or adjacent to the current patch) to generate the two or more cross-component predictions (CCPs). When generating the hybrid prediction, the two or more cross-component predictions may be combined using equal weighting.

[0021] In some implementations, the video encoder also includes a set of CCP parameters associated with one of the two or more regular candidates indicated by the selected hybrid candidate for the current block, for reference in subsequent blocks. This set of CCP parameters for the regular candidate can be selected based on its cost or its cross-component prediction type. [Attached Image Description] The accompanying drawings are included in this disclosure to further understand it and form part of it. The drawings illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the disclosure. It will be understood that the drawings are not necessarily drawn to scale, as some components may be shown in a manner not proportional to actual implementation dimensions to clearly illustrate the concepts of the present disclosure.

[0023] Figure 1 It showcases 67 intra-frame prediction modes, including 65 directional or angular intra-frame prediction modes (from 2 to 66).

[0024] Figure 2 This demonstrates an example of dividing neighboring samples into two groups in a cross-component linear model (CCLM).

[0025] Figure 3 The reference region used to derive the CCCM filter coefficients is shown.

[0026] Figure 4 A 3x2 filter is shown for predicting chromaticity samples based on six non-downsampled neighboring luminance samples.

[0027] Figure 5 This demonstrates the Inter-Component Convolutional Model (InterCCCM) process at the decoder end. The inter-component filter is derived from the prediction blocks of luminance and chrominance.

[0028] Figure 6 The neighbor template used to calculate model error is shown.

[0029] Figure 7 This example shows a cross-component prediction (CCP) merged candidate list, where the merged candidate is inserted after the inherited candidate and before the default candidate.

[0030] Figure 8A -C indicates the reordering of the CCP merge candidate list with mixed candidates.

[0031] Figure 9 An example video encoder that enables hybrid cross-component prediction is shown.

[0032] Figure 10 This demonstrates the relevant parts of the video encoder that enables cross-component predictive mixing.

[0033] Figure 11 This concept demonstrates the process of encoding pixel blocks using a list of one or more CCP merging candidates that includes merging candidates.

[0034] Figure 12 An example of a video decoder that enables hybrid cross-component prediction is shown.

[0035] Figure 13 This demonstrates the relevant parts of the video decoder that enable cross-component prediction mixing.

[0036] Figure 14 This concept demonstrates the process of decoding pixel blocks using a list of one or more CCP merging candidates that includes mixed candidates.

[0037] Figure 15 This invention conceptually demonstrates an electronic system for implementing certain embodiments of the present disclosure.

Detailed Implementation Methods

[0039] I. Intra-frame prediction A. Intra-frame prediction mode and most probable mode (MPM) Intra-prediction methods utilize one or more reference lines adjacent to the current prediction unit (PU) and an intra-prediction mode to generate a predictor for the current PU. The intra-prediction direction can be selected from a set of modes containing multiple prediction directions, a DC mode, and a planar mode. An intra-prediction mode can also refer to any intra-prediction mode that uses spatially reconstructed samples to determine the current block predictor. The number of directional intra-prediction modes can be 33 or extended to 65. Including DC and planar modes, the number of intra-prediction modes is 35, but can also be extended to 67. Figure 1 It showcases 67 intra-frame prediction modes, including 65 directional or angular intra-frame prediction modes (from 2 to 66).

[0040] Certain intra-prediction modes (e.g., 3 or 5) are identified as the set of most probable modes (MPMs) for intra-prediction of the current prediction block, and thus a signaling transmission index can be used to select one of the MPMs.

[0041] B. Location-Related Intra-Prediction Combination (PDPC) In VVC, intra-prediction results for DC, planar, and certain angle modes can be further refined using the position-dependent intra-prediction combination (PDPC) method. PDPC is an intra-prediction method that combines boundary reference samples with HEVC-style intra-prediction using filtered boundary reference samples. PDPC can be applied without signaling transmission in the following intra-modal modes: planar, DC, intra-angle less than or equal to the horizontal direction, and intra-angle greater than or equal to the vertical direction and less than or equal to 80 degrees. PDPC is not applied if the current block is in Bdpcm mode or the multiple reference line (MRL) index is greater than 0.

[0042] C. Template-based intra-frame mode derivation (TIMD) In mode selection, template matching methods can be applied by calculating the cost between reconstructed samples and predicted samples. One example is template-based intra-mode derivation (TIMD). TIMD is an encoding method in which both the encoder and decoder implicitly derive the intra-predicted mode of the CU using neighboring templates, rather than the encoder signaling the exact intra-predicted mode to the decoder.

[0043] For each intra-prediction mode in the MPM, and for the wide-angle mode when upper-right and / or lower-left reference samples are available, the SATD between the predicted and reconstructed samples of the template is calculated as the cost. The two intra-prediction modes with the smallest SATD are selected as TIMD modes. These two TIMD modes are fused with corresponding weights after applying PDPC processing, and the resulting weighted intra-prediction is used to encode the current CU. Position-related intra-prediction combination (PDPC) is incorporated into the derivation of the TIMD modes.

[0044] II. Cross-component model prediction A. Cross-component linear model prediction (CCLM) To reduce cross-component redundancy, a cross-component linear model (CCLM) prediction mode can be used in video coding. Here, chroma samples are based on reconstructed luminance samples from the same CU and predicted using the following linear model: Among them, pred C (i,j) represents the predicted chromaticity sample in CU, rec L ′(i,j) represents the downsampled reconstructed brightness sample of the same CU.

[0045] B. Multi-model CCLM The Multi-Model CCLM (MMLM) mode is used to predict chromaticity samples for the entire CU from luminance samples using two linear models. In MMLM, the neighboring luminance and chromaticity samples of the current block are divided into two groups, each serving as a training set to derive the linear model (i.e., derive specific α and β for a specific group). Furthermore, samples from the current luminance block are also classified according to the same rules used to classify neighboring luminance samples.

[0046] Figure 2 This demonstrates an example of dividing neighboring samples into two groups in a multi-model cross-component linear model (CCLM). The threshold is calculated as the average of the neighboring reconstructed brightness samples. If the neighboring sample Rec′ L If [x,y]≤threshold, then it belongs to group 1; if the neighboring sample Rec′ L If [x,y]>the threshold, then it is classified as group 2.

[0047] C. Convolutional Cross-Component Model (CCCM) Convolutional Cross-Component Models (CCCMs) can be used to predict chroma samples from reconstructed luminance samples in a manner similar to CCLMs. Like CCLMs, when chroma subsampling is employed, the reconstructed luminance samples are downsampled to match a lower-resolution chroma grid. Similar to CCLMs, top, left, or top and left reference samples are used as templates for model derivation.

[0048] The CCCM's convolutional 7-tap filter has a 5-tap plus sign spatial component, a nonlinear term, and a bias term. The input to the filter's 5-tap spatial component includes the center (C) luminance sample co-located with the chrominance sample to be predicted, as well as its top / north (N), bottom / south (S), left / west (W), and right / east (E) neighboring samples.

[0049] The nonlinear term P is represented as the square of the center brightness sample C, scaled to the range of sample values ​​for the content: P = (C*C+midVal)>>bitDepth That is, for 10-bit content, the calculation is as follows: P = (C*C+512)>>10 The filter output is calculated by convolving the filter coefficients ci with the input values, and then cropped to the range of effective chromaticity samples. predChromaVal=c0C+c1N+c2S+c3E+c4W+c5P+c6B The bias term B represents the scalar offset between the input and output (similar to the offset term in CCLM) and is set to an intermediate chroma value (512 for 10-bit content). The filter coefficients ci are calculated by minimizing the mean square error (MSE) between the predicted and reconstructed chroma samples in the reference region.

[0050] Figure 3 The reference region 300 used to derive the CCCM filter coefficients is shown. As shown, the reference region includes 2 or 6 rows of chroma samples above and to the left of the PU. Whether to use 6 or 2 rows of neighboring samples to derive the CCCM model parameters in a single-model CCCM is determined by the template cost. Similarly, for a multi-model CCCM mode, the two candidates are divided into two groups by deriving the mean using 6 rows of neighboring luma samples or luma samples collocated with the current chroma block. The cost is obtained by applying the candidate CCPs (2 or 6 rows) to the template and calculating the sum of absolute differences (SAD) between the CCP prediction samples and the reconstructed samples in the template.

[0051] The reference region 300 extends to the right by one PU width and downwards by one PU height, beyond the PU boundary. The region is adjusted to include only usable samples. The extended portion of the region (labeled the "extended region") is used to support the "side samples" of the plus-shaped spatial filter, filling in unusable areas. Reference sample values ​​immediately outside the top-left corner of the PU are used as offsets (offsetLuma, offsetCb, and offsetCr) to simplify processing. These fixed values ​​are subtracted from the sample values ​​used for model creation and final prediction (i.e., luminance and chrominance in the reference region, and luminance in the current PU), as follows: C'=C–offsetLuma N'=N–offsetLuma S' = S – offsetLuma E'=E–offsetLuma W' = W – offsetLuma P'=nonLinear(C') B =midValue = 1<<(bitDepth - 1) Chromaticity values ​​are predicted using the following formulas, where offsetChroma is equal to the offsetCr and offsetCb of the Cr and Cb components, respectively: predChromaVal=c0C'+c1N'+c2S'+c3E'+c4W' + c5P'+c6B+offsetChroma CCCM mode using a 3x2 filter with non-downsampled brightness samples can be adopted. Figure 4 A 3x2 filter for predicting chroma samples based on six undownsampled neighboring luminance samples is demonstrated. This 3x2 filter has a 6-tap spatial term, four nonlinear terms, and one bias term. The 6-tap spatial term corresponds to the six neighboring luminance samples (L0, L1, ..., L5) surrounding the chroma sample to be predicted (C), and the four nonlinear terms are derived from samples L0, L1, L2, and L3. The chroma sample C is predicted by the 3x2 filter as follows: in β is the coefficient, and β is the offset. In CCCM, multiple downsampling filters can be applied to a set of reconstructed brightness samples.

[0052] D. Gradient linear model For the YUV 4:2:0 color format, the gradient linear model (GLM) method can be used to predict chrominance samples from luminance sample gradients. Two modes are supported: two-parameter GLM and three-parameter GLM. Compared to CCLM, two-parameter GLM uses luminance sample gradients to replace downsampled luminance values ​​to derive the linear model. Specifically, when applying two-parameter GLM, the input to the CCLM process, i.e., the downsampled luminance sample L, is replaced by the luminance sample gradient G. Other parts of CCLM (such as parameter derivation and linear transformation of predicted samples) remain unchanged.

[0053] In the three-parameter GLM, chromaticity samples can be predicted based on the gradient of luminance samples and downsampled luminance values ​​(using different parameters). The model parameters of the three-parameter GLM are derived from the immediate 6 rows and columns of samples using the same LDL decomposition-based MSE minimization method as CCCM.

[0054] E. Inter-frame CCCM InterCCCM (InterCCCM) is used when the CU employs inter-frame prediction or intra-block copy (IBC) to predict chromaticity samples from reconstructed luminance samples. Figure 5 The inter-frame CCCM process at the decoder end is demonstrated. The cross-component filter is derived from the prediction blocks of luma and chroma. The derived filter is applied to the reconstructed luma block and mixed with the chroma prediction block to generate the final chroma prediction block. During the mixing process, the filtered reconstructed luma block uses a mixing weight of 0.75, and the chroma prediction block uses a mixing weight of 0.25.

[0055] An 8-tap filter with 6 luminance sample input taps, one nonlinear term, and one bias term can be used to obtain the predicted chromaticity value according to the following formula: predChromaVal=c0L0+c1L1+c2L2+c3L3+c4L4+c5L5 + c6nonlinear((L0+L3+1)>>1) + c7B Spatial luminance samples (L0,…,L5) are selected from the luminance grid, choosing the six luminance samples closest to the chrominance position C without requiring downsampling. "nonlinear" refers to the nonlinear operator of CCCM, and B is the bias. Filter coefficients can be derived using Gaussian elimination without division; the required offset is applied to the samples before filter derivation. The offset from Gaussian elimination without division is obtained by averaging four points from the luminance and chrominance prediction blocks, corresponding to the top-left, top-right, bottom-left, and bottom-right corners of the block. A maximum of 256 chrominance samples are used in the filter coefficient derivation.

[0056] III. Cross-Component Prediction (CCP) Merging Mode A. Cross-Component Prediction (CCP) Merging Mode Cross-component prediction merging mode is also known as non-local CCP mode. For chroma coding, a signaling flag can be transmitted to indicate whether the current block is encoded using a CCP mode (including CCLM, CCCM, GLM, and their variants) or a non-CCP mode (traditional chroma intra-prediction mode, fusion of chroma intra-prediction modes). If CCP mode is selected, a signaling flag is also transmitted to indicate how the CCP type and parameters are derived, i.e., obtained from the CCP merging list or transmitted / derived on the spot. In some embodiments, a signaling flag is transmitted to indicate whether CCP merging mode is applied. If CCP merging mode is applied, an index is transmitted to indicate which candidate model is used for the current block. Cross-component model (CCM) information of neighboring blocks can be inherited / referenced by the current chroma block.

[0057] For CCP merging mode, the CCP merge candidate list (or cross-component merge candidate list) is constructed from spatially proximate, temporally, spatially non-proximate, history-based, or shift-time candidates. After including these candidates, a default model is further included to fill any remaining empty slots in the merge list. Pruning is performed to remove redundant CCP models from the list. After the list is constructed, the CCP models in the list are reordered according to their SAD cost, which is obtained from the neighboring templates of the current block. Cross-component model (CCM) information of neighboring blocks can be inherited / referenced by the current chroma block.

[0058] Spatial proximity and non-proximity candidates: The location and inclusion order of spatial proximity and non-proximity candidates are the same as those defined for regular inter-frame merge prediction candidates.

[0059] Time and Shift Time Candidates: Time candidates are selected from the collocated picture. The position and inclusion order of time candidates are defined the same as those of regular inter-frame merge prediction candidates. Shift time candidates are also selected from the collocated picture. The position of the time candidate is shifted by a selected motion vector derived from the motion vectors of neighboring blocks.

[0060] History-based candidate selection: A history-based table is maintained to contain recently used cross-component prediction (CCP) models, and this table is reset at the beginning of each coding tree unit (CTU) row. If the current list is not full after including spatially neighboring and non-adjacent candidates, CCP models from the history-based table are added to the list.

[0061] Default Candidate: CCLM candidates with default scaling parameters are considered only if the list is not full after including spatially adjacent, spatially non-adjacent, or history-based candidates. If there are no candidates with a single-model CCLM mode in the current list, the default scaling parameters are {0, 1 / 8, -1 / 8, 2 / 8, -2 / 8, 3 / 8, -3 / 8, 4 / 8, -4 / 8, 5 / 8, -5 / 8, 6 / 8}. Otherwise, the default scaling parameters are {0, the scaling parameter of the first CCLM candidate + {1 / 8, -1 / 8, 2 / 8, -2 / 8, 3 / 8, -3 / 8, 4 / 8, -4 / 8, 5 / 8, -5 / 8, 6 / 8}}.

[0062] B. CCP merging of chroma inter-frame blocks In some embodiments, the cross-component prediction merging mode described above can be extended to chroma inter-frame coding. CCP models, including CCLM, MMLM, CCCM, GLM, chroma fusion, CCP merging mode, and inter-frame CCCM, can be stored and inherited for subsequent chroma intra-frame and inter-frame coding.

[0063] Similar to CCP merging for chroma intra-blocks, the line signaling transmits a flag indicating whether the chroma inter-block is encoded using this mode. If CCP merging mode is used, the CCP merging list is constructed similarly to that for chroma intra-blocks, except that the CCP merging list also includes additional shift-temporal candidates and in-situ derivation candidates. The additional shift-temporal candidates are derived from co-located images. The positions of these candidates can be the same as those defined for regular inter-frame merge prediction candidates, with the shift obtained from the motion vector of the current block. In-situ derivation candidates are used only for low-latency images and are obtained from neighboring reconstructed samples of the current block. In some embodiments, at most one in-situ derivation candidate is added to the CCP merging list, including single / multi-model CCCM and single / multi-model CCLM.

[0064] After constructing the CCP merging (candidate) list, the candidate with the lowest template cost is selected for the chroma inter-frame block. Then, the prediction method for the chroma inter-frame block is the same as that for inter-frame CCCM. That is, motion-compensated prediction samples are mixed with cross-component prediction samples to form the final prediction.

[0065] C. Construct a candidate list for CCP merging. In some embodiments, the CCP merge candidate list is constructed by adding candidates in a predefined order until a maximum number of candidates is reached. The added candidates may include, but are not limited to, all or some of the candidates described in the CCP merge patterns above. For example, the following candidates are added to the candidate list in the following order: spatially proximate candidates, temporally proximate candidates, non-proximate proximate candidates, and history-based candidates. Another example is that the candidate list may contain the same candidates as in the foregoing examples, but the candidates are added to the list in a different order.

[0066] In some embodiments, if all predefined neighboring and history-based candidates have been added but the maximum number of candidates has not been reached, some default candidates are added to the CCP merge candidate list until the maximum number of candidates is reached.

[0067] For CCP merging mode, candidates in the list can be reordered to reduce the syntax overhead when transmitting the selected candidate index in signaling. The reordering rules can rely on the encoding information or model error of neighboring blocks. For example, if the upper or left neighboring block is encoded using MMLM, MMLM candidates in the list can be moved to the front of the current list. Similarly, if the upper or left neighboring block is encoded using single-model LM (i.e., CCLM) or CCCM, single-model LM or CCCM candidates in the list can be moved to the front of the current list. Likewise, if the upper or left neighboring block uses GLM, GLM-related candidates in the list can be moved to the front of the current list.

[0068] In some embodiments, the reordering rule is based on model error (i.e. template cost), which is achieved by applying a candidate model to the neighboring templates of the current block and then comparing the resulting error with the reconstructed samples of the neighboring templates. Figure 6 This illustrates the neighbor template used to calculate model error. As shown, the size of the neighbor template above the current block is w. a × h a The size of the template adjacent to the left of the current block is w. b × h b .

[0069] Suppose there are K models in the current candidate list, and β k These are the final scaling and offset parameters obtained after inheriting candidate k. The model error of candidate k based on the upper neighboring template is: in, and These are the luminance reconstruction sample (e.g., after downsampling or applying GLM mode) and the chrominance reconstruction sample at the upper template position (i, j), respectively, where 0 ≤ i <w a And 0 ≤ j <h a Similarly, the model error of candidate k through the left-side neighboring template is: in, and These are the luminance reconstructed sample (e.g., after downsampling or applying GLM mode) and the chrominance reconstructed sample at the left template position (m, n), respectively, and ≤ m. <w b And 0 ≤ n <h b Then the model error of candidate k is: After calculating the model errors for all candidates, a list of model errors can be obtained. Then, the candidate indices in the inherited candidate list can be reordered by sorting the model error list in ascending order.

[0070] In some embodiments, if candidate k is predicted using CCCM, then and Defined as Where c0 k c1 k c2 k c3 k c4 k c5 k c6 k These are the final filter coefficients after inheriting candidate k. P and B are the nonlinear term and bias term, respectively. In some embodiments, if the aforementioned neighbor template is unavailable, then Similarly, if the adjacent template on the left is unavailable, then If neither template is available, the candidate index reordering method using model error should not be applied.

[0071] In some embodiments, after candidates are reordered according to template cost, redundancy of candidates can be further checked. If the template cost difference between a candidate and its predecessor in the list is less than or equal to a threshold, the candidate is considered redundant. If a candidate is redundant, it can be removed from the list or moved to the end of the list.

[0072] D. Multi-hypothesis mixing of cross-component model merging In some embodiments, the final prediction of the current block can be blended with N hypotheses using predefined weights, where N is greater than or equal to 2. Each hypothesis is a current block prediction generated based on a mode. This mode can be any intra-frame or inter-frame mode. At least one hypothesis is generated based on a cross-component prediction merging mode (CCP merging mode). CCP merging mode refers to a cross-component model that is inherited from spatially, historically, or temporally neighboring blocks / locations. A flag can be transmitted at the block level via signaling to indicate whether a blending mode is applied. For example, this flag can be signaled at the coding unit (CU) level and / or prediction unit (PU) level and / or coding tree unit (CTU) level. Alternatively, the flag can be signaled at the color block (CB) level and / or prediction block (PB) level and / or coding tree block (CTB) level and / or transform unit / transform block (TU / TB) level and / or any predefined region level.

[0073] In some embodiments, the final prediction for the current block can be achieved by blending two hypotheses with predefined weights. The hypothesis generation mode can be any intra-frame or inter-frame mode. At least one hypothesis is generated based on the CCP merging mode. A signaling flag is transmitted at the block level to indicate whether the blending mode is applied.

[0074] In some embodiments, the final prediction for the current block may be a mixture of two hypotheses, H1 and H2, both generated based on the CCP merging mode. A flag is signaled at the block level to indicate whether the merging mode is applied. In some embodiments, when the flag indicates that the merging mode is disabled, the original syntax for signaling / parsing the intra-prediction mode for the current block is followed. When the flag indicates that the merging mode is enabled, signaling is performed at the encoder, and the following syntax related to the modes and / or weights for generating H1 and H2 is determined at the decoder.

[0075] In some embodiments, both modes for generating H1 and H2 are selected from a cross-component merging candidate list. The inherited candidate index of the selected mode can be explicitly or implicitly indicated. For example, the candidates in the list can be reordered using the methods described above. The candidate index can be explicitly signaled. Alternatively, if only the first k candidates (k ≥ 3) in the list can be selected, the candidate index can be explicitly signaled. Another example is that the first two candidates are implicitly selected. Yet another example is that an index is explicitly signaled to indicate the candidate index for the first mode, and the candidate index for the second mode is the signaled index + k, where k can be 1, 2, 3, 4, or 5.

[0076] In some embodiments, both patterns for generating H1 and H2 are selected from a cross-component merging candidate list. The inheritance candidate index of the selected pattern is implicitly determined. The two candidates with the lowest template matching cost or boundary matching cost in the list can be implicitly selected.

[0077] In some embodiments, both modes generating H1 and H2 are selected from a cross-component merging candidate list. The candidates in the cross-component merging candidate list are categorized. Modes can be categorized according to several classification criteria, such as whether the model is linear or convolutional, whether the model is a single model or multiple models, the sample locations used to derive the model, the filter shape of the model, whether the model uses gradient brightness input, the gradient filter used, whether the brightness input of the model is downsampled, the downsampled filter used, and whether the model contains nonlinear terms. The selection of the two modes is based on category information. For example, the two selected modes are not allowed to belong to certain categories. More specifically, neither of the two selected modes can be a linear model (CCLM or MMLM). Alternatively, the two selected modes must belong to certain categories. More specifically, the two selected modes must be modes that use non-downsampled brightness as input (e.g., interCCCM and CCCM using non-downsampled brightness samples). Furthermore, some modes cannot be selected unless one of the two modes is a different specific mode. More specifically, linear models (e.g., CCLM and MMLM) cannot be selected unless the alternative mode is one that uses non-downsampled luminance as input (e.g., interCCCM and CCCM using non-downsampled luminance samples). (For example, the CCLM-interCCCM combination is allowed, but the CCLM-MMLM combination is not.) In some embodiments, the weights of H1 and H2 are selected from a set of weight parameters. An index can be explicitly signaled to indicate the selected weights. For example, the weight set (wh1, wh2) can also be (3, 5), (5, 3), (-2, 10), (10, -2), (4, 4), or any subset of the weights described above.

[0078] In some embodiments, the weights can be implicitly determined based on the pattern information of neighboring blocks. For example, if H1 is a linear model (such as CCLM and MMLM) and H2 is a convolutional model (such as CCCM), and more neighboring blocks use the convolutional model pattern, then H2 has a larger mixed weight.

[0079] In some implementations, the weights can be implicitly determined based on template matching cost or boundary matching cost. Weights associated with the minimum template matching cost or boundary matching cost are selected. Methods for calculating template matching cost and boundary matching cost have been described in the preceding paragraphs. For example, if template matching cost is used, for each weight in the weight parameter set, a predicted sample value within the chroma template of the current chroma block is generated by calculating wh1H1 + wh2H2. The template cost is the error between the reference data (the reconstructed chroma sample on the current block template) and the predicted sample within the chroma template.

[0080] In some implementations, the weight parameters (wh1, wh2) are implicitly derived based on template matching cost (TIMD cost) or boundary matching cost. Methods for calculating template matching cost and boundary matching cost have been described in the preceding paragraphs. Template matching cost and boundary matching cost can be determined at the decoder without signaling transmission. For example, if the pattern used to generate H1 has a high TIMD cost, H1 uses a smaller weight when blending. If the pattern used to generate H2 has a high TIMD cost, H2 uses a smaller weight when blending. If the cost of the predicted result is less than the assumed cost, α uses a smaller weight. Another example follows: w h1 = TIMD_cost h2 / (TIMD_cost h1 + TIMD_cost h2 ) w h2 = TIMD_cost h1 / (TIMD_cost h1 + TIMD_cost h2 ) In some implementations, the weighting parameter (w) h1 w h2 The CCCM parameters can be implicitly derived using a method similar to that used in Section II.C above. Let P final = w h1 H1+ w h2 H2+β, where β is an offset value. The weighting parameter α can be minimized by the predicted chromaticity sample. The mean square error (MSE) between the reconstructed chromaticity samples in the reference region is derived, as described and illustrated above regarding CCCM.

[0081] In some implementations, when combining two cross-component models, a final cross-component model can be generated by combining / weighting the parameters of the two models. If a term appears only in one model, the parameter associated with that term in the other model is considered zero. For example, if you want to combine a 7-tap CCCM model with a 3-parameter GLM model, let the 7-tap CCCM model be... P CCCM = c0×C+c1×N+c2×S+c3×E+c4×W+c5×P+c6×B The 3-parameter GLM model is Let the combined weights be (w_h1, w_h2). Since the C term in the 7-tap CCCM model is equivalent to the L term in the 3-parameter GLM model, and the c_6×B in the 7-tap CCCM model is equivalent to the L term in the 3-parameter GLM model... All are bias terms, and the final model combination is as follows (essentially an 8-tap model): In some implementations, when generating a final cross-component model from two cross-component models, some parameters are obtained by combining / weighting the parameters of the two cross-component models, while the remaining parameters can be re-derived. For example, if a 7-tap CCCM model is to be combined with a 3-parameter GLM model to generate an 8-tap model, let the 7-tap CCCM model be P. CCCM =c0×C+c1×N+c2×S+c3×E+c4×W+c5×P+c6×B, this 3-parameter GLM model is The final model is then combined according to the following formula, where Calculate using the method described in the preceding paragraphs. (Bias term) is based on And the reconstructed sample values ​​of brightness and chromaticity from neighboring areas are re-derived. For example, By minimizing the predicted chromaticity sample P final The mean square error (MSE) between the reconstructed chromaticity samples and the reference region is used to derive the result, as described above. Figure 3 As shown. P final Calculate as follows: E. Inheriting multiple cross-component models In some implementations, two cross-component models are combined into a final model by weighted averaging of their respective model parameters. These two cross-component models are derived from an upper spatial neighbor candidate and a left spatial neighbor candidate, respectively. An upper spatial neighbor candidate is a candidate whose vertical position is less than or equal to the top boundary of the current block. A left spatial neighbor candidate is a candidate whose horizontal position is less than or equal to the left boundary of the current block. The weighting factor α is determined based on the horizontal and vertical spatial positions within the current block. For example, if two candidate predictions (denoted as p) are combined... above and p left If the current block is merged, the final prediction for the current block at position (x, y) is: ,in .

[0082] In some implementations, the candidate in the upper space proximity is the first candidate in the list that satisfies the condition that its vertical position is less than or equal to the position of the top boundary of the current block. The candidate in the left space proximity is the first candidate in the list that satisfies the condition that its horizontal position is less than or equal to the position of the left boundary of the current block.

[0083] F. Mixed Candidates In some implementations, the final prediction for the current block can be a combination of predictions from two or more (conventional) CCP merging patterns (which can be two conventional candidates from the CCP merging candidate list). Such a combination can be considered a "hybrid candidate" or "fusion candidate".

[0084] In some implementations, the hybrid candidate is a candidate that incorporates information from multiple CCP merging candidates. When a hybrid candidate is selected, the final prediction is a mixture of predictions (hypotheses) generated from multiple CCP merging candidates. The multiple hypotheses generated from multiple CCP merging candidates can be mixed using the methods described in Sections III.D and III.E above.

[0085] In some implementations, each merging candidate contains information on two (regular) CCP merging candidates. When a merging candidate is selected, the final prediction is a mixture of predictions generated based on these two CCP merging candidates. In some implementations, the two predictions generated based on these two CCP merging candidates are combined using an equal-weighted approach to form the final prediction. In some implementations, the candidate indices of the two CCP merging candidates are information stored in the merging candidate.

[0086] In some implementations, when a block is encoded in a CCP merging mode and a hybrid candidate is selected, one of the two CCP merging candidates indicated by the hybrid candidate is selected and stored in the block for later reference. This selection can be based on the cost of the two CCP merging candidates. For example, the selected CCP merging candidate is the one with the lower cost of the two CCP merging modes. The cost can be the template matching cost or boundary matching cost for reordering the candidate list as described in Section III.C.

[0087] In some implementations, when a block is encoded in cross-component prediction (CCP) merging mode and a merging candidate is selected, one of the two CCP merging candidates indicated by that merging candidate is chosen and stored in the block for later reference. This selection is based on the modes of the two CCP merging candidates. For example, convolutional modes (such as CCCM and its variants) have a higher priority than linear modes (such as CCLM and MMLM). As another example, modes using non-downsampled brightness values ​​as input (such as interCCCM) have a higher priority than modes using downsampled brightness values ​​as input.

[0088] In some implementations, hybrid candidates are also inserted into the candidate list of the CCP merge mode. In some implementations, hybrid candidates are inserted into the candidate list after other candidates (such as candidates inherited from neighboring locations / blocks, e.g., spatial candidates, temporal candidates, non-neighboring candidates, or candidates inherited from the history table) and before the default candidate. Figure 7 A CCP merge candidate list of 700 is shown, where the merge candidate is inserted after the inherited candidate and before the default candidate.

[0089] In some implementations, the number of hybrid candidates inserted into the CCP merge candidate list is limited. For example, a maximum of two hybrid candidates can be inserted. In some implementations, assuming there is k remaining space in the CCP merge candidate list before the hybrid candidates are inserted, if k is greater than a predefined value m, then k hybrid candidates can be inserted, and m spaces are reserved for inserting other candidates (such as the default candidate). If k is less than m, then no hybrid candidates are inserted.

[0090] In some implementations, the two CCP merging candidates indicated by the hybrid candidate are also in the same candidate list as the hybrid candidate. Figure 7In the example, each hybrid candidate (candidates 5, 6, and 7) in list 700 indicates two or more CCP merge candidates in the same list (hybrid candidate 5 indicates candidates 0 and 1 in the list; hybrid candidate 6 indicates candidates 0 and 2 in the list; hybrid candidate 7 indicates candidates 3 and 9 in the list). In some implementations, the two CCP merge candidates indicated by the hybrid candidate must be candidates inserted immediately before the hybrid candidate. In some implementations, the two CCP merge candidates indicated by the hybrid candidate can be any candidates in the (original) candidate list other than the hybrid candidate. In some implementations, the two CCP merge candidates indicated by the hybrid candidate must be two inherited candidates, for example, these two CCP merge candidates are inherited from neighboring blocks / locations or history tables. In some implementations, the two CCP merge candidates indicated by the hybrid candidate can be any CCP merge candidates in the (original) candidate list other than the hybrid candidate and the default candidate.

[0091] In some implementations, the entire candidate list can be reordered after the mixed candidate is inserted. In some implementations, candidates that precede the mixed candidate are reordered before the mixed candidate is inserted. This reordering can be based on the methods described in Section III.C above. In some implementations, candidates that precede the mixed candidate and candidates that will be inserted after the mixed candidate are reordered together before the mixed candidate is inserted. For example, inherited candidates are inserted before the mixed candidate, and default candidates are inserted after the mixed candidate. When the number of mixed candidates is determined (perhaps due to a limitation on the number of mixed candidates), the number of default candidates that will be inserted after the mixed candidate can be calculated. Where the mixed candidate may indicate that the default candidate is indicated as one of its two CCP merge candidates, inherited candidates and default candidates can be reordered together in the candidate list before the mixed candidate is inserted.

[0092] Figure 8A -C indicates a reordering of the CCP merge candidate list 800 with mixed candidates. Candidate list 800 will include inherited candidates, mixed candidates, and default candidates. Figure 8A This shows the result of reordering the entire candidate list after all candidates (including mixed candidates) are inserted. Figure 8B This shows the case where the candidate insertion list is inherited and reordered before the mixed candidate insertion. Figure 8C This shows the situation where, before the mixed candidate is inserted, the inherited candidate and the default candidate are inserted into the candidate list and reordered together with the space reserved for the mixed candidate.

[0093] In some implementations, the insertion order of the hybrid candidates can be determined as follows: (1) Assume that the CCP merging candidates that the hybrid candidates are allowed to indicate are cand_1, cand_2, cand_3, ..., cand_N (numbers 1, 2, 3, ..., N indicate the candidate order). These candidates can be sorted according to certain criteria, such as based on their insertion order, based on their template matching cost, or consistent with their order in the original candidate list.

[0094] (2) Mixed candidates can be inserted into the candidate list according to the order of the CCP merged candidate pairs they indicate, for example: (1, 2), (1, 3), (1, 4), ..., (1, N), (2, 3), (2, 4), ..., (2, N), (3, 4), ..., (N-1, N); another example: (1, 2), (1, 3), (2, 3), (1, 4), (2, 4), (3, 4), (1, 5), ..., (1, N), ..., (N-1, N); yet another example: (1, 2), (1, 3), (2, 3), (1, 4), (2, 4), (1, 5), (3, 4), (2, 5), (1, 6), ..., (N-1, N).

[0095] In some implementations, the insertion order of hybrid candidates is determined as follows: assuming the hybrid candidates can indicate CCP merging candidates cand_1, cand_2, cand_3, ..., cand_N, then N*(N-1) / 2 hybrid candidates are created, each corresponding to a different combination of two CCP merging candidates. All hybrid candidates are sorted according to a cost metric. For example, the cost metric could be template matching cost, and the insertion order is determined by cost. Hybrid candidates with lower costs are inserted first.

[0096] In some implementations, a second candidate list is constructed, and hybrid candidates are inserted into the second list instead of the original candidate list (described in Section III.C). In some implementations, a signaling flag is transmitted to indicate which candidate list to use. In some implementations, the second candidate list contains only hybrid candidates. In some implementations, if no hybrid candidates can be created (e.g., hybrid candidates can only mix space candidates, but no space candidates are available for the current block), the flag can be skipped, and the original list can be used. In some implementations, the second candidate list is reordered. For example, the second candidate list is reordered based on template cost. In some implementations, the second list is constructed after the original candidate list has been reordered.

[0097] In some implementations, a signaling candidate index is used to indicate the position of the selected hybrid candidate in the second candidate list. In other implementations, the candidate index is implicitly derived. For example, the candidate index may be implicitly derived to be 0 to indicate the first candidate in the selection list, since the second candidate list is reordered according to template cost, and the first candidate is the one with the lowest template cost.

[0098] In some implementations, the size of the second candidate list differs from the size of the original candidate list. In other implementations, the size of the second candidate list is the same as the size of the original candidate list. In some implementations, the two CCP merge candidates indicated by the mixed candidate in the second candidate list can be CCP merge candidates from the original list. In some implementations, the two CCP merge candidates indicated by the mixed candidate in the second candidate list can be CCP merge candidates from the original list other than the default candidate.

[0099] In some implementations, only specific combinations of two CCP merging candidates are allowed to form a merging candidate. For example, in some implementations, linear models (such as CCLM and MMLM) cannot be selected as CCP merging candidates for merging. As another example, in some implementations, the two CCP merging candidates must use non-downsampled luminance samples as input (e.g., InterCCCM and CCCM use non-downsampled luminance samples). As yet another example, in some implementations, linear models (such as CCLM and MMLM) cannot be selected as CCP merging candidates for merging unless they are paired with a model that uses non-downsampled luminance samples as input (e.g., interCCCM and CCCM use non-downsampled luminance samples; CCLM-interCCCM pairing is allowed, but CCLM-MMLM pairing is not).

[0100] Any of the methods proposed above can be implemented in the encoder and / or decoder. For example, any of the proposed methods can be implemented in the inter-frame / intra-frame / prediction module of the encoder and / or the inter-frame / intra-frame / prediction module of the decoder. Alternatively, any of the proposed methods can be implemented as a circuit coupled to the inter-frame / intra-frame / prediction module of the encoder and / or the inter-frame / intra-frame / prediction module of the decoder to provide the required information to the inter-frame / intra-frame / prediction module.

[0101] IV. Example Video Encoder Figure 9An example video encoder 900 capable of hybrid cross-component prediction is shown. As shown, the video encoder 900 receives an input video signal from a video source 905 and encodes it into a bitstream 995. The video encoder 900 has multiple components or modules for encoding the signal from the video source 905, including at least some components selected from a transform module 910, a quantization module 911, an inverse quantization module 914, an inverse transform module 915, an intra-frame image estimation module 924, an intra-frame prediction module 925, a motion compensation module 930, a motion estimation module 935, a loop filter 945, a reconstructed image buffer 950, an MV buffer 965, an MV prediction module 975, and an entropy encoder 990. The motion compensation module 930 and the motion estimation module 935 belong to the inter-frame prediction module 940. The intra-frame prediction module 925 and the intra-frame prediction estimation module 924 belong to the current image prediction module 920, which uses the reconstructed samples of the current image as reference samples for predicting the current block.

[0102] In some embodiments, modules 910–990 are software instruction modules executed by one or more processing units (e.g., processors) and running on a computing or electronic device. In some embodiments, modules 910–990 are hardware circuit modules implemented by one or more integrated circuits (ICs) of an electronic device. Although modules 910–990 are illustrated as independent modules, some of these modules may be combined into a single module.

[0103] Video source 905 provides an uncompressed raw video signal, presenting pixel data for each video frame. Subtractor 908 calculates the difference between the raw video pixel data from video source 905 and the predicted pixel data 913 provided by motion compensation module 930 or intra-frame prediction module 925, using this as the prediction residual 909. Transform module 910 converts this difference (or residual pixel data or residual signal 909) into transform coefficients (e.g., by performing a discrete cosine transform, DCT). Quantization module 911 quantizes the transform coefficients into quantized data (or quantization coefficients) 912, which is then encoded into the bitstream 995 by entropy encoder 990.

[0104] The inverse quantization module 914 dequantizes the quantized data (or quantization coefficients) 912 to obtain transform coefficients 918. The inverse transform module 915 performs an inverse transform on the transform coefficients 918 to generate a reconstruction residual 919. The reconstruction residual 919 is added to the predicted pixel data 913 to generate reconstructed pixel data 917. In some embodiments, the reconstructed pixel data 917 is temporarily stored in a line buffer 927 (or an intra-frame prediction buffer) for intra-frame image prediction and spatial MV prediction. The reconstructed pixels are filtered by a loop filter 945 and then stored in a reconstructed image buffer 950. In some embodiments, the reconstructed image buffer 950 is external memory to the video encoder 900. In some embodiments, the reconstructed image buffer 950 is internal memory to the video encoder 900.

[0105] Intra-frame image estimation module 924 performs intra-frame prediction based on reconstructed pixel data 917 to generate intra-frame prediction data. The intra-frame prediction data is provided to entropy encoder 990 for encoding into bitstream 995. The intra-frame prediction data is also used by intra-frame prediction module 925 to generate predicted pixel data 913.

[0106] The motion estimation module 935 performs inter-frame prediction by generating MVs (Motion Videos) that reference pixel data from previously decoded frames stored in the reconstructed image buffer 950. These MVs are then provided to the motion compensation module 930 to generate predicted pixel data.

[0107] The video encoder 900 does not encode the complete actual MV into the bitstream. Instead, it uses MV prediction to generate a predicted MV and encodes the difference between the MV used for motion compensation and the predicted MV as residual motion data and stores it in the bitstream 995.

[0108] The MV prediction module 975 generates a predicted MV based on a reference MV (i.e., a motion-compensated MV used for motion compensation) generated for encoding a previous video frame. The MV prediction module 975 retrieves the reference MV of the previous video frame from the MV buffer 965. The video encoder 900 stores the MV generated for the current video frame in the MV buffer 965 as a reference MV for generating the predicted MV.

[0109] The MV prediction module 975 generates a predicted MV using a reference MV. The predicted MV can be calculated through spatial MV prediction or temporal MV prediction. The difference (residual motion data) between the predicted MV and the motion compensation MV (MC MV) of the current frame is encoded into the bitstream 995 by the entropy encoder 990.

[0110] The entropy encoder 990 uses entropy coding techniques (such as context-adaptive binary arithmetic coding (CABAC) or Huffman coding) to encode various parameters and data into the bitstream 995. The entropy encoder 990 encodes various header elements, flags, quantization transform coefficients 912, and residual motion data as syntax elements into the bitstream 995. The bitstream 995 is then stored in a storage device or transmitted to the decoder via a communication medium such as a network.

[0111] Loop filter 945 filters or smooths the reconstructed pixel data 917 to reduce coded artifacts, especially at pixel block boundaries. In some embodiments, the filtering or smoothing operations performed by loop filter 945 include deblocking filter (DBF), sample adaptive offset (SAO), and / or adaptive loop filter (ALF). In some embodiments, luma mapping chroma scaling (LMCS) is performed before the loop filter.

[0112] Figure 10 The relevant portions of the video encoder 900 that implements cross-component predictive blending are illustrated. As shown, the predictive blending module 1050 generates a blended prediction 1060 for the chroma component (Cr or Cb) to be incorporated into the predicted pixel data 913. The blended prediction 1060 is a weighted sum of two or more chroma predictions. These two or more chroma predictions are derived from at least two cross-component predictive modules 1041 and 1042. Each cross-component predictive module generates a chroma prediction based on (i) reconstructed luma samples (generated by the intra-prediction module 920 in combination with luma residuals, or provided by the line buffer 927) and (ii) cross-component predictive (CCP) parameters of a CCP merging candidate indicated by a blending candidate. In some embodiments, the reconstructed luma samples are reconstructed using intra-prediction modes of samples that are neighboring or adjacent to the current block, as described in Section I (rather than based on a reference block identified by motion vectors).

[0113] Based on the candidate selection index, the candidate acquisition module 1030 accesses the CCP merging candidate list 1010 to provide one or more CCP parameter sets (e.g., linear model or convolution parameters) for cross-component prediction of the current block. The CCP merging candidate list 1010 may include one or more regular candidates 1011 and one or more hybrid candidates 1012. When a regular CCP merging candidate 1011 is selected, the candidate acquisition module 1030 retrieves a set of CCP parameters from the CCP parameter memory 1015 and provides them to the cross-component prediction module 1041 without prediction merging. When a hybrid candidate 1012 indicating that two or more CCP merging candidates need to be merged is selected, the candidate acquisition module 1030 retrieves the parameters of the two or more CCP merging candidates from the CCP parameter memory 1015 and provides them to the cross-component prediction modules 1041 and 1042.

[0114] CCP merge mode manager 1020 manages the CCP merge candidate list, including CCP parameter memory 1015. CCP merge mode manager 1020 can populate candidate list 1010 with various types of regular candidates (e.g., spatial proximity, temporal proximity, history, default, etc.) and one or more mixed candidates, each mixed candidate pointing to two or more regular candidates for merging. Regular candidates can be candidates inherited from predefined locations or history tables. Candidate list 1010 can be organized as a single list, storing both regular CCP merge mode candidates 1011 and mixed candidates 1012. Candidate list 1010 can also be organized as a first list (“original list”) storing only regular CCP merge mode candidates 1011 and a second list storing only mixed candidates 1012. (Therefore, for some embodiments, it can be considered that two or more CCP merge candidate lists are generated and used for the current block.) In the case of generating two or more CCP merge candidate lists, a signaling flag is transmitted to select which of the two or more CCP merge candidate lists is used for the current block. When only one CCP merger candidate list is generated, the method for inserting candidates into the CCP merger candidate list 1010 and the reordering of the list can be found in [reference needed]. Figure 7 and Figure 8A -C is used for description.

[0115] The CCP merging mode manager 1020 can derive new CCP linear or convolutional models and store them in the CCP parameter memory 1015. The CCP merging mode manager 1020 can update the contents of the CCP parameter memory 1015, providing parameter information for CCP merging candidates in the candidate list. The CCP merging mode manager 1020 can also manage the transfer of CCP models between images or blocks. For example, the CCP merging mode manager 1020 can store a set of CCP parameters used by one of two or more cross-component prediction modules (1041 and 1042) in memory 1015 for the current block, so that these CCP parameters can be referenced or inherited by subsequent blocks. The set of CCP parameters used for future reference can be selected based on its cross-component prediction type (linear vs. convolutional, downsampling vs. non-downsampling, etc.) or its cost.

[0116] Figure 11 A process 1100 for encoding a pixel block using one or more CCP merging candidate lists containing merging candidates is conceptually illustrated. In some embodiments, one or more processing units (e.g., processors) of a computing device implementing encoder 900 execute process 1100 by executing instructions stored in a computer-readable medium. In some embodiments, an electronic device implementing encoder 900 executes process 1100.

[0117] The encoder receives data of the current pixel block in the current image to be encoded at block 1110. The current block includes a first color block (e.g., a luma component block) and a second color block (e.g., a chroma component block, Cr or Cb).

[0118] At block 1120, the encoder constructs one or more cross-component prediction (CCP) merging candidate lists for the current block. Each candidate list includes one or more regular candidates and one or more hybrid candidates. Each regular candidate is associated with a set of CCP parameters. Each hybrid candidate indicates two or more regular candidates. In some embodiments, two or more CCP merging candidate lists are constructed for the current block, comprising one or more regular candidates and one or more hybrid candidates, with the selected candidate chosen from one of these two or more CCP merging candidate lists.

[0119] In some embodiments, the hybrid candidate indicates the two or more regular candidates by providing an index assigned to the one or more CCP merge candidate lists. In some embodiments, the two or more regular candidates indicated by the hybrid candidate are restricted to not being linear models. In some embodiments, the two or more regular candidates indicated by the hybrid candidate are restricted to using non-downsampled luminance samples as input, or restricted to not being linear models, unless at least one of the indicated regular candidates uses non-downsampled luminance samples as input.

[0120] In some embodiments, the two or more regular candidates indicated by the hybrid candidate are limited to candidates inherited from the position of the adjacent current block or from the history table. In some embodiments, the two or more regular candidates indicated by the hybrid candidate are limited to candidates in one of the one or more CCP merge candidate lists. In some embodiments, the two or more regular candidates indicated by the hybrid candidate are limited to those excluding the default candidate with default parameters.

[0121] In some embodiments, hybrid candidates are inserted into one of the one or more CCP merge candidate lists after the non-default regular candidates and before the default candidates. In some embodiments, the regular candidates in the one or more CCP merge candidate lists are reordered before the hybrid candidates are inserted into the list. In some embodiments, the one or more CCP merge candidate lists are reordered as a whole after the hybrid candidates are inserted into the list. In some embodiments, each hybrid candidate is inserted into one of the one or more CCP merge candidate lists in an order determined based on the cost of the hybrid candidates.

[0122] In some embodiments, two CCP merging candidate lists are constructed and organized: a first list containing only regular candidates and a second list containing only hybrid candidates. The second list is constructed after the first list is constructed and reordered according to the (template) cost of the candidates in the first list. The first and second lists may have the same size or different sizes. The two or more regular candidates indicated by each hybrid candidate in the second list may be limited to candidates in the first list. Hybrid candidates in the second list can indicate their two or more regular candidates by using an index from the first list. In some embodiments, hybrid candidates in the second list are limited to indicating only regular candidates that are not default candidates in the first list. In some embodiments, hybrid candidates in the second list are reordered according to the cost of the candidates in the second list, and the hybrid candidate with the lowest cost in the second list may be implicitly selected without signaling transmission.

[0123] In some embodiments, each hybrid candidate is inserted into one of the one or more CCP merge candidate lists according to an order determined based on the indices of the two or more regular candidates indicated by the hybrid candidate. The indices of the regular candidates may be determined for the CCP merge candidate list based on the cost of the candidates.

[0124] At block 1130, the encoder selects a candidate from the one or more CCP merging candidate lists. In some embodiments, a signaling flag is transmitted to indicate whether the candidate is selected from a first list or a second list. (If merging prediction cannot be used, this flag may be skipped and no signaling transmission is performed.) In some embodiments, the encoder selects a merging candidate by transmitting a candidate index to indicate the position of the merging candidate in the second list.

[0125] At block 1135, the encoder determines whether the selected candidate is a regular candidate or a mixed candidate. If the selected candidate is a regular candidate, the process proceeds to block 1140. If the selected candidate is a mixed candidate, the process proceeds to block 1160.

[0126] At block 1140, the encoder generates a cross-component prediction by applying the CCP parameter set of the selected general candidates to the first color patch reconstructed via intra-frame prediction. The encoder uses this cross-component prediction to encode the current block at block 1150.

[0127] At block 1160, the encoder generates a blending prediction for the second color block based on two or more cross-component predictions generated by applying the two or more regular candidates indicated by the selected blending candidates. At block 1170, the encoder uses the generated blending prediction to encode the current block by producing a prediction residual.

[0128] In some embodiments, the two or more conventional candidates (or their CCP parameters) are used to generate the two or more cross-component predictions by applying intra-frame predictions of samples that are neighboring or adjacent to the current block to the reconstruction of the first color block. When generating the mixed predictions, the two or more cross-component predictions may be combined using equal weighting.

[0129] In some embodiments, the encoder also stores for the current block a set of CCP parameters associated with one of the two or more regular candidates indicated by the selected hybrid candidate, for reference in subsequent blocks. The set of CCP parameters for the selected candidate among the two or more regular candidates can be chosen based on its cost or its cross-component prediction type.

[0130] V. Example Video Decoder In some embodiments, the encoder may signal (or generate) one or more syntax elements in the bitstream so that the decoder can parse the one or more syntax elements from the bitstream.

[0131] In some embodiments, the encoder may transmit (or generate) one or more syntax elements in the bitstream so that the decoder may parse the one or more syntax elements from the bitstream.

[0132] Figure 12 An example of a video decoder 1200 capable of hybrid cross-component prediction is shown. As shown, the video decoder 1200 is an image or video decoding circuit that receives a bitstream 1295 and decodes its contents into pixel data for video frames to be displayed. The video decoder 1200 has multiple components or modules for decoding the bitstream 1295, including several components selected from an inverse quantization module 1214, an inverse transform module 1215, an intra-frame prediction module 1225, a motion compensation module 1230, an in-loop filter 1245, a decoded image buffer 1250, an MV buffer 1265, an MV prediction module 1275, and a parser 1290. The motion compensation module 1230 is part of the inter-frame prediction module 1240. The intra-frame prediction module 1225 is part of the current image prediction module 1220, which uses reconstructed samples of the current image as reference samples for predicting the current block.

[0133] In some embodiments, modules 1214–1290 are software instruction modules executed by one or more processing units (e.g., processors) of a computing device. In some embodiments, modules 1214–1290 are hardware circuit modules implemented by one or more integrated circuits (ICs) of an electronic device. Although modules 1214–1290 are illustrated as independent modules, some of these modules may be combined into a single module.

[0134] Parser 1290 (or entropy decoder) receives bitstream 1295 and performs preliminary parsing according to the syntax defined by the video codec or image codec standard. The parsed syntax elements include various header elements, flags, and quantization data (or quantization coefficients) 1212. Parser 1290 then uses entropy coding techniques such as context-adaptive binary arithmetic coding (CABAC) or Huffman coding to parse out the various syntax elements.

[0135] Inverse quantization module 1214 performs inverse quantization on quantized data (or quantization coefficients) 1212 to obtain transform coefficients, and inverse transform module 1215 performs inverse transform on transform coefficients 1218 to generate reconstructed residual signal 1219. Reconstructed residual signal 1219 is added to predicted pixel data 1213 from intra-frame prediction module 1225 or motion compensation module 1230 to generate decoded pixel data 1217. Decoded pixel data is filtered by in-loop filter 1245 and stored in decoded image buffer 1250. In some embodiments, decoded image buffer 1250 is external memory to video decoder 1200. In some embodiments, decoded image buffer 1250 is internal memory to video decoder 1200.

[0136] Intra-prediction module 1225 receives intra-prediction data from bitstream 1295 and generates predicted pixel data 1213 from decoded pixel data 1217 stored in decoded image buffer 1250 accordingly. In some embodiments, decoded pixel data 1217 is also stored in line buffer 1227 (or intra-prediction buffer) for intra-image prediction and spatial MV prediction.

[0137] In some embodiments, the contents of the decoded image buffer 1250 are used for display. The display device 1205 can directly retrieve the contents of the decoded image buffer 1250 for display, or retrieve the contents of the decoded image buffer into a display buffer. In some embodiments, the display device receives pixel values ​​from the decoded image buffer 1250 via a pixel transfer interface.

[0138] The motion compensation module 1230 generates predicted pixel data 1213 from the decoded pixel data 1217 stored in the decoded image buffer 1250 based on the motion compensation MV (MC MV). These motion compensation MVs are obtained by adding the residual motion data received from the bitstream 1295 to the predicted MV received from the MV prediction module 1275.

[0139] The MV prediction module 1275 generates a predicted MV based on a reference MV generated for decoding a previous video frame, such as a motion-compensated MV used to perform motion compensation. The MV prediction module 1275 retrieves the reference MV of the previous video frame from the MV buffer 1265. The video decoder 1200 stores the motion-compensated MV generated for decoding the current video frame in the MV buffer 1265 as a reference MV for generating the predicted MV.

[0140] The in-loop filter 1245 performs filtering or smoothing operations on the decoded pixel data 1217 to reduce coding artifacts, especially at pixel block boundaries. In some embodiments, the filtering or smoothing operations performed by the in-loop filter 1245 include a deblock filter (DBF), a sample adaptive offset (SAO), and / or an adaptive loop filter (ALF). In some embodiments, luma mapping chroma scaling (LMCS) is performed before the loop filter.

[0141] Figure 13 The relevant portions of the video decoder 1200 that implements cross-component predictive mixing are illustrated. As shown, the predictive mixing module 1350 generates a mixed prediction 1360 for the chroma component (Cr or Cb) to become part of the predicted pixel data 1213. The mixed prediction 1360 is a weighted sum of two or more chroma predictions. These two or more chroma predictions come from at least two cross-component predictive modules 1341 and 1342. Each cross-component predictive module generates a chroma prediction based on: (i) a reconstructed luma sample (generated by the intra-prediction module 1220 in combination with the luma residual, or provided by the line buffer 1227); and (ii) the CCP parameter of a CCP merging candidate indicated by a mixing candidate. In some embodiments, the reconstructed luma sample is reconstructed using the intra-prediction mode of samples that are neighboring or adjacent to the current block, as described in Section I (rather than based on a reference block identified by motion vectors).

[0142] The entropy decoder 1290 may indicate an index to select a candidate from the CCP merging candidate list 1310. Based on the indicated candidate selection index, the candidate extraction module 1330 accesses the CCP merging candidate list 1310 to provide one or more CCP parameter sets (e.g., linear model or convolution parameters) for cross-component prediction of the current block. The CCP merging candidate list 1310 may include one or more regular candidates 1311 and one or more mixed candidates 1312. When the selection index selects a regular CCP merging candidate 1311, the candidate extraction module 1330 extracts a set of CCP parameters from the CCP parameter memory 1315 and provides them to the cross-component prediction module 1341 without prediction mixing. When the selection index selects a mixed candidate 1312 indicating two or more CCP merging candidates to be mixed, the candidate extraction module 1330 extracts the parameters of the two or more CCP merging candidates from the CCP parameter memory 1315 and provides them to the cross-component prediction modules 1341 and 1342 respectively.

[0143] CCP merge mode manager 1320 manages the CCP merge candidate list, including CCP parameter memory 1315. CCP merge mode manager 1320 can populate the current block with regular candidates of various types (e.g., spatial proximity, temporal proximity, history, default, etc.) and one or more mixed candidates, each pointing to two or more regular candidates for merging. Regular candidates can be candidates inherited from predefined locations or history tables. Candidate list 1310 can be organized into a single list storing regular CCP merge mode candidates 1311 and mixed candidates 1312. Candidate list 1310 can also be organized into a first list (“original list”) storing only regular CCP merge mode candidates 1311 and a second list storing only mixed candidates 1312. (Therefore, for some embodiments, it can be considered that two or more CCP merge candidate lists are generated and used for the current block.) In the case of generating two or more CCP merge candidate lists, a signaling flag is transmitted to select which of the two or more CCP merge candidate lists is used for the current block. When only one CCP merger candidate list is generated, the method for inserting candidates into the CCP merger candidate list 1310 and the reordering of the list have been referenced. Figure 7 and Figure 8A -C is described.

[0144] The CCP merging mode manager 1320 can derive new CCP linear or convolutional models and store them in the CCP parameter memory 1315. The CCP merging mode manager 1320 can update the contents of the CCP parameter memory 1315, providing parameter information for CCP merging candidates in the candidate list. The CCP merging mode manager 1320 can also manage the transfer of CCP models between images or blocks. For example, the CCP merging mode manager 1320 can store a set of CCP parameters used by one of two or more cross-component prediction modules (1341 and 1342) for the current block in memory 1315, so that these CCP parameters can be referenced or inherited by subsequent blocks. The set of CCP parameters used for future reference can be selected based on its cross-component prediction type (linear vs. convolutional, downsampling vs. non-downsampling, etc.) or its cost.

[0145] Figure 14 This conceptually illustrates a process 1400 for decoding a pixel block using one or more CCP merging candidate lists containing merging candidates. In some embodiments, one or more processing units (e.g., processors) of a computing device implementing decoder 1200 execute process 1400 by executing instructions stored in a computer-readable medium. In some embodiments, an electronic device implementing decoder 1200 executes process 1400.

[0146] At block 1410, the decoder receives data to be decoded as the current pixel block in the current image. The current block includes a first color block (e.g., a luminance component block) and a second color block (e.g., a chrominance component block, Cr or Cb).

[0147] At block 1420, the decoder constructs one or more cross-component prediction (CCP) merging candidate lists for the current block. These candidate lists include one or more regular candidates and one or more hybrid candidates. Each regular candidate is associated with a set of CCP parameters. Each hybrid candidate points to two or more regular candidates. In some embodiments, two or more CCP merging candidate lists are constructed for the current block, comprising one or more regular candidates and one or more hybrid candidates, with the selected candidate chosen from one of these two or more CCP merging candidate lists.

[0148] In some embodiments, the hybrid candidate indicates the two or more regular candidates by providing an index assigned to one of the one or more CCP merging candidate lists. In some embodiments, the two or more regular candidates indicated by the hybrid candidate are limited to not being linear models. In some embodiments, the two or more regular candidates indicated by the hybrid candidate are limited to using non-downsampled luminance samples as input, or are limited to not being linear models, unless at least one of the indicated regular candidates uses non-downsampled luminance samples as input.

[0149] In some embodiments, the two or more regular candidates indicated by the hybrid candidate are defined as candidates inherited from the position adjacent to the current block or from the history table. In some embodiments, the two or more regular candidates indicated by the hybrid candidate are defined as candidates in one of the one or more CCP merge candidate lists. In some embodiments, the two or more regular candidates indicated by the hybrid candidate are defined as excluding the default candidate with default parameters.

[0150] In some embodiments, hybrid candidates are inserted into one of the one or more CCP merge candidate lists after the non-default regular candidates and before the default candidates. In some embodiments, the regular candidates in the one or more CCP merge candidate lists are reordered before the hybrid candidates are inserted into the list. In some embodiments, the one or more CCP merge candidate lists are reordered as a whole after the hybrid candidates are inserted into the list. In some embodiments, each hybrid candidate is inserted into one of the one or more CCP merge candidate lists in an order determined based on the cost of the hybrid candidate.

[0151] In some embodiments, two CCP merging candidate lists are constructed and organized, namely a first list containing only the regular candidates and a second list containing only the hybrid candidates. The second list is constructed after the first list is constructed and reordered according to the (template) cost of the candidates in the first list. The first and second lists may be the same size or different. The two or more regular candidates indicated by each hybrid candidate in the second list may be limited to candidates in the first list. The hybrid candidates in the second list can indicate their two or more regular candidates by using the index of the first list. In some embodiments, the hybrid candidates in the second list are limited to indicating only regular candidates that are not default candidates in the first list. In some embodiments, the hybrid candidates in the second list are reordered according to the cost of the candidates in the second list, and the hybrid candidate with the lowest cost in the second list may be implicitly selected without signaling transmission.

[0152] In some embodiments, each hybrid candidate is inserted into one of the one or more CCP merge candidate lists according to an order determined based on the indices of the two or more regular candidates indicated by the hybrid candidate. The indices of the regular candidates may be determined for the CCP merge candidate list based on the cost of the candidates.

[0153] At block 1430, the decoder selects a candidate from the one or more CCP merging candidate lists. In some embodiments, a flag is signaled to indicate whether the candidate is selected from the first list or the second list (this flag may be skipped and no signaling is performed if merging prediction cannot be used). In some embodiments, the decoder selects a merging candidate by signaling a candidate index to indicate the position of the merging candidate in the second list.

[0154] At block 1435, the decoder determines whether the selected candidate is a regular candidate or a mixed candidate. If the selected candidate is a regular candidate, the process proceeds to block 1440. If the selected candidate is a mixed candidate, the process proceeds to block 1460.

[0155] At block 1440, the decoder generates a cross-component prediction by applying the CCP parameter set of the selected regular candidates to the first color patch reconstructed via intra-frame prediction. The decoder uses this cross-component prediction to decode the current block at block 1450.

[0156] At block 1460, the decoder generates a blended prediction for the second color patch based on two or more cross-component predictions generated by applying the two or more regular candidate options indicated by the selected blending candidate options. At block 1470, the decoder reconstructs the current patch using the generated blended predictions combined with the prediction residuals. The decoder can then provide the reconstructed current patch as part of the reconstructed current image for display.

[0157] In some embodiments, the two or more conventional candidates (or their CCP parameters) are applied to the reconstruction of the first color patch using intra-frame predictions of samples that are neighboring or adjacent to the current patch to generate the two or more cross-component predictions. When generating the mixed predictions, the two or more cross-component predictions may be combined using equal weighting.

[0158] In some embodiments, the decoder also stores for the current block a set of CCP parameters associated with one of the two or more regular candidates indicated by the selected hybrid candidate, for reference in subsequent blocks. The set of CCP parameters for the selected candidate among the two or more regular candidates can be chosen based on its cost or its cross-component prediction type.

[0159] VI. Example Electronic System Many of the features and applications described above are implemented as software processes, which are specified as a set of instructions recorded on a computer-readable storage medium (also known as a computer-readable medium). When these instructions are executed by one or more computing or processing units (e.g., one or more processors, processor cores, or other processing units), the processing unit performs the operations indicated in the instructions. Examples of computer-readable media include, but are not limited to, CD-ROMs, flash drives, random access memory (RAM) chips, hard disks, erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), etc. Computer-readable media do not include carrier waves and electronic signals transmitted via wireless or wired connections.

[0160] In this specification, the term "software" is intended to include firmware residing in read-only memory or application programs stored in magnetic storage media, which can be read into memory and processed by a processor. Furthermore, in some embodiments, multiple software innovations may be implemented as sub-parts of a larger program while still remaining as independent software innovations. In some embodiments, multiple software innovations may also be implemented as independent programs. Finally, any combination of independent programs that collectively implement the software innovations described herein is within the scope of this disclosure. In some embodiments, when a software program is installed and run on one or more electronic systems, it defines one or more specific machine implementations that execute and perform the operations of the software program.

[0161] Figure 15 This conceptual illustration illustrates an electronic system 1500 for implementing certain embodiments of the present disclosure. The electronic system 1500 may be a computer (e.g., a desktop computer, personal computer, tablet computer, etc.), a telephone, a PDA, or any other type of electronic device. Such electronic systems include various types of computer-readable media and interfaces for other various types of computer-readable media. The electronic system 1500 includes a bus 1505, a processing unit 1510, a graphics processing unit (GPU) 1515, system memory 1520, a network interface 1525, a read-only memory 1530, a permanent storage device 1535, an input device 1540, and an output device 1545.

[0162] Bus 1505 refers to all system, peripheral, and chipset buses within electronic system 1500. These buses are used for communication connections between numerous internal devices of electronic system 1500. For example, bus 1505 communicates with processing unit 1510 and GPU 1515, read-only memory 1530, system memory 1520, and permanent storage device 1535.

[0163] From these different storage units, the processing unit 1510 retrieves instructions to be executed and data to be processed, thereby executing the processes of this disclosure. The processing unit may be a single processor or a multi-core processor in different embodiments. Some instructions are passed to and executed by the graphics processing unit (GPU) 1515. The GPU 1515 can offload various computing tasks or supplement the image processing functions provided by the processing unit 1510.

[0164] Read-only memory (ROM) 1530 stores static data and instructions used by processing unit 1510 and other modules of the electronic system. Permanent storage device 1535 is a read-write storage device. This device is a non-volatile storage unit that can store instructions and data even when the electronic system 1500 is powered off. Some embodiments of this disclosure use mass storage devices (such as disks or optical discs and their corresponding disk drives) as permanent storage device 1535.

[0165] Other embodiments use removable storage devices (such as floppy disks, flash memory devices, etc., and their corresponding drives) as permanent storage devices. Similar to permanent storage device 1535, system memory 1520 is a read-write storage device. However, unlike storage device 1535, system memory 1520 is a volatile read-write memory, such as random access memory. System memory 1520 stores some instructions and data used by the processor during runtime. In some embodiments, processes according to this disclosure are stored in system memory 1520, permanent storage device 1535, and / or read-only memory 1530. For example, various storage units include instructions for processing multimedia segments according to certain embodiments. From these different storage units, processing unit 1510 retrieves instructions to be executed and data to be processed to perform processes of certain embodiments.

[0166] Bus 1505 is also connected to input device 1540 and output device 1545. Input device 1540 enables a user to pass information and issue commands to the electronic system. Input device 1540 includes an alphanumeric keypad and pointing device (also known as a "cursor control device"), a camera (such as a webcam), a microphone, or similar devices for receiving voice commands. Output device 1545 displays images generated by the electronic system or otherwise outputs data. Output device 1545 includes printers and display devices such as cathode ray tube (CRT) or liquid crystal display (LCD), as well as speakers or similar audio output devices. Some embodiments include devices such as touchscreens that function as both input and output devices.

[0167] Finally, as Figure 15 As shown, bus 1505 also connects electronic system 1500 to network interface 1525 via a network adapter (not shown). In this way, the computer can become part of a computer network (such as a local area network (LAN), wide area network (WAN), intranet, or network of networks, such as the Internet). Any or all components of electronic system 1500 can be used in conjunction with this disclosure.

[0168] Some embodiments include electronic components such as microprocessors, memories, and storage devices that store computer program instructions in a machine-readable or computer-readable medium (also referred to as a computer-readable storage medium, machine-readable medium, or machine-readable storage medium). Examples of such computer-readable media include random access memory (RAM), read-only memory (ROM), read-only optical disc (CD-ROM), recordable optical disc (CD-R), rewritable optical disc (CD-RW), read-only digital versatile optical discs (such as DVD-ROM, dual-layer DVD-ROM), various recordable / rewritable DVDs (such as DVD-RAM, DVD-RW, DVD+RW, etc.), flash memory (such as SD cards, mini-SD cards, micro-SD cards, etc.), magnetic and / or solid-state drives, read-only and recordable Blu-ray discs, ultra-high-density optical discs, any other optical or magnetic media, and floppy disks. The computer-readable medium may store a computer program executable by at least one processing unit and contain a set of instructions for performing various operations. Examples of computer programs or computer code include machine code (such as code generated by a compiler) and files containing high-level code that are executed by a computer, electronic component, or microprocessor via an interpreter.

[0169] While the foregoing discussion primarily concerns microprocessors or multi-core processors executing software, many of the aforementioned features and applications are executed by one or more integrated circuits, such as application-specific integrated circuits (ASICs) or field-programmable gate arrays (FPGAs). In some embodiments, such integrated circuits execute instructions stored on the circuit itself. Furthermore, some embodiments execute software stored in a programmable logic device (PLD), ROM, or RAM device.

[0170] In this specification and any claim of this application, the terms "computer," "server," "processor," and "memory" refer to electronic or other technical devices. These terms do not encompass any person or group. For the purposes of this specification, the term "display" or "displayed" means display on an electronic device. In this specification and any claim of this application, the terms "computer-readable medium," "computer-readable media," and "machine-readable medium" are fully defined as tangible, physical objects that store information in a computer-readable form. These terms do not include any wireless signals, wired download signals, or other transient signals.

[0171] Although this disclosure has been described in conjunction with numerous specific details, those skilled in the art will recognize that this disclosure may be presented in other specific forms without departing from its spirit. Furthermore, several figures (including...) Figure 11 and Figure 14 The process is conceptually illustrated. The specific operations of these processes may not be performed in the exact order illustrated and described. Specific operations may not be performed as a series of consecutive operations; different specific operations may be performed in different embodiments. Furthermore, the process may be implemented through multiple sub-processes or as part of a larger macro-process. Therefore, those skilled in the art will understand that this disclosure should not be limited to the foregoing illustrative details but should be defined by the appended claims.

[0172] Additional Notes The topics described herein sometimes exemplify situations where different components are contained within or connected to different other components. It should be understood that such illustrated architectures are merely examples, and many other architectures can actually be implemented to achieve the same functionality. Conceptually, any arrangement of components to achieve the same functionality is effectively “associated” to achieve the desired function. Therefore, any two components combined in this document to achieve a specific function can be considered “associated” to achieve the desired function, regardless of the architecture or intermediate components. Similarly, any two such associated components can be considered “operably connected” or “operably coupled” to each other to achieve the desired function; any two components that can be suchly associated can also be considered “operably coupled” to each other to achieve the desired function. Specific examples of operational coupling include, but are not limited to, physically matable and / or physically interactive components and / or wirelessly interactive and / or logically interactive and / or logically interactive components.

[0173] Furthermore, regarding the use of almost all plural and / or singular terms in this document, those skilled in the art can appropriately convert plural to singular and / or singular to plural as needed by the context and / or application. For clarity, various singular / plural transformations are explicitly listed herein.

[0174] Furthermore, those skilled in the art will understand that terms commonly used in this specification, particularly in appended claims, such as the body portion of appended claims, are generally considered "open-ended" terms. For example, the word "comprising" should be interpreted as "including but not limited to," the word "having" should be interpreted as "having at least," and the word "including" should be interpreted as "including but not limited to," etc. Those skilled in the art will also further understand that if a specific number of claim elements is expressly expressed in a claim, that intention is expressly stated in the claim; if such a statement is not made, then such an intention does not exist. For example, for ease of understanding, the appended claims below may contain the introductory phrases "at least one" and "one or more" to introduce claim elements. However, the use of such phrases should not be interpreted as limiting any particular claim containing that claim element to containing only one of that element when introduced by the indefinite article "a" or "an," even if the same claim contains both the introductory phrase "one or more" or "at least one" and the indefinite article such as "a" or "an," for example, "a" and / or "an" should be interpreted as "at least one" or "one or more"; the same principle applies to the use of definite articles to introduce claim elements. Furthermore, even if a specific number of elements are explicitly stated in the claims, those skilled in the art will recognize that such a statement should be interpreted as at least that number. For example, stating "two elements" alone, without further modification, implies at least two elements, or two or more elements. Additionally, when using conventions such as "at least one A, B, and C," such structures should generally be interpreted in the manner understood by those skilled in the art. For example, "a system having at least one A, B, and C" includes, but is not limited to, systems having only A, only B, only C, both A and B, both A and C, both B and C, and systems having both A, B, and C. Similarly, when using conventions such as "at least one A, B, or C," such structures should generally be interpreted in the manner understood by those skilled in the art. For example, "a system having at least one A, B, or C" includes, but is not limited to, systems having only A, only B, only C, both A and B, both A and C, both B and C, and systems having both A, B, and C. Those skilled in the art will further understand that virtually any disjunctive words and / or phrases appearing in the specification, claims, or drawings, presenting two or more alternative terms, should be understood to include the possibility of including only one term, any term, or all terms. For example, the phrase "A or B" should be understood to include the possibility of including "A" or "B" or "A and B".

[0175] As can be seen from the foregoing, various embodiments of this disclosure have been described herein for illustrative purposes, and various modifications can be made without departing from the scope and spirit of this disclosure. Therefore, the various embodiments disclosed in this specification are not intended to be limiting, and the true scope and spirit are defined by the following claims.

Claims

1. A video encoding / decoding method, comprising: Receive input data related to a current block of a current image of a video to be encoded or decoded, wherein the current block includes a first color block and a second color block; For the current block, construct one or more cross-component prediction (CCP) merging candidate lists, which include one or more regular candidates and one or more fusion candidates, wherein each regular candidate is associated with a set of CCP parameters and each fusion candidate corresponds to two or more regular candidates; Select one candidate from the list of one or more CCP merger candidates; A prediction of the second color patch is generated based on the selected candidate, wherein if the selected candidate is a fusion candidate, the prediction is generated based on two or more cross-component predictions (CCP) generated by applying the two or more regular candidates corresponding to the selected fusion candidate to the first color patch reconstructed by intra-frame prediction referencing samples adjacent to or adjacent to the current block. Use the generated prediction to encode or decode the current block.

2. The video encoding / decoding method of claim 1, wherein two or more CCP merging candidate lists are constructed for the current block, the CCP merging candidate lists including one or more regular candidates and one or more fusion candidates, and the selected candidate is selected from one of the two or more CCP merging candidate lists.

3. The video encoding / decoding method of claim 2, wherein a flag is transmitted via signaling to indicate which of the two or more CCP merged candidate lists the selected candidate is from.

4. The video encoding / decoding method as described in claim 1, wherein when generating the fusion prediction, the two or more cross-component predictions are processed with equal weight.

5. The video encoding / decoding method of claim 1, wherein the fusion candidate indicates the two or more regular candidates by providing an index assigned to the one or more CCP fusion candidate lists.

6. The video encoding / decoding method of claim 1, wherein the two or more conventional candidates indicated by the fusion candidate are defined as nonlinear models.

7. The video encoding / decoding method of claim 1, wherein the two or more conventional candidates indicated by the fusion candidate are limited to using non-downsampled luminance samples as input.

8. The video encoding / decoding method of claim 1, wherein the two or more conventional candidates indicated by the fusion candidate are defined as nonlinear models, unless at least one of the indicated conventional candidates uses a non-downsampled luminance sample as input.

9. The video encoding / decoding method of claim 1, wherein the two or more conventional candidates indicated by the fusion candidate are defined as candidates inherited from the position of the adjacent current block or from the history table.

10. The video encoding / decoding method of claim 1, wherein the two or more conventional candidates indicated by the fusion candidate are limited to candidates in the one or more CCP merging candidate lists.

11. The video encoding and decoding method as claimed in claim 1, wherein constructing and organizing two CCP merging candidate lists comprises a first list containing only the regular candidate and a second list containing only the fusion candidate.

12. The video encoding / decoding method of claim 11, wherein the second list is constructed after the first list has been constructed and reordered.

13. The video encoding / decoding method of claim 11, wherein the two or more conventional candidates indicated by each fusion candidate in the second list are limited to candidates in the first list.

14. The video encoding / decoding method of claim 11, wherein the fusion candidates in the second list are indicated by using the index of the first list to indicate two or more regular candidates.

15. The video encoding / decoding method of claim 11, wherein the fusion candidates in the second list are reordered according to the cost of the candidates in the second list.

16. The video encoding / decoding method of claim 11, wherein selecting a fusion candidate includes implicitly selecting the lowest-cost fusion candidate from the second list without signaling transmission.

17. The video encoding / decoding method of claim 11, wherein selecting a fusion candidate includes transmitting a candidate index via signaling to indicate the position of the fusion candidate in the second list.

18. The video encoding / decoding method of claim 11, wherein the sizes of the first and second lists are different.

19. The video encoding / decoding method of claim 11, wherein the first and second lists are of the same size.

20. The video encoding / decoding method of claim 1, wherein each fusion candidate is inserted into one of the one or more CCP fusion candidate lists in an order determined based on the indices of the two or more regular candidates indicated by the fusion candidate.

21. The video encoding / decoding method of claim 1, wherein each fusion candidate is inserted into one of the one or more CCP fusion candidate lists in an order determined based on the cost of the fusion candidate.

22. The video encoding / decoding method of claim 1 further includes storing for the current block a set of CCP parameters associated with one of the two or more conventional candidates indicated by the selected fusion candidate, for reference by subsequent blocks.

23. The video encoding / decoding method of claim 22, wherein one of the two or more conventional candidates is selected based on its cost or its cross-component prediction type, along with an associated set of CCP parameters.

24. An electronic device, comprising: A video codec circuit configured to perform the following operations includes: Receive input data related to a current block of a current image of a video to be encoded or decoded, wherein the current block includes a first color block and a second color block; For the current block, construct one or more cross-component prediction (CCP) merging candidate lists, which include one or more regular candidates and one or more fusion candidates, wherein each regular candidate is associated with a set of CCP parameters and each fusion candidate corresponds to two or more regular candidates; Select one candidate from the list of one or more CCP merger candidates; A prediction of the second color patch is generated based on the selected candidate, wherein if the selected candidate is a fusion candidate, the prediction is generated based on two or more cross-component predictions (CCP) generated by applying the two or more regular candidates corresponding to the selected fusion candidate to the first color patch reconstructed by intra-frame prediction referencing samples adjacent to or adjacent to the current block. Use the generated prediction to encode or decode the current block.

25. A video decoding method, comprising: Receive input data related to a current block of a current image of a video to be decoded, wherein the current block includes a first color block and a second color block; For the current block, construct one or more cross-component prediction (CCP) merging candidate lists, which include one or more regular candidates and one or more fusion candidates, wherein each regular candidate is associated with a set of CCP parameters and each fusion candidate corresponds to two or more regular candidates; Select one candidate from the list of one or more CCP merger candidates; A prediction of the second color patch is generated based on the selected candidate, wherein if the selected candidate is a fusion candidate, the prediction is generated based on two or more cross-component predictions (CCP) generated by applying the two or more regular candidates corresponding to the selected fusion candidate to the first color patch reconstructed by intra-frame prediction referencing samples adjacent to or adjacent to the current block. Use the generated prediction to decode the current block.

26. A video encoding method, comprising: Receive input data related to a current block of a current image of a video to be encoded, wherein the current block includes a first color block and a second color block; For the current block, construct one or more cross-component prediction (CCP) merging candidate lists, which include one or more regular candidates and one or more fusion candidates, wherein each regular candidate is associated with a set of CCP parameters and each fusion candidate corresponds to two or more regular candidates; Select one candidate from the list of one or more CCP merger candidates; A prediction of the second color patch is generated based on the selected candidate, wherein if the selected candidate is a fusion candidate, the prediction is generated based on two or more cross-component predictions (CCP) generated by applying two or more regular candidates corresponding to the selected fusion candidate to the first color patch reconstructed by intra-frame prediction referencing samples adjacent to or adjacent to the current block. Use the generated prediction to encode the current block.

Images