Extrapolated intra prediction model for chroma coding

Abstract

A method of using extended intra prediction (EIP) or prediction based on multiple source samples to generate cross-component prediction of chroma samples is provided. A video codec receives data of a current block of pixels of a current picture of a video for encoding or decoding. The video codec determines a model for predicting at least one target sample based on one or more sets of neighboring source samples. The at least one target sample is a chroma sample. The video codec applies the model to generate a predictor for the current block, in particular for generating a prediction of the current block chroma samples. The video codec encodes or decodes the current block using the generated predictor.

Description

[0001] Cross-references

[0002] This disclosure is a non-provisional application and claims priority to U.S. Provisional Patent Application No. 63 / 592,930, filed October 25, 2023. The contents of the above application are incorporated herein by reference. Technical Field

[0003] This disclosure generally relates to video encoding and decoding. Specifically, this disclosure relates to methods for encoding pixel blocks using extrapolation intra-prediction (EIP) and cross-component prediction. Background Technology

[0004] Unless otherwise stated, the methods described in this section are not prior art to the claims and are not acknowledged as prior art by way of inclusion in this section.

[0005] High-Efficiency Video Coding (HEVC) is an international video coding standard developed by the Joint Collaborative Team on Video Coding (JCT-VC). HEVC is based on a hybrid block-based motion-compensated discrete cosine transform (DCT-like) coding architecture. The basic unit of compression is called a coding unit (CU), which is a 2Nx2N square pixel block. Each CU can be recursively divided into four smaller CUs until a predefined minimum size is reached. Each CU contains one or more prediction units (PUs).

[0006] Versatile Video Coding (VVC) is the latest international video coding standard developed by the Joint Video Experts Team (JVET) of ITU-T SG16 WP3 and ISO / IEC JTC1 / SC29 / WG11. The input video signal is predicted from a reconstructed signal, which originates from the coded image region. The prediction residual signal is processed through block transform. The transform coefficients, along with other auxiliary information, are quantized and entropy-coded in the bitstream. The reconstructed signal is generated from the prediction signal and the inversely transformed reconstructed residual signal, with the inverse transform applied to the dequantized transform coefficients. The reconstructed signal is further filtered within a loop to remove coding artifacts. The decoded image is stored in a frame buffer and used to predict subsequent images in the input video signal.

[0007] In VVC, the encoded image is segmented into non-overlapping square block regions represented by relevant coding tree units (CTUs). Leaf nodes of the coding tree correspond to coding units (CUs). The encoded image can consist of multiple slices, each containing an integer number of CTUs. The CTUs within a slice are processed in raster scan order. Bi-predictive (B) slices are decoded via either intra-frame or inter-frame prediction; inter-frame prediction uses at most two motion vectors (MVs) and a reference index to predict the sample values ​​for each block. Predictive (P) slices are decoded via either intra-frame or inter-frame prediction; inter-frame prediction uses at most one motion vector and a reference index to predict the sample values ​​for each block. Intra (I) slices are decoded only via intra-frame prediction.

[0008] The Coding Unit (CTU) can be segmented into one or more non-overlapping Coding Units (CUs) using a quadtree (QT) or nested multi-type-tree (MTT) structure to accommodate different local motion and texture characteristics. CUs can be further segmented into smaller CUs using one of five segmentation types: quadtree segmentation, vertical binary tree segmentation, horizontal binary tree segmentation, vertical center-side ternary tree segmentation, and horizontal center-side ternary tree segmentation.

[0009] Each CU contains one or more prediction units (PUs). The prediction unit, along with the associated CU syntax, serves as the basic unit of signaling predictor information. The specified prediction process is used to predict the values ​​of associated pixel samples within the PU. Each CU may contain one or more transform units (TUs) to represent prediction residual blocks. A transform unit (TU) consists of a transform block (TB) for one luma sample and two corresponding chroma sample transform blocks; each TB corresponds to a residual sample block for one color component. Integer transforms are applied to the transform blocks. The level values ​​of the quantization coefficients, along with other auxiliary information, are entropy-encoded in the bitstream. Coding tree blocks (CTBs), coding blocks (CBs), prediction blocks (PBs), and transform blocks (TBs) are used to specify two-dimensional sample arrays of a single color component associated with the CTU, CU, PU, ​​and TU, respectively. Therefore, a CTU contains one luma CTB, two chroma CTBs, and associated syntax elements. CUs, PUs, and TUs have similar relationships.

[0010] For each CU in inter-frame prediction, motion parameters include motion vectors, reference image indices, reference image list usage indices, and other additional information used to generate inter-frame prediction samples. Motion parameters can be signaled explicitly or implicitly. When a CU is encoded in skip mode, it is associated with a PU and has no significant residual coefficients, no encoded motion vector increments, or reference image indices. Merging mode specifies that the motion parameters for the current CU are obtained from neighboring CUs, including spatial and temporal candidates, and additional scheduling introduced in VVC. Merging mode can be applied to any CU in inter-frame prediction. An alternative to merging mode is explicit transmission of motion parameters, where the motion vectors for each CU, the corresponding reference image indices for each reference image list, reference image list usage flags, and other necessary information are explicitly signaled. Summary of the Invention

[0011] The following content is for illustrative purposes only and is not intended to be limiting in any way. That is, the following content is intended to introduce the concepts, key points, benefits, and advantages of the novel and non-obvious techniques described herein. Some, but not all, embodiments will be further elaborated in the detailed description below. Therefore, the following content is not intended to identify the essential features of the claimed subject matter, nor is it intended to determine the scope of the claimed subject matter.

[0012] Certain embodiments of this disclosure provide a method for generating cross-component predictions of chroma samples using extended intra-frame prediction (EIP) or predictions based on multiple source samples. A video codec receives data of a current pixel block as a current image of a video for encoding or decoding. The video codec determines a model for predicting at least one target sample based on one or more neighboring source sample sets. The at least one target sample is a chroma sample. The video codec applies the model to generate a predictor for the current block, specifically for generating predictions of the chroma samples of the current block. The video codec uses the generated predictor to encode or decode the current block.

[0013] In some embodiments, the model is used to predict multiple target sample sets (which may be all chroma samples) based on neighboring source sample sets. The neighboring source sample sets can be downsampled or non-downsampled luminance samples. In some embodiments, a set of MxN samples provides the PxQ target samples, with the remainder serving as source samples. In some embodiments, the model is applied to samples in the current block and samples in template regions adjacent to the current block to generate a predictor for the current block.

[0014] In some embodiments, the model is provided by a pool of candidates selected from a plurality of candidates, including a merged list of spatially adjacent, spatially non-adjacent, history-based, or default candidates. In some embodiments, the model is obtained by using samples in a reference region as corresponding source and target samples for the filter. In some embodiments, the model includes filter coefficients obtained by minimizing the mean squared error (MSE) between the corresponding source and target samples (e.g., the model could be a 7-tap convolutional filter with 5-tap plus sign spatial components, a nonlinear term, and a bias term). Attached Figure Description

[0015] The accompanying drawings are included in this disclosure to further understand it and form part of it. The drawings illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the disclosure. It will be understood that the drawings are not necessarily drawn to scale, as some components may be shown out of proportion to their actual dimensions for clarity of the concepts of the disclosure.

[0016] Figure 1 shows some of the defined reconstructed region types.

[0017] Figure 2 shows some defined filter shape types.

[0018] Figure 3 An example is shown that predictions are generated for different positions in the current block in diagonal order.

[0019] Figure 4 The spatial portion of the convolution filter is shown.

[0020] Figure 5 This shows the reference region used to compute the filtering coefficients of the convolution cross-component model (CCCM).

[0021] Figure 6 Four Sobel-based gradient modes for use in gradient linear models are shown.

[0022] Figure 7 This demonstrates the non-downsampled adjacent luminance samples used to derive the chrominance samples.

[0023] Figure 8A -B displays the corresponding luminance sample and adjacent chrominance samples of the current chrominance sample.

[0024] Figure 9A -H shows an example pattern used to predict the corresponding luminance sample for the chromaticity sample.

[0025] Figure 10A -B shows the prediction of the current chroma sample as a combination of the corresponding downsampled luminance sample and the adjacent chroma sample.

[0026] Figure 11A -D indicates some additional modes used to predict the corresponding downsampled luminance sample for the current chroma sample.

[0027] Figure 12 This demonstrates how to derive multiple target samples using multi-source sample prediction with the same / shared kernel block.

[0028] Figure 13 An example video encoder that enables multiple source sample-based prediction (MSSBP) is shown.

[0029] Figure 14 The video encoder portion of the cross-component MSSBP implementation is shown.

[0030] Figure 15 This conceptually demonstrates the process of encoding pixel blocks using cross-component MSSBP.

[0031] Figure 16 An example video decoder that enables Multi-Source Sample Prediction (MSSBP) is shown.

[0032] Figure 17 The video decoder portion of the cross-component MSSBP implementation is shown.

[0033] Figure 18 This conceptually demonstrates the process of decoding pixel blocks using cross-component MSSBP.

[0034] Figure 19 An electronic system that implements certain embodiments of the present disclosure is conceptually shown. Detailed Implementation

[0035] In the following detailed description, numerous specific details are illustrated by way of example to provide a comprehensive understanding of the related techniques. Any variations, derivatives, and / or extensions of the techniques described herein are within the scope of this disclosure. In some cases, to avoid unnecessarily obscuring certain aspects of the techniques disclosed, well-known methods, processes, components, and / or circuits involving one or more exemplary embodiments may be presented at a relatively high level without detailed description.

[0036] I. Extrapolation Intra Prediction (EIP)

[0037] Extrapolated intra-slice prediction (EIP) is a process that generates a predictor for each sample within a block. Therefore, it is also called multi-source sample prediction (MSSBP). In some applications, EIP is limited to block sizes no larger than 32x32 and is used only for intra slices.

[0038] For EIP, an offset value can be removed when the input is fed into the EIP filter. The top-left reconstructed pixel of the current block's predefined neighborhood is used as the offset value for EIP prediction. The minimum and maximum values ​​can be obtained from the reconstructed pixels of the current block's predefined reconstruction region.

[0039] In some embodiments, there are three types of reconstruction regions and three filter shapes. Figure 1 illustrates the three defined reconstruction region types. Figure 2 illustrates the three defined filter shapes, which have 15 inputs and produce one output.

[0040] When the current block is predicted using EIP mode, the decoder decodes relevant syntax elements to determine the type of reconstructed region and the filter shape selected for the current block. The size of the reconstructed region depends on min(block width, block height) and the selected filter shape. For example, when the current block is an 8x16 block and the selected filter shape is 4x4, the upper size of the reconstructed region is min(8, 16) + 4 – 1 = 11, and the left size of the reconstructed region is min(8, 16) + 4 – 1 = 11.

[0041] The selected filter moves within the selected reconstruction region in single-pixel steps to collect input and output samples for the EIP. After removing the offsets from the input and output samples, the autocorrelation matrix and cross-correlation vector are constructed. Then, the EIP coefficients are obtained using the CCCM filter coefficient derivation method described in Section III.B below, specifically by minimizing the mean-square error (MSE) between the corresponding input and output samples. Specifically, the filter coefficients c... i The MSE is calculated by minimizing the difference between the predicted chromaticity sample and the reconstructed chromaticity sample within the reference region.

[0042] EIP mode generates predictions for the current block by predicting diagonally from the top left to the bottom right position, such as... Figure 3 As shown in the figure, an example of generating predictions for different positions of the current block in diagonal order is illustrated. To reduce prediction error, the minimum and maximum values ​​of the neighborhood reconstruction region can be applied to limit the output range of each prediction. The calculation of the predicted samples in the test is as follows:

[0043] Among them, pred (x,y) It is the predicted value of the current block at position (x, y) (min, max, and offset are the values ​​mentioned in this section). c i It is the i-th coefficient of the derived EIP filter, with coefficient indices from 0 to 14, t (x-xoffset,y-yoffset) It is used for the reconstruction or prediction of the current predicted location.

[0044] II. Intra-mode coding with 67 intra-prediction modes

[0045] To capture arbitrary edge directions in natural video, the number of directional intra-frame modes has been expanded from 33 (in HEVC) to 65 (in VVC). In VVC, several traditional angular intra-frame prediction modes can be adaptively replaced with wide-angle intra-frame prediction modes for non-square blocks. In HEVC, each intra-coded block is a square, and the length of each side is a power of 2. Therefore, no division operation is required when generating the intra-frame predictor using DC modes. In VVC, blocks can be rectangular, and typically a division operation is required for each block. To avoid the division operation in DC prediction, for non-square blocks, only the longer side is used to calculate the average.

[0046] III. Cross-component prediction

[0047] A. Cross-Component Linear Model (CCLM)

[0048] To reduce cross-component redundancy, VVC employs a cross-component linear model (CCLM) prediction mode, which predicts chrominance samples based on reconstructed luminance samples from the same coding unit (CU) using the following linear model: (1) in This represents the predicted chromaticity samples within the CU. This represents the downsampled reconstructed luminance samples from the same CU. CCLM parameters ( and It is derived by using up to four neighboring chromaticity samples and their corresponding downsampled luminance samples.

[0049] The brightness samples of four neighborhoods at the selected location are downsampled and compared four times to find the two larger values: x 0 A and x 1 A and two smaller values: x 0 B andx 1 B The corresponding chromaticity sample values ​​are denoted as follows: y 0 A , y 1 A , y 0 B and y 1 B .Then x A , x B , y A and y B The derivation is as follows: X a =( x 0 A + x 1 A +1)>>1; X b =( x 0 B + x 1 B +1)>>1 Y a =( y 0 A + y 1 A +1)>>1; Y b =( y 0 B + y 1 B +1)>>1 Finally, the linear model parameters and It is obtained from the following formula.

[0050]

[0051] In addition to the upper and left templates being used together to calculate linear model coefficients, they can also be used for two other LM modes, called LM_A and LM_L modes respectively.

[0052] In LM_A mode, only the upper template is used to calculate the linear model coefficients. To obtain more samples, the upper template is expanded to (W+H) samples. In LM_L mode, only the left template is used to calculate the linear model coefficients. To obtain more samples, the left template is expanded to (H+W) samples.

[0053] In LM_LA mode, the left and top templates are used to calculate the linear model coefficients. To match the chroma sample positions of a 4:2:0 video sequence, two types of downsampling filters are applied to the luminance samples to achieve a 2:1 downsampling ratio in both the horizontal and vertical directions.

[0054] B. Convolutional Cross-Component Intra-Frame Prediction Model (CCCM)

[0055] In some embodiments, a convolutional cross-component model (CCCM) is used to predict chroma samples from reconstructed luminance samples in a manner similar to the current cross-component linear model (CCLM) mode. Similar to CCLM, when chroma subsampling is employed, the reconstructed luminance samples are downsampled to match a low-resolution chroma grid. As with CCLM, upper, left, or upper and left reference samples serve as templates for model derivation.

[0056] Similar to CCLM, CCCM offers the option of a single model or a multi-model variant. The multi-model variant uses two models: one for samples above the average luminance reference value and the other for the remaining samples (following the CCLM design philosophy). The multi-model CCCM mode can be used in prediction units (PUs) with at least 128 reference samples.

[0057] In some embodiments, CCCM uses a 7-tap convolutional filter. This 7-tap convolutional filter includes a 5-tap plus sign spatial component, a nonlinear term, and a bias term. Figure 4 The spatial portion of the convolutional filter is shown. As shown in the figure, the input of the 5-tap component of the filter space includes the center (C) luminance sample corresponding to the chrominance sample to be predicted, as well as its above / north (N), below / south (S), left / west (W), and right / east (E) neighbors.

[0058] The nonlinear term P is represented as a square power of the center brightness sample C, scaled to the range of sample values ​​for the content: P = (C) C + midVal) >> bitDepth That is, for 10 bits of content, the calculation is as follows: P = (C) (C + 512) >> 10 The bias term B represents the scalar offset between the input and output (similar to the offset term in CCLM) and is set to an intermediate chroma value (512 for 10 bits). The filter output is passed through the filter coefficients c. i Perform convolution with the input value and crop to the range of valid chromaticity samples: predChromaVal = c0C + c1N + c2S + c3E + c4W + c5P + c6B Filter coefficients c i It is obtained by minimizing the mean square error (MSE) between the predicted chromaticity sample and the reconstructed chromaticity sample within the reference region. Figure 5 The reference region used to calculate the CCCM filter coefficients is shown. As shown in the figure, reference region 500 includes 2 or 6 rows of chroma samples above and to the left of the PU. Whether to use 6 or 2 rows of neighbor samples to derive the CCCM model parameters in a single-model CCCM is determined by the template cost. Similarly, for a multi-model CCCM mode, the samples are divided into two groups by deriving the mean from either 6 rows of neighbor luminance samples or the luminance sample corresponding to the current chroma block. The cost is obtained by applying the candidate cross-component prediction (CCP) (2 or 6 rows) to the template and calculating the sum of absolute differences (SAD) between the CCP predicted samples and the reconstructed samples in the template.

[0059] The reference region 500 extends to the right by one PU width and downwards by one PU height to the PU boundary. The region is adjusted to include only usable samples. The region extension shown in blue is used to support the "side samples" of the plus-shaped spatial filter, filling in unavailable areas.

[0060] Minimizing the mean square error (MSE) is achieved by calculating the autocorrelation matrix of the luminance input and the cross-correlation vector between the luminance input and the chrominance output. The autocorrelation matrix is ​​decomposed using LDL, and the final filter coefficients are obtained through back-substitution. This calculation is similar to that of the adaptive loop filter (ALF) coefficients in ECM, except that LDL decomposition is chosen instead of Cholesky decomposition to avoid the use of square root operations.

[0061] The autocorrelation matrix is ​​calculated using reconstructed values ​​from luma and chromaticity samples. These samples are full-range (e.g., 10 bits from 0 to 1023), resulting in large values ​​in the autocorrelation matrix. This necessitates high-bit-depth operations during model parameter calculations. It is recommended to remove the fixed offset of the luma and chromaticity samples from each model in each PU. This reduces the magnitude of the values ​​used during model creation and allows for a reduction in the precision required for fixed-point operations. Therefore, it is recommended to replace the original CCCM implementation's 22-bit precision with 16-bit fractional precision.

[0062] Reference sample values ​​are used as offsets (offsetLuma, offsetCb, and offsetCr) outside the top left corner of the PU to simplify processing. The sample values ​​used for model creation and final prediction (i.e., luminance and chrominance in the reference region, and luminance in the current PU) are subtracted from these fixed values ​​by: C' = C – offsetLuma N' = N – offsetLuma S' = S – offsetLuma E' = E – offsetLuma W' = W – offsetLuma P' = nonLinear(C') B = midValue = 1 << (bitDepth - 1) Chromaticity values ​​are predicted using the following formula, where offsetChroma represents the offsetCr and offsetCb of the Cr and Cb components, respectively: predChromaVal = c0C' + c1N' + c2S' + c3E' + c4W' + c5P' + c6B +offsetChroma To avoid any additional sample-level operations, luminance offsets are removed during luminance reference sample interpolation. For example, this can be achieved by replacing the rounding term used in luminance reference sample interpolation with an update offset that includes the rounding term and offsetLuma. Chroma offsets can be removed by directly subtracting the chroma offset from the reference chroma sample. Alternatively, the effect of chroma offsets can be removed from the cross-component vectors, with the same result. To add the chroma offset back to the output of the convolutional prediction operation, the chroma offset is added to the bias term of the convolutional model.

[0063] The calculation of CCCM model parameters requires division operations. Division operations are not always easy to implement. Division can be replaced by multiplication (with a scaling factor) and shift operations, where the scaling factor and the number of shifts are calculated based on the denominator, similar to the method used in CCLM parameter calculation.

[0064] C. Gradient Linear Model (GLM)

[0065] For the YUV 4:2:0 color format, the Gradient Linear Model (GLM) method can be used to predict chrominance samples based on the luminance sample gradient. Two modes are supported: two-parameter GLM and three-parameter GLM. Compared to CCLM, two-parameter GLM uses the luminance sample gradient to derive the linear model, rather than downsampling the luminance value. Specifically, when applying two-parameter GLM, the input to the CCLM process, i.e., the downsampled luminance sample L, is replaced by the luminance sample gradient G. Other parts of CCLM (such as parameter derivation and linear transformation of predicted samples) remain unchanged. C = ⍺·G + β In the three-parameter GLM, chromaticity samples can be predicted based on the gradient of luminance samples and downsampled luminance values ​​(with different parameters). The model parameters of the three-parameter GLM are derived from neighbor samples in 6 rows and columns using the MSE minimization method of LDL decomposition (the same method used in CCCM): C = ⍺0 · G + ⍺1 · L + ⍺2 · β For signaling, when CCLM mode is enabled in the current coding unit (CU), a flag is signaled to indicate whether GLM is enabled for the Cb and Cr components. If GLM is enabled, another flag is signaled to indicate which GLM mode is selected, and a syntax element is further signaled to select one of the four gradient filters used for gradient computation. Figure 6 Four Sobel-based gradient patterns 601-604 are shown for the Gradient Linear Model (GLM).

[0066] D. CCCM using non-downsampled brightness samples

[0067] In some embodiments, a 3x2 filter CCCM mode using unsampled luminance samples may be employed, comprising a 6-tap spatial term, four nonlinear terms, and a bias term. The 6-tap spatial term corresponds to the 6 neighboring luminance samples (i.e., luminance samples) surrounding the chrominance sample to be predicted (i.e., C). L 0 , L 1 … L 5 ), 4 nonlinear terms are derived from the samples L 0, L 1 , L 2 and L 3 Derivation. Figure 7 This demonstrates the non-downsampled neighbor luminance samples used to derive the chroma samples. The chroma samples are derived as follows:

[0068] Among them ⍺ i β is the coefficient, and β is the offset. Similar to existing CCCM designs, up to 6 rows / columns of chroma samples above and to the left of the current coding unit (CU) can be applied to obtain the filter coefficients. The filter coefficients are obtained based on the same LDL decomposition method used in CCCM. In some embodiments, CCCM using non-downsampled luma samples is signaled as an additional CCCM model besides the existing CCCM model. When CCCM is selected, a flag is simultaneously signaled for both chroma components to indicate whether the default CCCM model or the proposed CCCM model is applied. Furthermore, a sequence parameter set (SPS) signaling is introduced to indicate whether CCCM using non-downsampled luma samples is enabled.

[0069] E. Cross-component prediction merging mode

[0070] In some embodiments, for chroma coding, the cross-component prediction (CCP) model may be provided by candidates selected from a merge list. In some embodiments, a flag is signaled to indicate whether a CCP mode (including CCLM, CCCM, GLM and their variants) or a non-CCP mode (traditional chroma intra-prediction mode, chroma intra-prediction mode fusion) is used. If a CCP mode is selected, an additional flag is signaled to indicate how the CCP type and parameters are obtained, i.e., whether from the CCP merge list or through immediate signaling / derivation. The CCP merge candidate list is constructed from spatially adjacent, spatially non-adjacent, or history-based candidates. After including these candidates, a default model is further included to fill any remaining gaps in the merge list. To remove redundant CCP models from the list, pruning is applied. After the list is constructed, the CCP models in the list are reordered according to the absolute difference (SAD) cost obtained using the neighboring templates of the current block. The different types of candidates are described below: Spatial adjacency and non-adjacency candidates Neighboring candidates are spatially adjacent blocks or location-related candidates that are adjacent to the current block, while non-neighboring candidates are blocks or location-related candidates in the current image that are not adjacent to the current block. The location and inclusion order of spatially adjacent and non-neighboring candidates are the same as those defined for regular inter-frame merge prediction candidates.

[0071] Historical candidates

[0072] Maintain a history-based table containing recently used CCP models, and reset the table at the start of each Coding Tree Unit (CTU) column. If the current list is not full after including spatially adjacent and non-adjacent candidates, add CCP models based on the history table to the list.

[0073] Default candidate

[0074] CCLM candidates with default scaling parameters are considered only if the list is not full after including spatially adjacent, spatially non-adjacent, or history-based candidates. If there are no single-model CCLM candidates in the current list, the default scaling parameters are {0, 1 / 8, -1 / 8, 2 / 8, -2 / 8, 3 / 8, -3 / 8, 4 / 8, -4 / 8, 5 / 8, -5 / 8, 6 / 8}. Otherwise, the default scaling parameters are {0, the scaling parameter of the first CCLM candidate + {1 / 8, -1 / 8, 2 / 8, -2 / 8, 3 / 8, -3 / 8, 4 / 8, -4 / 8, 5 / 8, -5 / 8, 6 / 8}}. The offset parameters are obtained based on the default scaling parameters, the average value of adjacent reconstructed luminance samples, and the average value of adjacent reconstructed Cb / Cr samples.

[0075] A flag is signaled to indicate whether CCP merging mode is applied. If CCP merging mode is applied, an index is signaled to indicate which candidate model is used for the current block. Furthermore, CCP merging mode is not allowed for the current chroma codec block when the current CU is encoded using intra-sub-partitions (ISPs) of a single tree, or when the current chroma codec block size is less than or equal to 16.

[0076] IV. Application of EIP in Colorimetric Encoding

[0077] In some embodiments, the current chroma prediction is generated using corresponding luma samples and neighboring chroma samples. This method may be referred to as EIP-chroma or cross-component multiple-source sample-based prediction (CC-MSSBP for short). For example, as shown in the figure below, the prediction of the current chroma "C" is a combination of corresponding luma samples (L0-L5) and neighboring chroma samples (C6-C8). For example, the prediction of the current chroma "C" is:

[0078] Figure 8A -B shows the corresponding luma samples (L0-L5) and neighboring chroma samples (C6-C8) of the current chroma 800. Figure 9A -H shows an example pattern of the corresponding luma samples (L0-L n ) for predicting the chroma sample (C). In some embodiments, the current chroma prediction is generated using corresponding downsampled luma samples (e.g., obtained through a low-pass or high-pass downsampling filtering process according to the current color format) and neighboring chroma samples.

[0079] Figure 10A -B shows that the prediction of the current chroma "C" is a combination of corresponding downsampled luma samples (L0 3 ) and neighboring chroma samples (C3-C5). The overlapping luma and chroma samples may be in the same phase. For some embodiments, Figure 11A -D shows other patterns of the corresponding downsampled luma samples.

[0080] In some embodiments, multiple target samples are derived using the same / shared kernel block of multiple-source sample prediction. Figure 12 shows that multiple target samples are derived using the same / shared kernel block of multiple-source sample prediction. The figure shows several examples 1201-1209, and each example uses a shared kernel of source samples to derive multiple target samples. As shown in examples 1201-1205, the target (to be predicted) samples can be any Mx1 or 1xN samples within the MxN block, and the source samples are the remaining Mx(N - 1) or (M - 1)xN samples. More generally, as shown in examples 1206-1209, the target samples can be any PxQ samples within the MxN block (PxQ < MxN, and 1 ≤ P ≤ M and 1 ≤ Q ≤ N), and these samples provide the source samples.

[0081] Still taking the same example, the source samples can be used to generate predictors for the target samples according to the following formula:

[0082] In other words, multiple linear models with the same input but different parameters can be used to predict multiple target samples. When solving for the optimal parameters of these linear models, the autocorrelation matrix of all linear models... A i The only difference is the cross-correlation vector of all linear models. They are different. Therefore, by applying several column operations to the following combined matrix, the optimal parameters of multiple models can be obtained simultaneously using Gaussian elimination: [A b 0 … b N-1 ]

[0083] V.EIP Information Inheritance

[0084] In some embodiments, all or part of the EIP information of the inherited EIP model may be stored together with the inherited EIP model parameters. In some embodiments, the EIP information may include, but is not limited to, template region selection type (e.g., EIP_T, EIP_L, or EIP_LT), template region size, kernel type (e.g., 4x4 square kernel, 8x2 rectangular kernel, or 2x8 rectangular kernel), multi-model flags, multi-model classification method, multi-model threshold, fusion flags, fusion method, post-filter flags, or model parameters.

[0085] In some embodiments, a propagation flag may be stored as part of the EIP information. This information describes whether the EIP information was copied from a reference block. If a block uses an EIP filter, the EIP information for the current block is derived and stored. The propagation flag for the current block's EIP information is set to false. If a block does not use an EIP filter, the EIP information for the current block is derived by copying the EIP information from a reference block. The propagation flag for the current block's EIP information is then set to true.

[0086] In some embodiments, EIP information may include information indicating the source of the EIP information. For example, the reference block to which the EIP information is copied may be a block in the current image located by the block vector of the current block, a block located by the motion vector of the current block, or a block at the same location as the current block.

[0087] In some embodiments, EIP information can be added as candidates to a merge list or history list, which may further include cross-component prediction (CCP) candidates for intra- or inter-chroma prediction. For example, EIP information can be one of the candidates in an intra-MPM list, an intra- or inter-CCP history list, an intra- or inter-CCP merge list, a regular inter-frame history list, or a regular inter-frame merge list. Another example is that EIP candidates have EIP information from previous coded blocks and can serve as a shortcut to indicate the EIP mode (i.e., deriving the EIP model using the current neighboring reconstructed samples) instead of inheriting parameters from neighbors. For example, the default candidates for the CCP merge list could be EIP_LT, EIP_L, or EIP_T.

[0088] In some embodiments, the parameters of the inheritance model can be further optimized based on the inherited EIP information. The inherited EIP information may include the derivation method of the inheritance model, such as the template region selection type and / or the size of the template region. The optimized parameters are derived based on local information. The optimization process can follow the derivation method of the inheritance model and use templates of the same type and / or template regions of the same size.

[0089] In some embodiments, the parameters of the inheritance model can be further optimized based on a predefined process. For all types of inheritance EIP models, predefined template types and predefined template region sizes are used.

[0090] Any of the methods suggested above can be applied independently or in combination. Furthermore, any of the methods suggested above can be implemented in the encoder and / or decoder. For example, any of the suggested methods can be implemented in the inter-frame prediction module of the encoder and / or decoder. Alternatively, any of the suggested methods can be implemented as circuitry coupled to the inter-frame prediction module of the encoder and / or decoder.

[0091] VI. Example Video Encoder

[0092] Figure 13An example video encoder 1300 capable of multiple source sample-based prediction (MSSBP) is shown. As shown, the video encoder 1300 receives an input video signal from a video source 1305 and encodes the signal into a bitstream 1395. The video encoder 1300 has multiple components or modules for encoding the signal from the video source 1305, including at least some components selected from the following: transform module 1310, quantization module 1311, inverse quantization module 1314, inverse transform module 1315, intra-frame image estimation module 1324, intra-frame prediction module 1325, motion compensation module 1330, motion estimation module 1335, loop filter 1345, reconstructed image buffer 1350, MV buffer 1365, MV prediction module 1375, and entropy encoder 1390. The motion compensation module 1330 and motion estimation module 1335 are part of the inter-frame prediction module 1340. Intra-frame prediction module 1325 and intra-frame prediction estimation module 1324 are part of current image prediction module 1320, which uses current image reconstruction samples as reference samples for predicting the current block.

[0093] In some embodiments, modules 1310–1390 are software instruction modules executed by one or more processing units (e.g., processors) of a computing device or electronic device. In some embodiments, modules 1310–1390 are hardware circuit modules implemented by one or more integrated circuits (ICs) of an electronic device. Although modules 1310–1390 are illustrated as independent modules, some of these modules may be combined into a single module.

[0094] Video source 1305 provides an uncompressed raw video signal, presenting pixel data for each video frame. Subtractor 1308 calculates the difference between the raw video pixel data of video source 1305 and the predicted pixel data 1313 of motion compensation module 1330 or intra-frame prediction module 1325, as the prediction residual 1309. Transform module 1310 converts this difference (or residual pixel data or residual signal 1308) into transform coefficients (e.g., by performing a Discrete Cosine Transform (DCT)). Quantization module 1311 quantizes the transform coefficients into quantized data (or quantization coefficients) 1312, which are then encoded into bitstream 1395 by entropy encoder 1390.

[0095] The inverse quantization module 1314 performs inverse quantization on the quantized data (or quantization coefficients) 1312 to obtain transform coefficients, and the inverse transform module 1315 performs inverse transform on the transform coefficients to generate a reconstruction residual 1319. The reconstruction residual 1319 is added to the predicted pixel data 1313 to generate reconstructed pixel data 1317. In some embodiments, the reconstructed pixel data 1317 is temporarily stored in the line buffer 1327 (or the intra-frame prediction buffer) for intra-frame image prediction and spatial MV prediction. The reconstructed pixels are filtered by the loop filter 1345 and stored in the reconstructed image buffer 1350. In some embodiments, the reconstructed image buffer 1350 is external memory to the video encoder 1300. In some embodiments, the reconstructed image buffer 1350 is internal memory to the video encoder 1300.

[0096] Intra-frame image estimation module 1324 performs intra-frame prediction based on reconstructed pixel data 1317 to generate intra-frame prediction data. The intra-frame prediction data is provided to entropy encoder 1390 for encoding into bitstream 1395. The intra-frame prediction data is also used by intra-frame prediction module 1325 to generate predicted pixel data 1313.

[0097] The motion estimation module 1335 performs inter-frame predictions on reference pixel data of previously decoded frames stored in the reconstructed image buffer 1350 by generating MVs. These MVs are then provided to the motion compensation module 1330 to generate predicted pixel data.

[0098] Instead of encoding the complete actual MV into the bitstream, the video encoder 1300 uses MV prediction to generate a predicted MV and encodes the difference between the MV used for motion compensation and the predicted MV as residual motion data and stores it in the bitstream 1395.

[0099] The MV prediction module 1375 generates a predicted MV based on a reference MV (i.e., the MV used for motion compensation) generated for encoding the previous video frame. The MV prediction module 1375 retrieves the reference MV of the previous video frame from the MV buffer 1365. The video encoder 1300 stores the MV generated for the current video frame in the MV buffer 1365 as a reference MV for generating the predicted MV.

[0100] The MV prediction module 1375 creates a predicted MV using a reference MV. The predicted MV can be calculated through spatial MV prediction or temporal MV prediction. The difference (residual motion data) between the predicted MV and the motion compensation MV (MC MV) of the current frame is encoded into the bitstream 1395 by the entropy encoder 1390.

[0101] The entropy encoder 1390 encodes various parameters and data into a bitstream 1395 using entropy coding techniques such as context-adaptive binary arithmetic coding (CABAC) or Huffman coding. The entropy encoder 1390 encodes various header elements, flags, quantization transform coefficients 1312, and residual motion data as syntax elements into the bitstream 1395. The bitstream 1395 is then stored in a storage device or transmitted to the decoder via a communication medium such as a network.

[0102] The loop filter 1345 performs filtering or smoothing operations on the reconstructed pixel data 1317 to reduce coding artifacts, particularly at pixel block boundaries. In some embodiments, the filtering or smoothing operations performed by the loop filter 1345 include a deblock filter (DBF), a sample adaptive offset (SAO), and / or an adaptive loop filter (ALF). In some embodiments, luma mapping chroma scaling (LMCS) is performed before the loop filter.

[0103] Figure 14 A partial video encoder 1300 is shown, demonstrating cross-component multi-source sample prediction (MSSBP). The cross-component MSSBP model builder 1405 retrieves source and target samples for building the CC-MSSBP model. Some examples of the relative positions of the source and target samples have been provided above. Figure 12 The target sample is a chroma sample. Source samples may include luminance samples and / or chroma samples. Luminance samples may be downsampled or non-downsampled. Some examples of the relative positions of the source luminance / chroma samples and the target chroma sample have been provided above. Figure 7 As described in -11. The source sample can be a reconstructed sample (from line buffer 1327 and / or from reconstructed image buffer 1350) and may include luminance samples and / or chrominance samples. The source sample can be located in a template region defined by the selected template region type, such as EIP_T, EIP_L, EIP_LT.

[0104] The retrieved template samples can be used to derive one or more cross-component MSSBP models via regression based on the selected filter kernel shape. Each MSSBP model is a filter containing the coefficients of a set of filter taps. Example filter kernel shapes have been described in Section 3 above. In some embodiments, the entropy encoder 1390 can specify the filter type or shape to be derived.

[0105] In some embodiments, the previously derived cross-component MSSBP model is stored in memory 1410 and can be retrieved for the current block. In some embodiments, the cross-component MSSBP model may be provided by candidates selected from a merge candidate list. The merge candidate list may include spatially adjacent, spatially non-adjacent, history-based, and / or default candidates. Selector 1425 may select one of the merge candidates and retrieve the corresponding CC-MSSBP model for the current block. The CC-MSSBP merge candidate selection may be provided to entropy encoder 1390 for signaling in bitstream 1395.

[0106] Filter 1420 applies the selected CC-MSSBP model to generate chroma predictions 1430 for each target location in the current block, as described in Section 4 above. Filter 1420 can use adjacent reconstructed samples (provided by the reconstructed image buffer 1350 or line buffer 1327), MSSBP predicted samples (provided by the MSSBP predictor 1425), or predicted samples (provided by the intra-frame prediction module 1325) as filtering inputs at the source sample locations specified by the selected filter shape. The filter can traverse the current block, according to the above... Figure 3 The diagonal pattern generates chromaticity samples for the predictor. Predictions for different target positions in the current block are provided as MSSBP predictor 1430 and used as predicted pixel data 1313.

[0107] Figure 15 A conceptual illustration of process 1500 for encoding pixel blocks using cross-component MSSBP is presented. In some embodiments, one or more processing units (e.g., processors) of a computing device implementing video encoder 1300 execute process 1500 by executing instructions stored in a computer-readable medium. In some embodiments, an electronic device implementing video encoder 1300 executes process 1500.

[0108] At block 1510, the encoder receives data as the current pixel block in the current image for encoding. At block 1520, the encoder determines a model for predicting at least one target sample based on one or more neighboring source sample sets. The at least one target sample is a chroma sample. In some embodiments, the model is used to predict multiple target samples (which may be all chroma samples) based on the neighboring source sample set. The neighboring source sample set may include downsampled or non-downsampled luminance samples. In some embodiments, a set of MxN samples provides the PxQ target sample, with the remainder serving as source samples.

[0109] At block 1530, the encoder applies a model to generate a predictor for the current block, specifically generating predictions for the chroma samples of the current block. In some embodiments, the model is applied to samples in the current block as well as samples in template regions adjacent to the current block to generate the predictor for the current block.

[0110] In some embodiments, the model is provided by candidates selected from a plurality of candidates, i.e., a merged list including spatially adjacent, spatially non-adjacent, historical, or default candidates. In some embodiments, the model is obtained by using samples in a reference region as corresponding source and target samples for the filter. In some embodiments, the model includes filter coefficients obtained by minimizing the mean squared error (MSE) between the corresponding source and target samples (e.g., the model may be a 7-tap convolutional filter with 5-tap plus sign spatial components, a nonlinear term, and a bias term).

[0111] At block 1540, the encoder uses the generated predictor to encode the current block to generate a prediction block and produce prediction residuals.

[0112] VII. Example Video Decoder

[0113] In some embodiments, the encoder may signal (or generate) one or more syntax elements in the bitstream so that the decoder may parse the one or more syntax elements from the bitstream.

[0114] Figure 16 An example video decoder 1600 capable of implementing Multi-Source Sample Prediction (MSSBP) is shown. As shown, the video decoder 1600 is an image or video decoding circuit that receives a bitstream 1695 and decodes the bitstream content into pixel data of video frames for display. The video decoder 1600 has multiple components or modules for decoding the bitstream 1695, including some components selected from an inverse quantization module 1611, an inverse transform module 1610, an intra-frame prediction module 1625, a motion compensation module 1630, an in-loop filter 1645, a decoded image buffer 1650, an MV buffer 1665, an MV prediction module 1675, and a parser 1690. The motion compensation module 1630 is part of the inter-frame prediction module 1640. The intra-frame prediction module 1625 is part of the current image prediction module 1620, which uses current image reconstructed samples as reference samples for predicting the current block.

[0115] In some embodiments, modules 1610–1690 are software instruction modules executed by one or more processing units (e.g., processors) of a computing device. In some embodiments, modules 1610–1690 are hardware circuit modules implemented by one or more integrated circuits of an electronic device. Although modules 1610–1690 are shown as independent modules, some modules may be combined into a single module.

[0116] Parser 1690 (or entropy decoder) receives bitstream 1695 and performs preliminary parsing according to the syntax defined by the video coding or image coding standard. The parsed syntax elements include various header elements, flags, and quantization data (or quantization coefficients) 1612. Parser 1690 then parses out the various syntax elements using entropy coding techniques, such as context-adaptive binary arithmetic codec (CABAC) or Huffman coding.

[0117] The inverse quantization module 1611 performs inverse quantization on the quantized data (or quantization coefficients) 1612 to obtain transform coefficients, and the inverse transform module 1610 performs inverse transform on the transform coefficients 1616 to generate a reconstructed residual signal 1619. The reconstructed residual signal 1619 is added to the predicted pixel data 1613 from the intra-frame prediction module 1625 or the motion compensation module 1630 to generate decoded pixel data 1617. The decoded pixel data is filtered by the in-loop filter 1645 and then stored in the decoded image buffer 1650. In some embodiments, the decoded image buffer 1650 is an external memory of the video decoder 1600. In some embodiments, the decoded image buffer 1650 is an internal memory of the video decoder 1600.

[0118] Intra-prediction module 1625 receives intra-prediction data from bitstream 1695 and generates predicted pixel data 1613 from decoded pixel data 1617 stored in decoded image buffer 1650 accordingly. In some embodiments, decoded pixel data 1617 is also stored in line buffer 1627 (or intra-prediction buffer) for intra-prediction and spatial MV prediction.

[0119] In some embodiments, the contents of the decoded image buffer 1650 are used for display. The display device 1605 may directly retrieve the contents of the decoded image buffer 1650 for display, or retrieve the contents of the decoded image buffer into a display buffer. In some embodiments, the display device receives pixel values ​​from the decoded image buffer 1650 via pixel transfer.

[0120] The motion compensation module 1630 generates predicted pixel data 1613 from the decoded pixel data 1617 stored in the decoded image buffer 1650 based on the motion compensation MV (MC MV). These motion compensation MVs are decoded by adding the residual motion data from the bitstream 1695 to the predicted MV from the MV prediction module 1675.

[0121] The MV prediction module 1675 generates a predicted MV based on a reference MV generated for decoding a previous video frame, such as a motion-compensated MV used to perform motion compensation. The MV prediction module 1675 retrieves the reference MV of the previous video frame from the MV buffer 1665. The video decoder 1600 stores the motion-compensated MV generated for decoding the current video frame in the MV buffer 1665 as a reference MV for generating the predicted MV.

[0122] The loop filter 1645 performs filtering or smoothing operations on the decoded pixel data 1617 to reduce coding artifacts, especially at pixel block boundaries. In some embodiments, the filtering or smoothing operations performed by the loop filter 1645 include deblocking filtering (DBF), sample adaptive offset (SAO), and / or adaptive loop filtering (ALF). In some embodiments, luminance-mapped chrominance scaling (LMCS) is performed before loop filtering.

[0123] Figure 17 The diagram illustrates a portion of the video decoder 1600 that implements cross-component MSSBP. The cross-component MSSBP model builder 1705 retrieves source and target samples to construct a CC-MSSBP model. See above for some examples of the relative positions of the source and target samples. Figure 12 The target sample is a chroma sample. The source sample may include luminance and / or chroma samples. The luminance sample may be downsampled or non-downsampled. See [link to relevant documentation] for some examples of the relative positions of the source luminance / chroma samples and the target chroma sample. Figure 7 -11. The source sample can be a reconstructed sample (from line buffer 1627 and / or reconstructed image buffer 1650) and may include luminance and / or chrominance samples. The source sample can be located in a template region defined by a selected template region type, such as EIP_T, EIP_L, EIP_LT.

[0124] The retrieved template samples can be used to derive one or more cross-component MSSBP models via regression based on a selected filter kernel shape. Each MSSBP model is a filter containing the coefficients of a set of filter taps. Example filter kernel shapes are shown in Section 3 above. In some implementations, the entropy decoder 1690 can specify the type or shape of the filter to be derived.

[0125] In some implementations, the previously derived cross-component MSSBP model is stored in memory 1710 and can be retrieved for the current block. In some implementations, the cross-component MSSBP model may be provided by candidates selected from a merge candidate list. The merge candidate list may include spatially adjacent, spatially non-adjacent, history-based, and / or default candidates. Selector 1725 may select a merge candidate and retrieve the corresponding CC-MSSBP model for the current block. The selection of CC-MSSBP merge candidates may be provided by entropy decoder 1690 by parsing relevant syntax elements in bitstream 1695.

[0126] Filter 1720 applies the selected CC-MSSBP model to generate chroma predictions 1730 for each target location in the current block, as described in Section 4 above. Filter 1720 can use adjacent reconstructed samples (provided by the reconstructed image buffer 1650 or line buffer 1627), MSSBP predicted samples (provided by the MSSBP predictor 1725), or predicted samples (provided by the intra-frame prediction module 1625) as filtering inputs at the source sample locations specified by the selected filter shape. The filter can traverse the current block, as described above. Figure 3 The diagonal pattern generates chromaticity samples for the predictor. Predictions for different target positions in the current block are provided as MSSBP predictor 1730 and used as predicted pixel data 1613.

[0127] Figure 18 The conceptual illustration illustrates a process 1800 for decoding pixel blocks using cross-component MSSBP. In some embodiments, one or more processing units (e.g., processors) of a computing device implementing decoder 1600 execute process 1800 by executing instructions stored in a computer-readable medium. In some embodiments, an electronic device implementing decoder 1600 executes process 1800.

[0128] At block 1810, the decoder receives data as the current pixel block in the current image for decoding. At block 1820, the decoder determines a model for predicting at least one target sample based on one or more neighboring source sample sets. The at least one target sample is a chroma sample. In some implementations, the model is used to predict multiple target samples (which may be all chroma samples) based on the neighboring source sample set. The neighboring source sample set may include downsampled or non-downsampled luminance samples. In some implementations, a set of MxN samples provides the PxQ target sample, with the remainder serving as source samples.

[0129] At block 1830, the decoder applies a model to generate a predictor for the current block, specifically, to generate predictions for the chroma samples of the current block. In some implementations, the model is applied to samples in the current block as well as samples in template regions adjacent to the current block to generate the predictor for the current block.

[0130] In some implementations, the model is provided by a pool of candidates selected from multiple candidates, including a merged list of spatially adjacent, spatially non-adjacent, historical, or default candidates. In some implementations, the model is obtained by using samples in a reference region as corresponding source and target samples for the filter. In some implementations, the model includes filter coefficients obtained by minimizing the mean squared error (MSE) between the corresponding source and target samples (e.g., the model could be a 7-tap convolutional filter with 5-tap plus sign spatial components, a nonlinear term, and a bias term).

[0131] At block 1840, the decoder uses the generated predictor to reconstruct the current block, generating a predicted block and combining it with the prediction residual. The decoder can then provide the reconstructed current block as part of the reconstructed current image for display.

[0132] VIII. Example Electronic Systems

[0133] Many of the features and applications described above are implemented as software processes, which are specified as a set of instructions recorded on a computer-readable storage medium (also known as a computer-readable medium). When these instructions are executed by one or more computing or processing units (such as one or more processors, processor cores, or other processing units), they cause the processing unit to perform the operations indicated in the instructions. Examples of computer-readable media include, but are not limited to, CD-ROMs, flash drives, random-access memory (RAM) chips, hard disks, erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), etc. Computer-readable media do not include carrier waves and electronic signals transmitted via wireless or wired connections.

[0134] In this specification, the term "software" includes firmware residing in read-only memory or application programs stored in magnetic memory, which can be read into memory and processed by a processor. Furthermore, in some embodiments, multiple software inventions may be implemented as sub-parts of a larger program while still remaining distinct software inventions. In some embodiments, multiple software inventions may also be implemented as independent programs. Finally, any combination of independent programs that implement the software inventions described herein falls within the scope of this disclosure. In some embodiments, when a software program is installed on one or more electronic systems and runs, one or more specific machine implementations are defined to execute and perform the operations of the software program.

[0135] Figure 19 An electronic system 1900 is conceptually illustrated, with some embodiments implemented on this electronic system. The electronic system 1900 of this disclosure can be a computer (e.g., a desktop computer, personal computer, tablet computer, etc.), a telephone, a PDA, or any other type of electronic device. Such electronic systems include various types of computer-readable media and their interfaces. The electronic system 1900 includes a bus 1905, a processing unit 1910, a graphics-processing unit (GPU) 1915, system memory 1920, a network 1925, read-only memory 1930, permanent storage device 1935, input device 1940, and output device 1945.

[0136] Bus 1905 represents the various system, peripheral, and chipset buses within electronic system 1900, responsible for connecting numerous devices within the system. For example, bus 1905 connects processing unit 1910 with GPU 1915, read-only memory 1930, system memory 1920, and permanent storage device 1935.

[0137] Processing unit 1910 retrieves instructions from these different memory units for execution, and retrieves data for processing, thereby performing the processes of this disclosure. The processing unit may be a single-processor or a multi-core processor in different embodiments. Some instructions are passed to GPU 1915 and executed thereon. GPU 1915 may offload various computing or supplemental image processing provided by processing unit 1910.

[0138] Read-only memory (ROM) 1930 stores static data and instructions used by processing unit 1910 and other modules of the electronic system. Permanent storage device 1935 is a read-write storage device. Permanent storage device 1935 is a non-volatile storage unit that can store instructions and data even when the electronic system 1900 is powered off. Some embodiments of this disclosure use mass storage devices (such as magnetic disks or optical disks and their corresponding disk drives) as permanent storage device 1935.

[0139] Other embodiments use removable storage devices (such as floppy disks, flash memory devices, etc., and their corresponding disk drives) as permanent storage devices. Similar to permanent storage device 1935, system memory 1920 is also a read-write storage device. However, unlike storage device 1935, system memory 1920 is volatile read-write memory, such as random access memory. System memory 1920 stores some instructions and data used by the processor during runtime. In some embodiments, processes according to this disclosure are stored in system memory 1920, permanent storage device 1935, and / or read-only memory 1930. For example, various storage units include instructions for processing multimedia segments, according to some embodiments. From these different storage units, processing unit 1910 retrieves instructions for execution and retrieves data for processing, thereby performing the processes of some embodiments.

[0140] Bus 1905 also connects input device 1940 and output device 1945. Input device 1940 enables a user to pass information to the electronic system and select commands. Input device 1940 includes an alphanumeric keypad and pointing device (also known as a "cursor control device"), a camera (such as a webcam), a microphone, or similar devices for receiving voice commands. Output device 1945 displays images generated by the electronic system or otherwise outputs data. Output device 1945 includes printers and display devices such as cathode ray tubes (CRTs) or liquid crystal displays (LCDs), as well as speakers or similar audio output devices. Some embodiments include devices such as touchscreens that function as both input and output devices.

[0141] Finally, as Figure 19 As shown, bus 1905 also connects electronic system 1900 to network 1925 via a network adapter (not shown). In this way, the computer can become part of a computer network (such as a local area network (LAN), wide area network (WAN), or intranet, or a network of networks such as the Internet). Any or all components of electronic system 1900 can be used in conjunction with this disclosure.

[0142] Some embodiments include electronic components, such as microprocessors, storage, and memory, that store computer program instructions in a machine-readable or computer-readable medium (also referred to as a computer-readable storage medium, machine-readable medium, or machine-readable storage medium). Examples of such computer-readable media include random access memory (RAM), read-only memory (ROM), read-only compact discs (CD-ROM), recordable compact discs (CD-R), rewritable compact discs (CD-RW), read-only digital versatile discs (DVD-ROM, dual-layer DVD-ROM), various recordable / rewritable DVDs (such as DVD-RAM, DVD-RW, DVD+RW, etc.), flash memory (such as SD cards, mini-SD cards, micro-SD cards, etc.), magnetic and / or solid-state drives, read-only and recordable Blu-ray discs, ultra-high density optical discs, any other optical or magnetic media, and floppy disks. Computer-readable media may store a computer program executable by at least one processing unit and contain a set of instructions for performing various operations. Examples of computer programs or computer code include machine code, such as code generated by a compiler, and files containing high-level code that are executed by a computer, electronic component, or microprocessor via an interpreter.

[0143] While the foregoing discussion primarily concerns microprocessors or multi-core processors executing software, many of the aforementioned features and applications are implemented by one or more integrated circuits, such as application-specific integrated circuits (ASICs) or field-programmable gate arrays (FPGAs). In some embodiments, such integrated circuits execute instructions stored on the circuit itself. Furthermore, some embodiments execute software stored in programmable logic devices (PLDs), ROM, or RAM devices.

[0144] In this specification and any claim of this application, the terms "computer," "server," "processor," and "memory" refer to electronic or other technical devices. These terms do not include individuals or groups. For the purposes of this specification, the term "display" or "show" means to be displayed on an electronic device. In this specification and any claim of this application, the terms "computer-readable medium," "computer-readable media," and "machine-readable medium" are fully defined as tangible, physical objects that store information and exist in a computer-readable form. These terms do not include any wireless signals, wired download signals, or other temporary signals.

[0145] Although this disclosure has been described in conjunction with numerous specific details, those skilled in the art will recognize that this disclosure may take other specific forms without departing from its spirit. Furthermore, many figures (including...) Figure 15 and Figure 18 The process is conceptually described. The specific operations of these processes may not be performed in the order shown and described. Specific operations may not be performed in a continuous sequence of operations; different specific operations may be performed in different embodiments. Furthermore, a process may be implemented through multiple sub-processes or as part of a larger macro-process. Therefore, those skilled in the art should understand that this disclosure should not be limited to the foregoing illustrative details but should be defined by the claims.

[0146] Additional notes

[0147] The topics described herein sometimes exemplify different components contained in or connected to other different components. It should be understood that such illustrated architectures are merely examples, and many other architectures can actually be implemented to achieve the same functionality. Conceptually, any arrangement of components to achieve the same functionality is effectively “associated” to achieve the desired function. Therefore, any two combinations of components in this document to achieve a particular function can be considered “associated” to achieve the desired function, regardless of the architecture or intermediate components. Similarly, any two such associated components can also be considered “operably connected” or “operably coupled” to achieve the desired function, and any two components that can be suchly associated can also be considered “operably coupled” to achieve the desired function. Specific examples of operable coupling include, but are not limited to, physically matable and / or physically interactive components and / or wirelessly interactive and / or logically interactive components.

[0148] Furthermore, regarding the use of almost all plural and / or singular terms in this document, those skilled in the art can appropriately convert from plural to singular and / or from singular to plural depending on the context and / or application. For clarity, various singular / plural permutations are explicitly listed herein.

[0149] Furthermore, those skilled in the art will understand that terms commonly used in this specification, particularly in claims, such as the body of a claim, are generally considered "open-ended" terms. For example, "comprising" should be interpreted as "including but not limited to," "having" should be interpreted as "having at least," and "including" should be interpreted as "including but not limited to," etc. Those skilled in the art will also further understand that if a particular quantity introduced in a claim is intentional, that intention will be explicitly stated in the claim; if no such statement is made, the intention does not exist. For example, for ease of understanding, a claim may include the use of the introductory phrases "at least one" and "one or more" to introduce the content of the claim. However, the use of such phrases should not be construed as limiting any particular claim containing that content to containing only one instance of that content, even if the same claim contains the introductory phrases "one or more" or "at least one" and indefinite articles such as "one" or "one," for example, "one" and / or "one" should be interpreted as "at least one" or "one or more"; the same applies to definite articles used to introduce the content of the claim. Furthermore, even if a specific number of introduced elements is explicitly stated in the claims, those skilled in the art will recognize that such a statement should be interpreted as at least the number stated. For example, stating "two elements" alone, without further modification, means at least two elements, or two or more elements. Additionally, when using conventions such as "at least one A, B, and C, etc.", such structures are generally intended for those skilled in the art to understand the meaning of the convention. For example, "a system having at least one A, B, and C" includes, but is not limited to, systems with only A, only B, only C, A and B, A and C, B and C, and systems where A, B, and C are present simultaneously. Similarly, when using conventions such as "at least one A, B, or C, etc.", such structures are generally intended for those skilled in the art to understand the meaning of the convention. For example, "a system having at least one A, B, or C" includes, but is not limited to, systems with only A, only B, only C, A and B, A and C, B and C, and systems where A, B, and C are present simultaneously. Those skilled in the art will further understand that virtually any disjunctive words and / or phrases appearing in the specification, claims, or drawings, presenting two or more alternative terms, should be understood to include the possibility of including one term, any two terms, or all of the terms. For example, the phrase “A or B” should be understood as including the possibility of “A” or “B” or “A and B”.

[0150] As can be seen from the foregoing, various embodiments of this disclosure have been described for illustrative purposes, and various modifications can be made without departing from the scope and spirit of this disclosure. Therefore, the various embodiments disclosed in this specification are not intended to be limiting, and the true scope and spirit are determined by the claims.

Claims

1. A video encoding / decoding method, comprising: Receive data of a current pixel block as a current image of a video for encoding or decoding; Based on one or more neighboring source sample sets, determine a model for predicting multiple target sample sets; The model is applied to generate a predictor that includes chromaticity sample predictions for the current block; as well as The generated predictor is used to encode or decode the current block.

2. The video encoding and decoding method as described in claim 1, wherein the adjacent source sample set includes downsampled luminance samples.

3. The video encoding and decoding method as described in claim 1, wherein the adjacent source sample set includes non-downsampled luminance samples.

4. The video encoding / decoding method as described in claim 1, wherein the model is applied to samples in the current block and samples in a template region adjacent to the current block to generate the predictor for the current block.

5. The video encoding / decoding method as described in claim 1, wherein the model includes filter coefficients obtained by minimizing the mean square error between corresponding source samples and target samples.

6. The video encoding / decoding method as described in claim 1, wherein the model is provided by a candidate selected from a plurality of candidate options.

7. The video encoding / decoding method as claimed in claim 1, wherein the model is obtained by using samples in a reference region as corresponding source and target samples of a filter.

8. The video encoding and decoding method as described in claim 1, wherein a set of MxN samples provides PxQ target samples, and the remainder are used as source samples.

9. An electronic device, comprising: A video codec circuit is configured to perform the following operations: Receive data of a current pixel block as a current image of a video for encoding or decoding; Based on one or more neighboring source sample sets, determine a model for predicting multiple target sample sets; The model is applied to generate a predictor that includes chromaticity sample predictions for the current block; as well as The generated predictor is used to encode or decode the current block.

10. A video decoding method, comprising: Receive data of a current pixel block as a current image of a video for decoding; Based on one or more neighboring source sample sets, determine a model for predicting multiple target sample sets; The model is applied to generate a predictor that includes chromaticity sample predictions for the current block; as well as Reconstruct the current block using the generated predictor.

11. A video encoding method, comprising: Receive data of a current pixel block as a current image of a video for encoding; Based on one or more neighboring source sample sets, determine a model for predicting multiple target sample sets; The model is applied to generate a predictor that includes chromaticity sample predictions for the current block; as well as The generated predictor is used to encode the current block.

Images