Adjacent sampling access for cross-component non-linear mode derivation

Abstract

A method for deriving a cross-component model or a convolution model when encoding or decoding a current block of a current picture of a video is provided. A video codec may retrieve a set of reconstructed component samples for a first set of pixel positions required to derive a component prediction model for a current block. The video codec generates a set of padding component samples for a second set of pixel locations, which is required to derive a component prediction model based on the retrieved set of reconstructed component samples. The video codec derives a component prediction model by using the captured set of reconstructed component samples and the generated set of padding component samples. A video codec encodes or decodes a current block by using a component prediction model to generate a predictor of the current block.

Description

[0001] Cross-references

[0002] This application is part of a non-provisional application claiming priority to U.S. Provisional Patent Application No. 63 / 369,086 filed on July 22, 2022. The above application is hereby incorporated by reference in its entirety. Technical Field

[0003] The present application relates to video coding and decoding, and in particular to a method for coding and decoding pixel blocks by inferring a cross-component model. Background Art

[0004] Unless otherwise indicated herein, the methods described in this section are not prior art to the claims listed below and are not admitted to be prior art by inclusion in this section.

[0005] High Efficiency Video Codec (HEVC) is an international video codec standard developed by the Joint Collaboration Group on Video Codecs (JCT-VC). HEVC is based on a hybrid block-based motion compensated DCT-like transform codec architecture. The basic unit of compression, called a codec unit (CU), is a 2Nx2N square block, and each CU can be recursively split into four smaller CUs until a predetermined minimum size is reached. Each CU contains one or more prediction units (PUs).

[0006] Versatile Video Codec (VVC) is the latest international video codec standard developed by the Joint Video Experts Group (JVET) of ITU-T SG16 WP3 and ISO / IEC JTC1 / SC29 / WG11. The input video signal is predicted from the reconstructed signal, which is derived from the coded picture area. The predicted residual signal is processed by block transform. The transform coefficients are quantized and entropy coded together with other side information in the bitstream. The reconstructed signal is generated by the prediction signal and the reconstructed residual signal after inverse transforming the dequantized transform coefficients. The reconstructed signal is further processed by in-loop filtering to remove coding artifacts. The decoded pictures are stored in the frame buffer and used to predict future pictures in the input video signal.

[0007] In VVC, the coded picture is divided into non-overlapping square block areas, represented by related codec tree units (CTUs). The leaf nodes of the codec tree correspond to codec units (CUs). The coded picture can be represented by a set of slices, each slice consisting of an integer number of CTUs. The CTUs in a slice are processed continuously by raster scanning. Bi-predictive (abbreviated as B) slices can be decoded using intra prediction or inter prediction, using up to two motion vectors and reference indices to predict the sample values ​​of each block. Predictive (abbreviated as P) slices are decoded using intra prediction or inter prediction, using up to one motion vector and reference index to predict the sample values ​​of each block. Intra (abbreviated as I) slices are decoded only using intra prediction.

[0008] A CTU can be partitioned into one or more non-overlapping codec units (CUs) using a quadtree (QT) with a nested multi-type tree (MTT) structure to accommodate various local motion and texture characteristics. A CU can be further partitioned into smaller CUs using one of five partition types (quadtree partition, vertical binary tree partition, horizontal binary tree partition, vertical center-side ternary tree partition, horizontal center-side ternary tree partition).

[0009] Each CU contains one or more prediction units (PUs). The prediction unit, together with the associated CU syntax, works as a basic unit for signaling prediction sub-information. A specified prediction process is used to predict the values ​​of the relevant pixel samples within the PU. Each CU may contain one or more transform units (TUs) for representing prediction residual blocks. A transform unit (TU) consists of a transform block (TB) of luma samples and two corresponding transform blocks of chroma samples, each TB corresponding to a sampled residual block of a color component. Integer conversion is applied to the transform block. The level value of the quantized coefficient is entropy encoded in the bitstream along with other auxiliary information. The terms codec tree block (CTB), codec block (CB), prediction block (PB), and transform block (TB) are defined as a two-dimensional array of samples specifying a monochrome component associated with a CTU, CU, PU, ​​and TU, respectively. Therefore, a CTU consists of a luma CTB, two chroma CTBs, and related syntax elements. Similar relationships apply to CUs, PUs, and TUs.

[0010] For each inter-predicted CU, motion parameters consisting of motion vector, reference picture index, reference picture list usage index, and additional information are used for inter-prediction sample generation. The motion parameters can be communicated explicitly or implicitly. When a CU is encoded in skip mode, the CU is associated with one PU and has no significant residual coefficients, no coded motion vector increments or reference picture indexes. A merge mode is specified in which the motion parameters of the current CU are obtained from neighboring CUs, including spatial and temporal candidates and additional timetables introduced in VVC. This merge mode can be applied to any inter-predicted CU. An alternative to the merge mode is the explicit transmission of motion parameters, in which motion vectors, corresponding reference picture indexes for each reference picture list, reference picture list usage flags, and other required information are explicitly communicated in each CU. Summary of the invention

[0011] The following summary is illustrative only and is not intended to be limiting in any way. That is, the following summary is provided to introduce the concepts, highlights, benefits, and advantages of the novel and non-obvious technologies described herein. Selected but not all embodiments are further described in the detailed description below. Therefore, the following summary is not intended to identify essential features of the claimed subject matter, nor is it intended to be used to determine the scope of the claimed subject matter.

[0012] Some embodiments provide a method for deriving a cross-component model or a convolutional model when encoding or decoding a current block of a current picture of a video. A video codec may retrieve a set of reconstructed component samples for a first set of pixel positions required to derive a component prediction model for the current block. The video codec generates a set of padding component samples for a second set of pixel positions, which is required to derive the component prediction model based on the retrieved set of reconstructed component samples. The encoder may generate each padding component sample by copying the nearest reconstructed component sample retrieved, or a predefined value, or an intermediate value defined by the bit depth of the current sample. The video codec derives the component prediction model by using the retrieved set of reconstructed component samples and the generated set of padding component samples. The video codec encodes or decodes the current block using the component prediction model to generate a predictor for the current block.

[0013] In some embodiments, a second set of pixel positions is designated as having unavailable or prohibited reconstruction samples based on a boundary. The second set of pixel positions may correspond to component samples within the current block. The second set of pixel positions may correspond to component samples in an upper left adjacent region of the current block. The second set of pixel positions may be located outside of a data pipeline unit surrounding the current block. The boundary may be a range boundary of a buffer storing one or more rows of reconstructed codec tree units (CTUs).

[0014] In some embodiments, the component prediction model is derived from the corresponding target component samples and the prediction component samples, where each prediction component sample is generated based on the collocated component sample and a set of surrounding component samples (e.g., "N", "S", "W", "E", "NW", "NE", "SE", "SW" samples surrounding the collocated sample "C"). In some embodiments, the surrounding component samples of the pixel positions beyond the boundary are replaced by padding component samples. The component prediction model can be a cross-component model, so that the target component sample is a chroma sample, and the collocated component sample and the surrounding component samples are luma samples.

[0015] In some embodiments, the target component samples and the collocated component samples are samples of a reference region adjacent to the current block. Some of the surrounding component samples may be samples in an extended region just beyond the reference region. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The accompanying drawings are included to provide a further understanding of the present disclosure and are incorporated into and constitute a part of the present disclosure. The accompanying drawings illustrate embodiments of the present disclosure and, together with the description, are used to explain the principles of the present disclosure. It is worth noting that the accompanying drawings are not necessarily drawn to scale, because in order to clearly illustrate the concepts of the present disclosure, some components may be shown as being out of proportion to the sizes in actual implementation.

[0017] Figure 1 Conceptual illustration of chrominance and luma sampling for deriving linear model parameters.

[0018] Figure 2 An example of classifying adjacent samples into two groups is shown.

[0019] Figure 3 Conceptual illustration of the spatial components of a convolution filter.

[0020] Figure 4 A reference region for deriving filter coefficients of a convolutional model for the current block is shown.

[0021] Figure 5 Reconstructed samples of the current block that are not available for deriving model parameters are shown.

[0022] Figure 6 Reconstructed samples outside of a codec tree unit (CTU) row are shown, which are not available for deriving model parameters for the current block.

[0023] Figure 7 The reconstructed samples in the upper left neighboring area of ​​the current block that are not available for deriving model parameters are shown.

[0024] Fig. 8A -C shows the reconstruction samples that are not available in the current data pipeline unit for exporting model parameters.

[0025] Fig. 9 An example video encoder that may implement a component prediction model is illustrated.

[0026] Fig.10 Portions of a video encoder that derives and uses a component prediction model are described.

[0027] Fig.11 Conceptually illustrates the process of handling unavailable samples when an encoder derives component prediction models.

[0028] Fig.12 An example video decoder that may implement a component prediction model is illustrated.

[0029] Fig.13 Describes portions of a video decoder that derives and uses a component prediction model.

[0030] Fig.14 Conceptually illustrates the process of handling unavailable samples when the decoder derives the component prediction model.

[0031] Fig.15 An electronic system that implements some embodiments of the present disclosure is conceptually illustrated. DETAILED DESCRIPTION

[0032] In the detailed description below, many specific details are set forth by way of example to provide a thorough understanding of the relevant teachings. Any variation, derivation and / or extension based on the teachings described herein are within the scope of protection of the present disclosure. In some cases, well-known methods, processes, components and / or circuits related to one or more example implementations disclosed herein may be described at a relatively high level without detail to avoid unnecessarily obscuring aspects of the teachings of the present disclosure.

[0033] Cross-Component Linear Model (CCLM)

[0034] Cross Component Linear Model (CCLM) or Linear Model (LM) mode is a cross-component chroma prediction mode in which the chroma components of a block are predicted from the collocated reconstructed luma samples by a linear model. The parameters of the linear model (e.g., scale and offset) are derived from the already reconstructed luma and chroma samples adjacent to the block. For example, in VVC, the CCLM mode predicts chroma samples from reconstructed luma samples using inter-channel dependencies. The prediction is performed using a linear model of the following form:

[0035] P(i,j)=α·rec′ L (i,j)+β Equation (1)

[0036] P(i,j) in equation (1) represents the predicted chroma sample in a CU (or the predicted chroma sample of the current CU), rec′ L (i, j) represents the downsampled reconstructed luminance sample of the same CU (or the CU of the reconstructed luminance sample corresponding to the current CU).

[0037] The CCLM model parameters α (scaling parameter) and β (offset parameter) are derived based on up to four adjacent chroma samples and their corresponding downsampled luma samples. In LM_A mode (also called LM-T mode), only the above or top adjacent template is used to calculate the linear model coefficients. In LM_L mode (also called LM-L mode), only the left template is used to calculate the linear model coefficients. In LM-LA mode (also called LM-LT mode), both the left and above templates are used to calculate the linear model coefficients.

[0038] Figure 1 The chrominance and luma samples used to derive linear model parameters are conceptually illustrated. The diagram shows a current block 100 having luma component samples and chroma component samples in a 4:2:0 format. The luma and chroma samples adjacent to the current block are reconstructed samples. These reconstructed samples are used to derive the cross-component linear model (parameters α and β). Since the current block is in a 4:2:0 format, the luma samples are downsampled before being used for linear model derivation. In this example, there are 16 pairs of reconstructed luma (downsampled) and chroma samples adjacent to the current block. These 16 pairs of luma and chroma values ​​are used to derive linear model parameters.

[0039] Assuming the current chroma block dimension is W×H, W' and H' are set to

[0040] – When LM-LT mode is applied, W'=W, H'=H;

[0041] – When LM-T mode is applied, W'=W+H;

[0042] – When LM-L mode is applied, H'=H+W

[0043] The adjacent positions above are represented as S[0,-1]...S[W'-1,-1], and the adjacent positions on the left are represented as S[-1,0]...S[-1,H'-1]. Then four samples are selected as

[0044] –S[W' / 4,-1],S[3*W' / 4,-1],S[-1,H' / 4],S[-1,3*H' / 4] when LM mode is applied (both upper and left adjacent samples are available);

[0045] –S[W' / 8,-1],S[3*W' / 8,-1],S[5*W' / 8,-1],S[7*W' / 8,-1] when LM-T mode is applied (only top adjacent samples are available);

[0046] –S[-1,H' / 8],S[-1,3*H' / 8],S[-1,5*H' / 8],S[-1,7*H' / 8] when LM-L mode is applied (only left adjacent samples are available);

[0047] The four adjacent brightness samples at the selected position are downsampled and compared four times to find the two larger values: x 0A and x 1A , and two smaller values: x 0B and x 1B Their corresponding chroma sampling values ​​are represented by y 0A ,y 1A ,y 0B and 1B Then X A , X B , Y A and Y B It is derived as follows:

[0048] X a =(x 0A +x 1A +1)>>1;X b =(x 0B +x 1B +1)>>1; Equation (2)

[0049] Y a =(y 0A +y 1A +1)>>1;Y b =(y 0B +y 1B +1)>>1 Equation (3)

[0050] The linear model parameters α and β are obtained according to the following equations

[0051]

[0052] Equation (4)

[0053] β=Y b -α·X b Equation (5)

[0054] The operation of calculating the α and β parameters according to equations (4) and (5) can be implemented by a lookup table. In some embodiments, in order to reduce the memory required to store the lookup table, the diff value (the difference between the maximum and minimum values) and the parameter α are represented by exponential notation. For example, diff is approximated by a 4-bit significant part and an exponent. Therefore, for 16 significant digit values, the table of 1 / diff is reduced to 16 elements, as shown below:

[0055] DivTable[] = {0,7,6,5,5,4,4,3,3,2,2,1,1,1,1,0} Equation (6)

[0056] This reduces the complexity of the calculations and the amount of memory required to store the required tables.

[0057] In some embodiments, in order to obtain more samples for calculating CCLM model parameters α and β, the above template is expanded to include (W+H) samples for LM-T mode, and the left template is expanded to include (H+W) samples for LM-L mode. For LM-LT mode, both the expanded left template and the expanded upper template are used to calculate the linear model coefficients.

[0058] In order to match the chroma sampling positions of the 4:2:0 video sequence, two types of downsampling filters are applied to the luma samples to achieve a 2:1 horizontal and vertical downsampling ratio. The choice of downsampling filter is specified by the sequence parameter set (SPS) level flag. The two downsampling filters are as follows, corresponding to "type-0" and "type-2" content, respectively.

[0059] rec′ L (i, j) = [rec L (2i-1, 2j-1)+2j-1)+2*rec L (2i-1, 2j-1)+rec L (2i+1,2j-1)+rec L (2i-1,2j)+rec L (2i+1,2j)+4]>>3 Equation (7)

[0060] rec′ L (i, j) = [rec L (2i, 2j-1)+rec L (2i-1,2j)+4*rec L (2i,2j)+rec L (2i+1,2j)+rec L (2i, 2j+1)+4]>>3 Equation (8)

[0061] In some embodiments, the α and δ parameter calculations are performed as part of the decoding process, rather than just as an encoder search operation. Therefore, there is no syntax for conveying α and δ values ​​to the decoder.

[0062] For chroma intra mode coding, a total of 8 intra modes are allowed. These modes include five traditional intra modes and three cross-component linear model modes (LM_LA, LM_A, and LM_L). Chroma intra mode encoding and decoding can directly depend on the intra prediction mode of the corresponding luma block. Chroma intra mode signaling and the corresponding luma intra prediction mode are according to the following table:

[0063]

[0064]

[0065] Since separate block partitioning structures for luma and chroma components are enabled in I slices, one chroma block can correspond to multiple luma blocks. Therefore, for the chroma DM (**What is DM**) mode, the intra prediction mode of the corresponding luma block covering the center position of the current chroma block is directly inherited.

[0066] A single unified binarization table (mapped to bin string) is used for chroma intra prediction modes according to the following table:

[0067]

[0068] In the table, the first bin indicates whether it is normal (0) or LM mode (1). If it is LM mode, the next bin indicates whether it is LM_chroma (0). If it is not LM_chroma, the next 1bin indicates whether it is LM_L (0) or LM_A (1). For this case, when sps_cclm_enabled_flag is 0, the first bin of the binarization table of the corresponding intra_chroma_pred_mode can be discarded before entropy coding. Or, in other words, the first bin is inferred to be 0 and therefore not encoded. This single binarization table is used for the cases where sps_cclm_enabled_flag is equal to 0 and 1. The first two bins in the table are context encoded using their own context model, and the remaining bins are bypass encoded.

[0069] Additionally, to reduce luma-chroma latency in dual trees, when the 64x64 luma codec tree node is not split (and ISP is not used for 64x64 CUs) or QT partitioning is applied, the chroma CU tree nodes in 32x32 / 32x16 chroma encoding can use CCLM in the following ways:

[0070] If the 32x32 chroma node is not split or QT is split partitioned, then all chroma CUs in the 32x32 node can use CCLM

[0071] If a 32x32 chroma node uses horizontal BT splitting and the 32x16 child node does not split or uses vertical BT splitting, all chroma CUs in the 32x16 chroma node can use CCLM.

[0072] In all other luma and chroma codec tree split conditions, CCLM is not allowed for chroma CUs.

[0073] II. Multi-Model CCLM (MMLM)

[0074] The multi-model CCLM mode (MMLM) uses two models to predict chroma samples from the luma samples of the entire CU. Similar to CCLM, three multi-model CCLM modes (MMLM_LA, MMLM_A, and MMLM_L) are used to indicate whether to use both the upper and left neighboring samples, only the upper neighboring samples, or only the left neighboring samples in the derivation of model parameters.

[0075] In MMLM, the adjacent luma samples and adjacent chroma samples of the current block are divided into two groups, each of which is used as a training set to derive a linear model (i.e., derive a specific α and β for a specific group). In addition, the samples of the current luma block are also classified according to the same rules as the classification of adjacent luma samples.

[0076] Figure 2 An example of classifying adjacent samples into two groups is shown. The threshold is calculated as the average value of adjacent reconstructed brightness samples. For adjacent samples at [x, y], if Rec' L [x, y] <= Threshold, it is classified into the first group; and the adjacent samples at [x, y], if Rec' L [x,y]>Threshold, it is classified into the second group. Therefore, the multi-model CCLM prediction of chroma sampling is:

[0077] Pred c [x,y]=α1×Rec′ L [x,y]+β1 if Rec' L [x,y]≤Threshold

[0078] Pred c [x,y]=α2×Rec′ L [x,y]+β2 if Rec' L [x,y]>Threshold

[0079] III. Convolutional Cross-Component Model (CCCM)

[0080] In some embodiments, a convolutional cross-component model (CCCM) is applied to improve cross-component prediction performance. For some embodiments, the convolution model has a 7-tap filter with a 5-tap plus a signed shape spatial component, a nonlinear term, and a bias term. The input to the spatial 5-tap component of the filter includes a center (C) luma sample (collocated with the chroma sample to be predicted), and the neighboring samples of the upper / north (N), lower / south (S), left / west (W), and right / east (E) of the center luma sample. Figure 3 Conceptual illustration of the spatial components of a convolution filter. The nonlinear term (denoted as P) is expressed as a power of two of the center luma sample C, scaled by the sample value range of the content:

[0081] P=(C*C+midVal)>>bitDepth (9)

[0083] Therefore, for 10-bit content, the nonlinear term P is calculated as:

[0084] P=(C*C+512)>>10

[0085] The bias term (denoted as B) represents a scalar offset between the input and output (similar to the bias term in CCLM) and is set to the intermediate chrominance value (512 for 10-bit content). The output of the filter is calculated as the filter coefficient c i The convolution between the input value and is clipped to the valid chroma sampling range:

[0086] predChromaVal=c0C+c1N+c2S+c3E+c4W+c5P+c6B (10)

[0088] Filter coefficient c iIt is calculated by minimizing the MSE between the reconstructed (or target) chroma sample and the corresponding predicted chroma sample in the reference area. Each predicted chroma sample is generated from the juxtaposed luma sample and its surrounding luma samples using the derived component prediction model (such as formula (10)). Formula (10) is a convolution model based on the center sample and the surrounding 4 samples (C, N, S, E, W). Formula (10) can be extended to include taps for the center sample and the surrounding 8 samples (C, N, S, E, W, NE, NW, SE, SW). Formula (10) and its extended form can be called a component prediction model (because it can be used for cross-component prediction or intra-component prediction).

[0089] Figure 4 A reference region for deriving filter coefficients for a convolution model for a current block is shown. The reference region includes (reference) rows of (chroma) samples above and to the left of the current block 400. (In this example, the current block 400 is a PU). The reference region extends a reference width to the right and a reference height below the PU boundary. The reference region can be adjusted to include only available samples. An extended area of ​​the reference region is used to support "auxiliary sampling" of a plusshaped spatial filter (e.g., Figure 3 The N, E, W, S samples, and NW, NE, SW, SE samples) are filled in when they are unavailable.

[0090] Minimization of MSE can be achieved by calculating the autocorrelation matrix of the luminance input and the cross-correlation vector between the luminance input and the chrominance output. The autocorrelation matrix is ​​decomposed by LDL, and the final filter coefficients are calculated using the inverse permutation method. This process is similar to the calculation of the ALF filter coefficients in ECM, however, in some embodiments, LDL decomposition is selected instead of Cholesky decomposition to avoid the use of square root operations.

[0091] In some embodiments, a high-order model may be used instead of a linear model to predict chroma sampling. The high-order model may include a k-tap space term, a nonlinear term (denoted as P), and a bias term (denoted as B). The high-order model may be expressed as:

[0092] (11)

[0094] in is the downsampled reconstructed brightness sample at position (i, j), yes A neighboring sample around, a0, a x, b, and c are model parameters. The high-order model can be used to derive model parameters between color components, or between reference samples of the current frame / picture and a reference frame / picture.

[0095] IV. Model-derived reconstruction sampling boundaries

[0096] A higher level model mode (eg, Equation 11) may use the above, left, and / or above-left neighboring regions from the current block (eg, Figure 4 In some embodiments, when a desired neighboring area is outside a restriction boundary or "unavailable", the desired reconstructed samples may be replaced by other available / valid samples, set to a predefined value, or set to an intermediate chroma value of the current sampling bit depth. The restriction boundary may be a CTU, slice, tile, picture boundary, data pipeline unit boundary, or any other boundary that results in a restriction or shortage when referencing neighboring reconstructed samples. "Unavailable" may mean that reconstructed samples within the current block are unavailable, or reconstructed samples within the upper left, left, or upper neighboring area are unavailable (e.g., not yet reconstructed).

[0097] In some embodiments, at least some of the reconstructed samples of the current luma block are not available for deriving model parameters. These unavailable reconstructed samples are replaced by other available / valid samples. Figure 5 Reconstructed samples of the current block 500 that are not available for deriving model parameters are shown. In this example, the model derivation requires the reconstruction of samples "NW", "N", "NE", "W", "C", "E", "SW", "S", "SE". However, the reconstruction of samples "SW", "S", "SE" is prohibited / unavailable for deriving model parameters because samples "SW", "S", "SE" are samples of the current block 500 and may not have been reconstructed. In some embodiments, when deriving model parameters for CCLM or CCCM, the unavailable reconstruction samples are replaced by their closest available neighboring samples. Therefore, "S" is replaced by "C", "SE" is replaced by "E", and "SW" is replaced by "W".

[0098] In some embodiments, when deriving model parameters, reconstructed samples outside the CTU line buffer are prohibited / unavailable. These prohibited reconstructed samples can be replaced by other available / valid samples to derive model parameters. Figure 6Reconstructed samples outside the CTU row are shown, which are not available for deriving model parameters for the current block 600. In the figure, the video codec uses a buffer that supports two CTU row lines. Model derivation requires reconstructed samples "NW", "N", "NE", "W", "C", "E", "SW", "S", "SE", but reconstructed samples "NW", "N" and "NE" are outside the CTU row buffer. If the current model requires "NW", "N", "NE" reconstructed samples to derive model parameters, then "NW" is replaced by "W", "N" is replaced by "C", and "NE" is replaced by "E".

[0099] In some embodiments, when deriving model parameters, the reconstruction samples in the upper left neighboring area of ​​the current block are prohibited / unavailable. When deriving model parameters, these prohibited reconstruction samples will be replaced by other available / valid samples. Figure 7 Reconstructed samples in the upper left adjacent area of ​​the current block 700 that are not available for deriving model parameters are shown. As shown, model derivation requires reconstruction samples "NW", "N", "NE", "W", "C", "E", "SW", "S", "SE", but the reconstruction samples "NW" and "N" are located in the upper left adjacent area 710 and are therefore not allowed. Unavailable reconstructed samples are replaced by their respective closest valid reconstructed samples. Therefore, "NW" is replaced by "W" and "N" is replaced by "NE". In addition, if the reconstructed luminance samples from the current block 700 cannot be used (for example, not yet reconstructed), the reconstructed samples "E" and "SE" are prohibited / unavailable for model derivation because they are located in the current luminance block 700 and may not be available. The video codec can replace "E" with "C" and "SE" with "S" when deriving the model.

[0100] Data pipeline units (e.g., VPDUs) are defined as non-overlapping square / rectangular units in the figure. In a hardware decoder, consecutive data pipeline units are processed by multiple pipeline levels simultaneously. Different stages process different pipeline units simultaneously. In some embodiments, reconstruction samples outside the current data pipeline unit are prohibited / unavailable for luma reconstruction downsampling process, derivation of model parameters, or generation of chroma prediction. These prohibited reconstruction samples are replaced by other available / valid samples.

[0101] Fig. 8A -C shows the reconstruction samples that are not available in the current data pipeline unit to derive model parameters. The current model requires "NW", "N", "NE", "W", "C", "E", "SW", "S", "SE" reconstruction samples for model derivation.

[0102] Fig. 8AAn example is shown where reconstructed samples "NW", "W", and "SW" are outside the current data pipeline unit 850. These unavailable reconstructed samples are replaced by their respective closest valid reconstructed samples. Therefore, the video codec replaces "NW" with "N", "W" with "C", and "SW" with "S" because samples "N", "C", and "S" are located within the current data pipeline unit 850.

[0103] Figure 8B An example is shown in which samples "NE", "E", and "SE" are outside the current data pipeline unit 860, and samples "SW" and "S" are in the current block 800. Therefore, these reconstructed samples are not available. The unavailable reconstructed samples are replaced by their respective closest valid reconstructed samples. Therefore, the video codec replaces "NE" with "N", "E" with "C", and "SE" with "C". The video codec also replaces "SW" with "W", and "S" with "C".

[0104] Figure 8C An example is shown in which required reconstructed samples "NE", "E", "SE", "S", and "SW" are outside the current data pipeline unit 870 and are unavailable / prohibited. In this example, the derived model can be a CCLM model, which is used to predict the chroma samples of the current block based on the already reconstructed luma samples of the current block. The unavailable reconstructed samples are replaced by their respective closest valid / available reconstructed samples. Therefore, in order to generate the chroma prediction, "NE" is replaced with "N", "SW" is replaced with "W", and "E", "SE", and "S" are replaced with "C". In this example, the reconstructed luma samples "NW", "N", "W", and "C" of the current block 800 are available.

[0105] Any of the above-mentioned methods can be implemented in an encoder and / or a decoder. For example, any of the methods proposed can be implemented in an inter / intra / prediction module of an encoder, and / or an inter / intra / prediction module of a decoder. Alternatively, any of the methods proposed can be implemented as a circuit coupled to an inter / intra / prediction module of an encoder and / or an inter / intra / prediction module of a decoder to provide information required by the inter / intra / prediction module.

[0106] V. Example Video Encoder

[0107] Fig. 9An example video encoder 900 implementing a component prediction model is illustrated. As shown, the video encoder 900 receives a video source 905 input video signal and encodes the signal into a bitstream 995. The video encoder 900 has several components or modules for encoding the signal from the video source 905, including at least some components selected from the following: a transform module 910, a quantization module 911, an inverse quantization module 914, an inverse transform module 915, an intra-frame estimation module 920, an intra-frame prediction module 925, a motion compensation module 930, a motion estimation module 935, a loop filter 945, a reconstructed picture buffer 950, an MV buffer 965, an MV prediction module 975, and an entropy encoder 990. The motion compensation module 930 and the motion estimation module 935 are part of the inter-frame prediction module 940.

[0108] In some embodiments, modules 910-990 are software instruction modules executed by one or more processing units (e.g., processors) of a computing device or electronic device. In some embodiments, modules 910-990 are hardware circuit modules implemented by one or more integrated circuits (ICs) of an electronic device. Although modules 910-990 are shown as separate modules, some modules can be combined into a single module.

[0109] The video source 905 provides an uncompressed raw video signal that presents pixel data for each video frame. The subtractor 908 calculates the difference between the raw video pixel data of the video source 905 and the predicted pixel data 913 from the motion compensation module 930 or the intra-frame prediction module 925 as a prediction residual 909. The transform module 910 converts the difference (or residual pixel data or residual signal 908) into transform coefficients (e.g., by performing a discrete cosine transform or DCT). The quantization module 911 quantizes the transform coefficients into quantized data (or quantized coefficients) 912, which are encoded into a bitstream 995 by the entropy encoder 990.

[0110] The inverse quantization module 914 inversely quantizes the quantized data (or quantized coefficients) 912 to obtain transform coefficients, and the inverse transform module 915 inversely transforms the transform coefficients to generate a reconstructed residual 919. The reconstructed residual 919 is added to the predicted pixel data 913 to generate reconstructed pixel data 917. In some embodiments, the reconstructed pixel data 917 is temporarily stored in a line buffer (not shown) for intra-frame prediction and spatial MV prediction. The reconstructed pixels are filtered by the loop filter 945 and stored in the reconstructed picture buffer 950. In some embodiments, the reconstructed picture buffer 950 is a storage external to the video encoder 900. In some embodiments, the reconstructed picture buffer 950 is a storage internal to the video encoder 900.

[0111] The intra estimation module 920 performs intra prediction based on the reconstructed pixel data 917 to generate intra prediction data. The intra prediction data is provided to the entropy encoder 990 to be encoded into a bitstream 995. The intra prediction data is also used by the intra prediction module 925 to generate predicted pixel data 913.

[0112] The motion estimation module 935 performs inter-frame prediction by generating MVs to refer to pixel data of a previously decoded frame stored in the reconstructed picture buffer 950. These MVs are provided to the motion compensation module 930 to generate predicted pixel data.

[0113] Instead of encoding the complete actual MV in the bitstream, the video encoder 900 uses MV prediction to generate a predicted MV, and the difference between the MV used for motion compensation and the predicted MV is encoded as residual motion data and stored in the bitstream 995.

[0114] The MV prediction module 975 generates a predicted MV based on a reference MV generated for a previously encoded video frame, i.e., a motion compensated MV for performing motion compensation. The MV prediction module 975 retrieves a reference MV from a previous video frame from the MV buffer 965. The video encoder 900 stores the MV generated for the current video frame in the MV buffer 965 as a reference MV for generating a predicted MV.

[0115] The MV prediction module 975 uses the reference MV to create a predicted MV. The predicted MV can be calculated by spatial MV prediction or temporal MV prediction. The entropy encoder 990 encodes the difference (residual motion data) between the predicted MV and the motion compensation MV (MC MV) of the current frame into a bitstream 995.

[0116] The entropy encoder 990 encodes various parameters and data into a bitstream 995 by using an entropy coding technique such as context adaptive binary arithmetic coding (CABAC) or Huffman coding. The entropy encoder 990 encodes various header elements, flags along with quantized transform coefficients 912 and residual motion data as syntax elements into the bitstream 995. The bitstream 995 is in turn stored in a storage device or transmitted to a decoder via a network communication medium, such as a network.

[0117] The loop filter 945 performs filtering or smoothing operations on the reconstructed pixel data 917 to reduce encoding artifacts, particularly at the boundaries of pixel blocks. In some embodiments, the filtering or smoothing operations performed by the loop filter 945 include deblocking filtering (DBF), sample adaptive offset (SAO) and / or adaptive loop filter (ALF).

[0118] Fig.10Portions of a video encoder 900 that derives and uses a component prediction model are shown. As shown, an initial predictor generation module 1020 provides an initial predictor 1015 to a component predictor model 1010. The initial predictor 1015 may include component samples (luminance or chrominance) of a reference block used to predict component samples of a current block, or samples of the current block used for cross-component prediction. The component prediction model 1010 is applied to the initial predictor 1015 to generate a refined predictor 1025. Samples of the refined predictor 1025 may be used as predicted pixel data 913.

[0119] When the current block is decoded through inter prediction, the motion estimation module 935 provides an MV used by the motion compensation module 930 to identify a reference block in a reference picture. When the current block is encoded and decoded through intra prediction, the intra prediction estimation module 920 provides an intra mode or BV used by the intra prediction module 925 to identify a reference block in the current picture. The initial predictor generation module can provide component samples of the reference block as initial predictors 1015 for the current block. In some embodiments, the luma component samples of the current block are used as the initial predictors for the current block.

[0120] In order to derive the component prediction model 1010, the regression data selection module 1030 retrieves the required component samples from the reconstructed picture buffer 950 to be used as regression data. In some embodiments, the required component samples are component samples of pixels in the reference area and the extended area. Figure 4 To describe the reference area and the extended area. The reference area and the extended area may be in and / or around the current block and in and / or around the reference block. The retrieved regression data (i.e., the required component samples) include a reference sample (X) and a current sample (Y) for determining coefficients or parameters of the component prediction model 1010. The current sample may refer to a target chroma sample, and the reference sample may refer to a luma sample used to generate a corresponding predicted chroma sample, including a collocated luma sample and its surrounding luma samples.

[0121] The regression data selection module 1030 may determine whether a component sample required for model derivation is unavailable or prohibited based on a specific boundary. For example, in some embodiments, the reconstructed samples within the current block may not be available for model derivation; in some embodiments, the reconstructed samples outside the current data pipeline unit may not be available for model derivation. If a component sample is required for model derivation but is unavailable, the regression data selection module may generate a filler sample to replace the unavailable sample by, for example, copying the nearest valid component sample. See above Figure 5 -8 to describe the handling of unavailable component sampling required for model derivation.

[0122] Model builder 1005 uses the regression data (X and Y) to derive parameters of component prediction model 1010 using a technique of elimination, iteration, or decomposition. Component prediction model 1010 may be a higher order model with multiple filter taps, such as equation (10) or equation (11) above.

[0123] Fig.11 A process 1100 for deriving a component prediction model by processing unavailable samples is conceptually illustrated. In some embodiments, one or more processing units (e.g., processors) of a computing device implementing encoder 900 perform process 1100 by executing instructions stored in a computer-readable medium. In some embodiments, an electronic device implementing encoder 900 performs process 1100.

[0124] The encoder receives (at step 1110) data to be encoded as a current pixel block in a current picture. The encoder retrieves (at step 1120) a set of reconstructed component samples for a first set of pixel positions required to derive a component prediction model for the current block. The encoder generates (at step 1130) a set of padding component samples for a second set of pixel positions required to derive a component prediction model based on the retrieved set of reconstructed component samples. The encoder may generate each padding component sample by copying the nearest retrieved reconstructed component sample, or a predefined value, or an intermediate value defined by the bit depth of the current sample.

[0125] In some embodiments, the second set of pixel positions is designated as having unavailable or prohibited reconstruction samples based on a boundary. The second set of pixel positions may correspond to component samples within the current block. The second set of pixel positions may correspond to component samples in an upper left adjacent area of ​​the current block. The second set of pixel positions may be located outside of a data pipeline unit surrounding the current block. This boundary may be a boundary of a buffer storing one or more rows of reconstructed codec tree units (CTUs).

[0126] In some embodiments, a component prediction model is derived from corresponding target component samples and prediction component samples, where each prediction component sample is generated based on a collocated component sample and a set of surrounding component samples (e.g., "N", "S", "W", "E", "NW", "NE", "SE", "SW" samples surrounding the collocated sample "C"). In some embodiments, surrounding component samples of pixel positions beyond the boundary are replaced by padded component samples. The component prediction model can be a cross-component model, so that the target component sample is a chroma sample, and the collocated and surrounding component samples are luma samples.

[0127] In some embodiments, the target component samples and the collocated component samples are samples of a reference region adjacent to the current block. Some of the surrounding component samples may be samples in an extended region just beyond the reference region.

[0128] The encoder (at step 1140) derives a component prediction model by using the captured reconstructed component sample set and the generated padding component sample set. The encoder (at step 1150) encodes the current block by using the component prediction model to generate a predictor for the current block. The predictor for the current block may include predicted chrominance samples generated by applying the component prediction model to luma samples of the current block. The predictor is used to generate a prediction residual.

[0129] VI. Example Video Decoder

[0130] In some embodiments, the encoder may signal (or generate) one or more syntax elements in a bitstream so that the decoder may parse the one or more syntax elements from the bitstream.

[0131] Fig.12 An example video decoder 1200 that can implement a component prediction model is illustrated. As shown, the video decoder 1200 is an image decoding or video decoding circuit that receives a bitstream 1295 and decodes the contents of the bitstream into pixel data of a video frame for display. The video decoder 1200 has several components or modules for decoding the bitstream 1295, including some components selected from an inverse quantization module 1211, an inverse transform module 1210, an intra-frame prediction module 1225, a motion compensation module 1230, a loop filter 1245, a decoded picture buffer 1250, an MV buffer 1265, an MV prediction module 1275, and a parser 1290. The motion compensation module 1230 is part of the inter-frame prediction module 1240.

[0132] In some embodiments, modules 1210-1290 are software instruction modules executed by one or more processing units (e.g., processors) of a computing device. In some embodiments, modules 1210-1290 are hardware circuit modules implemented by one or more ICs of an electronic device. Although modules 1210-1290 are shown as separate modules, some modules may be combined into a single module.

[0133] The parser 1290 (or entropy decoder) receives the bitstream 1295 and performs initial parsing according to the syntax defined by the video coding or image coding standard. The parsed syntax elements include various header elements, flags, and quantization data (or quantization coefficients) 1212. The parser 1290 parses out the various syntax elements using entropy coding techniques such as context adaptive binary arithmetic coding (CABAC) or Huffman coding.

[0134] The inverse quantization module 1211 inversely quantizes the quantized data (or quantized coefficients) 1212 to obtain transform coefficients, and the inverse transform module 1210 inversely transforms the transform coefficients 1216 to obtain a reconstructed residual signal 1219. The reconstructed residual signal 1219 is added to the predicted pixel data 1213 from the intra prediction module 1225 or the motion compensation module 1230 to generate decoded pixel data 1217. The decoded pixel data is filtered by the loop filter 1245 and stored in the decoded picture buffer 1250. As shown in the figure, in some embodiments, the decoded picture buffer 1250 is a storage outside the video decoder 1200. In some embodiments, the decoded picture buffer 1250 is a storage inside the video decoder 1200.

[0135] The intra prediction module 1225 receives intra prediction data from the bitstream 1295 and generates predicted pixel data 1213 from decoded pixel data 1217 stored in the decoded picture buffer 1250. In some embodiments, the decoded pixel data 1217 is also stored in a line buffer (not shown) for intra prediction and spatial MV prediction.

[0136] In some embodiments, the contents of the decoded picture buffer 1250 are used for display. The display device 1255 either retrieves the contents of the decoded picture buffer 1250 for direct display or retrieves the contents of the decoded picture buffer to a display buffer. In some embodiments, the display device receives pixel values ​​from the decoded picture buffer 1250 via pixel transfer.

[0137] The motion compensation module 1230 generates predicted pixel data 1213 from the decoded pixel data 1217 stored in the decoded picture buffer 1250 according to the motion compensated MV (MC MV). These motion compensated MVs are decoded by adding the residual motion data received from the bitstream 1295 to the predicted MV received from the MV prediction module 1275.

[0138] The MV prediction module 1275 generates a predicted MV, for example, a motion compensated MV for performing motion compensation, based on a reference MV generated for decoding a previous video frame. The MV prediction module 1275 retrieves the reference MV of the previous video frame from the MV buffer 1265. The video decoder 1200 stores the motion compensated MV generated for decoding a current video frame in the MV buffer 1265 as a reference MV for generating a predicted MV.

[0139] The loop filter 1245 performs filtering or smoothing operations on the decoded pixel data 1217 to reduce coding artifacts, particularly at the boundaries of pixel blocks. In some embodiments, the filtering operations performed by the loop filter 1245 include deblocking filtering, sample adaptive offset (SAO), and / or adaptive loop filter (ALF).

[0140] Fig.13 Portions of a video decoder 1200 that derives and uses a component prediction model are shown. As shown, an initial predictor generation module 1320 provides an initial predictor 1315 to a component predictor model 1310. The initial predictor 1315 may contain component samples (luminance or chrominance) of a reference block for predicting component samples of a current block, or component samples of a current block for cross-component prediction. The component prediction model 1310 is applied to the initial predictor 1315 to generate a refined predictor 1325. Samples of the refined predictor 1325 may be used as predicted pixel data 1213.

[0141] When the current block is decoded through inter prediction, the entropy decoder 1290 may provide an MV used by the motion compensation module 1230 to identify a reference block in a reference picture. When the current block is decoded through intra prediction, the entropy decoder 1290 may provide an intra mode or BV used by the intra prediction module 1225 to identify a reference block in the current picture. The initial predictor generation module may provide component samples of the reference block as initial predictors 1315 of the current block. In some embodiments, the luma component samples of the current block are used as the initial predictor of the current block.

[0142] To derive the component prediction model 1310, the regression data selection module 1330 retrieves the required component samples from the decoded picture buffer 1250 to be used as regression data. In some embodiments, the required component samples are component samples of pixels in the reference area and the extended area. Figure 4 to describe the reference area and the extended area. The reference area and the extended area may be in and / or around the current block and in and / or around the reference block. The retrieved regression data (i.e., the required component samples) include a reference sample (X) and a current sample (Y) for determining coefficients or parameters of the component prediction model 1310. The current sample may refer to a target chroma sample, and the reference sample may refer to a luma sample used to generate a corresponding predicted chroma sample, including a collocated luma sample and its surrounding luma samples.

[0143] The regression data selection module 1330 may determine whether a component sample required for model derivation is unavailable or prohibited based on a specific boundary. For example, in some embodiments, a reconstructed sample within the current block may not be available for model derivation; in some embodiments, a reconstructed sample outside the current data pipeline unit may not be available for model derivation. If a component sample is required for model derivation but is unavailable, the regression data selection module may generate a filler sample to replace the unavailable sample by, for example, copying the nearest valid or captured component sample. See above Figure 5 -8 to describe the handling of unavailable component sampling required for model derivation.

[0144] Model builder 1305 uses the regression data (X and Y) to derive parameters for component prediction model 1310 using a technique of elimination, iteration, or decomposition. Component prediction model 1310 may be a higher order model with multiple filter taps, such as equation (10) or equation (11) above.

[0145] Fig.14 A process 1400 for deriving a component prediction model by processing unavailable samples is conceptually illustrated. In some embodiments, one or more processing units (e.g., processors) of a computing device implementing the decoder 1200 perform the process 1400 by executing instructions stored in a computer-readable medium. In some embodiments, an electronic device implementing the decoder 1200 performs the process 1400.

[0146] The decoder receives (at step 1410) data to be decoded as a current pixel block in a current picture. The decoder retrieves (at step 1420) a set of reconstructed component samples for a first set of pixel positions required to derive a component prediction model for the current block. The decoder generates (at step 1430) a set of padding component samples for a second set of pixel positions required to derive a component prediction model based on the retrieved set of reconstructed component samples. The decoder may generate each padding component sample by copying the most recently retrieved reconstructed component sample, a predefined value, or an intermediate value defined by the bit depth of the current sample.

[0147] In some embodiments, the second set of pixel positions is designated as having unavailable or prohibited reconstruction samples based on a boundary. The second set of pixel positions may correspond to component samples within the current block. The second set of pixel positions may correspond to component samples in an upper left adjacent area of ​​the current block. The second set of pixel positions may be located outside of a data pipeline unit surrounding the current block. This boundary may be a boundary of a buffer storing one or more rows of reconstructed codec tree units (CTUs).

[0148] In some embodiments, a component prediction model is derived from corresponding target component samples and prediction component samples, where each prediction component sample is generated based on a collocated component sample and a set of surrounding component samples (e.g., "N", "S", "W", "E", "NW", "NE", "SE", "SW" samples surrounding the collocated sample "C"). In some embodiments, surrounding component samples of pixel positions beyond the boundary are replaced by padded component samples. The component prediction model can be a cross-component model, so that the target component sample is a chroma sample, and the collocated and surrounding component samples are luma samples.

[0149] In some embodiments, the target component samples and the collocated component samples are samples of a reference region adjacent to the current block. Some of the surrounding component samples may be samples in an extended region just beyond the reference region.

[0150] The decoder (at step 1440) derives a component prediction model by using the retrieved set of reconstructed component samples and the generated set of padding component samples. The decoder (at step 1450) reconstructs the current block by using the component prediction model to generate a predictor for the current block. The predictor for the current block may include predicted chrominance samples generated by applying the component prediction model to luma samples of the current block. The decoder may then provide the reconstructed current block for display as part of a reconstructed current picture.

[0151] VII. Example Electronic Systems

[0152] Many of the above features and applications are implemented as software processes that are specified as a set of instructions recorded on a computer-readable storage medium (also referred to as a computer-readable medium). When these instructions are executed by one or more computing or processing units (e.g., one or more processors, processor cores, or other processing units), they cause the processing units to perform the actions indicated in the instructions. Examples of computer-readable media include, but are not limited to, CD-ROMs, flash drives, random access memory (RAM) chips, hard drives, erasable programmable read-only memories (EPROMs), electrically erasable programmable read-only memories (EEPROMs), etc. Computer-readable media do not include carrier waves and electronic signals transmitted wirelessly or via wired connections.

[0153] In this specification, the term "software" is meant to include firmware residing in read-only storage or application programs stored in magnetic storage, which can be read into storage for processing by a processor. In addition, in some embodiments, multiple software inventions can be implemented as sub-parts of a larger program while retaining different software inventions. In some embodiments, multiple software inventions can also be implemented as separate programs. Finally, any combination of separate programs that together implement the software inventions described herein are within the scope of this disclosure. In some embodiments, when the software program is installed to run on one or more electronic systems, one or more specific machine implementations are defined for executing and performing the operations of the software program.

[0154] Fig.15An electronic system 1500 is conceptually illustrated to implement some embodiments of the present disclosure. The electronic system 1500 may be a computer (e.g., a desktop computer, a personal computer, a tablet computer, etc.), a phone, a PDA, or any other type of electronic device. Such an electronic system includes various types of computer-readable media and interfaces for various other types of computer-readable media. The electronic system 1500 includes a bus 1505, a processing unit 1510, a graphics processing unit (GPU) 1515, a system memory 1520, a network 1525, a read-only memory 1530, a permanent storage device 1535, an input device 1540, and an output device 1545.

[0155] The bus 1505 collectively represents all system, peripheral, and chipset buses that communicatively connect the numerous internal devices of the electronic system 1500. For example, the bus 1505 communicatively connects the processing unit 1510 and the GPU 1515, the read-only memory 1530, the system memory 1520, and the permanent storage device 1535.

[0156] From these various memory units, the processing unit 1510 retrieves instructions to be executed and data to be processed in order to perform the processes of the present disclosure. In different embodiments, the processing unit may be a single processor or a multi-core processor. Some instructions are passed to and executed by the GPU 1515. The GPU 1515 may offload various calculations or supplement the image processing provided by the processing unit 1510.

[0157] Read-only memory (ROM) 1530 stores static data and instructions used by processing unit 1510 and other modules of the electronic system. On the other hand, permanent storage device 1535 is a read-write storage device. This device is a non-volatile storage unit that stores instructions and data even when electronic system 1500 is turned off. Some embodiments of the present disclosure use a mass storage device (such as a magnetic disk or optical disk and its corresponding magnetic disk drive) as permanent storage device 1535.

[0158] Other embodiments use removable storage devices (e.g., floppy disks, flash memory devices, etc. and their corresponding disk drives) as permanent storage devices. Like permanent storage device 1535, system storage 1520 is a read-write storage device. However, unlike storage device 1535, system storage 1520 is a volatile read-write storage, such as random access memory. System storage 1520 stores some instructions and data used by the processor at runtime. In some embodiments, processing according to the present disclosure is stored in system storage 1520, permanent storage device 1535 and / or read-only storage 1530. For example, various storage units include instructions for processing multimedia clips and some embodiments. From these different storage units, processing unit 1510 retrieves instructions to be executed and data to be processed in order to perform processing of some embodiments.

[0159] The bus 1505 is also connected to input and output devices 1540 and 1545. The input device 1540 enables a user to transmit information and select commands to the electronic system. The input device 1540 includes an alphanumeric keyboard and a pointing device (also referred to as a "cursor control device"), a camera (e.g., a webcam), a microphone or similar device for receiving voice commands, etc. The output device 1545 displays images generated by the electronic system or outputs data in other ways. The output device 1545 includes a printer and a display device, such as a cathode ray tube (CRT) or a liquid crystal display (LCD), and a speaker or similar audio output device. Some embodiments include devices that function as both input and output devices, such as a touch screen.

[0160] Finally, if Fig.15 As shown, bus 1505 also couples electronic system 1500 to a network 1525 via a network adapter (not shown). In this manner, the computer can be part of a computer network, such as a local area network ("LAN"), a wide area network ("WAN") or an intranet, or a network of networks. Any or all components of electronic system 1500 may be used in conjunction with the present disclosure.

[0161] Some embodiments include electronic components, such as microprocessors, storage, and memory, which store computer program instructions in machine-readable or computer-readable media (or alternatively referred to as computer-readable storage media, machine-readable media, or machine-readable storage media). Some examples of such computer-readable media include RAM, ROM, compact disk-read only (CD-ROM), compact disk-recordable (CD-R), compact disk-rewritable (CD-RW), digital versatile disk-read only (e.g., DVD-ROM, dual-layer DVD-ROM), various recordable / rewritable DVDs (e.g., DVD-RAM, DVD-RW, DVD+RW, etc.), flash memory (e.g., SD card, mini-SD card, micro SD card, etc.), magnetic and / or solid-state hard drives, read-only and recordable CD-ROM, ultra-density CD-ROM, any other optical or magnetic medium, and floppy disk. Computer-readable media can store a computer program that is executable by at least one processing unit and includes a set of instructions for performing various operations. Examples of computer programs or computer codes include machine code, such as that generated by a compiler, and files including high-level code that is executed by a computer, electronic component, or microprocessor using an interpreter.

[0162] Although the above discussion primarily refers to microprocessors or multi-core processors executing software, many of the above functions and applications are performed by one or more integrated circuits, such as application specific integrated circuits (ASICs) or field programmable gate arrays (FPGAs). In some embodiments, such integrated circuits execute instructions stored on the circuits themselves. In addition, some embodiments execute software stored in programmable logic devices (PLDs), ROM, or RAM devices.

[0163] As used in this specification and any claims of this application, the terms "computer", "server", "processor" and "storage" refer to electronic or other technical devices. These terms do not include people or groups of people. For the purpose of this specification, the term display or display means display on an electronic device. As used in this specification and any claims of this application, the terms "computer-readable medium", "computer-readable medium" and "machine-readable medium" are entirely limited to tangible physical objects that store information in a readable form. These terms do not include any wireless signals, wired download signals, and any other temporary signals.

[0164] Although the present disclosure has been described with reference to many specific details, a person skilled in the art will recognize that the present disclosure may be implemented in other specific forms without departing from the spirit of the present disclosure. Fig.12 and Fig.14 ) conceptually illustrate the processes. The specific operations of these processes may not be performed in the exact order shown and described. Specific operations may not be performed in a continuous series of operations, and different specific operations may be performed in different embodiments. In addition, the process may be implemented using multiple sub-processes or as part of a larger macro process. Therefore, it will be understood by those skilled in the art that the present disclosure is not limited by the foregoing illustrative details, but is defined by the appended claims.

[0165] Additional Notes

[0166] The subject matter described herein sometimes illustrates different components contained in or connected to different other components. It should be understood that the architectures depicted in this way are merely examples, and many other architectures that implement the same function can actually be implemented. Conceptually, any arrangement of components that implement the same function is effectively "associated", thereby implementing the desired function. Therefore, any two component combinations herein to implement no matter how the architecture or intermediate components are, can be regarded as "associated" with each other, thereby implementing the desired function. Similarly, any two components so associated can also be regarded as "operably connected" or "operably coupled" to each other to achieve the desired function, and any two components that can be so associated can also be regarded as "operably connected" "coupled", to achieve the desired function. Specific examples of operably coupled include, but are not limited to, physically matable and / or physically interactive components and / or wirelessly interactive and / or wirelessly interactive components and / or logically interactive and / or logically interactive components.

[0167] Furthermore, with respect to the use of substantially any plural and / or singular terms herein, those of ordinary skill in the art may translate from the plural to the singular and / or from the singular to the plural, depending on the context and / or application. For the sake of clarity, the various singular / plural arrangements may be expressly set forth herein.

[0168] Furthermore, it will be understood by those of ordinary skill in the art that, in general, the terms used herein, and especially in the appended claims, such as the bodies of the appended claims, are generally intended as "open" terms, e.g., the term "comprising" should be interpreted as "including but not limited to", "having" should be interpreted as "having at least". It will be further understood by those skilled in the art that if a specific number of claim statements are intended to be introduced, such intent will be explicitly stated in the claim, and such intent is absent in the absence of such statement. For example, to aid understanding, the following appended claims may contain the use of the introductory phrases "at least one" and "one or more" to introduce the recitation of the claims. However, the use of such phrases should not be interpreted as implying that a claim introduced by the indefinite article "a" or "an" will limit any particular claim containing such introduced claim to the implementation containing only one such statement, even when the same claim includes the introductory phrases "one or more" or "at least one" and an indefinite article such as "an", which should be interpreted as "at least one" or "one or more". The same applies to the use of definite articles to introduce claim statements. Furthermore, even if a specific number of claims recited in an introduced claim is explicitly cited, a person of ordinary skill in the art will recognize that such a recitation should be interpreted as indicating at least the number of citations, e.g., a reference to "two iterations," without other modifiers, means at least two iterations, or two or more iterations. Furthermore, in those cases where the convention uses terms like "at least one of A, B, and C, etc.", generally, such a construction is intended to be understood by a person of ordinary skill in the art in the sense that the convention is understood, e.g., "a system having at least one of A, B, and C" will include, but is not limited to, such systems: A alone, B alone, C alone, A and B together, A and C together, B and C together, and / or A, B, and C together, etc. In those cases where the convention uses terms like "at least one of A, B, or C," generally, such a construction is intended to be understood by a person of ordinary skill in the art in the sense that the convention is understood, e.g., "a system having at least one of A, B, or C" will include, but is not limited to, systems: A alone, B alone, C alone, A and B together, A and C together, B and C together, and / or A, B, and C together, etc. Those with ordinary knowledge in the art will further understand that in fact, whether in the specification, claims or drawings, any disjunctive words and / or phrases that appear as two or more alternative terms should be understood to include the possibility of one term, one term or two terms. For example, the phrase "A or B" will be understood to include the possibility of "A" or "B" or "A and B".

[0169] It can be understood from the above that various embodiments of the present disclosure have been described herein for illustrative purposes, and various modifications may be made without departing from the scope and spirit of the present disclosure. Therefore, the various embodiments disclosed herein are not intended to be limiting, and the true scope and spirit are indicated by the appended claims.

Claims

1. A video encoding and decoding method, comprising: Receiving data of a pixel block to be encoded or decoded as a current block of a current picture of a video; Retrieving a set of reconstructed component samples at a first set of pixel positions required to derive a component prediction model for the current block; generating a set of padding component samples for a second set of pixel positions required for deriving the component prediction model based on the acquired set of reconstructed component samples; deriving the component prediction model using the acquired set of reconstructed component samples and the generated set of padding component samples; as well as The current block is encoded or decoded using the component prediction model to generate a predictor of the current block.

2. The video encoding and decoding method according to claim 1, characterized in that: Generating the padding component sample includes copying one of: (i) the closest captured reconstructed component sample, (ii) a predefined value, or (iii) an intermediate value defined by the bit depth of the current sample.

3. The video encoding and decoding method according to claim 1, characterized in that: The second set of pixel locations is designated as having boundary-based unavailable reconstruction samples.

4. The video encoding and decoding method according to claim 3, characterized in that: The second set of pixel positions corresponds to component samples within the current block. 5 . The video decoding method according to claim 3 , wherein the second set of pixel positions corresponds to component samples in an upper left neighboring area of ​​the current block.

6. The video encoding and decoding method according to claim 3, characterized in that: The second set of pixel positions is located outside of the data pipeline unit surrounding the current block.

7. The video encoding and decoding method according to claim 3, characterized in that: The boundary is a range boundary of a buffer storing one or more rows of reconstructed codec tree units.

8. The video encoding and decoding method according to claim 1, characterized in that: The component prediction model is derived by corresponding target component samples and prediction component samples, each prediction component sample being generated based on a collocated component sample and a set of surrounding component samples.

9. The video encoding and decoding method according to claim 8, characterized in that: Surrounding component samples at pixel locations beyond this boundary are replaced with padding component samples.

10. The video encoding and decoding method according to claim 8, characterized in that: The target component sample is a chroma sample, wherein the collocated component sample and the surrounding component sample are luma samples.

11. The video encoding and decoding method according to claim 8, characterized in that: The target component sample and the collocated component sample are samples of a reference area adjacent to the current block.

12. The video encoding and decoding method according to claim 1, characterized in that: The predictor of the current block includes predicted chrominance samples generated by applying the component prediction model to luma samples of the current block.

13. An electronic device comprising: The video codec circuit is configured to perform operations including: Receiving data of a pixel block to be encoded or decoded as a current block of a current picture of a video; Retrieving a set of reconstructed component samples at a first set of pixel positions required to derive a component prediction model for the current block; generating a set of padding component samples for a second set of pixel positions required for deriving the component prediction model based on the acquired set of reconstructed component samples; deriving the component prediction model using the acquired set of reconstructed component samples and the generated set of padding component samples; as well as The current block is encoded or decoded using the component prediction model to generate a predictor of the current block.

14. A video decoding method, comprising: Receiving data of a pixel block to be decoded as a current block of a current picture of a video; Retrieving a set of reconstructed component samples at a first set of pixel positions required to derive a component prediction model for the current block; generating a set of padding component samples for a second set of pixel positions required for deriving the component prediction model based on the acquired set of reconstructed component samples; deriving the component prediction model using the acquired set of reconstructed component samples and the generated set of padding component samples; as well as The current block is reconstructed using the component prediction model to generate a predictor of the current block.

15. A video encoding method, comprising: Receiving data of a pixel block to be encoded as a current block of a current picture of a video; Retrieving a set of reconstructed component samples at a first set of pixel positions required to derive a component prediction model for the current block; generating a set of padding component samples for a second set of pixel positions required for deriving the component prediction model based on the acquired set of reconstructed component samples; deriving the component prediction model using the acquired set of reconstructed component samples and the generated set of padding component samples; as well as The current block is encoded using the component prediction model to generate a predictor for the current block.

Images