Regression-based matrix-based intra prediction

Abstract

A method of encoding or decoding a pixel block using regression-based matrix intra prediction (MIP) is provided. A video codec receives data to be encoded or decoded as a current pixel block of a current picture of a video. The video codec identifies corresponding input training samples and output training samples from at least one pixel reconstruction region and its neighboring L-shaped region. The corresponding input training samples and output training samples can include samples identified from one or more predefined positions relative to the current block. The video codec derives an intra prediction matrix by regression based on the identified training samples. The video codec performs matrix multiplication based on the intra prediction matrix and samples neighboring the current block to generate a predictor for the current block. The video codec encodes or decodes the current block using the generated predictor.

Description

[0001] Cross-references

[0002] This disclosure is a non-provisional application and claims priority to U.S. Provisional Patent Application No. 63 / 622,742, filed January 19, 2024. The contents of the above application are incorporated herein by reference. Technical Field

[0003] This disclosure relates to video encoding and decoding. Specifically, this disclosure relates to a method for encoding pixel blocks using matrix-based intra-frame prediction. Background Technology

[0004] Unless otherwise stated herein, the methods described in this section are not prior art to the following claims and are not acknowledged as prior art by virtue of their inclusion in this section.

[0005] High-Efficiency Video Coding (HEVC) is an international video coding standard developed by the Joint Collaborative Team on Video Coding (JCT-VC). HEVC is based on a hybrid block-based motion-compensated DCT-like transform coding and decoding architecture. The basic unit used for compression is called a coding unit (CU), which is a 2N×2N pixel block. Each CU can be recursively divided into four smaller CUs until a predefined minimum size is reached. Each CU contains one or more prediction units (PUs).

[0006] Versatile Video Coding (VVC) is the latest international video coding standard developed by the Joint Video Experts Team (JVET) of ITU-T SG16 WP3 and ISO / IEC JTC1 / SC29 / WG11. The input video signal is predicted from the reconstructed signal, which is derived from the coded image region. The prediction residual signal is processed through block transform. The transform coefficients are quantized and entropy-coded in the bitstream along with other auxiliary information. The reconstructed signal is generated from the prediction signal and the reconstructed residual signal after inverse transform, which is applied to the dequantized transform coefficients. The reconstructed signal is further filtered within a loop to remove coding artifacts. The decoded image is stored in a frame buffer and used to predict subsequent images in the input video signal.

[0007] In VVC, the encoded image is segmented into non-overlapping block regions represented by relevant coding tree units (CTUs). Leaf nodes of the coding tree correspond to coding units (CUs). The encoded image can consist of multiple slices, each containing an integer number of CTUs. The CTUs within a slice are processed in raster scan order. Bidirectional prediction (B) slices are decoded via intra-frame or inter-frame prediction, using at most two motion vectors (MVs) and a reference index to predict the sample value for each block. Predictive (P) slices are decoded via intra-frame or inter-frame prediction, using at most one motion vector and a reference index to predict the sample value for each block. Intra-frame (I) slices are decoded only via intra-frame prediction.

[0008] The Code Unit (CTU) can be segmented into one or more non-overlapping Codec Units (CUs) using a quadtree (QT) or nested multi-type-tree (MTT) structure to accommodate different local motion and texture characteristics. CUs can be further segmented into smaller CUs using one of five segmentation types: quadtree segmentation, vertical binary tree segmentation, horizontal binary tree segmentation, vertical center-side ternary tree segmentation, and horizontal center-side ternary tree segmentation.

[0009] Each CU contains one or more prediction units (PUs). The prediction unit, along with the associated CU syntax, serves as the basic unit of signaling prediction sub-information. The specified prediction process is used to predict the values ​​of associated pixel samples within the PU. Each CU may contain one or more transform units (TUs) to represent prediction residual blocks. A transform unit (TU) consists of one luma transform block (TB) and two corresponding chroma transform blocks, with each TB corresponding to a residual sample block for one color component. Integer transforms are applied to the transform blocks. The level values ​​of the quantization coefficients are entropy-encoded in the bitstream along with other auxiliary information. Coding tree blocks (CTBs), coding blocks (CBs), prediction blocks (PBs), and transform blocks (TBs) are used to specify two-dimensional sample arrays of a single color component associated with the CTU, CU, PU, ​​and TU, respectively. Therefore, a CTU contains one luma CTB, two chroma CTBs, and associated syntax elements. CUs, PUs, and TUs have similar relationships.

[0010] For each inter-frame predicted CU, motion parameters include motion vectors, reference image indices, reference image list usage indices, and other additional information used for inter-frame predicted sample generation. Motion parameters can be signaled explicitly or implicitly. When a CU is encoded in skip mode, it is associated with a PU and has no significant residual coefficients, no encoded motion vector increments, or reference image indices. A merge mode is specified in the VVC, where the motion parameters of the current CU are obtained from neighboring CUs, including spatial and temporal candidates and additional mechanisms / arrangements introduced in the VVC. The merge mode can be applied to any inter-frame predicted CU. An alternative to the merge mode is explicit transmission of motion parameters, where the motion vectors of each CU, the corresponding reference image indices for each reference image list, the reference image list usage flags, and other necessary information are explicitly signaled.

[0011] Intra-block copy (IBC) or current picture referencing (CPR) refers to encoding pixel blocks by referencing pixel locations within the same current picture as the current block using block vector references. Summary of the Invention

[0012] The following abstract is for illustrative purposes only and is not intended to be limiting in any way. That is, the abstract aims to introduce the concepts, key points, benefits, and advantages of the novel and non-obvious techniques described herein. Some implementations will be further elaborated in the detailed description below, not all implementations. Therefore, the following abstract is not intended to identify the essential features of the claimed subject matter, nor to determine the scope of the claimed subject matter.

[0013] Certain embodiments of this disclosure provide a method for encoding or decoding pixel blocks using regression-based matrix-based intra prediction (MIP). A video codec receives data for encoding or decoding the current pixel block of the current image to be used as a video. The video codec identifies corresponding input training samples and output training samples from at least one pixel reconstruction region and its neighboring L-shaped regions. The input training samples come from the neighboring L-shaped regions of the reconstruction region, and the output (target) training samples come from the pixel reconstruction region. Based on the identified training samples, the video codec derives an intra prediction matrix through regression. The video codec performs matrix multiplication based on the intra prediction matrix and samples neighboring the current block to generate a predictor for the current block. The video codec uses the generated predictor to encode or decode the current block.

[0014] In some embodiments, training samples are identified from samples in multiple reconstructed regions and their neighboring L-shaped regions. Each reconstructed region may have the same or different size as the current block. In some embodiments, output training samples are identified by downsampling the reconstructed region.

[0015] In some embodiments, the corresponding input training samples and output training samples may include samples identified relative to the current block from one or more predefined locations (e.g., non-adjacent locations). In some embodiments, the corresponding input training samples and output training samples may include samples identified from locations within one or more predefined search regions of the current image, such as locations identified from an intra-template matching prediction (IntraTMP) candidate list. In some embodiments, the corresponding input training samples and output training samples may include samples identified from a reference block and its neighboring L-shaped regions in a reference image, wherein the reference image is a reference image pointed to by a motion vector or a predefined image at the same location. In some embodiments, the corresponding input training samples and output training samples may include samples identified from a reference block and its neighboring L-shaped regions in the current image located by a block vector (e.g., for Block Copy Mode (IBC mode)).

[0016] In some embodiments, the video codec derives the intra-prediction matrix by deriving one or more linear models and performs matrix multiplication by using samples from neighboring blocks of the current block as input to the linear model. The video codec can generate predictors by adding a bias term, and the regression used to derive the intra-prediction matrix is ​​also used to derive the coefficients of the bias term.

[0017] In some embodiments, when a first flag is signaled in the bitstream to indicate whether an intra-prediction predictor is derived by performing matrix multiplication on samples adjacent to the current block, a second flag may be signaled to indicate whether regression for matrix multiplication is performed based on the training samples, or whether the coefficients of the matrix are derived by regression based on the training samples. In some embodiments, when a first flag is signaled to indicate whether intra-template matching prediction is used to generate the predictor for the current block, a second flag may be signaled to indicate whether the intra-prediction predictor is derived by performing matrix multiplication on samples adjacent to the current block, and the coefficients are derived by regression based on the training samples. In some embodiments, whether regression for matrix multiplication is performed based on the training samples is determined based on at least one of the following: the size of the current block, the scale of the current block, the slice type of the current block, and the color components of the current block (e.g., whether the current block is a luma block or a chroma block). Attached Figure Description

[0018] The accompanying drawings are included in this disclosure to provide a further understanding of the disclosure and form part of it. The drawings illustrate embodiments of the disclosure and, together with the description, serve to explain the principles of the disclosure. It will be understood that the drawings are not necessarily drawn to scale, as some components may be shown out of proportion to their actual dimensions in order to clearly illustrate the concepts of the disclosure.

[0019] Figure 1 It showcases 67 intra-frame prediction modes, including 65 directional or angular intra-frame prediction modes.

[0020] Figure 2A-2D This demonstrates the corresponding reference samples (R) applied to intra-frame prediction modes at various angles. (x,-1) and R (-1,y) )) is used for PDPC.

[0021] Figure 3 It demonstrates the predefined search region for intra-frame template matching.

[0022] Figure 4 The matrix-based intra prediction (MIP) process is demonstrated.

[0023] Figure 5 This concept demonstrates the collection of data samples used to train a linear MIP model.

[0024] Figure 6 A sample video encoder that enables MIP is shown.

[0025] Figure 7 This demonstrates the implementation of a regression-based MIP video encoder.

[0026] Figure 8 This concept demonstrates the process of encoding pixel blocks using regression-based MIPs.

[0027] Figure 9 An example video decoder that can implement MIP is shown.

[0028] Figure 10 This demonstrates the implementation of a regression-based MIP video decoder.

[0029] Figure 11 This concept demonstrates the process of decoding pixel blocks using a regression-based MIP.

[0030] Figure 12 This invention provides a conceptual illustration of an electronic system that implements certain embodiments of the present disclosure. Detailed Implementation

[0031] In the following detailed description, numerous specific details are set forth by way of example to provide a comprehensive understanding of the related techniques. Any variations, derivatives, and / or extensions of the techniques described herein are within the scope of this disclosure. In some cases, to avoid unnecessarily obscuring certain aspects of the techniques disclosed herein, well-known methods, processes, components, and / or circuits involved in one or more example embodiments may be described at a relatively high level without going into detail.

[0032] I. Intra-frame prediction

[0033] A. Intra-frame prediction mode and most probable mode (MPM)

[0034] Intra-prediction methods utilize one or more reference lines from neighboring prediction units (PUs) and one of the intra-prediction modes to generate predictors for the current PU. Intra-prediction directions can be selected from a set of modes containing multiple prediction directions, DC modes, and planar modes. An intra-prediction mode can also refer to any intra-prediction mode that uses spatial reconstruction samples to determine the predictors for the current block. The number of directional intra-prediction modes can be 33 or extended to 65. Including DC and planar modes, the number of intra-prediction modes is 35 (or 67). Certain intra-prediction modes (e.g., 3 or 5) are determined as the set of most probable modes (MPMs) for intra-prediction of the current prediction block, and one of these MPMs can be selected using a signaling index. Figure 1 It showcases 67 intra-frame prediction modes, including 65 directional or angular intra-frame prediction modes (from 2 to 66).

[0035] B. Location-Related Intra-Prediction Combination (PDPC)

[0036] In VVC, intra-prediction results for DC, planar, and certain angle modes can be further refined using the position-dependent intra-prediction combination (PDPC) method. PDPC is an intra-prediction method that combines boundary reference samples with HEVC-style intra-prediction (using filtered boundary reference samples). PDPC can be applied to the following intra-modal modes without signaling: planar, DC, intra-angles with angles less than or equal to horizontal, and intra-angles greater than or equal to vertical and less than or equal to 80 degrees. PDPC is not applied if the current block is in Bdpcm mode or the multi-reference line (MRL) index is greater than 0.

[0037] The predicted sample pred(x',y') is predicted using a linear combination of the intra-frame prediction mode (DC, plane, angle) and the reference sample, as follows: pred(x',y')= Clip(0, (1< <BitDepth ) –1, (wL×R -1,y + wT× R x,-1 + (64-wL-wT) × pred(x',y') + 32 )>>6) in R x,-1 R -1,y These represent the corresponding reference samples located at the top and left boundaries of the current sample (x, y) using the intra-frame prediction angle mode. wL and wT are weighting factors. pred(x',y') on the right side of the equation is the predicted sample value before PDPC correction, and pred(x',y') on the left side of the equation is its corrected value.

[0038] If PDPC is applied to DC, planar, horizontal, and vertical intra-frame modes, no additional boundary filters are required, such as the HEVC DC mode boundary filter or the horizontal / vertical mode edge filter. The PDPC processing is the same for DC and planar modes. For angular intra-frame prediction modes, if the current angular mode is HOR_IDX or VER_IDX, the left or top reference samples are not used, respectively. The weights and scaling factors of PDPC depend on the prediction mode and block size. PDPC is applied to blocks with both width and height greater than or equal to 4.

[0039] Figure 2A-2D This demonstrates the corresponding reference samples when PDPC is applied to intra-frame prediction modes at different angles. R x,-1 and R -1,y ). Figure 2A A reference sample of the top-right diagonal pattern is displayed. Figure 2B A reference sample of the bottom left diagonal pattern is shown. Figure 2C The reference sample of the pattern in the upper right corner of the adjacent diagonal is shown. Figure 2D The reference sample for the lower left pattern of the adjacent diagonal is shown.

[0040] The predicted sample pred(x', y') is located at position (x', y') in the predicted block. For example, in diagonal mode, the reference sample... R x,-1 The coordinates x are: x = x' + y' + 1, reference sample R. -1,yThe coordinates y are also: y = x' + y' + 1. For other angle modes, refer to the sample. R x,-1 and R -1,y It can be located at a fractional sample position. In this case, the sample value at the nearest integer sample position is used.

[0041] C. Intra-frame prediction fusion

[0042] Intra-frame prediction fusion methods derive prediction samples by weighted combination of multiple predictors from different reference lines. In this process, multiple intra-frame predictors are generated and then fused using a weighted average. The derivation process of the predictors used in the fusion process is as follows: (1) For angle-based intra-frame prediction modes (including single-mode cases of TIMD and DIMD), the intra-frame prediction fusion method derives the intra-frame prediction by weighting the intra-frame predictions from multiple reference lines, expressed as: ,in It is an intra-frame prediction from the default reference line. This is a prediction from the reference line above the default reference line. The weights are set to... and .

[0043] (2) For TIMD modes with mixing, For the first mode ( ), For the second mode ( ).

[0044] (3) For the DIMD model with mixing, the number of predictors participating in the weighted average increases from 3 to 6.

[0045] When the intra-frame prediction mode has a non-integer slope (requiring reference sample interpolation) and the block size is greater than 16, intra-frame prediction fusion can be applied to the luma block and used in conjunction with MRL, but not to the ISP codec block. PDPC can be applied to the intra-frame prediction mode that uses the reference line closest to the current block.

[0046] D. Intra-Template Matching (IntraTMP)

[0047] Intra-Template Matching Prediction (IntraTMP) is a special intra-prediction mode that achieves prediction by copying the best prediction block in the reconstructed portion of the current frame that matches the current template in an L-shape. For a predefined search range, the encoder searches for the template most similar to the current template in the reconstructed portion of the current frame and uses the corresponding block as the prediction block. Subsequently, the encoder signals the use of this mode and performs the same prediction operation at the decoder.

[0048] The prediction signal is generated by matching the L-shaped, top-only, or left-only causal neighbors of the current block with another block in a predefined search area. Figure 3 This figure illustrates the predefined search regions for intra-frame template matching. It shows reconstructed samples from the top and left codec tree units (CTUs), as well as partial reconstructed samples within the current CTU located above, to the left, lower left, and upper right of the current block 310. As shown, there are six predefined search regions, R1 to R6.

[0049] A given search order of 6 regions is used: R4, R5, R6, R1, R2, and R3. Within each region, the decoder constructs a candidate list of up to 19 template-matching block vectors, ordered in ascending order based on template cost (sum of absolute differences, with SAD as the cost function). The following modes are supported: Single predictor: Select a predictor from the candidate list.

[0050] Multi-predictor fusion: Multiple predictors are fused to derive the final prediction block. The fusion weights can be calculated based on the template matching cost of each predictor, or derived using the Wiener filter weight derivation method.

[0051] Subpixel precision: When using a single predictor, 1 / 2 pixel precision, 1 / 4 pixel precision, and 3 / 4 pixel precision are available, each with 8 possible directions.

[0052] Linear filtering model: A linear filter is learned between the reference template and the current template and applied to the reference block. This mode can be used for a single predictor when sub-pixel precision is not used.

[0053] The dimensions of all regions (SearchRange_w, SearchRange_h) are set proportionally to the block size (BlkW, BlkH) to ensure a fixed number of SAD comparisons per pixel. That is: SearchRange_w = min(64, a BlkW), SearchRange_h = min(64, a BlkH), Here, "a" is a constant that controls the gain / complexity tradeoff. In practical applications, "a" equals 5. To accelerate the template matching process, the search range of all search regions is subsampled by a factor of 3. After finding the best match, a refinement process is performed. Refinement involves a second template matching search around the best match, narrowing the range.

[0054] The intra-template matching tool is applicable to codec units (CUs) with a width and height of 64 or less. The maximum CU size for intra-template matching is configurable. When the current CU is not using decoder-side intra-mode derivation (DIMD), intra-template matching predicts the mode at the CU level via a dedicated flag.

[0055] E. Decoder-side Intra-Frame Mode Derivation (DIMD)

[0056] Decoder-Side Intra Mode Derivation (DIMD) is a technique in which one or more (e.g., two) intra-prediction modes, such as angles or orientations, are derived from reconstructed neighbor samples (templates) of a block, and these two predictors are combined with non-angle predictors (such as planar mode predictors), with weights derived from gradients. DIMD modes are used as alternative prediction modes and / or are always checked in high-complexity rate-distortion optimization (RDO) modes. To implicitly derive the intra-prediction modes of a block, texture gradient analysis is performed at both the encoder and decoder. This process begins with an empty Histogram of Gradients (HoG), which has 65 entries corresponding to 65 angle / orientation intra-prediction modes. The magnitudes of these entries are determined during texture gradient analysis.

[0057] In some embodiments, when applying DIMD, up to five intra-frame patterns can be derived from reconstructed neighbor samples, and these five predictors are combined with planar pattern predictors, with weights derived from gradient histograms. The division operations in weight derivation employ a lookup table (LUT) integerization scheme. For example, the division operation Orient = G in orientation calculation... y / G x The following LUT scheme can be used for calculation: x = Floor( Log2( G x ) ) normDiff = ( ( Gx<< 4 ) >> x ) & 15 x += ( 3 + ( normDiff != 0 ) ? 1 : 0 ) Orient = (G y ( DivSigTable[ normDiff ] | 8 ) + ( 1<<( x-1 ) )) >> x in DivSigTable

[16] = {0, 7, 6, 5,5, 4, 4, 3, 3, 2, 2, 1, 1, 1, 1, 0}.

[0058] For a block of size W × H, if the histogram magnitude of the upper or left side is twice that of the other side, the weights of each of the five derivation patterns will be modified. In this case, the weights are position-dependent and calculated as follows: If the histogram above is twice the size of the one on the left, then: .

[0059] If the histogram on the left is twice the size of the one above, then: , in The uniform weight for the unmodified DIMD selected mode, Δ i This is predefined and set to 10. The derived intra-mode is included in the main intra-mode most probable (MPM) list, so the DIMD process is performed before the MPM list is built. The main derived intra-mode of the DIMD block is stored with the block and used for the MPM list construction of neighboring blocks.

[0060] Depending on the availability of reconstructed samples, the decoding reference sample area of ​​the current W×H luma codec block (luma CB) can be extended to the upper right (if available), up to a maximum of W columns. If available, it can also be extended to the lower left, up to a maximum of H rows.

[0061] F. Template-based Intra-Frame Mode Derivation (TIMD)

[0062] For mode selection, template matching methods can be applied by calculating the cost between reconstructed samples and predicted samples. One example is template-based intra-mode derivation (TIMD). TIMD is an encoding method in which the intra-predicted mode of the coding unit (CU) is implicitly deduced by both the encoder and decoder using neighboring templates, rather than the encoder signaling a specific intra-predicted mode to the decoder.

[0063] For each intra-prediction mode in the MPM, and for the wide-angle mode when upper-right and / or lower-left reference samples are available, the SATD between the predicted and reconstructed samples of the template is calculated as the cost. The two intra-prediction modes with the smallest SATD are selected as TIMD modes. These two TIMD modes are weighted and fused after applying position-dependent intra-prediction combination (PDPC) processing, and this weighted intra-prediction is used to encode the current CU. PDPC is included in the derivation of the TIMD modes.

[0064] The costs of the two selected modes are compared with the threshold, and cost factor 2 is applied as follows: costMode2 < 2 costMode1.

[0065] If this condition is true, fusion is applied; otherwise, only mode 1 is used. The weights of the modes are calculated based on their SATD costs as follows: weight1 = costMode2 / (costMode1+ costMode2), weight2 = 1 - weight1.

[0066] Division operations can be performed using an integerization scheme based on a lookup table (LUT).

[0067] G. Regression-based GPM fusion

[0068] The regression-based GPM fusion pattern is designed as an additional implicit GPM pattern, where two integer fusion matrices ( W 0 and W 1 The fusion matrix is ​​derived from the template (top row, left column). It is modeled as an affine linear function of the sample position (x, y) in the current CU. W0 (x,y) = ax + by + c W1 (x,y) = 1 - W0 (x,y) The parameters (a, b, c) are derived from the reference template using a solver that minimizes the mean-square error (MSE) between the corresponding input and output samples in the current block of the template region. MSE minimization is achieved by calculating the autocorrelation matrix of the input samples and the cross-correlation vector between the input and output samples (containing parameters a, b, c). The autocorrelation matrix is ​​decomposed using LDL, and the final filter coefficients are calculated via back-substitution. The autocorrelation matrix is ​​calculated using reconstructed values ​​from the luma and / or chroma samples.

[0069] The candidate pair list is constructed from regular GPM candidates and reordered according to template cost. The GPM implicit pattern is signaled via a CU-level flag (gpm_implicit_flag). If gpm_implicit_flag is true, the merge index (merge-idx) is encoded with the GPM candidate pairs used in the signaling. If gpm_implicit_flag is false, regular GPM syntax elements are signaled.

[0070] II. Matrix-based Intra-frame Prediction (MIP)

[0071] Matrix-based intra prediction (MIP) is an intra-frame prediction technique. For a sample of a rectangular pixel block with width W and height H, the video codec performing MIP takes a row of H reconstructed neighborhood samples from the left side of the block and a row of W reconstructed neighborhood samples from the top of the block as input. If reconstructed samples are unavailable, they are generated using the traditional intra-frame prediction method. The generation of the predicted signal is based on (1) averaging, (2) matrix-vector multiplication, and / or (3) linear interpolation.

[0072] Figure 4 This demonstrates a matrix-based intra-frame prediction process for the current block 400, utilizing its neighboring samples 415. As shown in the figure, the first step (average) involves reconstructing the neighboring samples bdry for the current block. top and bdry left Averaging is performed to generate the input vector bdry for the MIP process. red 425. The second step (matrix-vector multiplication) involves multiplying matrix A by the input vector of the neighboring samples (bdry). red Multiply by , and add the bias b k The first step (linear interpolation) is to generate an initial MIP predictor 410. The second step (linear interpolation) is to interpolate or upsample based on the initial MIP predictor 410 and the neighboring samples to generate the final MIP prediction 420.

[0073] More generally, in some embodiments, to generate a MIP predictor, the MIP predictor is obtained by multiplying the input vector (denoted as p) with a matrix (denoted as A) and adding a bias value b. pred MIP If the size of the current block is not equal to the size of the MIP predictor, another upsampling is required to obtain the final predictor. pred final .

[0074] pred MIP = p A + b

[0075] pred final = upsample( pred MIP )

[0076] The above matrix equation can be rewritten as the following formula, where p n It is the nth element of the input vector p. a n,i It is the element in the nth row and i-th column of matrix A. pred MIP (i) is a MIP predictor pred MIP The i-th element.

[0077]

[0078] The MIP predictor can be observed from the above equations. pred MIP (i) The linear relationship between the input vector p and the MIP predictor. pred MIP If there are N elements, then the matrix equation can also be represented as N linear models.

[0079] In some embodiments, the MIP pattern matrix is ​​trained offline and predefined, which may not generate good predictions for all blocks. To improve prediction performance, some embodiments of this disclosure provide a regression-based MIP method that uses samples from neighboring templates as training pairs to train matrix A. For example, in some embodiments, the elements of matrix A (i.e., a n,iThe MIP matrix equation is derived from a reference template using a solver that minimizes the mean squared error (MSE) between the corresponding input and output samples in the collected training pairs. MSE minimization is achieved by computing the autocorrelation matrix of the input samples and the cross-correlation vector between the input and output samples. The autocorrelation matrix is ​​decomposed using LDL, and the final filter coefficients are calculated via back-substitution. The autocorrelation matrix is ​​calculated using reconstructed values ​​from luminance and / or chrominance samples. In another example, in some embodiments, the MIP matrix equation can be derived by solving N linear models.

[0080] In some embodiments, if a regression-based MIP mode is selected, the matrix A used to generate the MIP predictors is implicitly derived by a regression-based method, rather than reading / loading a predefined matrix from memory.

[0081] In some embodiments, if a regression-based MIP mode is selected, the N linear models used to generate MIP predictors are implicitly derived through a regression-based method. For each linear model, the applied input samples are derived from the neighboring reconstructed samples of the current block, and the corresponding output is the MIP predictor. pred MIP One sample.

[0082] In some embodiments, the dimension of the input vector of the MIP regression model can be equal to the dimension of the input vector of the MIP pattern in the VVC. In some embodiments, the dimension of the input vector of the MIP regression model can be greater than the dimension of the input vector of the MIP pattern in the VVC. In one embodiment, the bias term b in the MIP regression model can have a coefficient, and this coefficient can be derived through the regression process along with other parameters.

[0083] In some embodiments, multiple training pairs can be found from the neighboring template regions of the current block and used in the regression process of multiple MIP linear models. For example, multiple blocks with the same size as the current block and their neighboring L-shaped region samples can be found in the neighboring template regions. The input data of a training pair can be derived from the collected L-shaped region samples as the MIP process, and the output target of the training pair can be derived from the collected blocks. If the dimension of the MIP predictor is not equal to the dimension of the current block, a downsampling operation is applied to the collected block to make it have the same dimension as the MIP predictor, and the output target is the downsampled collected block.

[0084] In some embodiments, multiple training pairs can be found from predefined non-adjacent positions (i.e., positions not adjacent to the current block) and used in the regression process of multiple MIP linear models. In some embodiments, all or part of the candidates in the IntraTMP candidate list (i.e., from the above reference) Figure 3The candidates identified in the search region can be used as part or all of multiple training pairs for the regression process of multiple MIP linear models. In some embodiments, the multiple training pairs can come from reference blocks in a reference image, or any predefined co-located image of the current image (an image different from the current image).

[0085] In some embodiments, if the current block is predicted via intra-block copy (IBC) or intra-TMP, multiple training pairs can be determined using the block vector information of the current block. In some embodiments, multiple training pairs can be determined using block vector information and / or motion vector information from nearby spatially adjacent or non-adjacent blocks / locations in the current image / slice.

[0086] Figure 5 The diagram conceptually illustrates the collection of data samples used to train a MIP linear model. The figure shows the current block 510. To determine the MIP linear model used for intra-frame prediction of the current block, training pairs can be collected from blocks or regions 520, 530, 540, 550, 560, 570 and their corresponding neighboring L-shaped region samples 525, 535, 545, 555, 565, 575. For example, the input data for a training pair can be derived from L-shaped region sample 555, and the output target of the training pair can be derived from its corresponding block 550. In this example, blocks or regions 520-560 (and their corresponding neighboring L-shaped region samples 525-565) are located in the current image and can be reconstructed samples identified by block vectors via IBC, candidates from IntraTMP, spatially non-adjacent predefined locations, or other predefined locations. Block 570 and its neighboring L-shaped region samples 575 are located in a reference image and can be located by the motion vectors of the current block or as a predefined image at the same location.

[0087] In one embodiment, the regression-based MIP method can be a sub-pattern of the original MIP pattern. If the MIP flag of the current block is true, the regression-based MIP flag can be further signaled to indicate whether the regression-based MIP method is used. If the regression-based MIP flag is false, the MIP index and MIP transpose flags can be signaled. Otherwise, the MIP index and MIP transpose flags can be skipped.

[0088] In some embodiments, the regression-based MIP method can be a sub-pattern of IntraTMP. If the IntraTMP flag of the current block is true, the regression-based MIP flag can be further signaled to indicate whether the proposed regression-based MIP method is used. If the regression-based MIP flag is true, all or part of the candidates in the IntraTMP candidate list can be used for the regression process of multiple MIP linear models, and the final predictor can be generated by multiple MIP linear models.

[0089] In some embodiments, whether a regression-based MIP method is allowed for the current block may depend on the size of the CU, the scale of the CU, the slice type of the current block, or the color component. In some embodiments, high-level signaling syntax in SPS, PPS, PH, or SH can be used to indicate whether a regression-based MIP method is allowed for the current sequence, image, or slice.

[0090] Any of the proposed methods can be implemented in the encoder and / or decoder. For example, any of the proposed methods can be implemented in the inter-frame / intra-frame prediction module of the encoder and / or the inter-frame / intra-frame prediction module of the decoder. Alternatively, any of the proposed methods can be implemented as circuitry coupled to the inter-frame / intra-frame prediction module of the encoder and / or the inter-frame / intra-frame prediction module of the decoder to provide the required information to the inter-frame / intra-frame prediction module.

[0091] III. Example Video Encoder

[0092] Figure 6 An example video encoder 600 capable of implementing matrix-based intra-frame prediction (MIP) is illustrated. As shown, the video encoder 600 receives an input video signal from a video source 605 and encodes the signal into a bitstream 695. The video encoder 600 has multiple components or modules for encoding the signal from the video source 605, including at least some selected from the following components: a transform module 610, a quantization module 611, an inverse quantization module 614, an inverse transform module 615, an intra-frame image estimation module 624, an intra-frame prediction module 625, a motion compensation module 630, a motion estimation module 635, a loop filter 645, a reconstructed image buffer 650, an MV buffer 665, an MV prediction module 675, and an entropy encoder 690. The motion compensation module 630 and the motion estimation module 635 belong to the inter-frame prediction module 640. The intra-frame prediction module 625 and the intra-frame prediction estimation module 624 belong to the current image prediction module 620, which uses reconstructed samples of the current image as reference samples for predicting the current block.

[0093] In some embodiments, modules 610-690 are software instruction modules executed by one or more processing units (e.g., processors) of a computing device or electronic device. In some embodiments, modules 610-690 are hardware circuit modules implemented by one or more integrated circuits (ICs) of an electronic device. Although modules 610-690 are illustrated as independent modules, some of these modules may be combined into a single module.

[0094] Video source 605 provides the raw video signal, presenting pixel data for each video frame without compression. Subtractor 608 calculates the difference between the raw video pixel data of video source 605 and the predicted pixel data 613 of motion compensation module 630 or intra-frame prediction module 625, as the prediction residual 609. Transform module 610 converts this difference (or residual pixel data or residual signal 608) into transform coefficients (e.g., by performing a Discrete Cosine Transform (DCT)). Quantization module 611 quantizes the transform coefficients into quantized data (or quantization coefficients) 612, and entropy encoder 690 encodes them into a bitstream 695.

[0095] The inverse quantization module 614 inverse-quantizes the quantized data (or quantization coefficients) 612 to obtain transform coefficients 618. The inverse transform module 615 performs an inverse transform on the transform coefficients 618 to generate a reconstruction residual 619. The reconstruction residual 619 is added to the predicted pixel data 613 to generate reconstructed pixel data 617. In some embodiments, the reconstructed pixel data 617 is temporarily stored in a line buffer 627 (or an intra-frame prediction buffer) for intra-frame image prediction and spatial motion vector (MV) prediction. The reconstructed pixels are filtered by a loop filter 645 and stored in a reconstructed image buffer 650. In some embodiments, the reconstructed image buffer 650 is external memory to the video encoder 600. In some embodiments, the reconstructed image buffer 650 is internal memory to the video encoder 600.

[0096] Intra-frame image estimation module 624 performs intra-frame prediction based on reconstructed pixel data 617 to generate intra-frame prediction data. The intra-frame prediction data is provided to entropy encoder 690 and encoded as a bitstream 695. The intra-frame prediction data is also used by intra-frame prediction module 625 to generate predicted pixel data 613.

[0097] The motion estimation module 635 performs inter-frame prediction on reference pixel data of previously decoded frames stored in the reconstructed image buffer 650 by generating motion vectors (MVs). These motion vectors are provided to the motion compensation module 630 to generate predicted pixel data.

[0098] Instead of encoding the complete actual motion vectors into the bitstream, the video encoder 600 uses motion vector prediction to generate predicted motion vectors and encodes the difference between the motion vectors used for motion compensation and the predicted motion vectors as residual motion data, which is stored in the bitstream 695.

[0099] The motion vector prediction module 675 generates predicted motion vectors based on reference motion vectors generated for encoding previous video frames, i.e., motion-compensated motion vectors used to perform motion compensation. The motion vector prediction module 675 retrieves the reference motion vectors of previous video frames from the motion vector buffer 665. The video encoder 600 stores the motion vectors generated for the current video frame in the motion vector buffer 665 as reference motion vectors for generating the predicted motion vectors.

[0100] The motion vector prediction module 675 generates predicted motion vectors using reference motion vectors. The predicted motion vectors can be calculated through spatial motion vector prediction or temporal motion vector prediction. The difference (residual motion data) between the predicted motion vectors and the motion-compensated motion vectors (MC MV) of the current frame is encoded into a bitstream 695 by the entropy encoder 690.

[0101] The entropy encoder 690 encodes various parameters and data into a bitstream 695 using entropy encoding / decoding techniques such as context-adaptive binary arithmetic coding (CABAC) or Huffman coding. The entropy encoder 690 encodes various header elements, flags, quantization transform coefficients 612, and residual motion data as syntax elements into the bitstream 695. The bitstream 695 is then stored in a storage device or transmitted to the decoder via a communication medium such as a network.

[0102] Loop filter 645 performs filtering or smoothing operations on the reconstructed pixel data 617 to reduce coded artifacts, especially at pixel block boundaries. In some embodiments, the filtering or smoothing operations performed by loop filter 645 include deblocking filter (DBF), sample adaptive offset (SAO), and / or adaptive loop filter (ALF). In some embodiments, luma mapping chroma scaling (LMCS) is performed before loop filtering.

[0103] Figure 7 This diagram illustrates a portion of the modules implementing a regression-based MIP video encoder 600. In some embodiments, this diagram illustrates components of the intra-frame prediction module 620 of the video encoder 600. As shown, the intra-frame prediction module 620 uses neighboring samples 705 of the current block provided by the line buffer 627 as input samples. A matrix multiplication module 730 performs matrix multiplication on the input samples to generate MIP predictors 740, which serve as predicted pixel data 613.

[0104] Matrix multiplication 730 uses matrix parameters 725 as elements of the matrix or a linear model to perform matrix multiplication. Matrix parameters 725 can also be generated by regression module 720, which performs data regression on training samples 715 to generate matrix parameters.

[0105] The training sample collector 710 identifies and collects a specific set of reconstructed samples stored in the reconstructed image buffer 650 or the line buffer 627 as training samples 715 according to certain predetermined rules. Specifically, the training sample collector 710 identifies one or more blocks and their corresponding neighboring L-shaped regions at specific locations as sources of corresponding training data. Training output target samples are collected from these blocks, and training input samples are collected from the neighboring L-shaped regions of these blocks. Examples of locations used to identify blocks and their neighboring L-shaped regions include neighboring template regions, intra-frame TMP candidates, non-neighboring neighbors, IBC reference blocks identified by block vectors, reference blocks in the same location image or reference image identified by motion vectors, or other predefined locations.

[0106] Figure 8 The conceptual illustration illustrates a process 800 for encoding pixel blocks using a regression-based MIP. In some embodiments, one or more processing units (e.g., processors) of a computing device implementing encoder 600 execute process 800 by executing instructions stored in a computer-readable medium. In some embodiments, an electronic device implementing encoder 600 executes process 800.

[0107] In block 810, the encoder receives data of the current pixel block to be encoded into the current image. In block 820, the encoder identifies corresponding input training samples and output training samples from at least one pixel reconstruction region and its neighboring L-shaped regions. The input training samples come from the neighboring L-shaped regions of the reconstruction region, and the output (target) training samples come from the reconstruction pixel region.

[0108] In some embodiments, training samples are identified from multiple reconstructed regions and their neighboring L-shaped region samples, as shown in the reference. Figure 5 Each reconstructed region may have the same or different size as the current block. In some embodiments, output training samples are identified by downsampling the reconstructed regions.

[0109] In some embodiments, the corresponding input training samples and output training samples may include samples identified from one or more predefined non-adjacent locations relative to the current block. In some embodiments, the corresponding input training samples and output training samples may include samples identified from locations within one or more predefined search regions of the current image, such as those referenced in [reference]. Figure 3The locations identified in the candidate list for IntraTMP (Intra-Template Matching Prediction) are described above. In some embodiments, the corresponding input and output training samples may include samples identified from a reference block and its neighboring L-shaped regions in a reference image, which is located by motion vectors or is a predefined co-location image. In some embodiments, the corresponding input and output training samples may include samples identified from a reference block and its neighboring L-shaped regions in the current image located by block vectors (e.g., for Intra-Block Copy (IBC) mode).

[0110] The encoder derives the intra-prediction matrix at block 830 by performing regression based on the identified training samples. In some embodiments, the encoder derives the intra-prediction matrix by deriving one or more linear models and performs matrix multiplication by taking samples neighboring the current block as input to the linear model. The encoder can generate predictors by adding a bias term, and the regression used to derive the intra-prediction matrix is ​​also used to derive the coefficients of the bias term.

[0111] In some embodiments, when a first flag is signaled in the bitstream to indicate whether an intra-prediction predictor is derived by performing matrix multiplication on samples adjacent to the current block, a second flag may be signaled to indicate whether regression for matrix multiplication is performed based on the training samples, or whether the coefficients of the matrix are derived by regression based on the training samples. In some embodiments, when a first flag is signaled to indicate whether intra-template matching prediction is used to generate a predictor for the current block, a second flag may be signaled to indicate whether an intra-prediction predictor is derived by performing matrix multiplication on samples adjacent to the current block, and the coefficients are derived by regression based on the training samples. In some embodiments, whether regression for matrix multiplication is performed based on the training samples is determined based on at least one of the following: the size of the current block, the scale of the current block, the slice type of the current block, and the color components of the current block (e.g., whether the current block is a luma block or a chroma block).

[0112] At block 840, the encoder performs matrix multiplication based on the intra-prediction matrix and samples from neighboring blocks to generate a predictor for the current block. At block 850, the encoder uses the generated predictor to encode the current block to produce a prediction residual.

[0113] IV. Example Video Decoder

[0114] In some embodiments, the encoder may signal (or generate) one or more syntax elements in the bitstream, such that the decoder may parse the one or more syntax elements from the bitstream.

[0115] Figure 9An example video decoder 900 is shown that can implement matrix-based intra-frame prediction (MIP). As shown, the video decoder 900 is an image decoding or video decoding circuit that receives a bitstream 995 and decodes the contents of the bitstream into video frame pixel data for display. The video decoder 900 has several components or modules for decoding the bitstream 995, including some components selected from an inverse quantization module 914, an inverse transform module 915, an intra-frame prediction module 925, a motion compensation module 930, a loop filter 945, a decoded image buffer 950, a motion vector buffer 965, a motion vector prediction module 975, and a parser 990. The motion compensation module 930 is part of the inter-frame prediction module 940. The intra-frame prediction module 925 is part of the current image prediction module 920, which uses the reconstructed samples of the current image as reference samples for predicting the current block.

[0116] In some embodiments, modules 914–990 are software instruction modules executed by one or more processing units (e.g., processors) of a computing device. In some embodiments, modules 914–990 are hardware circuit modules implemented by one or more integrated circuits (ICs) of an electronic device. Although modules 914–990 are illustrated as independent modules, some of these modules may be combined into a single module.

[0117] Parser 990 (or entropy decoder) receives bitstream 995 and performs initial parsing according to the syntax defined by the video or image codec standard. The parsed syntax elements include various header elements, flags, and quantization data (or quantization coefficients) 912. Parser 990 then uses entropy coding techniques, such as context-adaptive binary arithmetic coding (CABAC) or Huffman coding, to further parse the syntax elements.

[0118] The inverse quantization module 914 dequantizes the quantized data (or quantization coefficients) 912 to obtain transform coefficients 918. The inverse transform module 915 performs an inverse transform on the transform coefficients 918 to generate a reconstructed residual signal 919. The reconstructed residual signal 919 is added to the predicted pixel data 913 from the intra-frame prediction module 925 or the motion compensation module 930 to generate decoded pixel data 917. The decoded pixel data is filtered by the loop filter 945 and stored in the decoded image buffer 950. In some embodiments, the decoded image buffer 950 is an external memory of the video decoder 900. In some embodiments, the decoded image buffer 950 is an internal memory of the video decoder 900.

[0119] The intra-prediction module 925 receives intra-prediction data from the bitstream 995 and generates prediction pixel data 913 from the decoded pixel data 917 stored in the line buffer 927 (or intra-prediction buffer) for intra-prediction and spatial motion vector prediction.

[0120] In some embodiments, the contents of the decoded image buffer 950 are used for display. The display device 905 can directly retrieve the contents of the decoded image buffer 950 for display, or retrieve the contents of the decoded image buffer into a display buffer. In some embodiments, the display device receives pixel values ​​from the decoded image buffer 950 via pixel transfer.

[0121] The motion compensation module 930 generates predicted pixel data 913 from the decoded pixel data 917 stored in the decoded image buffer 950 based on the motion-compensated motion vectors (MC MV). These motion-compensated motion vectors are decoded by adding the residual motion data received from the bitstream 995 to the predicted motion vectors received from the motion vector prediction module 975.

[0122] The motion vector prediction module 975 generates predicted motion vectors based on reference motion vectors generated for decoding the previous video frame, such as motion-compensated motion vectors used for performing motion compensation. The motion vector prediction module 975 retrieves the reference motion vectors of the previous video frame from the motion vector buffer 965. The video decoder 900 stores the motion-compensated motion vectors generated for decoding the current video frame in the motion vector buffer 965 as reference motion vectors for generating the predicted motion vectors.

[0123] Loop filter 945 performs filtering or smoothing operations on the decoded pixel data 917 to reduce coding artifacts, particularly at pixel block boundaries. In some embodiments, the filtering or smoothing operations performed by loop filter 945 include deblocking filter (DBF), sample adaptive offset (SAO), and / or adaptive loop filter (ALF). In some embodiments, luma mapping chroma scaling (LMCS) is performed before the loop filter.

[0124] Figure 10A partial video decoder 900 implementing regression-based MIP is shown. For some embodiments, components of the intra-prediction module 920 of the video decoder 900 are shown. As illustrated, the intra-prediction module 920 uses neighboring samples 1005 of the current block provided by the line buffer 927 as input samples. A matrix multiplication module 1030 performs matrix multiplication on the input samples to generate MIP predictors 1040 as predicted pixel data 913.

[0125] Matrix multiplication 1030 performs matrix multiplication using matrix parameter 1025 as an element of the matrix or as a linear model. Matrix parameter 1025 can also be generated by regression module 1020, which performs data regression on training samples 1015 to generate matrix parameters.

[0126] Training sample collector 1010 provides training samples 1015 by identifying and collecting a specific set of reconstructed samples stored in decoded image buffer 950 or line buffer 927 as training samples, following certain predetermined rules. Specifically, training sample collector 1010 identifies one or more blocks and their corresponding neighboring L-shaped regions at certain locations as sources of corresponding training data, wherein training output target samples are collected from the block, and training input samples are collected from the neighboring L-shaped regions of the block. Examples of locations used to identify blocks and their neighboring L-shaped regions include neighboring template regions, intra-frame TMP candidates, non-neighboring neighbors, IBC reference blocks identified by block vectors, reference blocks in reference images located by motion vectors, or reference blocks in images at the same location, or other predefined locations.

[0127] Figure 11 A conceptual illustration of process 1100 for decoding pixel blocks using regression-based MIP is presented. In some embodiments, one or more processing units (e.g., processors) of a computing device implementing decoder 900 execute process 1100 by executing instructions stored in a computer-readable medium. In some embodiments, an electronic device implementing decoder 900 executes process 1100.

[0128] In step 1110, the decoder receives data to be decoded into the current pixel block in the current image. In step 1120, the decoder identifies corresponding input training samples and output training samples from at least one pixel reconstruction region and its neighboring L-shaped regions. The input training samples come from the neighboring L-shaped regions of the reconstruction region, and the output (target) training samples come from the pixel reconstruction region.

[0129] In some embodiments, as referenced Figure 5 The training samples are identified from multiple reconstructed regions and their neighboring L-shaped samples. Each reconstructed region may have the same or different size as the current block. In some embodiments, output training samples are identified by downsampling the reconstructed regions.

[0130] In some embodiments, the corresponding input training samples and output training samples may include samples identified from one or more predefined non-adjacent locations relative to the current block. In some embodiments, the corresponding input training samples and output training samples may include samples identified from locations within one or more predefined search regions of the current image, such as those referenced in [reference]. Figure 3 The position in the intra-frame TMP candidate list. In some embodiments, the corresponding input training samples and output training samples may include samples identified from a reference block located by motion vectors in a reference image and its neighboring L-shaped regions, or a reference block in a predefined image at the same location. In some embodiments, the corresponding input training samples and output training samples may include samples identified from a reference block located by block vectors in the current image and its neighboring L-shaped regions (e.g., for IBC mode).

[0131] In step 1130, the decoder derives the intra-prediction matrix via regression based on the identified training samples. In some embodiments, the decoder derives the intra-prediction matrix by deriving one or more linear models and performs matrix multiplication by taking samples from neighboring current blocks as input to the linear models. The decoder can generate predictors by adding a bias term, and the regression used to derive the intra-prediction matrix is ​​also used to derive the coefficients of that bias term.

[0132] In some embodiments, when a first flag is signaled in the bitstream to indicate whether an intra-predictive predictor is derived by performing matrix multiplication on samples adjacent to the current block, a second flag may be signaled to indicate whether regression for matrix multiplication is performed based on training samples, or whether the coefficients of the matrix are derived by regression based on training samples. In some embodiments, when a first flag is signaled to indicate whether intra-template matching prediction is used to generate a predictor for the current block, a second flag may be signaled to indicate whether an intra-predictive predictor is derived by performing matrix multiplication on samples adjacent to the current block, and the coefficients are derived by regression based on training samples. In some embodiments, whether regression for matrix multiplication is performed based on training samples is determined based on at least one of the following: the size of the current block, the scale of the current block, the slice type of the current block, and the color components of the current block (e.g., whether the current block is a luma block or a chroma block).

[0133] In step 1140, the decoder performs matrix multiplication based on the intra-prediction matrix and samples from neighboring current blocks to generate a predictor for the current block. In step 1150, the decoder reconstructs the current block using the generated predictor and the corresponding prediction residual. The decoder can then display the reconstructed current block as part of the reconstructed current image.

[0134] V. Example Electronic System

[0135] Many of the features and applications described above are implemented as software processes, which are specified as a set of instructions recorded on a computer-readable storage medium (also known as a computer-readable medium). When these instructions are executed by one or more computing or processing units (such as one or more processors, processor cores, or other processing units), the processing unit performs the operations indicated in the instructions. Examples of computer-readable media include, but are not limited to, CD-ROMs, flash drives, random access memory (RAM) chips, hard disks, erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), etc. Computer-readable media do not include carrier waves and electronic signals transmitted via wireless or wired connections.

[0136] In this specification, the term "software" includes firmware residing in read-only memory or application programs stored in magnetic memory, which can be read into memory and processed by a processor. Furthermore, in some embodiments, multiple software inventions may be implemented as sub-parts of a larger program while still remaining distinct software inventions. In some embodiments, multiple software inventions may also be implemented as independent programs. Finally, any combination of independent programs that collectively implement the software inventions described herein falls within the scope of this disclosure. In some embodiments, when a software program is installed and runs on one or more electronic systems, one or more specific machine implementations are defined that execute and perform the operations of the software program.

[0137] Figure 12 An electronic system 1200 that may implement certain embodiments of the present disclosure is illustrated schematically. The electronic system 1200 may be a computer (e.g., a desktop computer, personal computer, tablet computer, etc.), a telephone, a personal digital assistant (PDA), or any other type of electronic device. Such electronic systems include various types of computer-readable media and interfaces for various other types of computer-readable media. The electronic system 1200 includes a bus 1205, a processing unit 1210, a graphics processing unit (GPU) 1215, system memory 1220, a network 1225, read-only memory 1230, permanent storage device 1235, an input device 1240, and an output device 1245.

[0138] Bus 1205 generally represents all system, peripheral, and chipset buses used for communication connections to numerous internal devices of electronic system 1200. For example, bus 1205 communicates with processing unit 1210 and GPU 1215, read-only memory 1230, system memory 1220, and permanent storage device 1235.

[0139] Processing unit 1210 retrieves instructions from these different memory units for execution and retrieves data for processing, thereby executing the processes of this disclosure. The processing unit may be a single-processor or a multi-core processor in different embodiments. Some instructions are passed to and executed by GPU 1215. GPU 1215 may offload various computing or supplemental image processing provided by processing unit 1210.

[0140] Read-only memory (ROM) 1230 stores static data and instructions used by processing unit 1210 and other modules of the electronic system. Permanent storage device 1235 is a read-write storage device. This device is a non-volatile storage unit that can store instructions and data even when the electronic system 1200 is turned off. Some embodiments of this disclosure use mass storage devices (such as magnetic disks or optical disks and their corresponding disk drives) as permanent storage device 1235.

[0141] Other embodiments use removable storage devices (such as floppy disks, flash memory devices, etc., and their corresponding disk drives) as permanent storage devices. Similar to permanent storage device 1235, system memory 1220 is also a read-write storage device. However, unlike storage device 1235, system memory 1220 is volatile read-write memory, such as random access memory. System memory 1220 stores some instructions and data used by the processor during runtime. In some embodiments, the processes according to this disclosure are stored in system memory 1220, permanent storage device 1235, and / or read-only memory 1230. For example, various storage units include instructions for processing multimedia segments, according to some embodiments. Processing unit 1210 retrieves instructions from these different storage units for execution and retrieves data for processing, thereby executing the processes of some embodiments.

[0142] Bus 1205 also connects input device 1240 and output device 1245. Input device 1240 enables a user to pass information to the electronic system and select commands. Input device 1240 includes an alphanumeric keypad and pointing device (also known as a "cursor control device"), a camera (e.g., a webcam), a microphone, or similar device for receiving voice commands. Output device 1245 displays images generated by the electronic system or otherwise outputs data. Output device 1245 includes a printer and display device, such as a cathode ray tube (CRT) or liquid crystal display (LCD), and a speaker or similar audio output device. Some embodiments include devices such as touchscreens that function as both input and output devices.

[0143] Finally, as Figure 12As shown, bus 1205 also connects electronic system 1200 to network 1225 via a network adapter (not shown). In this way, the computer can become part of a computer network (such as a local area network (LAN), wide area network (WAN), or intranet, or a network of networks, such as the Internet). Any or all components of electronic system 1200 can be used in conjunction with this disclosure.

[0144] Some embodiments include electronic components, such as microprocessors, storage, and memory, that store computer program instructions in a machine-readable or computer-readable medium (also referred to as a computer-readable storage medium, machine-readable medium, or machine-readable storage medium). Some examples of such computer-readable media include random access memory (RAM), read-only memory (ROM), read-only optical disc (CD-ROM), recordable optical disc (CD-R), rewritable optical disc (CD-RW), read-only digital versatile optical discs (such as DVD-ROM, dual-layer DVD-ROM), various recordable / rewritable DVDs (such as DVD-RAM, DVD-RW, DVD+RW, etc.), flash memory (such as SD cards, mini-SD cards, micro-SD cards, etc.), magnetic and / or solid-state drives, read-only and recordable Blu-ray discs, ultra-high-density optical discs, any other optical or magnetic media, and floppy disks. The computer-readable medium may store a computer program executable by at least one processing unit and include a set of instructions for performing various operations. Examples of computer programs or computer code include machine code (such as code generated by a compiler) and files containing high-level code that are executed by a computer, electronic component, or microprocessor via an interpreter.

[0145] While the foregoing discussion primarily concerns microprocessors or multi-core processors that execute software, many of the aforementioned features and applications are implemented by one or more integrated circuits, such as application-specific integrated circuits (ASICs) or field-programmable gate arrays (FPGAs). In some embodiments, such integrated circuits execute instructions stored within the circuit itself. Furthermore, some embodiments execute software stored in programmable logic devices (PLDs), read-only memory (ROM), or random access memory (RAM).

[0146] In this specification and any claim of this application, the terms "computer," "server," "processor," and "memory" refer to electronic or other technical devices. These terms do not include individuals or groups. For the purposes of this specification, the term "display" or "show" means display on an electronic device. In this specification and any claim of this application, the terms "computer-readable medium," "computer-readable media," and "machine-readable medium" are fully defined as tangible, physical objects that store information and exist in a computer-readable form. These terms do not include any wireless signals, wired download signals, or any other transient signals.

[0147] Although this disclosure has been described in conjunction with numerous specific details, those skilled in the art will recognize that this disclosure can be implemented in other specific forms without departing from the spirit of this disclosure. Furthermore, many figures (including...) Figure 8 and Figure 11 The process is illustrated schematically. The specific operations of these processes may not be performed in the order shown and described. Specific operations may not be performed in a continuous sequence of operations; different specific operations may be performed in different embodiments. Furthermore, the process may be implemented through multiple sub-processes or as part of a larger macro-process. Therefore, those skilled in the art will understand that this disclosure should not be limited to the exemplary details described above, but should be defined by the appended claims.

[0148] Additional notes

[0149] The topics described herein sometimes demonstrate different components contained within or connected to other different components. It should be understood that such architectures are merely examples, and many other architectures can actually be implemented to achieve the same functionality. Conceptually, any arrangement of components that achieves the same function is considered "associated" in order to achieve the desired functionality. Therefore, any two components combined in this document to achieve a specific function can be considered "associated" with each other in order to achieve the desired functionality, regardless of the architecture or intermediate components. Similarly, any two such associated components can be considered "operably connected" or "operably coupled" in order to achieve the desired functionality, and any two components that can be suchly associated can be considered "operably coupled" in order to achieve the desired functionality. Specific examples of operable coupling include, but are not limited to, physically matable and / or physically interactive components, wirelessly interactive and / or wirelessly interactive components, and logically interactive and / or logically interactive components.

[0150] Furthermore, regarding the use of almost all plural and / or singular terms in this document, those skilled in the art can appropriately convert plural to singular and / or singular to plural depending on the context and / or application. For clarity, various singular / plural permutations may be explicitly listed herein.

[0151] Furthermore, those skilled in the art will understand that the terms used herein, particularly in appended claims, such as the body portion of appended claims, are generally intended as “open” terms. For example, the word “comprising” should be interpreted as “including but not limited to,” the word “having” should be interpreted as “having at least,” and the word “including” should be interpreted as “including but not limited to,” etc. Those skilled in the art will also understand that if a claim expressly intends to introduce a particular quantity, that intention will be expressly stated in the claim; if no such statement is made, then such an intention does not exist. For example, for ease of understanding, the following appended claims may contain the use of the introductory phrases “at least one” and “one or more” to introduce the claim recitation. However, the use of such phrases should not be construed as limiting any particular claim containing such recitation to containing only one such recitation, even if the same claim contains the introductory phrases “one or more” or “at least one” and indefinite articles such as “one” or “a,” for example, “one” and / or “a” should be interpreted as “at least one” or “one or more”; the same applies to definite articles used to introduce the claim recitation. Furthermore, even when a specific quantity is explicitly stated in the introductory claims, those skilled in the art will recognize that such a statement should be interpreted as at least the stated quantity. For example, the single statement "two statements" without other modifiers means at least two statements, or two or more statements. Additionally, when using conventions such as "at least one A, B, and C, etc.", such structures are generally intended for those skilled in the art to understand the meaning of the convention. For example, "a system having at least one A, B, and C" includes, but is not limited to, systems having only A, only B, only C, both A and B, both A and C, both B and C, and systems having both A, B, and C, etc. Similarly, when using conventions such as "at least one A, B, or C, etc.", such structures are generally intended for those skilled in the art to understand the meaning of the convention. For example, "a system having at least one A, B, or C" includes, but is not limited to, systems having only A, only B, only C, both A and B, both A and C, both B and C, and systems having both A, B, and C, etc. Those skilled in the art will also understand that almost all extractives and / or phrases presenting two or more alternative terms in the specification, claims, or drawings should be understood to include one, any, or both terms. For example, the phrase “A or B” should be understood to include the possibility of “A” or “B” or “A and B”.

[0152] As can be seen from the foregoing, various embodiments of this disclosure have been described herein for illustrative purposes, and various modifications can be made without departing from the scope and spirit of this disclosure. Therefore, the various embodiments disclosed herein are not intended to be limiting, and the true scope and spirit are defined by the following claims.

Claims

1. A video encoding / decoding method, comprising: Receive data that is to be encoded or decoded as the current pixel block of the current image in the video; Identify the corresponding input and output training samples from at least one pixel reconstruction region and its adjacent L-shaped region; Based on the identified training samples, the intra-frame prediction matrix is ​​derived through regression. Matrix multiplication is performed based on the intra-frame prediction matrix and samples adjacent to the current block to generate a predictor for the current block; as well as Use the generated predictors to encode or decode the current block.

2. The video encoding / decoding method as described in claim 1, wherein, The input training sample comes from the neighboring L-shaped region of the reconstructed region, and the output training sample comes from the reconstructed pixel region.

3. The video encoding / decoding method as described in claim 1, wherein, Deriving the intra-prediction matrix involves deriving one or more linear models, and performing the matrix multiplication involves using samples from neighboring blocks as input to the linear models.

4. The video encoding / decoding method as described in claim 1, wherein, Generating the predictor for the current block includes adding a bias term, and deriving the intra-prediction matrix by regression includes deriving the coefficients of the bias term.

5. The video encoding / decoding method as described in claim 1, wherein, The corresponding input training sample and output training sample are identified from multiple reconstructed regions and their neighboring L-shaped region samples.

6. The video encoding / decoding method as described in claim 5, wherein, One of the multiple reconstructed regions has the same size as the current block.

7. The video encoding / decoding method as described in claim 1, wherein, The output training sample is identified by downsampling the reconstructed region.

8. The video encoding / decoding method as described in claim 1, wherein, The corresponding input training sample and output training sample include samples identified from one or more predefined non-adjacent positions relative to the current block.

9. The video encoding / decoding method as described in claim 1, wherein, The corresponding input training samples and output training samples include samples identified from locations within one or more predefined search regions of the current image.

10. The video encoding / decoding method as described in claim 1, wherein, The corresponding input training samples and output training samples include samples identified from the reference block and its neighboring L-shaped region in the reference image.

11. The video encoding / decoding method as described in claim 1, wherein, The corresponding input training samples and output training samples include samples identified from reference blocks and their neighboring L-shaped regions in the current image located by block vectors.

12. The video encoding / decoding method as described in claim 1, wherein, When the first signaling flag indicates whether the predictor of the intra-frame prediction is derived by performing matrix multiplication on samples adjacent to the current block, the second signaling flag indicates whether the coefficients of the matrix are derived by regression based on the training samples.

13. The video encoding / decoding method as described in claim 1, wherein, When the first signaling flag indicates whether to use intra-frame template matching prediction to generate the predictor for the current block, the second signaling flag indicates whether to derive the intra-frame prediction predictor by performing matrix multiplication on samples adjacent to the current block, and the coefficients of the matrix are derived by regression based on the training samples.

14. The video encoding / decoding method as described in claim 1, wherein, Whether to perform regression for matrix multiplication based on the training samples is determined by at least one of the following: the size of the current block, the scale of the current block, the slice type of the current block, and the color components of the current block.

15. An electronic device comprising: The video codec circuit is configured to perform the following operations: Receive data that is to be encoded or decoded as the current pixel block of the current image in the video; Identify the corresponding input and output training samples from at least one pixel reconstruction region and its adjacent L-shaped region; Based on the identified training samples, the intra-frame prediction matrix is ​​derived through regression. Matrix multiplication is performed based on the intra-frame prediction matrix and samples adjacent to the current block to generate a predictor for the current block; as well as Use the generated predictors to encode or decode the current block.

16. A video decoding method, comprising: Receive data of the current pixel block of the current image to be decoded into video; Identify the corresponding input and output training samples from at least one pixel reconstruction region and its adjacent L-shaped region; Based on the identified training samples, the intra-frame prediction matrix is ​​derived through regression. Matrix multiplication is performed based on the intra-frame prediction matrix and samples adjacent to the current block to generate a predictor for the current block; as well as Reconstruct the current block using the generated predictors.

17. A video coding method, comprising: Receive data of the current pixel block of the current image to be encoded into video; Identify the corresponding input and output training samples from at least one pixel reconstruction region and its adjacent L-shaped region; Based on the identified training samples, the intra-frame prediction matrix is ​​derived through regression. Matrix multiplication is performed based on the intra-frame prediction matrix and samples adjacent to the current block to generate a predictor for the current block; as well as The generated predictors are used to encode the current block.

Images