Representative prediction mode for pixel blocks

Abstract

A method of generating and using a representative prediction mode for a current codec block of pixels is provided. A video codec receives pixel block data to be encoded or decoded, the pixel block to be encoded or decoded as a current block of a current picture of a video. The video codec generates a predictor for a current block using a first prediction mode. The video codec determines the second prediction mode as a representative prediction mode for the current block. The video codec encodes or decodes the current block using the generated current block predictor and the representative prediction mode. Representative prediction modes may be used to select transformations of prediction residuals. Representative prediction modes may also be used to encode and decode subsequent blocks.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This disclosure is a part of a non-provisional application claiming priority to U.S. Provisional Patent Application Nos. 63 / 478,199 and 63 / 480,325, filed on January 3, 2023, and January 18, 2023, respectively. The contents of the above-listed applications are incorporated herein by reference in their entirety. Technical Field

[0003] The present disclosure generally relates to video coding and decoding, and more particularly to a method for coding and decoding pixel blocks by intra prediction or component prediction. Background Art

[0004] Unless otherwise indicated herein, the approaches described in this section are not prior art to the claims listed below and are not admitted to be prior art by inclusion in this section.

[0005] High-Efficiency Video Coding (HEVC) is an international video codec standard developed by the Joint Collaborative Team on Video Coding (JCT-VC). HEVC is based on a hybrid block-based motion-compensated DCT-like transform codec architecture. The basic unit of compression, called a coding unit (CU), is a 2Nx2N square pixel block. Each CU can be recursively divided into four smaller CUs until a predetermined minimum size is reached. Each CU contains one or more prediction units (PUs).

[0006] Versatile video coding (VVC) is the latest international video codec standard developed by the Joint Video Experts Team (JVET) of ITU-TSG16 WP3 and ISO / IEC JTC1 / SC29 / WG11. The input video signal is predicted from a reconstructed signal derived from the coded picture region. The prediction residual signal is processed using a block transform. The transform coefficients are quantized and entropy coded along with other auxiliary information in the bitstream. The reconstructed signal is generated from the prediction signal and the reconstructed residual signal obtained by inversely transforming the dequantized transform coefficients. The reconstructed signal is further processed using loop filtering to remove coding artifacts. The decoded picture is stored in a frame buffer and used to predict future pictures in the input video signal.

[0007] In VVC, a codec picture is divided into non-overlapping square block areas represented by associated coding tree units (CTUs). A codec picture can be represented by a set of fragments, each fragment containing an integer number of CTUs. The CTUs in a fragment are processed continuously in a raster scan. Intra-frame prediction or inter-frame prediction can be used to decode bi-predictive (B) fragments, where there are at most two motion vectors and reference indices to predict the sample values ​​of each block. Prediction (P) fragments are decoded using intra-frame prediction or inter-frame prediction with at most one motion vector and reference index to predict the sample values ​​of each block. Intra-frame (I) fragments are decoded using only intra-frame prediction.

[0008] A quadtree (QT) with a nested multi-type-tree (MTT) structure can be used to partition a CTU into one or more non-overlapping coding units (CUs) to accommodate various local motion and texture characteristics. A CU can be further partitioned into smaller CUs using one of five partitioning types: quadtree partitioning, vertical binary tree partitioning, horizontal binary tree partitioning, vertical center-side ternary tree partitioning, and horizontal center-side ternary tree partitioning.

[0009] Each CU contains one or more prediction units (PUs). A prediction unit, along with the associated CU syntax, serves as the basic unit for transmitting prediction information. A specified prediction process is used to predict the values ​​of the relevant pixel samples within the PU. Each CU may contain one or more transform units (TUs) to represent prediction residual blocks. A transform unit (TU) consists of a transform block (TB) of luma samples and two corresponding transform blocks of chroma samples, with each TB corresponding to a residual block of samples from one color component. An integer transform is applied to the transform blocks. The level values ​​of the quantized coefficients are entropy encoded and decoded in the bitstream along with other auxiliary information. The terms coding tree block (CTB), coding block (CB), prediction block (PB), and transform block (TB) are defined to specify a 2D sample array for one color component associated with a CTU, CU, PU, ​​and TU, respectively. Therefore, a CTU consists of one luma CTB, two chroma CTBs, and related syntax elements. Similar relationships apply to CUs, PUs, and TUs.

[0010] For each inter-predicted CU, motion parameters consisting of motion vector, reference picture index and reference picture list usage index, as well as additional information are used for inter-prediction sample generation. The motion parameters can be sent explicitly or implicitly. When the CU is encoded or decoded in skip mode, the CU is associated with one PU and has no significant residual coefficients, no motion vector increments or reference picture indices are encoded or decoded. Merge mode is specified and the motion parameters of the current CU are obtained from neighboring CUs, including spatial and temporal candidates, as well as additional scheduling introduced in VVC. Merge mode can be applied to any inter-predicted CU. An alternative to merge mode is the explicit transmission of motion parameters, where the motion vector, the corresponding reference picture index and reference picture list usage flag for each reference picture list, as well as other required information, are sent explicitly per CU. In addition to the inter-frame codec features in HEVC, VVC also includes many new and improved inter-frame prediction codec tools, as shown below: extended merge prediction, Merge mode with MVD (MMVD), Symmetric MVD (SMVD) signaling, affine motion compensated prediction, Subblock-based temporal motion vector prediction (SbTMVP), Adaptive motion vector resolution (AMVR), Motion field storage: 1 / 16 luma sample MV storage and 8x8 motion field compression, Bi-prediction with CU-level weight (BCW), Bi-directional optical flow (BDOF), Decoder side motion vector refinement (DMVR), Geometric partitioning mode (GPM), Combined inter and intra prediction (CIIP). Summary of the Invention

[0011] The following summary is illustrative only and is not intended to be binding in any way. That is, the following summary is provided to introduce the concepts, highlights, benefits, and advantages of the novel and non-obvious technologies described herein. Selected, but not all, embodiments are further described in the detailed description below. Therefore, the following summary is not intended to identify essential features of the claimed subject matter, nor is it intended to be used to determine the scope of the claimed subject matter.

[0012] Some embodiments of the present disclosure provide a method for generating and using a representative prediction mode for a currently coded pixel block. A video codec receives data of a pixel block to be encoded or decoded as a current block of a current picture of a video. The video codec generates a predictor for the current block using a first prediction mode. The video codec determines a second prediction mode as a representative prediction mode for the current block. The video codec encodes or decodes the current block using the generated current block predictor and the representative prediction mode. The representative prediction mode can be used to select a transform for a prediction residual. The representative prediction mode can also be used to encode or decode subsequent blocks.

[0013] In some embodiments, the first prediction mode is not a directional intra prediction mode. The first prediction mode may be a regular intra mode, a special intra mode, or a non-intra mode. In some embodiments, the predictor is generated by matrix multiplying a predetermined or derived matrix with a set of input samples derived from adjacent samples of the current block. In some embodiments, the current block is a chroma component block, and the predictor for the current block is generated by applying a cross-component model to a co-located luma component block. In some embodiments, the predictor for the current block is generated based on a reference block determined by matching a first template region adjacent to the current block with a second template region in the current picture or a reference picture. In some embodiments, the predictor is generated by combining predictions from multiple prediction hypotheses.

[0014] In some embodiments, the predictor is generated by using a block vector or a motion vector to determine a reference region in a current image or a reference image, and the representative prediction mode is an intra-frame prediction mode for encoding or decoding pixel samples within or adjacent to the reference region. In some embodiments, the video codec determines the representative prediction mode by searching a plurality of predetermined locations within or adjacent to the reference region in a predetermined order to determine the representative prediction mode.

[0015] In some embodiments, a representative prediction mode is selected from a plurality of intra prediction modes based on a cost, wherein the cost of the intra prediction mode is a difference between a reconstructed sample and a predicted sample of a neighboring region of the current block, and the predicted sample of the neighboring region of the current block is generated based on the intra prediction mode. In some embodiments, the representative prediction mode is determined by deriving a plurality of histograms of gradient (HoGs) of the plurality of intra prediction modes, and the HoGs of the intra prediction modes are derived based on a set of predetermined predictors of the current block.

[0016] In some embodiments, the representative prediction mode is used to select a primary transform and / or a secondary transform. In some embodiments, the representative intra prediction mode is used to select a transform set, a transposition flag, or both for a non-separable transform. In some embodiments, the video codec provides the representative prediction mode as a most probable mode (MPM) for encoding and decoding subsequent blocks. In some embodiments, the current block is a luma component block, and the representative prediction mode is used to encode a co-located chroma component block in a chroma DM mode, for example, by generating an intra prediction predictor for the chroma block. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings are included to provide a further understanding of the present disclosure and are incorporated into and constitute a part of this disclosure. The drawings illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the present disclosure. It should be noted that the drawings are not necessarily drawn to scale, as certain components may be shown out of proportion in actual implementations in order to clearly illustrate the concepts of the present disclosure.

[0018] Figure 1 Displays intra prediction modes in different directions.

[0019] Figure 2A -B conceptually shows a reference template with extended lengths on the top and left sides for supporting wide-angle directional mode for non-square blocks of different aspect ratios.

[0020] Figure 3 It is shown that the intra prediction mode of the current block is implicitly derived using decoder-side intra mode derivation (DIMD).

[0021] Figure 4 It is shown that the intra prediction mode of the current block is implicitly derived using template-based intra mode derivation (TIMD).

[0022] Figure 5 Conceptually shown are chrominance and luma samples used to derive linear model parameters.

[0023] Figure 6 An example of dividing adjacent samples into two groups is shown.

[0024] Figure 7 The spatial components of a convolution filter are conceptually shown.

[0025] Figure 8 Shows the reference region used to derive the filter coefficients of the convolution model of the current block.

[0026] Figure 9 Shows a table for transform set selection.

[0027] Figure 10 The template matching prediction (TMP) is conceptually illustrated.

[0028] Figure 11 The use of a predictor of a current block to determine a representative prediction mode of the current block is conceptually illustrated.

[0029] Figure 12 A predetermined prediction unit in a reference region used to derive a representative intra prediction of a current block is shown.

[0030] Figure 13 An example video encoder is shown that may implement representative prediction modes for a pixel block being encoded.

[0031] Figure 14 Portions of a video encoder that implement representative prediction modes are shown.

[0032] Figure 15 The process of generating and using a representative intra prediction for a current block is conceptually illustrated.

[0033] Figure 16 An example video decoder is shown that can implement representative prediction modes for a pixel block being decoded.

[0034] Figure 17 The portion of a video decoder that implements a representative prediction mode is shown.

[0035] Figure 18 The process of generating and using a representative intra prediction for a current block is conceptually illustrated.

[0036] Figure 19 An electronic system that implements some embodiments of the present disclosure is conceptually illustrated. DETAILED DESCRIPTION

[0037] In the following detailed description, many specific details are set forth by way of example in order to provide a thorough understanding of the relevant teachings. Any variations, derivatives, and / or extensions based on the teachings described herein are within the scope of protection of this disclosure. In some cases, well-known methods, processes, components, and / or circuits related to one or more example embodiments disclosed herein may be described at a relatively high level without detail to avoid unnecessarily obscuring aspects of the teachings of this disclosure.

[0038] I. Intra-frame prediction mode

[0039] Intra-frame prediction methods use one or more reference layers adjacent to the current prediction unit (PU) and at least one intra-frame prediction mode to generate a prediction sub-unit for the current PU. The intra-frame prediction direction can be selected from a mode set containing multiple prediction directions. For each PU encoded and decoded via intra-frame prediction, an index is used and encoded to select one of the intra-frame prediction modes. The corresponding prediction is generated, and the residual is then derived and transformed.

[0040] Figure 1Intra-frame prediction modes in different directions are shown. These intra-frame prediction modes are called directional modes, excluding DC mode or planar mode. As shown in the figure, there are 33 directional modes (V: vertical direction; H: horizontal direction), so H, H+1 to H+8, H-1 to H-7, V, V+1 to V+8, V-1 to V-8 are used. Generally, the directional mode can be expressed as H+k or V+k mode, where k = ±1, ±2, ..., ±8. Each such intra-frame prediction mode can also be called an intra-frame prediction angle. In order to capture arbitrary edge directions presented in natural video, the number of directional intra-frame modes can be expanded from 33 used in HEVC to 65, so k ranges from ±1 to ±16. These more dense directional intra-frame prediction modes are applicable to all block sizes and luminance and chrominance intra-frame prediction. By including DC and Planar modes, the number of intra-frame prediction modes is 35 (or 67). In one embodiment, the first intra-frame prediction mode used to generate the current block predictor refers to one or more of DC, Planar and / or intra-frame directional modes. In another embodiment, the first intra-frame prediction mode is not the only prediction mode used to determine the final predictor for the current block. For example, the final predictor is a hybrid predictor formed by a weighted average of multiple prediction hypotheses. Each prediction hypothesis is generated using an intra-frame prediction mode, which can be any intra-frame prediction mode and / or any mode used for prediction generation mentioned in the present invention. In another embodiment, the first intra-frame prediction mode used to generate the predictor for the current block is any predefined matrix and / or model. For example, the matrix of the first intra-frame prediction mode uses adjacent reconstructed or predicted samples of the current block as input to generate the predictor for the current block. For example, the model of the first intra-frame prediction mode includes model parameters derived using adjacent reconstructed or predicted samples of the current block. For example, when generating the predictor for the current block, the model of the first intra-frame prediction mode uses adjacent reconstructed or predicted samples of the current block, and / or adjacent reconstructed or predicted samples of the co-located color component block of the current block, and / or current reconstructed or predicted samples of the co-located color component block of the current block, and / or current prediction samples of the current block. When the current block corresponds to a chrominance component (e.g., Cb and / or Cr), the co-located color component block of the current block corresponds to a co-located luma block that is positioned using the position of the current block and / or downsampled to the size of the current block. When generating a sample to be predicted at position (x, y) of the current block, this position is used to determine the sample to be used.

[0041] Among the 35 (or 67) intra prediction modes, some modes (for example, 3 or 5) are identified as a set of most probable modes (MPMs) for intra prediction in the current prediction block. The encoder can reduce the bit rate by sending an index for selecting one of the MPMs (instead of sending an index for selecting one of the 35 (or 67) intra prediction modes). For example, the intra prediction mode used in the left prediction block and the intra prediction mode used in the above prediction block are used as MPMs. When the intra prediction modes in two adjacent blocks use the same intra prediction mode, the intra prediction mode can be used as an MPM. When only one of the two adjacent blocks is available and is encoded and decoded in a directional mode, the two adjacent directions adjacent to the directional mode can be used as MPMs. DC mode and Planar mode are also considered MPMs and are used to fill the available positions in the MPM set, especially when the above or top adjacent block is not available or is not encoded and decoded in intra prediction, or when the intra prediction mode of the adjacent block is not a directional mode. If the intra prediction mode of the current prediction block is one of the modes in the MPM set, 1 or 2 bits are used to indicate which one it is. Otherwise, if the intra prediction mode of the current block is different from any entry in the MPM set, the current block will be coded as a non-MPM mode. There are 32 such non-MPM modes, and a (5-bit) fixed-length codec method is used to indicate this mode.

[0042] In one embodiment, the MPM list is constructed based on the intra modes of the left neighboring block and the upper neighboring block. Assuming that the mode of the left neighboring block is represented as Left and the mode of the upper neighboring block is represented as Above, the unified MPM list can be constructed as follows:

[0043] – When neighboring blocks are not available, their intra mode is set to Planar by default.

[0044] – If both Left and Above modes are non-angle modes:

[0045] MPM list → {plane, DC, V, H, V-4, V+4}

[0046] – If one of the Left and Above modes is an angled mode and the other is a non-angled mode:

[0047] Set the Max mode to the larger of the Left and Top modes.

[0048] MPM list → {plane, Max, Max-1, Max+1, Max-2, Max+2}

[0049] – If both Left and Above modes are angle modes and are different:

[0050] Set the Max mode to the larger of the Left and Top modes.

[0051] Set the "Min" mode to the smaller of "Left" and "Top"

[0052] *If "Max-Min" equals 1:

[0053] – MPM list → {plane, left, top, Min-1, Max+1, Min-2}

[0054] Otherwise, if "Max-Min" is greater than or equal to 62:

[0055] –MPM list → {plane, left, top, Min+1, Max-1, Min+2}

[0056] Otherwise, if Max-Min is equal to 2:

[0057] –MPM list → {plane, left, top, Min+1, Min-1, Max+1}

[0058] Otherwise:

[0059] –MPM list → {plane, left, top, Min-1, Min+1, Max-1}

[0060] – If both left and top are in angle mode and are the same:

[0061] ●MPM list → {plane, left, left-1, left+1, left-2, left+2}

[0062] In some embodiments, the MPM list contains spatial neighboring candidates (including left neighboring blocks and / or above neighboring blocks) and / or spatial non-neighboring candidates, and / or history-based candidates, and / or temporal candidates, and / or propagation candidates, some default intra-frame prediction modes, some modes derived from promising intra-frame prediction modes (each mode index is derived using the mode index of a predetermined candidate and a predetermined offset), and / or any available subset of intra-frame prediction modes. Spatial neighboring candidates can come from the left / upper / upper-left / upper-right / lower-left neighboring blocks of the current block, and / or any subset of the above positions. Spatial non-neighboring candidates can come from any predetermined position in the search pattern around the current block, and / or any subset of the above positions. History-based candidates can come from a history buffer that stores multiple intra-frame prediction mode information of previously coded blocks that were coded before the current block and have valid intra-frame prediction mode information. At a predetermined time point, the history buffer is empty. For example, the history buffer is empty at the beginning or end of a slice, CTU / CTB, CTU / CTB row, picture, tile, sequence and / or any predetermined unit. The temporal candidate may come from a buffer that stores intra-prediction mode information at a reference position in a reference frame (or reference picture) and / or a predetermined co-located picture, and / or stores intra-prediction mode information at any predetermined position near the reference position. For example, the reference position is a co-located block in a co-located picture. For another example, the reference position is indicated using motion information of a neighboring block or any predetermined block associated with the current block. The propagation candidate may come from intra-prediction mode information of one or more reference positions, which are referenced by motion information of a neighboring block or any predetermined block associated with the current block.

[0063] Traditional intra-frame angular prediction directions are defined as running from 45 degrees to -135 degrees clockwise. In VVC, several traditional intra-frame angular prediction modes are adaptively replaced with wide-angle intra-frame prediction modes for non-square blocks. The replaced modes are sent using the original mode index, which is then remapped to the wide-angle mode index after parsing.

[0064] In some embodiments, the total number of intra prediction modes remains unchanged, i.e., 67, and the intra mode encoding and decoding method remains unchanged. To support these prediction directions, a top reference template of length 2W+1 and a left reference template of length 2H+1 are defined. Figure 2A -B conceptually shows the length-extended top and left reference templates for supporting wide-angle directional mode for non-square blocks of different aspect ratios.

[0065] The number of alternative modes in wide-angle mode depends on the aspect ratio of the block. The alternative intra prediction modes for different blocks with different aspect ratios are shown in Table 1 below.

[0066] Table 1: Intra prediction modes replaced by wide-angle mode

[0067]

[0068] II. Decoder Side Intra Mode Derivation (DIMD)

[0069] Decoder-Side Intra Mode Derivation (DIMD) is a technique that derives two intra prediction modes / angles / directions from the reconstructed neighborhood samples (template) of a block and combines these two or more predictors with the planar mode predictor using weights derived from the gradients. DIMD mode is used as an alternative prediction mode and is always checked in high-complexity RDO mode. To implicitly derive the intra prediction mode for a block, texture gradient analysis is performed on both the encoder and decoder sides. This process starts with an empty Histogram of Gradients (HoG) containing 65 entries (or a size equal to any predetermined number of entries) corresponding to the 65 angular / directional intra prediction modes (or angular intra prediction modes available for DIMD). The magnitudes of these entries are determined during texture gradient analysis.

[0070] A video codec that implements DIMD performs the following steps: First, the video codec selects a template containing T = 3 columns and T = 3 rows from the left and top of the current block, respectively. This region is used as a reference for gradient-based intra prediction mode derivation. Second, a Sobel filter is applied horizontally and vertically to all 3×3 window positions, centered around the centerline pixel of the template. At each window position, the Sobel filter computes the pure horizontal and vertical intensities, G and G, respectively. x and G y The texture angle of the window is then calculated as follows:

[0071] angle=arctan(G x / G y ),

[0072] It can be converted to one of 65 angular intra prediction modes. Once the intra prediction mode index of the current window is derived as idx, the magnitude of its entry in HoG[idx] is updated by adding the following:

[0073] ampl=|G x |+|G y |

[0074] Figure 3The figure shows how decoder-side intra mode derivation (DIMD) is used to implicitly derive the intra prediction mode for the current block. The figure shows an example histogram of gradients (HoG) 310, which is calculated after applying the above operations to all or any subset of pixel positions in a template 315, which contains rows of neighboring pixel samples around the current block 300. Once the HoG is calculated, the indices of the two or more highest histogram bins (M1 and M2) are selected as two or more implicitly derived intra prediction modes (IPMs) for the block. The predictions of these two or more IPMs are further combined with the prediction of the planar mode as the prediction of the DIMD mode. The prediction fusion is applied as a weighted average of the three or more predictors mentioned above (M1 prediction, M2 prediction (and / or more predictions from other IPMs) and the planar mode prediction). For this purpose, the weight of the plane can be set to 21 / 64 (about 1 / 3). The remaining 43 / 64 (about 2 / 3) weights are shared by two or more HoG IPMs, and their weight size is proportional to the amplitude of the HoG strip. For example, combining only the predictions of M1, M2 and plane, the prediction fusion or combined prediction of DIMD can be:

[0075] Pred DIMD =(43*(w1*pred M1 +w2*pred M2 )+21*pred planar )>>6

[0076] w1=amp M1 / (amp M1 +amp M2 )

[0077] w2=amp M2 / (amp M1 +amp M2 )

[0078] In addition, two or more implicitly derived intra prediction modes are added to the most probable mode (MPM) list, so DIMD processing is performed before building the MPM list. The primary derived intra mode of a DIMD block is stored with the block and used to build the MPM lists of neighboring blocks and / or subsequent blocks.

[0079] III. Template-based Intra Mode Derivation (TIMD)

[0080] For mode selection, template matching can be applied by computing the cost between the reconstructed and predicted samples. One example is Template-based Intra Mode Derivation (TIMD). TIMD is a codec where the intra prediction mode of a CU is implicitly derived at the encoder and / or decoder using neighboring templates, rather than the encoder informing the decoder of the exact intra prediction mode.

[0081] Figure 4The intra prediction mode of the current block 400 is implicitly derived using template-based intra mode derivation (TIMD). As shown, the neighboring pixels of the current block 400 are used as a template 410. For each candidate intra prediction mode, if the candidate intra prediction mode refers to any of DC, planar, and / or directional intra prediction modes, prediction samples for the template 410 are generated using reference samples located in an L-shaped reference region 420 to the upper left of the template 410. The TM cost of the candidate intra mode is calculated based on the difference (e.g., SATD) between the reconstructed samples of the template and the prediction samples of the template generated by the candidate intra mode. The candidate intra prediction mode with the lowest cost is selected (as in implicit intra prediction mode derivation in TIMD mode) and used for intra prediction of the CU. In some embodiments, the candidate intra prediction modes can include 67 intra prediction modes (as in VVC) or be expanded to 131 intra prediction modes. In some embodiments, an MPM list can be used to indicate the directional information of the CU. Therefore, in order to reduce the intra mode search space and utilize the characteristics of the CU, the intra prediction mode is implicitly derived from the MPM list. In some embodiments, the candidate intra prediction mode may include DC, planar and / or intra directional modes. In some embodiments, the candidate intra prediction mode may include any intra prediction mode mentioned in the present invention and / or any mode for prediction generation. In some embodiments, the candidate intra prediction mode may include any predetermined matrix and / or model. For example, the candidate intra prediction mode refers to a matrix that uses the adjacent reconstructed or predicted samples of the current block as input to generate the predictor of the current block, and / or uses the adjacent reconstructed or predicted samples of the current block as input to generate the predictor of the current block template, and / or uses the adjacent reconstructed or predicted samples of the current block template as input to generate the predictor of the current block template. For example, the candidate intra prediction mode refers to a model parameter derived using the adjacent reconstructed or predicted samples of the current block. For example, when generating prediction samples for the current block template, the candidate intra prediction modes use neighboring reconstructed or predicted samples of the current block, and / or neighboring reconstructed or predicted samples of the co-located color component blocks of the current block, and / or neighboring reconstructed or predicted samples of the co-located color component blocks of the current block template, and / or neighboring reconstructed or predicted samples of the template of the current block. For example, when generating the predictor of the current block, the candidate intra prediction modes use neighboring reconstructed or predicted samples of the current block, and / or neighboring reconstructed or predicted samples of the co-located color component blocks of the current block, and / or current reconstructed or predicted samples of the co-located color component blocks of the current block, and / or current prediction samples of the current block. When the current block corresponds to a chrominance component (e.g., Cb and / or Cr), the co-located color component blocks of the current block correspond to co-located luma blocks, which are positioned using the position of the current block and / or downsampled to the size of the current block. When generating a sample to be predicted at position (x, y) of the current block, the position is used to determine the sample to be used.

[0082] In some embodiments of intra prediction modes implicitly derived from an MPM list, for each intra prediction mode in the MPM list, the sum of absolute transformed differences (SATD) between the predicted samples and the reconstructed samples of the template (or any predetermined measurement, such as the sum of absolute differences (SAD)) is calculated as the template matching (TM) cost of the intra prediction mode. The first two intra prediction modes with the smallest SATD are selected as TIMD modes. If it is decided to apply PDPC, the two TIMD modes are fused with weights after applying the PDPC process, and the current CU is encoded and decoded using this weighted intra prediction. In response to applying PDPC, a position dependent intra prediction combination (PDPC) is incorporated into the template-based TIMD mode derivation.

[0083] The costs of the two selected modes (mode 1 and mode 2) are compared with the threshold. In the test, a cost factor of 2 is applied as follows:

[0084] costMode2<2*costMode1

[0085] If this condition is true, prediction fusion is applied, otherwise only mode 1 is used. The weight of the mode is calculated based on its SATD cost as follows:

[0086] weight1=costMode2 / (costMode1+costMode2)

[0087] weight2=1-weight1

[0088] IV. Cross-component predictions generated using one or more of the following intra-prediction modes

[0089] a. Cross Component Linear Model (CCLM)

[0090] The Cross Component Linear Model (CCLM) or Linear Model (LM) mode is a cross-component prediction mode in which the chrominance components of a block are predicted using a linear model based on the co-located reconstructed luminance samples. The parameters of the linear model (e.g., scale and offset) are derived from the reconstructed luminance and chrominance samples adjacent to the block. For example, in VVC, the CCLM mode uses inter-channel dependencies to predict chrominance samples from reconstructed luminance samples. The prediction is performed using a linear model of the following form:

[0091] P(i,j)=α·rec′ L (i,j)+β (1)

[0092] P(i,j) in formula (1) represents the predicted chroma sample in a certain CU (or the predicted chroma sample of the current CU), rec′ L (i, j) represents the downsampled reconstructed luminance sample of the same CU (or the reconstructed luminance sample corresponding to the current CU).

[0093] The CCLM model parameters α (scaling parameter) and β (offset parameter) are derived based on up to four adjacent chroma samples and their corresponding downsampled luma samples. In LM_A mode (also known as LM-T mode), only the upper or top adjacent template is used to calculate the linear model coefficients. In LM_L mode (also known as LM-L mode), only the left template is used to calculate the linear model coefficients. In LM-LA mode (also known as LM-LT mode), both the left and upper templates are used to calculate the linear model coefficients (or parameters).

[0094] Figure 5 The chroma and luma samples used to derive linear model parameters are conceptually illustrated. The figure shows a current block 100 with luma and chroma component samples in a 4:2:0 format. The luma and chroma samples adjacent to the current block are reconstructed samples. These reconstructed samples are used to derive the cross-component linear model (parameters α and β). Since the current block is in a 4:2:0 format, the luma samples are first downsampled before being used in linear model derivation. In this example, there are 16 pairs of reconstructed luma (downsampled) and chroma samples adjacent to the current block. These 16 pairs of luma and chroma values ​​are used to derive the linear model parameters.

[0095] Assuming the current chroma block size is W×H, W' and H' are set to:

[0096] – When using LM-LT mode, W'=W, H'=H;

[0097] – When using the LM-T mode, W'=W+H;

[0098] – When using LM-L mode, H'=H+W

[0099] The adjacent positions above are recorded as S[0,-1]...S[W'-1,-1], and the adjacent positions on the left are recorded as S[-1,0]...S[-1,H'-1]. Then the four samples are selected as

[0100] – When LM mode is applied (both upper and left adjacent samples are available), S[W' / 4,-1], S[3*W' / 4,-1], S[-1, H' / 4], S[-1, 3*H' / 4];

[0101] – When LMT mode is applied (only upper adjacent samples are available), S[W' / 8,-1], S[3*W' / 8,-1], S[5*W' / 8,-1], S[7*W' / 8,-1];

[0102] – When LM-L mode is applied (only left adjacent samples are available), S[-1,H' / 8],S[-1,3*H' / 8],S[-1,5*H' / 8],S[-1,7*H' / 8];

[0103] The four adjacent brightness samples at the selected position are downsampled and compared four times to find the larger of the two values: x 0 A and x 1 A , and two smaller values: x 0 B and x 1 B Their corresponding chrominance sample values ​​are represented as y 0 A 、y 1 A 、y 0 B and y 1 B Then, X A 、X B 、Y A and Y B The following formula can be derived:

[0104] X a =(x 0 A +x 1 A +1)>>1;X b =(x 0 B +x 1 B +1)>>1; (2)

[0105] Ya =(y 0 A +y 1 A +1)>>1;Y b =(y 0 B +y 1 B +1)>>1 (3)

[0106] The linear model parameters α and β are obtained according to the following equations:

[0107]

[0108] β=Y b -α·X b (5)

[0109] The calculation of the α and β parameters according to formulas (4) and (5) can be implemented using a lookup table. In some embodiments, to reduce the memory required to store the lookup table, the diff value (the difference between the maximum and minimum values) and the parameter α are expressed in exponential notation. For example, diff is approximated using a 4-bit significant part and an exponent. Therefore, the lookup table for 1 / diff is reduced to 16 elements for 16 significant digits, as shown below:

[0110] DivTable[]={0,7,6,5,5,4,4,3,3,2,2,1,1,1,1,0} (6)

[0111] This reduces the complexity of the calculations and the amount of memory required to store the required tables.

[0112] In some embodiments, to obtain more samples for calculating CCLM model parameters α and β, for LM-T mode, the upper template is expanded to include (W+H) samples; for LM-L mode, the left template is expanded to include (H+W) samples. For LM-LT mode, both the expanded left template and the expanded upper template are used to calculate the linear model coefficients.

[0113] In some embodiments, to match the chroma sample positions of a 4:2:0 video sequence, two types of downsampling filters are applied to the luma samples to achieve a 2:1 downsampling ratio in both the horizontal and vertical directions. The choice of downsampling filter is specified by a sequence parameter set (SPS) level flag. The two downsampling filters are shown below, corresponding to "Type 0" and "Type 2" content, respectively.

[0114] rec L '(i,j)=[rec L(2i-1,2j-1)+2*rec L (2i-1,2j-1)+rec L (2i+1,2j-1)+rec L (2i-1,2j)+2*rec L (2i,2j)+rec L (2i+1,2j)+4]>>3 (7)

[0115] rec L '(i,j)=[rec L (2i,2j-1)+rec L (2i-1,2j)+4*rec L (2i,2j)+rec L (2i+1,2j)+rec L (2i,2j+1)+4]>>3 (8)

[0116] In some embodiments, when the upper reference line is at a CTU boundary, only one luma line (common line buffer in intra prediction) is used to generate the downsampled luma samples.

[0117] In some embodiments, the calculation of the α and β parameters is performed as part of the decoding process, rather than just a search operation by the encoder. Therefore, there is no need to use any syntax to convey the α and β values ​​to the decoder.

[0118] For chroma intra mode codec, a total of 8 or more intra prediction modes or any subset of the 8 intra prediction modes are allowed. These modes include five traditional intra prediction modes and three cross-component linear model modes (LM_LA, LM_A, and LM_L). Chroma intra mode codec can directly depend on the intra prediction mode of the corresponding luma block. For example, the chroma intra mode transmission and the corresponding luma intra prediction mode are shown in the following table:

[0119] Table 2:

[0120]

[0121] Since the independent block partition structure of luma and chroma components is enabled in I-slices, a chroma block may correspond to multiple luma blocks. Therefore, for the chroma derived mode (DM), the intra prediction mode of the corresponding luma block covering the center position of the current chroma block will be directly inherited.

[0122] Taking the chroma intra mode codec using a total of 8 intra prediction modes as an example, the chroma intra prediction mode uses a unified binarization table (mapped to bin string), as shown in the following table:

[0123] Table 3

[0124]

[0125] In the table, the first bin indicates whether it is normal mode (0) or LM mode (1). If it is LM mode, the next bin indicates whether it is LM_CHROMA (LM_LA) (0). If it is not LM_CHROMA, the next bin indicates whether it is LM_L (0) or LM_A (1). For this case, when sps_cclm_enabled_flag is 0, the first bin of the binarization table corresponding to intra_chroma_pred_mode can be discarded before entropy coding and decoding. Or, in other words, the first bin is inferred to be 0 and therefore not coded or decoded. This single binarization table is applicable to the cases where sps_cclm_enabled_flag is equal to 0 and 1. The first two bins in the table use their own context model for context coding and decoding, and the remaining bins are bypass coded and decoded.

[0126] In some embodiments, to reduce luma-chroma latency in dual trees, when the 64x64 luma codec tree node is not split (and the 64x64 CU is not using ISP) or partitioned using QT, the chroma CUs in the 32x32 / 32x16 chroma codec tree nodes are allowed to use CCLM as follows:

[0127] If a 32x32 chroma node is not split or partitioned using QT, all chroma CUs in the 32x32 node can use CCLM

[0128] If a 32x32 chroma node uses horizontal BT splitting and its 32x16 child nodes are unsplit or use vertical BT splitting, all chroma CUs in the 32x16 chroma node can use CCLM.

[0129] In all other luma and chroma codec tree split conditions, CCLM is not allowed for chroma CUs.

[0130] b. Multi-model CCLM (MMLM)

[0131] The multiple model CCLM mode (MMLM) uses two models to predict the chroma samples corresponding to the luma samples in the entire CU. Similar to CCLM, MMLM uses three multi-model CCLM modes (MMLM_LA, MMLM_A, and MMLM_L) to indicate whether to use both upper and left neighboring samples, only upper neighboring samples, or only left neighboring samples in model parameter derivation.

[0132] In MMLM, the adjacent luma samples and chroma samples of the current block are divided into two groups, each of which is used as a training set to derive a linear model (i.e., a specific α and β are derived for a specific group). In addition, the samples of the current luma block are also classified based on the same rules as the classification of the adjacent luma samples.

[0133] Figure 6 An example of dividing adjacent samples into two groups is shown. The threshold is calculated as the average value of adjacent reconstructed brightness samples. At [x, y], if Rec′ L [x,y]<=Threshold, then the neighborhood samples are classified as group 1; at [x,y], if Rec′ L [x,y](or rec′ L ((x,y)>Threshold, then the adjacent samples are classified into group 2. Therefore, the multi-model CCLM prediction results of the chrominance samples are:

[0134] Pred c [x,y]=α1×Recˊ L [x,y]+β1 If Rec′ L [x,y]≤Threshold

[0135] Pred c [x,y]=α2×Recˊ L [x,y]+β2 If Rec′ L [x,y]>Threshold

[0136] c. Convolutional Cross-Component Model

[0137] In some embodiments, a convolutional cross-component model (CCCM) is used to improve cross-component prediction performance. For some embodiments, the convolutional model has a 7-tap filter that includes a 5-tap plus-shaped spatial component, a nonlinear term, and a bias term. The input to the filter's spatial 5-tap component includes a center (C) luminance sample that is co-located with the chrominance sample to be predicted and its upper / north (N), lower / south (S), left / west (W), and right / east (E) neighboring samples. Figure 7 Conceptual illustration of the spatial components of the convolution filter. The nonlinear term (denoted as P) is expressed as a power of 2 of the center luma sample C, and scaled to the sample value range of the content:

[0138] P=(C*C+midVal)>>bitDepth (9)

[0139] Therefore, for 10-bit content, the nonlinear term P is calculated as follows:

[0140] P=(C*C+512)>>10

[0141] The bias term (denoted as B) represents a scalar offset between the input and output (similar to the offset term in CCLM) and is set to the intermediate chrominance value (512 for 10-bit content). The output of the filter is calculated as the filter coefficient c i Convolution with the input values, clipped to the range of valid chroma samples:

[0142] predChromaVal=c0C+c1N+c2S+c3E+c4W+c5P+c6B (10)

[0143] Filter coefficient c i is calculated by minimizing the MSE or any distortion between the predicted chrominance samples and the reconstructed chrominance samples in the reference region using a predetermined regression method (e.g., Gaussian elimination). Figure 8 An example of a reference region used to derive the filter coefficients of the convolution model for the current block is shown. The reference region includes the (chroma) sample (reference) lines above and to the left of the current block 400. (In this example, the current block 400 is a PU). The reference region extends one PU width to the right and one PU height below the PU boundary. This region is adjusted to contain only available samples. The extended area of ​​the reference region is used to support the "side samples" of the conformal spatial filter and to fill in unavailable areas.

[0144] MSE minimization is achieved by calculating the autocorrelation matrix of the luma input and the cross-correlation vector between the luma input and the chroma output. The autocorrelation matrix is ​​decomposed using LDL decomposition or Gaussian elimination, and the final filter coefficients (or model parameters) are calculated by back substitution. This process is similar to the calculation of ALF filter coefficients in ECM. However, in some embodiments, LDL decomposition or Gaussian elimination is selected instead of Cholesky decomposition to avoid the use of square root operations.

[0145] V. Intra block copy (IBC) mode

[0146] Intra Block Copy (IBC) is also known as Current Picture Reference (CPR). An IBC (or CPR) motion vector refers to a motion vector that references a reconstructed reference sample in the current picture. In some embodiments, IBC prediction mode is considered a third prediction mode for encoding a CU, in addition to intra or inter prediction modes.

[0147] Since IBC mode is implemented as a block-level codec, block matching (BM) and / or template matching is performed at the encoder to find the optimal block vector (or motion vector) for each CU. Here, the block vector (BV) is used to indicate the displacement from the current block to a reference block that has been reconstructed within the current image. In some embodiments, the luma block vector of a CU encoded and decoded by IBC uses integer precision.

[0148] VI. Combined inter and intra prediction (CIIP)

[0149] Inter-frame and intra-frame combined prediction (CIIP) combines the inter-frame prediction signal with the intra-frame prediction signal. inter The intra prediction signal P is derived using the same inter-frame prediction process as the normal merge mode; intra According to conventional intra prediction processing, one or more intra prediction modes are derived through planar mode or from a predetermined mechanism. For example, in the following, the predetermined mechanism is based on the adjacent reference area (template) of the current block. The intra prediction mode of the CU is implicitly derived from the adjacent templates at the encoder and decoder, rather than being sent to the decoder as an exact intra prediction mode bit. For example, the intra prediction mode is implicitly derived using TIMD and / or DIMD and / or any variant of TIMD and / or DIMD. For the example using a TIMD, the prediction samples of the template are generated using the reference samples of the template of each candidate mode. The cost is calculated as the SATD between the prediction samples and the reconstructed samples of the template. The lowest cost intra prediction mode and / or part of the lower cost intra prediction mode is selected for intra prediction of the CU. The candidate mode can be all MPMs and / or any subset of MPMs, or the 67 intra prediction modes in VVC, or extended to 131 intra prediction modes. The intra and inter prediction signals are combined by weighted averaging, where the weight values ​​are calculated based on the encoding and decoding modes of the top and left neighboring blocks. CIIP predicts P CIIP The composition is as follows: (wt is the weight value)

[0150] P CIIP =((4–wt)*P inter +wt*P intra +2)>>2

[0151] In some embodiments, when a CU is encoded and decoded in merge mode, if the CU contains at least 64 luma samples (i.e., the CU width multiplied by the CU height is equal to or greater than 64), and the CU width and CU height are both less than 128 luma samples, an additional flag may be sent to indicate whether the current CU applies the CIIP mode.

[0152] VII. Matrix-weighted Intra Prediction (MIP)

[0153] The matrix-weighted intra prediction (MIP) method is an intra-frame prediction technique. To predict samples for a rectangular block of width W and height H, MIP takes as input a row of H reconstructed neighboring boundary samples to the left of the block and a row of W reconstructed neighboring boundary samples above the block. If reconstructed samples are not available, these samples are generated using traditional intra-frame prediction. The generation of the prediction signal is based on three steps: averaging, matrix-vector multiplication, and linear interpolation.

[0154] Among the boundary samples, predetermined samples (e.g., four samples or eight samples) are selected by averaging according to the block size and shape. Specifically, the input boundary bdry is averaged by averaging adjacent boundary samples according to a predetermined rule depending on the block size. top and bdry left Shrink to a smaller boundary and Then, in some embodiments, the two reduced boundaries and Connect into a reduced boundary vector bdry red , so for blocks of shape 4×4, the size of this vector is 4, and for blocks of all other shapes, the size of this vector is 8. If mode refers to a MIP mode, then this connection is defined as follows:

[0155]

[0156] In some embodiments, a matrix-vector multiplication is performed with the average sample as input, and then an offset is added. The result is a downscaled prediction signal for a set of subsampled samples in the original block. red Generate reduced prediction signal pred red , the signal is of width W red , height is H red The signal of the downsampling block. Among them, W red and H red Defined as:

[0157]

[0158] Reduced prediction signal pred red Computed by taking the matrix-vector product and adding the offset:

[0159] pred red =A·bdry red +b

[0160] A is the prediction matrix, when W=H=4, A has W red ·H red rows and 4 columns, in all other cases there are 8 columns. b is a vector of size . The prediction matrix a and the offset vector b are taken from one of the sets S0, S1, S2. For example, the index idx = idx(W,H) can be defined as follows:

[0161]

[0162] In some embodiments, each coefficient of the matrix A (prediction matrix) is represented with 8-bit precision. Set S0 consists of 16 matrices Each matrix has 16 rows and 4 columns, and 16 offset vectors Each vector has a size of 16. The matrices and offset vectors in this set are used for blocks of size 4×4. Set S1 consists of 8 matrices Each matrix has 16 rows and 8 columns, and 8 offset vectors The size of each vector is 16. The set S2 consists of 6 matrices Each matrix has 64 rows and 8 columns, and 6 offset vectors The size is 64.

[0163] In some embodiments, the prediction signals for the remaining positions are generated by linear interpolation of the prediction signals on the subsample set, where the linear interpolation is a single-step linear interpolation in each direction. Regardless of the block shape or block size, the interpolation is performed first in the horizontal direction and then in the vertical direction.

[0164] In some embodiments, the transform set and / or transposition flag of the predetermined transform process is determined by the intra prediction mode predModeIntra of the current transform block. In one embodiment, the predetermined transform process is a non-separable transform of the primary transform. In another embodiment, the predetermined transform process is a non-separable transform of the secondary transform. For example, a low-frequency non-separable transform (LFNST). In another embodiment, the predetermined transform process is a separable transform of the primary transform. For example, any transform type of DCT-II and / or MTS. In another embodiment, the predetermined transform process is a separable transform of the secondary transform. In some embodiments, predModeIntra is used to perform the following operations: (i) if the current block is a MIP codec block, predModeIntra is mapped to PLANAR; (ii) if the current block is a CCLM codec block, predModeIntra is mapped to a co-located luma intra prediction mode. In some embodiments, predModeIntra is further derived from a wide-angle intra prediction mapping in the range [-14, 83]. Taking LFNST as an example, the LFNST transform set is selected from 35 transform sets and 3 inseparable transform matrices (kernels) for each transform set in LFNST. The transform set index lfnstTrSetIdx is defined according to predModeIntra. Figure 9 A table showing LFNST transform set selection is shown. This table maps different intra prediction modes (or different predModeIntra) to different LFNST set indices.

[0165] In some embodiments, the LFNST transpose flag determines the scan order of the LFNST output (decoder). The LFNST transpose flag is determined by predModeIntra as follows: (i) if predModeIntra is less than or equal to 34, the LFNST transpose flag is set to 0; (ii) otherwise, the LFNST transpose flag is set to 1. In some embodiments, for MIP codec blocks, predModeIntra maps to PLANAR mode, uses LFNST transform set 0, and the LFNST transpose flag is always equal to 0. In some embodiments, LFNST is enabled for MIP codec blocks with a width and height greater than or equal to 16.

[0166] Matrix-weighted intra prediction (MIP) takes as input H reconstructed adjacent boundary samples in the row to the left of the block and W reconstructed adjacent boundary samples in the row above the block. The generation of the predicted samples is based on (i) boundary downsampling, (ii) matrix-vector multiplication, and (iii) MIP prediction upsampling. Specifically, a video codec performing MIP first downsamples the reference samples, then multiplies the downsampled reference samples by a prediction matrix (matrix A) to generate partial predicted samples. After that, the partial predicted samples are upsampled to generate the predicted samples at the remaining positions.

[0167] VIII. Enhanced Multiple Transform Selection (MTS) for Intra Coding / Decoding

[0168] In some embodiments, when performing Multiple Transform Selection (MTS) with the inverse transform coefficients of the residual / residual as the primary transform, only DST7 and DCT8 transform kernels are used, which are used for intra and / or inter coding / decoding. In some embodiments, other primary transforms are also employed, including DCT5, DST4, DST1, and the identity transform (IDT).

[0169] In some embodiments, the MTS set depends on the TU size and the intra prediction mode information. In some embodiments, 16 different TU sizes are considered, and for each TU size, 5 different categories are considered according to the intra mode information. For each category, 1, 4, or 6 different transform pairs are considered. The number of intra MTS candidates can be adaptively selected (between 1, 4, and 6 MTS candidates) according to the sum of the absolute values of the transform coefficients. The sum is compared with two fixed thresholds (th0 and th1) to determine the total number of allowed MTS candidates. For example, 1 candidate: sum ≤ th0; 4 candidates: th0 < sum ≤ th1; 6 candidates: sum > th1. Although a total of 80 different categories can be considered, some of these different categories usually share exactly the same transform set. Therefore, there may be 58 (less than 80) unique entries in the final look up table (LUT).

[0170] In some embodiments, for the angle modes, the joint symmetry of the TU shape and the intra prediction is taken into account. Thus, mode i (i>34) with a TU shape of AxB can be mapped to the same class corresponding to mode j=(68-i) with a TU shape of BxA. However, for each transform pair, the order of the horizontal and vertical transform kernels is swapped. For example, for a 16x4 block of mode 18 (horizontal prediction) and a 4x16 block of mode 50 (vertical prediction), they are mapped to the same class. However, the vertical and horizontal transform kernels are swapped. In some embodiments, for the wide-angle intra mode, the closest regular angle mode is used to determine the transform set. For example, mode 2 is used for all modes between -2 and -14. Similarly, mode 66 is used for modes 67 to 80.

[0171] IX. Template Matching Prediction (TMP)

[0172] Template matching prediction (TMP), also known as intra TMP, is a special intra prediction mode that copies the best prediction block from the reconstructed portion of the current frame. The best prediction block is the block whose L-shaped (including the upper and left) template and / or only the upper template and / or only the left template matches the current template. Figure 10 The template matching prediction (TMP) is conceptually illustrated. As shown, a current block 1010 in a current image 1000 has an L-shaped adjacent region, which serves as a current template 1015. Within a predetermined search range, the video encoder and / or decoder searches for a template 1025 that is most similar to the current template 1015 in the reconstructed portion of the current frame and uses the reconstructed samples of the corresponding block 1020 as the prediction block / predictor for the current block. The encoder then signals the use of this mode, and / or the decoder interprets this signal.

[0173] X. Representative prediction mode of the current block

[0174] Some embodiments of the present disclosure provide a method for determining a representative prediction mode of a pixel block, wherein the pixel block can be predicted and decoded using a regular intra mode, a special intra mode, or a non-intra mode.

[0175] Conventional intra mode refers to the use of one or more traditional intra prediction modes and spatially adjacent reference samples (located in adjacent or non-adjacent reference lines of the current block) to generate the prediction sub of the current block. Conventional intra mode can be used for luma and / or chroma components, and the traditional intra prediction mode used can be indicated by syntax elements and / or predetermined implicit derivation methods (such as DIMD and / or TIMD).

[0176] Special intra modes refer to the application of an alternative scheme (e.g., a matrix-based scheme and / or cross-component information, rather than the traditional 67 or 131 intra prediction modes) to spatially adjacent reference samples to generate a predictor for the current block. For example, a special intra mode may refer to matrix-weighted intra prediction (MIP). Special intra modes may also be used for luma and / or chroma components. For example, a special intra mode may refer to any LM mode (e.g., CCLM and / or MMLM) and / or any LM variant (an intra prediction mode for generating cross-component predictions, such as CCCM and / or Gradient Linear Model (GLM)), or TMP. When a GLM is applied to the current block, compared to CCLM, GLM utilizes luma sample gradients to derive a linear model, rather than downsampled luma values. Therefore, if the current block uses a model such as CCLM, CCCM, GLM, and / or any model derived using cross-component correlation to predict chroma components, in some embodiments, a representative prediction mode may be required at some encoding or decoding stages.

[0177] Non-intra mode may refer to intra block copy (IBC), inter mode, and / or any mode whose mode type is not equal to the intra mode type (MODE_TYPE_INTRA). Non-intra mode may be used for luma and / or chroma components.

[0178] During the prediction phase, in order to improve prediction efficiency, the video codec may adjust according to various prediction modes when generating the predictor for the current block or after generating the predictor for the current block. However, even if multiple different intra prediction modes are used to generate the predictor (e.g., conventional intra prediction using multiple assumptions based on multiple traditional intra prediction modes), or no intra prediction mode is used to generate the predictor (e.g., special or non-intra prediction that does not use traditional intra prediction modes), the encoding or decoding process of the current block may still require a representative prediction mode for the current block.

[0179] In some embodiments, the video codec may use a representative (intra-frame prediction) prediction mode for the current block in some encoding or decoding stages (possibly after or after the prediction process). For example, for the transform / inverse transform stage of the current block, the video codec may use the representative intra-frame prediction mode to select a transform set and / or transposition flag for a secondary transform (e.g., a low-frequency non-separable transform, or LFNST) or any predetermined separable / non-separable and / or low-frequency / non-low-frequency transform. (For the encoder, the secondary transform is applied to the coefficients after the primary transform; for the decoder, the inverse secondary transform is applied to the received dequantized coefficients before the inverse primary transform.)

[0180] In some embodiments, for the transform / inverse transform stage, the video codec can use a representative prediction mode to select a transform kernel for the primary transform (e.g., a default primary transform, such as DCT-II and / or MTS, and / or a non-separable primary transform). The primary transform can be any predetermined transform that is performed on the residual at the encoder or on the inverse secondary transform coefficients at the decoder.

[0181] In some embodiments, a representative (intra-frame prediction) mode of the current block can be used to derive the MPM list of the subsequent codec block. In other words, the codec block can use the intra-frame prediction mode information (e.g., a representative (intra-frame prediction) prediction mode) of adjacent and / or subsequent codec blocks to derive the MPM list.

[0182] In some embodiments, if the current block is a luma block, the video codec may use a representative prediction mode of the luma block to determine the chroma DM of the co-located chroma block. (In chroma DM mode, the representative (intra) prediction mode of the corresponding (co-located) luma block covering the center position of the current chroma block is directly inherited by the chroma block.)

[0183] In some embodiments, the representative prediction mode of the current block may be predetermined as a DIMD-derived mode (derived by applying histogram / gradient analysis to all or any subset of the prediction samples of the current block, and / or by applying histogram / gradient analysis to all or any subset of spatially adjacent reconstructed samples), a TIMD-derived mode (derived by applying analysis to all or any subset of the prediction samples of the current block, by comparing the distortion between the final predictor of the current block and the false predictor of the current block generated using each candidate representative prediction mode, and / or by applying template analysis to all or any subset of spatially adjacent reconstructed samples, by comparing the distortion between the reconstructed samples on the template and the predictors on the template generated using each candidate representative prediction mode), or DC, planar, horizontal, vertical, diagonal, or any predetermined mode from the available intra-frame prediction modes. For example, for a current block that is encoded and decoded by MIP and whose predefined intra-frame prediction mode is a DIMD derived mode, the DIMD derived mode is stored as a representative prediction mode for the current block and / or can be used to encode and decode subsequent blocks, such as deriving an MPM list for subsequent blocks, determining an intra-frame prediction mode for a co-located chroma block (e.g., if the current block is luma, deriving a chroma DM), and / or selecting a transform set and / or a transposition flag and / or a transform kernel for the primary transform and / or secondary transform of the current block.

[0184] In some embodiments, the representative prediction mode is determined by applying a predetermined process to all or any subset of prediction samples of the current block. The video codec performing the predetermined process may suggest the intra prediction mode as the representative prediction mode for the current block.

[0185] In some embodiments, the predetermined process for deciding the representative prediction mode is DIMD and / or TIMD.For some embodiments using DIMD as the predetermined process, a DIMD window is applied to the predictor of the current block. Figure 11 This figure conceptually illustrates how a predictor for a current block is used to determine a representative prediction mode for the current block. As shown, a current block 1100 is encoded and / or decoded using a predictor 1110, which is a prediction sample generated based on a prediction mode 1105 for the current block (which can be any prediction mode, such as regular intra, special intra, or non-intra). A DIMD window is applied to the samples of predictor 1110 to perform a cumulative histogram (HoG) analysis 1120, which is then used to determine an intra prediction mode 1130 to serve as the representative prediction mode for the current block 1100.

[0186] In some embodiments, the prediction samples in the predictor 1110 are temporary predictors (e.g., partial prediction samples in a MIP) or the first downsampled predictor as the reduced prediction samples, and the predetermined processing is applied to all or any subset of the reduced prediction samples. For example, the size of the current block can be downsampled from 2Mx2N to MxN. In some embodiments, when determining the representative prediction mode, the center of the DIMD window is applied to samples within the current block (or reduced current block), but not to samples located at the boundary of the current block. If the DIMD window requires any samples outside the current (or reduced) block, padding from the boundary is used or only a DIMD window with a window center position that is not at the boundary is applied, rather than referencing samples outside the current (or reduced) block.

[0187] For example, for a current block encoded by a MIP and a representative prediction mode from a DIMD, the representative prediction mode may be stored for the MIP and / or may be used to encode subsequent blocks, e.g., to derive an MPM list for subsequent blocks, to determine an intra prediction mode for a co-located chroma block (e.g., to derive a chroma DM if the current block is luma), and / or to select a transform set and / or a transpose flag and / or a transform kernel for the primary transform and / or secondary transform of the current block.

[0188] In some embodiments, for each codec block (or any predetermined codec unit) or each valid codec block (or any predetermined codec unit) that satisfies a predetermined condition (e.g., intra codec, non-intra codec, inter codec, traditional intra prediction mode codec, or non-traditional intra prediction mode codec), a representative prediction mode is derived by performing predetermined processing (e.g., any of the above-mentioned DIMD and / or TIMD derivations) on the reconstructed samples of the codec block or valid codec block. The representative prediction mode derived here can be stored and / or referenced by one or more subsequent blocks (e.g., MPM list construction and / or transform set selection and / or prediction generation of subsequent blocks) and / or one or more co-located chroma blocks (e.g., chroma DM of the co-located chroma block). In some embodiments, mode information of a reference region of a current block is used to derive the representative prediction mode of the current block. Compared to performing texture analysis (DIMD or TIMD) on the predictor within the current block or spatially adjacent samples, deriving the representative prediction mode of the current block using the pattern information of the reference area has the advantage of being simpler and / or being able to output the derived representative prediction mode of the current block earlier, without having to wait for the prediction stage of the current block to know the prediction or reconstructed samples to be used.

[0189] In some embodiments, the pattern information of the reference area used to derive the representative prediction mode of the current block can be the pattern information of any predetermined subset of prediction units in the reference area of ​​the current block or codec block (e.g., a reference block and / or an adjacent template / area of ​​the reference block) or any predetermined area. The pattern information may include mode type, intra-frame prediction mode, motion information, block width, block height, block area, block shape, block ratio, residual information, transform information, partition information and / or any subset / extension of the above information. In some embodiments, if the video codec cannot determine the representative prediction mode from the predetermined subset of prediction units, a default prediction mode is used as the representative prediction mode for the current block. The default prediction mode can be any available intra-frame prediction mode, such as DC, planar, normal DIMD mode (which can be derived at the decoder), DIMD-derived mode and / or TIMD-derived mode.

[0190] In some embodiments where the current block is encoded and decoded using the intra block copy (IBC) mode, the reference block is indicated by a block vector (BV), and one or more mode information (e.g., intra prediction mode) stored in the reference block is used to derive the representative prediction mode of the current block. If the reference block is any of the special intra modes and / or non-intra modes, the default intra prediction mode can be used as the representative prediction mode. Otherwise, the intra prediction mode of the reference block (e.g., the intra prediction mode used to generate the reference block predictor) is used as the representative prediction mode of the current block.

[0191] In some embodiments where the current block is encoded and decoded using the IBC mode, one or more prediction units in the reference area (which may include an IBC reference block and / or one or more prediction units spatially adjacent / non-adjacent to the reference block) are predetermined, and a scanning order is applied to the predetermined prediction units.

[0192] Figure 12 1. Predefined prediction units in a reference region used to derive a representative intra prediction for a current block. As shown, a current block 1200 references a reference region 1210 using a block vector (BV) 1205. The figure shows several predetermined prediction units in or near the reference region 1210, labeled P1, P2, P3, P4, and P5.

[0193] The video codec can select intra-frame prediction mode in prediction units. Prediction units P1 to P5 can be scanned / considered / checked in a predetermined order: P1, P2, P3, (P4, P5). P1 covers the center position of the reference block. P2 covers the lower right position of the reference block. P3 covers the upper left position of the reference block. P4 covers the upper adjacent predetermined position outside the reference block (e.g., the center position). P5 covers the left adjacent predetermined position outside the reference block (e.g., the center position). If the height of the block is greater than the width, P5 is checked first, then P4. Otherwise, P4 is checked first, then P5.

[0194] In some embodiments, the first predetermined prediction unit with an intra prediction mode arranged in scanning order provides the representative prediction mode for the current block. In some embodiments, an explicit index is sent / parsed to indicate the intra prediction mode in the predetermined prediction unit as the representative prediction mode.

[0195] In some embodiments where the current block uses IBC codec, one or more prediction units in the reference region (which may include an IBC reference block and / or one or more prediction units that are spatially adjacent to or non-adjacent to the reference block) are predetermined, and a voting method is used to determine a representative prediction mode from the predetermined prediction units. In these embodiments, the most popular prediction mode is used as the representative prediction mode for the current block.

[0196] When a predetermined prediction unit does not have a valid (intra) prediction mode, the unit is designated as an invalid prediction unit. Invalid prediction units can be skipped or a default prediction mode can be designated as the prediction mode for the invalid prediction unit. In some embodiments, predetermined prediction units using non-intra modes (e.g., IBC), intra TMP, inter prediction, and / or MIP modes are considered invalid prediction units.

[0197] In some embodiments, when a representative prediction mode (intra-frame prediction mode) is used to determine the transform set and / or transform kernel of the multiple transform selection (MTS), the MTS can be implicit or explicit, and the current block can be predicted by a conventional intra-frame mode, a special intra-frame mode and / or a non-intra-frame mode. Explicit MTS (such as enhanced MTS for intra-frame coding and decoding) refers to sending an MTS index to determine a transform candidate (a transform pair consisting of a horizontal transform direction and a vertical transform direction) from the selected MTS set. Implicit MTS refers to using an implicit mapping rule (not dependent on syntax elements) to determine the transform candidate. In some embodiments, for explicit MTS, the selection of the MTS set depends on the representative prediction mode; for implicit MTS, the transform candidate is determined based on the representative prediction mode according to the mapping rule.

[0198] Tables 4-6 below show example mapping rules for representative prediction modes to horizontal and vertical transforms: ("Ang." = angular intra mode)

[0199] Table 4:

[0200]

[0201] Table 5:

[0202] Intra prediction mode Horizontal Transform Vertical Transformation Even numbers (2, 4, 6, ...) DCT-VIII DCT-II Odd numbers (1, 3, 5, ...) DCT-II DST-VII

[0203] Table 6:

[0204] Intra prediction mode Horizontal Transform Vertical Transformation Even numbers (2, 4, 6, ...) DCT-II DCT-VIII Odd numbers (1, 3, 5, ...) DST-VII DCT-II

[0205] In some embodiments, representative intra prediction modes may be used for transform selection in a 4:4:4 color format. For example, when the color format is 4:4:4, MIP may be used for chroma. For chroma MIP blocks, representative prediction modes may be used for secondary transforms to select a transform set and / or transposition flag.

[0206] In some embodiments, when the current block uses a special intra-mode codec, the representative prediction mode is stored in a buffer for intra-prediction modes. Any subsequent codec can access this buffer to learn the representative prediction mode.

[0207] In some embodiments, the representative prediction mode can vary implicitly with block width, block height, block area, or according to explicit rules specified in syntax elements such as block, tile, slice, picture, SPS, or PPS. In some embodiments, any proposed method or any combination of proposed methods can be applied to any intra-frame mode, such as WAIP, intra-frame angle mode, intra sub-partition (ISP), MIP, or any intra-frame mode specified in VVC or HEVC. When the current block uses ISP, it means that the current block is divided into sub-partitions vertically or horizontally according to the block size. For each sub-partition, reconstructed samples are obtained by adding the residual signal to the prediction signal. Here, the residual signal is generated by processes such as entropy decoding, inverse quantization, and inverse transformation. Therefore, the reconstructed sample values ​​of each sub-partition can be used to generate the prediction of the next sub-partition, and each sub-partition is processed repeatedly. In addition, the first sub-partition to be processed contains the samples in the upper left corner of the CU, and then continues downward (horizontal partitioning) or to the right (vertical partitioning). Therefore, the reference samples used to generate the sub-partition prediction signal are only located to the left and above the row. All sub-partitions share the same intra-frame mode.

[0208] In some embodiments, to reduce latency when performing a main transform, a video encoder may determine and use an exact representative prediction mode for the current block only for vertical transforms, not for horizontal transforms; and / or a corresponding video decoder may determine and use an exact representative prediction mode for the current block only for vertical inverse transforms, not for horizontal inverse transforms. In some embodiments, for a main transform, a video encoder may determine and use an exact representative prediction mode for the current block only for horizontal transforms, not for vertical transforms; and / or a corresponding video decoder may determine and use an exact representative prediction mode for the current block only for horizontal inverse transforms, not for vertical inverse transforms. Exact representative prediction modes refer to DIMD-derived modes and / or TIMD-derived modes that use derivation / analysis of (prediction) samples associated with the current block to determine the representative prediction mode for the current block, rather than directly specifying a default intra-frame prediction mode or a stored intra-frame prediction mode (associated with a reference region) as the (simple) representative prediction mode for the current block. If the current block does not use an exact representative prediction mode, a simple representative prediction mode is used.

[0209] In some embodiments, to reduce latency, the corresponding encoder and / or decoder may determine and use the exact representative prediction mode of the current block to determine the prediction mode for the separable transform codec (e.g., primary transform), but not for determining the prediction mode for the inseparable transform (e.g., secondary transform). In some embodiments, the video codec does not use the exact representative prediction mode of the current block to determine the prediction mode for the secondary transform and / or inseparable transform. Exact representative prediction mode refers to DIMD-derived mode and / or TIMD-derived mode, which use the derivation / analysis of (prediction) samples associated with the current block to determine the representative prediction mode for the current block, rather than directly specifying the default intra-frame prediction mode or the stored intra-frame prediction mode (associated with the reference region) as the (simple) representative prediction mode for the current block. If the current block does not use the exact representative prediction mode, the simple representative prediction mode is used instead.

[0210] In some embodiments, a video codec determines and uses the exact representative prediction mode for the current block only when certain enabling conditions are met. In some embodiments, the enabling conditions include a mode setting, causing the video codec to check the mode setting before applying the representative prediction mode for the current block. For example, when the mode of the current block is complex, a simple representative prediction mode is used for the current block instead of the exact representative prediction mode. Complex mode can refer to a hybrid mode where the final predictor is formed from multiple prediction hypotheses. Hybrid mode can be CIIP / GPM and / or any GPM extension and / or any GPM variant, such as SGPM, multi-hypothesis prediction (MHP), TIMD, and / or DIMD codec modes. As another example, complex mode refers to a refined mode where the motion / predictor is refined through multiple passes or decoder-side derivation. Refined mode can be DMVR and / or intra TMP codec mode. When GPM is used for the current block, the current block is split into two geometric partitions by a geometrically positioned line (expressed as distance and angle). Each geometric partition in the current block uses its own motion for inter prediction. An extension of GPM is that one geometric partition performs intra prediction. When the current block uses SGPM, both geometric partitions perform intra prediction. When the current block uses MHP, in addition to the traditional bidirectional prediction signal, one or more additional motion compensated prediction signals are added to form the final overall prediction signal. Therefore, for the hybrid mode, the final prediction is formed by combining multiple prediction hypotheses and is relatively complex. Exact representative prediction mode refers to DIMD derivation mode and / or TIMD derivation mode, which uses the derivation / analysis of (prediction) samples associated with the current block to determine the representative prediction mode of the current block, instead of directly assigning a default intra prediction mode or a stored intra prediction mode (associated with the reference area) as the (simple) representative prediction mode of the current block.

[0211] In some embodiments, the enabling condition includes a size setting related to block width, block height, block area, or block shape. The size setting may specify that the size of the current block is checked first, and if the size of the current block does not meet the size setting, the precise representative prediction mode cannot be applied. For example, when the width and / or height of the current block is greater than a predetermined threshold, the video codec does not determine or use the precise representative prediction mode for the current block. For another example, when the width and / or height of the current block is less than a predetermined threshold, the video codec does not determine or use the precise representative prediction mode for the current block. For another example, if the area of ​​the current block is greater than a predetermined threshold, the video codec does not determine or use the precise representative prediction mode for the current block. For another example, if the area of ​​the current block is less than a predetermined threshold, the video codec does not determine or use the precise representative prediction mode for the current block. For another example, if the long side of the current block is much larger than the short side of the current block, the video codec does not determine or use the precise representative prediction mode for the current block. The predetermined threshold value can be any integer, such as 2, 4, 8, 16, etc. The precise representative prediction mode refers to a DIMD-derived mode and / or a TIMD-derived mode that uses derivation / analysis of (prediction) samples associated with the current block to determine the representative prediction mode for the current block, rather than directly assigning a default intra-frame prediction mode or a stored intra-frame prediction mode (associated with a reference region) as the (simple) representative prediction mode for the current block. If the precise representative prediction mode is not used for the current block, the simple representative prediction mode is used.

[0212] In some embodiments, to reduce the delay caused by determining the representative prediction mode for the current block, some preprocessing operations are performed before determining the representative prediction mode. In some embodiments, the preprocessing operations include checking block settings. Block setting can refer to first subsampling (e.g., downsampling) the used prediction / reconstruction samples (within the current block and / or within the spatially adjacent region of the current block and / or within the reference region of the current block), and then determining the representative prediction mode for the current block based on the subsampled samples. For another example, block setting can refer to determining the representative prediction mode for the current block based only on a subset of the used prediction / reconstruction samples (within the current block and / or within the spatially adjacent region of the current block and / or within the reference region of the current block). The subset of used samples can be the first N rows or the first N columns of used samples, where N can be any predetermined integer, such as 4, 8, 16, etc. The subset can be the first M samples of the used samples, where M can be any predetermined integer, such as 4, 8, 16, etc.

[0213] In some embodiments, the pre-processing operation includes a partitioning setting. The partitioning setting means dividing the prediction / reconstruction samples used (in the current block and / or the spatially adjacent area of ​​the current block and / or the reference area of ​​the current block) into K sub-blocks, and determining and / or using a representative prediction mode for each sub-block separately. (K can be any predetermined integer, such as 4, 16, etc.)

[0214] The methods proposed in this invention can be enabled and / or disabled based on implicit rules (e.g., block width, height, or area) or explicit rules (e.g., syntax at the block, tile, slice, image, SPS, or PPS level). For example, when the block area is less than / greater than a threshold, the representative prediction mode of the current block is determined and used. The term "block" in this invention can refer to a TU / TB, CU / CB, PU / PB, a predetermined area, or a CTU / CTB. Any combination of the methods proposed in this invention can be applied.

[0215] Any of the aforementioned proposed methods may be implemented in an encoder and / or a decoder. For example, any of the proposed methods may be implemented in an inter-frame / intra-frame / IBC / prediction / transform module of an encoder and / or an inter-frame / intra-frame / IBC / prediction / transform module of a decoder. Alternatively, any of the proposed methods may be implemented as circuitry coupled to an inter-frame / intra-frame / IBC / prediction / transform module of an encoder and / or an inter-frame / intra-frame / IBC / prediction / transform module of a decoder to provide information required by the inter-frame / intra-frame / IBC / prediction / transform module.

[0216] In some of the above embodiments, an IBC-coded block is used as an example for the current block. However, the proposed method for determining and using a representative prediction mode for the current block is not limited to IBC blocks, and can also be applied to a current block coded in any other mode (e.g., intra-frame TMP or inter-frame block). When the current block is coded using intra-frame TMP, a reference block can be found through template matching.

[0217] In some embodiments, when the current block is encoded and decoded using inter-frame prediction mode, the reference block may be located in a reference picture (a previously encoded and decoded picture different from the current picture), which is indicated by the motion information of the current block. For example, the motion information of the current block may be bidirectionally predicted, and the reference picture indicated by the reference index of list 0 and / or the reference picture indicated by the reference index of list 1 may be used. For another example, the motion information of the current block may be unidirectionally predicted, and the reference picture indicated by the reference index may be from list 0 or list 1. If multiple reference pictures are used, an order may be used to define which reference picture is used first. One possible order is to use the reference picture closer to the current picture (with a smaller POC) first. Another possible order is to use the reference picture from a predetermined list (list 0 or list 1) first.

[0218] XI. Sample Video Encoder

[0219] Figure 13 An example video encoder 1300 is shown that can implement representative prediction modes for a pixel block being encoded. As shown, the video encoder 1300 receives an input video signal from a video source 1305 and encodes the signal into a bitstream 1395. The video encoder 1300 has several components or modules for encoding the signal from the video source 1305, including at least some components selected from the following: a transform module 1310, a quantization module 1311, an inverse quantization module 1314, an inverse transform module 1315, an intra estimation module 1320, an intra prediction module 1325, a motion compensation module 1330, a motion estimation module 1335, a loop filter 1345, a reconstructed picture buffer 1350, an MV buffer 1365, an MV prediction module 1375, and an entropy encoder 1390. The motion compensation module 1330 and the motion estimation module 1335 are part of the non-intra prediction module 1340.

[0220] In some embodiments, modules 1310-1390 are software instruction modules executed by one or more processing units (e.g., processors) of a computing device or electronic device. In some embodiments, modules 1310-1390 are hardware circuit modules implemented by one or more integrated circuits (ICs) of an electronic device. Although modules 1310-1390 are shown as separate modules, some modules may be combined into a single module.

[0221] A video source 1305 provides a raw video signal representing the pixel data for each video frame without compression. A subtractor 1308 calculates the difference between the raw video pixel data from the video source 1305 and the predicted pixel data 1313 from the motion compensation module 1330 or the intra-frame prediction module 1325. A transform module 1310 converts the difference (or residual pixel data or residual signal) into transform coefficients (e.g., by performing a discrete cosine transform or DCT). A quantization module 1311 quantizes the transform coefficients into quantized data (or quantized coefficients) 1312, which are encoded into a bitstream 1395 by an entropy encoder 1390.

[0222] Inverse quantization module 1314 dequantizes quantized data (or quantized coefficients) 1312 to obtain transform coefficients, and inverse transform module 1315 performs an inverse transform on the transform coefficients to generate a reconstructed residual 1319. Reconstructed residual 1319 is added to predicted pixel data 1313 to generate reconstructed pixel data 1317. In some embodiments, reconstructed pixel data 1317 is temporarily stored in a line buffer (not shown) for intra prediction and spatial MV prediction. The reconstructed pixels are filtered by loop filter 1345 and stored in a reconstructed picture buffer 1350. In some embodiments, reconstructed picture buffer 1350 is a memory external to video encoder 1300. In some embodiments, reconstructed picture buffer 1350 is a memory internal to video encoder 1300.

[0223] The intra estimation module 1320 performs intra prediction based on the reconstructed pixel data 1317 to generate intra prediction data. The intra prediction data is provided to the entropy encoder 1390 to be encoded into a bitstream 1395. The intra prediction data is also used by the intra prediction module 1325 to generate predicted pixel data 1313.

[0224] The motion estimation module 1335 performs inter-frame prediction by generating MVs to refer to pixel data of a previously decoded frame stored in the reconstructed picture buffer 1350. These MVs are provided to the motion compensation module 1330 to generate predicted pixel data.

[0225] Instead of encoding the complete actual MV in the bitstream, the video encoder 1300 uses MV prediction to generate a predicted MV, and the difference between the MV used for motion compensation and the predicted MV is encoded as residual motion data and stored in the bitstream 1395.

[0226] The MV prediction module 1375 generates a predicted MV based on a reference MV generated for encoding a previous video frame, i.e., a motion compensated MV used to perform motion compensation. The MV prediction module 1375 retrieves the reference MV from the previous video frame from the MV buffer 1365. The video encoder 1300 stores the MV generated for the current video frame in the MV buffer 1365 as a reference MV for generating the predicted MV.

[0227] The MV prediction module 1375 uses the reference MV to create a predicted MV. The predicted MV can be calculated by spatial MV prediction or temporal MV prediction. The difference (residual motion data) between the predicted MV and the motion compensated MV (MC MV) of the current frame is encoded into the bitstream 1395 by the entropy encoder 1390.

[0228] The entropy encoder 1390 encodes various parameters and data into a bitstream 1395 using an entropy coding technique such as context-adaptive binary arithmetic coding (CABAC) or Huffman coding. The entropy encoder 1390 encodes various header elements and flags along with quantized transform coefficients 1312 and residual motion data as syntax elements into the bitstream 1395. The bitstream 1395 is then stored in a storage device or transmitted to a decoder via a communication medium such as a network.

[0229] The loop filter 1345 performs filtering or smoothing operations on the reconstructed pixel data 1317 to reduce coding artifacts, particularly at pixel block boundaries. In some embodiments, the filtering or smoothing operations performed by the loop filter 1345 include a deblocking filter (DBF), a sample adaptive offset (SAO), and / or an adaptive loop filter (ALF).

[0230] Figure 14 13. The portion of the video encoder 1300 that implements representative prediction modes is shown. As shown, a non-intra prediction module (e.g., an inter prediction module) 1340, an intra prediction module 1325, and a special intra prediction module 1425 obtain pixel samples from a reconstructed image buffer 1350 to generate a predictor for a current block in the predicted pixel data 1313. In some embodiments, the intra prediction module 1325 performs prediction in a conventional intra mode, where a DIMD or TIMD operation may be performed to select one or more intra prediction directions.

[0231] The non-intra prediction module 1340 performs motion estimation and compensation for inter prediction modes (e.g., merge mode). In some embodiments, the non-intra prediction module 1340 also performs prediction based on samples in the current picture as a reference, such as in IBC mode. More generally, the non-intra prediction module 1340 performs prediction in non-intra modes.

[0232] The special intra prediction module 1425 performs prediction on special intra modes (such as MIP, CCLM, hybrid mode, TMP mode, or other modes that use current image samples as a reference) to generate a predictor for the current block without using traditional 67 or 131 directional intra prediction modes (whether for the same component or different components).

[0233] The video encoder 1300 also includes a representative mode module 1440. The representative mode module 1440 generates or determines a representative intra prediction mode for the current block by selecting from a plurality of intra prediction modes based on cost / derivation / analysis, or by searching within or adjacent to a plurality of predetermined locations in a reference area identified by a block vector or motion vector (provided by the non-intra prediction module 1340).

[0234] The representative intra prediction mode is provided to the transform module 1310 and the inverse transform module 1315 to select a transform mode. In some embodiments, the representative intra prediction mode is used to select the primary transform (e.g., MTS) according to Tables 4-6. In some embodiments, the representative prediction mode is used to select the primary transform (e.g., MTS) according to Tables 4-6. Figure 9 The secondary transform (e.g., LFNST) is selected. The representative intra prediction mode is also stored in the storage buffer 1445 so that the representative mode may or may not be used to encode subsequent blocks as most probable mode (MPM) and / or prediction generation and / or transform selection, or to encode co-located chroma component blocks in chroma DM mode.

[0235] Figure 15 The process 1500 for generating and using a representative intra prediction for a current block is conceptually illustrated. In some embodiments, one or more processing units (e.g., processors) of a computing device implementing the encoder 1300 perform the process 1500 by executing instructions stored in a computer-readable medium. In some embodiments, the electronic device implementing the encoder 1300 performs the process 1500.

[0236] The encoder receives (at block 1510) data to be encoded as a current block of pixels in a current picture.

[0237] The encoder generates (at block 1520) a predictor for the current block using a first prediction mode. In some embodiments, the first prediction mode is not a directional intra prediction mode. The first prediction mode can be a regular intra mode, a special intra mode, and / or a non-intra mode. The encoder determines (at block 1530) a second prediction mode as a representative prediction mode for the current block.

[0238] In some embodiments, the predictor is generated in MIP mode, i.e., by matrix multiplication of a predetermined or derived matrix with a set of input samples derived (e.g., downsampled) from neighboring samples of the current block. In some embodiments, the current block is a chroma component block, and the predictor for the current block is generated by applying a cross-component model to a co-located luma component block. In some embodiments, the predictor for the current block is generated by intra TMP mode, which is based on a reference block determined by matching a first template region adjacent to the current block with a second template region in the current picture or a reference picture. In some embodiments, the predictor is generated in hybrid mode (e.g., CIIP) by combining predictions from multiple prediction hypotheses.

[0239] In some embodiments, the predictor is generated by determining a reference region in the current image or the reference image using a block vector or a motion vector, and the representative prediction mode is an intra-frame prediction mode for encoding pixel samples within or adjacent to the reference region. In some embodiments, the encoder determines the representative prediction mode by searching a plurality of predetermined locations within or adjacent to the reference region in a predetermined order to determine the representative prediction mode.

[0240] In some embodiments, a representative prediction mode is selected from a plurality of intra prediction modes based on cost / derivation / analysis, wherein the cost of the intra prediction mode is a difference between a reconstructed sample and a predicted sample of a neighboring region of the current block, and the predicted sample of the neighboring region of the current block is generated based on the intra prediction mode. In some embodiments, the representative prediction mode is determined by deriving a plurality of histograms of gradients (HoGs) of the plurality of intra prediction modes, and the HoGs of the intra prediction modes are derived based on a set of predetermined predictors of the current block.

[0241] The encoder encodes the current block (at block 1540) by generating a prediction residual using the predictor generated for the current block and selecting a transform for the residual using the representative prediction mode. In some embodiments, the representative intra prediction mode is used to select a primary transform (e.g., MTS) according to Tables 4-6. In some embodiments, the representative intra prediction mode is used to select a primary transform (e.g., MTS) according to Tables 4-6. Figure 9 Selecting a secondary transform (eg, LFNST) In some embodiments, the representative intra prediction mode is used to select a transform set and / or a transpose flag for the secondary transform.

[0242] The encoder stores a representative prediction mode for the current block and / or provides (at block 1550) a representative prediction mode for encoding a subsequent block. For example, in some embodiments, the encoder provides the representative prediction mode for use as a most probable mode (MPM), prediction generation, and / or transform selection for encoding a subsequent block. In some embodiments, the current block is a luma component block, and the representative prediction mode is used to encode a co-located chroma component block in a chroma DM mode, for example, by generating an intra prediction predictor for the chroma block.

[0243] XII. Example Video Decoder

[0244] In some embodiments, the encoder may send (or generate) one or more syntax elements in a bitstream so that the decoder may parse the one or more syntax elements from the bitstream.

[0245] Figure 16 An example video decoder 1600 is shown showing representative prediction modes for a pixel block being decoded. As shown, video decoder 1600 is an image decoding or video decoding circuit that receives a bitstream 1695 and decodes the contents of the bitstream into pixel data for a video frame for display. Video decoder 1600 has several components or modules for decoding bitstream 1695, including some selected from the following: an inverse quantization module 1611, an inverse transform module 1610, an intra prediction module 1625, a motion compensation module 1630, a loop filter 1645, a decoded picture buffer 1650, an MV buffer 1665, an MV prediction module 1675, and a parser 1690. Motion compensation module 1630 is part of non-intra prediction module 1640.

[0246] In some embodiments, modules 1610-1690 are software instruction modules executed by one or more processing units (e.g., processors) of a computing device. In some embodiments, modules 1610-1690 are hardware circuit modules implemented by one or more ICs of an electronic device. Although modules 1610-1690 are shown as separate modules, some modules may be combined into a single module.

[0247] The parser 1690 (or entropy decoder) receives the bitstream 1695 and performs initial parsing according to the syntax defined by the video codec or image codec standard. The parsed syntax elements include various header elements, flags, and quantization data (or quantization coefficients) 1612. The parser 1690 parses the various syntax elements using entropy coding techniques such as context-adaptive binary arithmetic coding (ABAC) or Huffman encoding.

[0248] The inverse quantization module 1611 dequantizes the quantized data (or quantized coefficients) 1612 to obtain transform coefficients, and the inverse transform module 1610 inversely transforms the transform coefficients 1616 to generate a reconstructed residual signal 1619. The reconstructed residual signal 1619 is added to the predicted pixel data 1613 from the intra prediction module 1625 or the motion compensation module 1630 to generate decoded pixel data 1617. The decoded pixel data is filtered by the loop filter 1645 and stored in the decoded picture buffer 1650. In some embodiments, the decoded picture buffer 1650 is a memory external to the video decoder 1600. In some embodiments, the decoded picture buffer 1650 is a memory internal to the video decoder 1600.

[0249] The intra prediction module 1625 receives intra prediction data from the bitstream 1695 and, based on this, generates predicted pixel data 1613 from the decoded pixel data 1617 stored in the decoded picture buffer 1650. In some embodiments, the decoded pixel data 1617 is also stored in a line buffer (not shown) for intra prediction and spatial MV prediction.

[0250] In some embodiments, the contents of decoded picture buffer 1650 are used for display. Display device 1655 either retrieves the contents of decoded picture buffer 1650 for direct display or retrieves the contents of the decoded picture buffer into a display buffer. In some embodiments, the display device receives pixel values ​​from decoded picture buffer 1650 via pixel transfer.

[0251] The motion compensation module 1630 generates predicted pixel data 1613 according to motion compensated MV (MC MV) from the decoded pixel data 1617 stored in the decoded picture buffer 1650. These motion compensated MVs are decoded by adding the residual motion data received from the bitstream 1695 to the predicted MV received from the MV prediction module 1675.

[0252] The MV prediction module 1675 generates a predicted MV based on a reference MV generated for decoding a previous video frame (e.g., a motion compensated MV used to perform motion compensation). The MV prediction module 1675 obtains the reference MV of the previous video frame from the MV buffer 1665. The video decoder 1600 stores the motion compensated MV generated for decoding the current video frame in the MV buffer 1665 as a reference MV for generating the predicted MV.

[0253] Loop filter 1645 performs filtering or smoothing operations on decoded pixel data 1617 to reduce coding artifacts, particularly at pixel block boundaries. In some embodiments, the filtering or smoothing operations performed by loop filter 1645 include a deblocking filter (DBF), sample adaptive offset (SAO), and / or an adaptive loop filter (ALF).

[0254] Figure 17 16. The portion of the video decoder 1600 that implements representative prediction modes is shown. As shown, a non-intra prediction module (e.g., an inter prediction module) 1640, an intra prediction module 1625, and / or a special intra prediction module 1725 retrieve pixel samples from a decoded picture buffer 1650 to generate a predictor for a current block in the predicted pixel data 1613. In some embodiments, the intra prediction module 1625 performs prediction in a conventional intra mode, where conventional intra prediction, DIMD, or TIMD operations may be performed to select one or more of DC, planar, and / or intra prediction directions.

[0255] The non-intra prediction module 1640 performs motion compensation for inter prediction modes (e.g., merge mode). In some embodiments, the non-intra prediction module 1640 also performs prediction based on samples in the current picture as a reference, such as in IBC mode. More generally, the non-intra prediction module 1640 performs prediction for non-intra modes.

[0256] The special intra prediction module 1725 performs prediction on special intra modes (such as MIP, CCLM, hybrid mode, TMP mode, or other modes that use current picture samples as reference) to generate a predictor for the current block without using traditional 67 or 131 directional intra prediction modes, whether for the same color component or different color components.

[0257] The video decoder 1600 also includes a representative mode module 1740. The representative mode module 1740 generates or determines a representative prediction mode for the current block by selecting from a plurality of intra prediction modes based on cost / derivation / analysis, or by searching a plurality of predetermined locations within or near a reference area identified by a block vector or motion vector (provided by the non-intra prediction module 1640).

[0258] The representative intra prediction mode is provided to the inverse transform module 1615 to select a transform mode. In some embodiments, the representative prediction mode is used to select the primary transform (e.g., MTS) according to Tables 4-6. In some embodiments, the representative intra prediction mode is used to select the primary transform (e.g., MTS) according to Tables 4-6. Figure 9The secondary transform (e.g., LFNST) is selected. The representative intra prediction mode is also stored in the storage buffer 1745 so that the representative mode may or may not be used to encode subsequent blocks into the most probable mode (MPM) and / or prediction generation and / or transform selection, or to encode and decode the co-located chroma component block in the chroma DM mode.

[0259] Figure 18 The process 1800 for generating and using a representative intra prediction for a current block is conceptually illustrated. In some embodiments, one or more processing units (e.g., processors) of a computing device implementing the decoder 1600 perform the process 1800 by executing instructions stored in a computer-readable medium. In some embodiments, the electronic device implementing the decoder 1600 performs the process 1800.

[0260] The decoder receives (at block 1810) data to be decoded as a current block of pixels in a current picture.

[0261] The decoder generates (at block 1820) a predictor for the current block using the first prediction mode. In some embodiments, the first prediction mode is not a directional intra prediction mode. The first prediction mode can be a regular intra mode, a special intra mode, and / or a non-intra mode. The decoder determines (at block 1830) a second prediction mode as a representative prediction mode for the current block.

[0262] In some embodiments, the predictor is generated in MIP mode, i.e., by performing a matrix multiplication operation on a predetermined or derived matrix with a set of input samples derived (e.g., downsampled) from neighboring samples of the current block. In some embodiments, the current block is a chroma component block, and the predictor for the current block is generated by applying a cross-component model to a co-located luma component block. In some embodiments, the predictor for the current block is generated in intraTMP mode based on a reference block determined by matching a first template region adjacent to the current block with a second template region in the current picture or a reference picture. In some embodiments, the predictor is generated in hybrid mode (e.g., CIIP) by combining predictions from multiple prediction hypotheses.

[0263] In some embodiments, the predictor is generated by determining a reference region in a current picture or a reference picture using a block vector or a motion vector, and the representative prediction mode is an intra-frame prediction mode for encoding or decoding pixel samples within or adjacent to the reference region. In some embodiments, the decoder determines the representative prediction mode by searching a plurality of predetermined locations within or adjacent to the reference region in a predetermined order to determine the representative prediction mode.

[0264] In some embodiments, a representative prediction mode is selected from a plurality of intra prediction modes based on cost / derivation / analysis, wherein the cost of the intra prediction mode is a difference between a reconstructed sample of a current block and a predicted sample of a neighboring region, and the predicted sample of the neighboring region of the current block is generated based on the intra prediction mode. In some embodiments, the representative prediction mode is determined by deriving a plurality of histograms of gradients (HoGs) of the plurality of intra prediction modes, and the HoGs of the intra prediction modes are derived based on a set of predetermined predictors of the current block.

[0265] The decoder reconstructs (at block 1840) the current block using the generated current block predictor and selects an inverse transform of the prediction residual using the representative prediction mode. In some embodiments, the representative intra prediction mode is used to select a primary transform (e.g., MTS) according to Tables 4-6. In some embodiments, the representative intra prediction mode is used to select a primary transform (e.g., MTS) according to Tables 4-6. Figure 9 A secondary transform (eg, LFNST) is selected. The decoder may then display the reconstructed current block as part of the reconstructed current image. In some embodiments, a representative intra prediction mode is used to select a transform set and / or a transposition flag for the secondary transform.

[0266] The decoder stores the representative prediction mode for the current block and / or provides (at block 1850) the representative prediction mode for use in decoding subsequent blocks. For example, in some embodiments, the decoder provides the representative prediction mode for use as a most probable mode (MPM), prediction generation, and / or transform selection for encoding subsequent blocks. In some embodiments, the current block is a luma component block, and the representative prediction mode is used to decode the co-located chroma component block in chroma DM mode, for example, by generating intra-prediction predictors for the chroma block.

[0267] XIII. Example Electronic System

[0268] Many of the above features and applications are implemented as software processes, which are specified as a set of instructions recorded on a computer-readable storage medium (also referred to as a computer-readable medium). When these instructions are executed by one or more computing or processing units (e.g., one or more processors, processor cores, or other processing units), they cause the processing units to perform the actions indicated in the instructions. Examples of computer-readable media include, but are not limited to, compact disc read-only memory (CD-ROM), flash drives, random access memory (RAM) chips, hard drives, erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), etc. Computer-readable media do not include carrier waves and electronic signals transmitted via wireless or wired connections.

[0269] In this specification, the term "software" is intended to include firmware residing in read-only memory or application programs stored in magnetic memory, which can be read into memory for processing by a processor. Furthermore, in some embodiments, multiple software inventions can be implemented as sub-parts of a larger program while retaining different software inventions. In some embodiments, multiple software inventions can also be implemented as separate programs. Finally, any combination of separate programs that together implement the software inventions described herein is within the scope of this disclosure. In some embodiments, the software program, when installed to run on one or more electronic systems, defines one or more specific machine implementations that process and execute the operations of the software program.

[0270] Figure 19 The electronic system 1900 for implementing some embodiments of the present disclosure is conceptually illustrated. The electronic system 1900 can be a computer (e.g., a desktop computer, a personal computer, a tablet computer, etc.), a phone, a PDA, or any other type of electronic device. Such an electronic system includes various types of computer-readable media and interfaces for various other types of computer-readable media. The electronic system 1900 includes a bus 1905, a processing unit 1910, a graphics processing unit (GPU) 1915, system memory 1920, a network 1925, read-only memory 1930, a permanent storage device 1935, an input device 1940, and an output device 1945.

[0271] Bus 1905 collectively represents all system, peripheral, and chipset buses that communicatively connect the numerous internal devices of electronic system 1900. For example, bus 1905 communicatively connects processing unit 1910 with GPU 1915, read-only memory 1930, system memory 1920, and permanent storage 1935.

[0272] Processing unit 1910 retrieves instructions to be executed and data to be processed from these various memory units in order to perform the processing of the present disclosure. In different embodiments, the processing unit can be a single processor or a multi-core processor. Some instructions are passed to and executed by GPU 1915. GPU 1915 can offload various calculations or supplement the image processing provided by processing unit 1910.

[0273] Read-only memory (ROM) 1930 stores static data and instructions used by processing unit 1910 and other modules of the electronic system. Persistent storage device 1935, on the other hand, is a read-write storage device. This device is a non-volatile storage unit that stores instructions and data even when electronic system 1900 is turned off. Some embodiments of the present disclosure use a mass storage device (e.g., a magnetic disk or optical disk and its corresponding disk drive) as persistent storage device 1935.

[0274] Other embodiments use removable storage devices (such as floppy disks, flash memory devices, etc., and their corresponding disk drives) as permanent storage devices. Like the permanent storage device 1935, the system memory 1920 is a read-write memory device. However, unlike the permanent storage device 1935, the system memory 1920 is a volatile read-write memory, such as a random access memory. The system memory 1920 stores some instructions and data used by the processor at runtime. In some embodiments, the processing according to the present disclosure is stored in the system memory 1920, the permanent storage device 1935 and / or the read-only memory 1930. For example, according to some embodiments of the present disclosure, various memory units include instructions for processing multimedia clips. From these various memory units, the processing unit 1910 obtains instructions to be executed and data to be processed in order to perform the processing of some embodiments.

[0275] The bus 1905 is also connected to input devices 1940 and output devices 1945. The input devices 1940 enable the user to convey information and select commands to the electronic system. The input devices 1940 include an alphanumeric keyboard and a pointing device (also known as a "cursor control device"), a camera (e.g., a webcam), a microphone, or a similar device for receiving voice commands. The output devices 1945 display images generated by the electronic system or output data. The output devices 1945 include printers and display devices, such as cathode ray tubes (CRTs) or liquid crystal displays (LCDs), as well as speakers or similar audio output devices. Some embodiments include devices used as input and output devices, such as a touch screen.

[0276] Finally, if Figure 19 As shown, bus 1905 also couples electronic system 1900 to a network 1925 via a network adapter (not shown). In this manner, the computer can be part of a computer network, such as a local area network ("LAN"), a wide area network ("WAN"), or an intranet, or a network of multiple networks, such as the Internet. Any or all components of electronic system 1900 may be used in conjunction with the present disclosure.

[0277] Some embodiments include electronic components, such as a microprocessor, storage devices, and memory, which store computer program instructions in a machine-readable or computer-readable medium (or alternatively referred to as a computer-readable storage medium, machine-readable medium, or machine-readable storage medium). Some examples of such computer-readable media include RAM, ROM, read-only compact discs (CD-ROMs), recordable compact discs (CD-Rs), rewritable compact discs (CD-RWs), read-only digital versatile discs (e.g., DVD-ROMs, dual-layer DVD-ROMs), various recordable / rewritable DVDs (e.g., DVD-RAMs, DVD-RWs, DVD+RWs, etc.), flash memory (e.g., SD cards, mini SD cards, micro SD cards, etc.), magnetic and / or solid-state hard drives, read-only and recordable A computer program may be a computer program that is executable by at least one processing unit and includes a set of instructions for performing various operations. Examples of computer programs or computer code include machine code such as produced by a compiler and files including high-level code that is executed by a computer, electronic component, or microprocessor using an interpreter.

[0278] While the above discussion primarily involves microprocessors or multi-core processors executing software, many of the aforementioned features and applications are implemented by one or more integrated circuits, such as application-specific integrated circuits (ASICs) or field-programmable gate arrays (FPGAs). In some embodiments, such integrated circuits execute instructions stored on the circuits themselves. Additionally, some embodiments execute software stored in programmable logic devices (PLDs), ROM, or RAM devices.

[0279] As used in this specification and in any claims of this application, the terms "computer," "server," "processor," and "memory" refer to electronic or other technological devices. These terms do not include people or groups of people. For the purposes of this specification, the terms display or showing refer to displaying on an electronic device. As used in this specification and in any claims of this application, the terms "computer-readable medium," "computer-readable medium," and "machine-readable medium" are entirely limited to tangible, physical objects that store information in a computer-readable form. These terms do not include any wireless signals, wired download signals, or any other transient signals.

[0280] Although the present disclosure has been described with reference to numerous specific details, those skilled in the art will recognize that the present disclosure may be embodied in other specific forms without departing from the spirit of the present disclosure. Figure 14 and Figure 17 ) conceptually illustrates a process. The specific operations of these processes may not be performed in the exact order shown and described. The specific operations may not be performed in a continuous series of operations, and different specific operations may be performed in different embodiments. In addition, the process may be implemented using several sub-processes or as part of a larger macro-process. Therefore, it will be understood by those skilled in the art that the present disclosure is not limited by the foregoing illustrative details, but is defined by the appended claims.

[0281] Supplementary Notes

[0282] The subject matter described herein sometimes represents different components that are contained in or connected to other different components. It will be understood that the described structures are merely examples and that many other structures can actually be implemented to achieve the same function. Conceptually, any arrangement of components that achieve the same function is actually "associated" to achieve the desired function. Therefore, regardless of the structure or intermediate components, any two components combined to achieve a specific function are considered to be "interrelated" to achieve the desired function. Similarly, any two associated components are considered to be "operably connected" or "operably coupled" to each other to achieve the specific function. Any two components that can be associated with each other are also considered to be "operably coupled" to each other to achieve the specific function. Any two components that can be associated with each other are also considered to be "operably coupled" to each other to achieve the specific function. Specific examples of operable connections include, but are not limited to, physically pairable and / or physically interacting components, and / or wirelessly interactable and / or wirelessly interacting components, and / or logically interacting and / or logically interacting components.

[0283] Furthermore, with respect to the use of substantially any plural and / or singular terms, one skilled in the art can translate from the plural to the singular and / or from the singular to the plural as appropriate to the context and / or application. For sake of clarity, the present invention expressly sets forth various singular / plural permutations.

[0284] Furthermore, those skilled in the art will appreciate that, generally, terminology used herein, particularly in the claims, and in the subject matter of the claims, is often used as "open" terms. For example, "including" should be interpreted as "including but not limited to," "having" should be interpreted as "at least having," and "comprising" should be interpreted as "including but not limited to," etc. Those skilled in the art will further appreciate that if a specific number of claim elements is intended to be introduced, it will be explicitly indicated within the claim, and in the absence of such information, it will not be displayed. For example, to aid understanding, the following claims may include the phrases "at least one" and "one or more" to introduce claim elements. However, the use of these phrases should not be construed to imply that the use of the indefinite article "a" or "an" to introduce claim elements is restrictive of any particular claim. Even when the same claim includes the introductory phrases "a or more" or "at least one," the indefinite article, such as "a" or "an," should be interpreted to mean at least one or more, and the same applies to the use of the explicit recitation to introduce the claim. Furthermore, even when introductory text explicitly refers to a specific number, those skilled in the art will recognize that such reference should be interpreted as referring to the number being referenced. For example, "two references" without further modification means at least two references, or two or more references. Furthermore, when expressions such as "at least one of A, B, and C" are used, they are generally so stated so that those skilled in the art will understand the expression. For example, "a system includes at least one of A, B, and C" will include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B, and C, etc. Those skilled in the art will further understand that any separated words and / or phrases, whether in the specification, claims, or drawings, that are presented as two or more alternative terms should be understood to include the possibility of one, one, or both of these terms. For example, "A or B" should be understood to include the possibility of "A," "B," or "A and B."

[0285] As can be seen from the foregoing, various embodiments of the present invention have been described for illustrative purposes, and various modifications may be made without departing from the scope and spirit of the present invention. Therefore, the various embodiments disclosed herein are not intended to be limiting, and the true scope and application are indicated by the claims.

Claims

1. A video encoding and decoding method, comprising: receiving data of a pixel block to be encoded or decoded as a current block of a current picture of a video; generating a prediction sub-block for the current block using a first prediction mode; Determining the second prediction mode as the representative prediction mode of the current block; as well as The current block is encoded or decoded using the generated predictor and the representative prediction mode of the current block.

2. The video encoding and decoding method according to claim 1, wherein: The first prediction mode is not a directional intra prediction mode.

3. The video encoding and decoding method according to claim 1, wherein: The representative prediction mode is selected from a plurality of intra-frame prediction modes based on a plurality of costs, wherein the cost of the intra-frame prediction mode is a difference between a plurality of reconstructed samples and a plurality of prediction samples of a neighboring region of the current block, and the plurality of prediction samples of the neighboring region of the current block are generated based on the intra-frame prediction mode.

4. The video encoding and decoding method according to claim 1, wherein: The predictor of the current block is generated by determining a reference region in the current picture or the reference picture using a block vector or a motion vector, and the representative prediction mode is an intra prediction mode for encoding or decoding a plurality of pixel samples within or adjacent to the reference region.

5. The video encoding and decoding method according to claim 4, wherein: Determining the representative prediction mode includes searching a plurality of predetermined locations in or near the reference area in a predetermined order to determine the representative prediction mode.

6. The video encoding and decoding method according to claim 1, wherein: Determining the representative prediction mode includes deriving a plurality of Histograms of Gradients (HoGs) for a plurality of intra prediction modes, wherein the HoGs of the intra prediction modes are derived based on a set of predetermined predictors for the current block.

7. The video encoding and decoding method according to claim 1, wherein: Encoding or decoding the current block includes transforming or inverse-transforming a plurality of residuals of a predictor of the current block or a plurality of transform coefficients of the plurality of residuals of the current block using a transform mode selected based on the representative prediction mode.

8. The video encoding and decoding method according to claim 7, wherein: The representative prediction mode is used to select a transform set, a transpose flag, or both of the non-separable transform.

9. The video encoding and decoding method according to claim 1, wherein: Also included is providing the representative prediction mode for use as a most probable mode (MPM) for encoding or decoding a subsequent block.

10. The video encoding and decoding method according to claim 1, wherein: The current block is a luma component block, and the representative prediction mode is used to encode or decode a co-located chroma component block.

11. The video encoding and decoding method according to claim 1, wherein: The predictor of the current block is generated by matrix multiplication of a predetermined or derived matrix and a set of input samples derived from a plurality of neighboring samples of the current block.

12. The video encoding and decoding method according to claim 1, wherein: The current block is a chroma component block, and the predictor of the current block is generated by applying a cross-component model to a co-located luma component block.

13. The video encoding and decoding method according to claim 1, wherein: The predictor of the current block is generated based on a reference block, where the reference block is determined by matching a first template region adjacent to the current block with a second template region in the current picture or the reference picture.

14. The video encoding and decoding method according to claim 1, wherein: The predictor for the current block is generated by combining predictions from multiple prediction hypotheses.

15. A video codec circuit configured to perform the following operations: receiving data of a pixel block to be encoded or decoded as a current block of a current picture of a video; generating a prediction sub-block for the current block using a first prediction mode; Identifying the second prediction mode as a representative prediction mode for the current block; and The current block is encoded or decoded using the generated predictor and the representative prediction mode of the current block.

16. A video encoding method, comprising: receiving data of a pixel block to be encoded as a current block of a current picture of a video; generating a prediction sub-block for the current block using a first prediction mode; Identifying the second prediction mode as a representative prediction mode for the current block; as well as The current block is encoded using the generated predictor and the representative prediction mode of the current block.

17. A video decoding method, comprising: receiving data of a pixel block to be decoded as a current block of a current picture of a video; generating a predictor for the current block using a first prediction mode; determining a second prediction mode as a representative prediction mode for the current block; as well as The current block is reconstructed using the generated predictor and the representative prediction mode of the current block.

Images