Counter-based intra prediction mode

Abstract

A method of video processing includes performing a conversion between a video comprising video units and a coded representation of the video, wherein, when processing a video unit in the conversion, one or more frequency tables are selectively updated to include information about frequencies of one or more intra-prediction modes of the video unit used in the processing, wherein the frequencies indicate occurrences of the one or more intra-prediction modes used for the conversion, and wherein, when processing the video unit, one or more ordered intra-prediction mode (IPM) tables are selectively updated to indicate the one or more intra-prediction modes used in the processing.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to and the benefit of International Patent Application No. PCT / CN2019 / 101443, filed on August 19, 2019, International Patent Application No. PCT / CN2019 / 103075, filed on August 28, 2019, and International Patent Application No. PCT / CN2019 / 103425, filed on August 29, 2019, in accordance with the applicable provisions of the Patent Law and / or the Paris Convention. The entire disclosures of the above applications are incorporated herein by reference and made a part of the disclosure of this application for all legal purposes. Technical Field

[0003] This patent document relates to video encoding and decoding technology, equipment and systems. Background Art

[0004] Despite advances in video compression, digital video still represents the largest use of bandwidth on the Internet and other digital communications networks. As the number of connected user devices capable of receiving and displaying video increases, bandwidth demands for digital video usage are expected to continue to grow. Summary of the Invention

[0005] The present invention relates to digital video coding and decoding, and more particularly to devices, systems, and methods for counter-based intra-frame coding and decoding of videos and images. The described methods can be applied to existing video coding standards (e.g., High Efficiency Video Codec (HEVC)) and future video / image coding standards or video / image codecs.

[0006] In one example aspect, an example method of video processing includes performing a conversion between a video comprising a video unit and a codec representation of the video, wherein, while processing the video unit in the conversion, one or more frequency tables are selectively updated to include information about the frequency of one or more intra-frame prediction modes of the video unit used in the processing, wherein the frequency indicates an occurrence of the one or more intra-frame prediction modes used for the conversion, and wherein, while processing the video unit, one or more sorted intra-frame prediction mode (IPM) tables are selectively updated to indicate the one or more intra-frame prediction modes used in the processing.

[0007] In yet another example aspect, an example method of video processing includes performing a conversion between a video unit of a video and a coded representation of the video using one or more frequency tables or one or more ordered intra prediction mode (IPM) tables, wherein the one or more frequency tables include information about frequencies of one or more intra prediction modes used in the conversion of the video, wherein the frequency indicates occurrences of the one or more intra prediction modes used for the conversion, wherein the one or more ordered IPM tables indicate the one or more intra prediction modes in an ordered order, and wherein the one or more frequency tables or the one or more ordered IPM tables are used in intra mode coding in a process of constructing a most probable mode (MPM) list of size N, where N is an integer.

[0008] In yet another example aspect, an example method of video processing includes performing a conversion between a video block of a video and a coded representation of the video, wherein the coded representation includes a syntax element that indicates a selected intra prediction mode used for the conversion, the coded representation does not include one or more syntax elements that indicate a most probable mode (MPM), or an index of an MPM list, or remaining intra prediction modes other than the intra prediction modes included in the MPM list, and wherein the selected intra prediction mode is based on historical information that indicates frequencies of one or more intra prediction modes used in another conversion between one or more video blocks of the video and the video prior to the conversion of the video block.

[0009] In yet another example aspect, a method of video processing is disclosed. The method includes performing a conversion between a video unit of a video and a coded representation of the video unit using one or more frequency tables, wherein the one or more frequency tables include information about frequencies of intra prediction modes used in the conversion of the video, and selectively updating, due to the conversion, the one or more frequency tables based on a coding mode of the video unit.

[0010] In another example aspect, another method of video processing is disclosed. The method includes performing a conversion between a video unit of a video and a coded representation of the video unit using a frequency table, wherein the frequency table includes a plurality of entries, each entry representing a frequency of occurrences of a corresponding intra coding mode in the conversion, and selectively updating, with the conversion, the frequency table based on coding information of the video unit.

[0011] In another aspect, another video processing method is disclosed. The method includes performing a conversion between a current video unit and a next video unit of a video and a coded representation of the video using one or more frequency tables and / or using one or more intra prediction mode tables ordered according to frequencies indicated in the one or more frequency tables; wherein the one or more frequency tables comprise information about frequencies of intra prediction modes used in the conversion; and wherein the one or more frequency tables and / or the one or more intra prediction mode tables are reset or initialized between a use of the conversion of the current video unit and a use of the conversion of the next video unit.

[0012] In yet another example aspect, another video processing method is disclosed. The method includes performing a conversion between a video unit of a video and a coded representation of the video unit using one or more frequency tables, wherein the one or more frequency tables comprise information about frequencies of intra prediction modes used in the conversion of the video and side information about occurrences of the intra prediction modes.

[0013] In another example aspect, another video processing method is disclosed. The method includes, for a conversion between a video unit of a video and a coded representation of the video, determining an intra prediction mode for the conversion; and performing the conversion based on the intra prediction mode; wherein the intra prediction mode is signaled in a syntax element in the coded representation.

[0014] In yet another representative aspect, the above-described method is embodied in the form of a processor-executable code and stored in a computer-readable program medium.

[0015] In yet another representative aspect, an apparatus configured to or operable to perform the above-described method is disclosed. The apparatus can include a processor programmed to implement the method.

[0016] In yet another representative aspect, a video decoder apparatus can implement the methods described herein.

[0017] The above and other aspects and features of the disclosed technology are more fully described in the accompanying drawings, specification, and claims. BRIEF DESCRIPTION OF DRAWINGS

[0018] Figure 1 Sixty-seven intra prediction modes are shown.

[0019] Figure 2 An example of ALWIP for 4x4 block is shown.

[0020] Figure 3 An example of ALWIP for 8x8 block is shown.

[0021] Figure 4is an example of ALWIP for 8x4 blocks.

[0022] Figure 5 is an example of ALWUP for 16x16 blocks.

[0023] Figure 6 An example of four reference lines adjacent to a prediction block is shown.

[0024] Figure 7 An example of the partitioning of 4x8 and 8x4 blocks is shown.

[0025] Figure 8 An example of the partitioning of all blocks except 4x8, 8x4 and 4x4 is shown.

[0026] Figure 9 An example of a neighboring block for the MPM list construction process is shown.

[0027] Figure 10 An example of the "CR" position of DM derived from the corresponding luma block is shown.

[0028] Figure 11 An example of the bilinear intra prediction mode is shown.

[0029] Figure 12 is an example illustration of updating the frequency table and the sorted IPM table.

[0030] Figures 13-14 An example of a decoding flowchart in some embodiments is shown.

[0031] Figure 15 An example of the updating and reordering process is shown.

[0032] Figure 16 An example of building the MPM list and the non-MPM list with the frequency table is shown.

[0033] Figure 17 An example of building all MPM modes with the frequency table and the non-MPM list is shown.

[0034] Figure 18 is a diagram of the MPM construction procedure with local and global sublists.

[0035] Figures 19A-19B is a flowchart of a video processing method.

[0036] Figure 20 is a block diagram of an example hardware platform for implementing video processing techniques.

[0037] Figure 21 is a flowchart of an example method for video processing.

[0038] Figure 22 An example of an update and reorder process in one example embodiment is shown.

[0039] Figures 23 to 25 An example method for video processing is shown.

[0040] Figure 26 is a block diagram illustrating an example video processing system in which various techniques disclosed herein can be implemented.

[0041] Figure 27 is a block diagram illustrating a video coding system in accordance with some embodiments of the disclosure.

[0042] Figure 28 is a block diagram illustrating an encoder in accordance with some embodiments of the disclosure.

[0043] Figure 29 is a block diagram illustrating a decoder in accordance with some embodiments of the disclosure. DETAILED DESCRIPTION

[0044] Embodiments of the disclosed technology can be applied to existing video coding standards (e.g., HEVC, H.265) and future standards to improve compression performance. In this document, section headings are used to improve readability of the specification and do not in any way limit the scope of the discussion or embodiments to only the section in which they are discussed.

[0045] 1. OVERVIEW

[0046] This document relates to image / video coding technology. In particular, it relates to intra mode coding in image / video coding. It can be applied to existing video coding standards such as HEVC, or standards under development (Versatile Video Coding). It can also be applicable to future video coding standards or video codecs.

[0047] 2. PRELIMINARY DISCUSSION

[0048] Video coding standards have evolved primarily through the development of the well-known ITU-T and ISO / IEC standards. The ITU-T produced H.261 and H.263, ISO / IEC produced MPEG-1 and MPEG-4 Visual, and the two organizations jointly produced the H.262 / MPEG-2 Video and H.264 / MPEG-4 Advanced Video Coding (AVC) and H.265 / High Efficiency Video Coding (HEVC) standards. Since H.262, the video coding standards are based on the hybrid video coding structure, where temporal prediction plus transform coding is employed. To explore future video coding technologies beyond HEVC, the Joint Video Exploration Team (JVET) was founded by VCEG and MPEG jointly in 2015. Since then, many new methods have been adopted by the JVET and put into the reference software named Joint Exploration Model (JEM). In April 2018, the Joint

[0049] The latest version of the VVC draft, namely Versatile Video Coding (Draft Proposal) can be found at:

[0050] http: / / phenix.it-sudparis.eu / jvet / doc_end_user / documents / 15_Gothenburg / wg11 / JVET-O2001-v14.zip

[0051] The latest reference software of VVC, called VTM, can be found at:

[0052] https: / / vcgit.hhi.fraunhofer.de / jvet / VVCSoftware_VTM / tags / VTM-5.0

[0053] 2.1 Luma Intra Prediction Method

[0054] 2.1.1 Intra Mode Coding with 67 Intra Prediction Modes

[0055] To capture the arbitrary edge directions that are present in natural video, the number of directional intra modes is extended from 33 used in HEVC to 65. The additional directional modes are in Figure 1The planar and DC modes remain unchanged, as indicated by the dashed arrows. Thus, there are 67 intra prediction modes in total. These more dense directional intra prediction modes apply to all block sizes and both luma and chroma intra prediction.

[0056] The regular angular intra prediction directions are defined as 45 degrees to -135 degrees in the clockwise direction, as shown in Figure 1 In VTM2, for non-square blocks, several regular angular intra prediction modes are adaptively replaced by wide-angle intra prediction modes. The replaced modes are signaled using the original method and remapped to the indices of the wide-angle modes after parsing. The total number of intra prediction modes is unchanged, i.e., 67, and the intra mode coding is unchanged.

[0057] In HEVC, each intra coded block has a square shape and the length of each side is a power of 2. Thus, no division operation is needed to generate the intra prediction factor using the DC mode. In VVV, the block can have a rectangular shape and, in general, a division operation has to be used for each block. To avoid the division operation for DC prediction, only the longer side is used to calculate the average for non-square blocks.

[0058] In addition to the 67 intra prediction modes, wide-angle intra prediction (WAIP) and position-dependent intra prediction combination (PDPC) methods for non-square blocks are enabled for certain blocks. PDPC applies to the following intra modes without signaling: planar, DC, horizontal, vertical, bottom-left angular mode and its eight neighboring angular modes, and top-right angular mode and its eight neighboring angular modes.

[0059] 2.1.2 Affine linear weighted intra prediction (ALWIP, also known as matrix-based intra prediction)

[0060] Affine linear weighted intra prediction (ALWIP, also known as matrix-based intra prediction (MIP)) was proposed in JVET-N0217.

[0061] 2.1.2.1 Generating the reduced prediction signal by matrix-vector multiplication

[0062] First, the neighboring reference samples are down-sampled by averaging to produce the reduced reference signal bdry red . Then, the reduced prediction signal pred red is calculated by computing the matrix-vector product and adding an offset.

[0063] pred red = A · bdry red + b

[0064] Here, A is a matrix having W in the case of W=H=4. red ·H red rows and 4 columns, while in all other cases it has 8 columns. b is the dimension W red ·H red Vector.

[0065] 2.1.2.2 Diagram of the entire ALWIP process

[0066] Figures 2 to 5 The entire process of averaging, matrix-vector multiplication, and linear interpolation is shown for different shapes in . Note that the remaining shapes are handled as one of the following cases.

[0067] 1. Given a 4×4 block, ALWIP takes two averages along each axis of the boundary. The resulting four input samples are fed into a matrix-vector multiplication. The matrix is ​​taken from set S0. After adding the offset, this yields 16 final prediction samples. Linear interpolation is not required to generate the prediction signal. Therefore, a total of (4·16) / (4·4) = 4 multiplications are performed per sample. Figure 2 is a diagram of ALWIP for a 4×4 block.

[0068] 2. Given an 8×8 block, ALWIP takes four averages along each axis of the boundary. The resulting eight input samples are fed into a matrix-vector multiplication. The matrix is ​​taken from set S1. This generates 16 samples at odd positions in the prediction block. Therefore, a total of (8·16) / (8·8) = 2 multiplications are performed per sample. After adding the offset, these samples are vertically interpolated using the reduced top boundary. Horizontally interpolated using the original left boundary. Figure 3 is a diagram of ALWIP for 8×8 blocks.

[0069] 3. Given an 8×4 block, ALWIP takes four averages along the horizontal axis of the boundary and four original boundary values ​​at the left boundary. The resulting eight input samples are fed into a matrix-vector multiplication. The matrix is ​​taken from set S1. This generates 16 samples at each odd horizontal position and vertical position in the prediction block. Therefore, a total of (8·16) / (8·4) = 4 multiplications are performed on each sample. After adding the offset, these samples are horizontally interpolated using the original left boundary. Figure 4 This is an illustration of ALWIP for an 8×4 block. The same applies to the case after transposition.

[0070] 4. Given a 16x16 block, ALWIP takes four averages along each axis of the boundary. The resulting eight input samples enter a matrix vector multiplication operation. The matrix is taken from the set S2. Sixty-four samples are produced at odd positions of the predicted block. Thus, a total of (8*64) / (16*16)=2 multiplications are performed per sample. After adding the offset, these samples are vertically interpolated using the eight averages of the top boundary. Subsequently, they are horizontally interpolated using the original left boundary. In this case, the interpolation process does not add any multiplication operations. Thus, in total, two multiplications per sample are needed to compute the ALWIP prediction. Figure 5 is an illustration of ALWIP for a 16x16 block.

[0071] The procedure is basically the same for larger shapes, and it is easy to check that each sample has fewer than four multiplications.

[0072] For Wx8 blocks (W>8), only horizontal interpolation is needed because the samples are given at horizontal odd positions and at each vertical position.

[0073] Finally, for Wx4 blocks (W>8), let A k be the matrix formed by discarding each row corresponding to an odd entry on the horizontal axis of the downsampled block. Thus, the output size is 32, and again, only horizontal interpolation is needed to be performed.

[0074] The transposed case is also handled accordingly.

[0075] 2.1.2.3 Intra mode setting

[0076] The total number of intra prediction modes used in MIP varies depending on the block size. More specifically, the following applies:

[0077] - For 4x4 blocks, there are 34 modes

[0078] - Otherwise, if the width and height of the block are not both larger than 8, there are 18 modes

[0079] - Otherwise (both the width and height of the block are larger than 8), there are 10 modes

[0080] In the MPM list construction process, if a neighboring block is coded with a MIP mode, the corresponding intra prediction mode is set to the planar mode.

[0081] 2.1.3 Multiple reference lines (MRL)

[0082] Multiple reference lines (MRL) intra prediction uses more reference lines for intra prediction. In Figure 6In the middle, an example of 4 reference lines is shown, where the samples of segments A and F are not taken from the reconstructed neighboring samples, but are padded by the closest samples of segments B and E, respectively. HEVC intra picture prediction uses the closest reference line, i.e. reference line 0. In MRL, 2 additional lines are used (reference line 1 and reference line 3).

[0083] The index of the selected reference line (mrl idx) is signaled and used to generate the intra prediction factor. For reference line indices larger than 0, only the additional reference line modes are included in the MPM list and only the MPM index is signaled without the rest of the modes. The reference line index is signaled before the intra prediction mode and, in case of a non-zero reference line index, the planar and DC modes are excluded from the intra prediction mode.

[0084] The first row of blocks within a CTU is disabled for MRL to prevent the use of extended reference samples outside the current CTU row. Also, PDPC is disabled when additional lines are used.

[0085] 2.1.4 Intra Subblock Partitioning (ISP)

[0086] In JVET-M0102, ISP was proposed as shown in Table 1, which partitions a luma intra prediction block vertically or horizontally into 2 or 4 sub-partitions depending on the block size. Figure 7 and Figure 8 An example of the two possibilities is shown. All sub-partitions satisfy the condition of having at least 16 samples. For block sizes, if 4xN or Nx4 (N>8) is allowed, there can be 1xN or Nx1 sub-partitions.

[0087] Table 1: Number of sub-partitions depending on block size (maxTBSize denotes the maximum transform size)

[0088]

[0089] For each of these sub-partitions, a residual signal is generated by entropy decoding the coefficients sent by the encoder, inverse quantizing and inverse transforming them. Then, the sub-partition is intra predicted and finally the corresponding reconstructed samples are obtained by adding the residual signal to the prediction signal. Thus, the reconstructed values of each sub-partition will be used to generate the prediction for the next partition, repeating the process, and so on. All sub-partitions share the same intra mode.

[0090] Table 2: Specification of trTypeHor and trTypeVer depending on predModeIntra

[0091]

[0092] 2.2 Luma intra mode coding

[0093] The allowed intra prediction modes are divided into two parts: intra prediction modes in the most probable mode (MPM) list; and the rest of the modes. Whether to use a mode in the MPM list or not is controlled by a flag (intra_luma_mpm_flag).

[0094] For the MPM list, the first is always set to the planar mode, therefore, a separate flag is first signaled to indicate whether the selected mode is planar or not. If not, it is assumed that the MPM list size is equal to 6, and the index of this MPM list is further signaled with a reduction of 1.

[0095] Alternatively, the size of the MPM list can be considered differently as 5, and the planar mode is not included in the list. However, the flag intra_luma_mpm_flag is still signaled to indicate whether the mode is planar or not in the case where it is equal to true. In this case, when the block selects a non-planar MPM mode, the index of this MPM list is signaled.

[0096] 2.2.1 Construction of the MPM list

[0097] Assuming the mode to the left is denoted as "Left" and the mode above is denoted as "Above", the unified MPM list is constructed in the following steps:

[0098] - When the intra prediction mode of the neighboring block is not available, its intra mode is set to "Planar" by default.

[0099] - If the "Left" and "Above" modes are the same and both are angular:

[0100] o MPM list -> {Planar, Left, Left-1, Left+1, DC, Left-2}

[0101] - If the "Left" and "Above" modes are different and both are angular:

[0102] o Set the "Max" mode to the larger one of the "Left" and "Above" modes

[0103] o If the difference of the "Left" and "Above" modes is between 2 and 62, inclusive

[0104] ■ MPM list -> {Planar, Left, Above, DC, Max-1, Max+1}

[0105] o Otherwise

[0106] ■ MPM list -> {Planar, Left, Above, DC, Max-2, Max+2}

[0107] - If the "left" and "above" modes are different and one of the "left" and "above" modes is an angular mode and the other is a non-angular mode:

[0108] o Set the "Max" mode to the larger one of the "left" and "above" modes

[0109] o MPM list -> {Planar, Max, DC, Max-1, Max+1, Max-2}

[0110] - If both the "left" and "above" modes are non-angular modes:

[0111] o MPM list -> {Planar, DC, V, H, V-4, V+4}

[0112] Figure 9 An example of a neighboring block used for the MPM list construction process is shown.

[0113] 2.2.2 Specification on intra mode coding

[0114] 7.3.8.5 Coding unit syntax

[0115]

[0116]

[0117]

[0118]

[0119]

[0120]

[0121] 8.4.2 Derivation process of the luma intra prediction mode

[0122] The inputs of this process are:

[0123] - A luma position (xCb, yCb) specifying the top-left sample of the current luma coding block relative to the top-left luma sample of the current picture,

[0124] - A variable cbWidth specifying the width of the current coding block in luma samples,

[0125] - A variable cbHeight specifying the height of the current coding block in luma samples.

[0126] In this process, the luma intra prediction mode IntraPredModeY[xCb][yCb] is derived.

[0127] Table 8-1 specifies the values of the intra prediction mode IntraPredModeY[ xCb ][ yCb ] and the associated names.

[0128] Table 8-1 Specification of intra prediction modes and associated names

[0129]

[0130] NOTE - The intra prediction modes INTRA LT CCLM, INTRA L CCLM, and INTRA T CCLM are only applicable for chroma components.

[0131] IntraPredModeY[ xCb ][ yCb ] is derived as follows:

[0132] - If intra luma not planar flag[ xCb ][ yCb ] is equal to 0, then IntraPredModeY[ xCb ][ yCb ] is set equal to INTRA PLANAR.

[0133] - Else if BdpcmFlag[ xCb ][ yCb ] is equal to 1, then IntraPredModeY[ xCb ][ yCb ] is set equal to BdpcmDir[ xCb ][ yCb ]? INTRA ANGULAR50 : INTRA ANGULAR18.

[0134] - Else (intra luma not planar flag[ xCb ][ yCb ] is equal to 1), the following ordered steps apply:

[0135] 1. The neighbouring positions ( xNbA, yNbA ) and ( xNbB, yNbB ) are set equal to ( xCb - 1, yCb + cbHeight - 1 ) and ( xCb + cbWidth - 1, yCb - 1 ), respectively.

[0136] 2. For X replaced by A or B, the variable candIntraPredModeX is derived as follows:

[0137] - The derivation process of the availability of a neighbouring block specified in clause 6.4.4 is invoked with the position ( xCurr, yCurr ) set equal to ( xCb, yCb ), the neighbouring position ( xNbY, yNbY ) set equal to ( xNbX, yNbX ), the checkPredModeY set equal to FALSE, and the cldx set equal to 0 as inputs, and the output is assigned to availableX.

[0138] - The following candidate intra prediction mode candIntraPredModeX is derived:

[0139] - If one or more of the following conditions are true, candIntraPredModeX is set equal to INTRA_PLANAR.

[0140] - The variable availableX is equal to FALSE.

[0141] - CuPredMode[0][xNbX][yNbX] is not equal to MODE_INTRA.

[0142] - intra_mip_flag[xNbX][yNbX] is equal to 1.

[0143] - X is equal to B and yCb - 1 is less than ((yCb » CtbLog2SizeY) « CtbLog2SizeY).

[0144] - Otherwise, candIntraPredModeX is set equal to IntraPredModeY[xNbX][yNbX].

[0145] 3. The following candModeList[x] for x = 0..4 is derived:

[0146] - If candIntraPredModeB is equal to candIntraPredModeA and candIntraPredModeA is greater than INTRA_DC, the following candModeList[x] for x = 0..4 is derived:

[0147] candModeList[0] = candIntraPredModeA (8-6)

[0148] candModeList[1] = 2 + ((candIntraPredModeA + 61) % 64) (8-7)

[0149] candModeList[2] = 2 + ((candIntraPredModeA - 1) % 64) (8-8)

[0150] candModeList[3] = 2 + ((candIntraPredModeA + 60) % 64) (8-9)

[0151] candModeList[4] = 2 + (candIntraPredModeA % 64) (8-10)

[0152] - Otherwise, if candIntraPredModeB is not equal to candIntraPredModeA, and candIntraPredModeA or candIntraPredModeB is greater than INTRA DC, the following applies:

[0153] - The variables minAB and maxAB are derived as follows:

[0154] minAB = Min(candIntraPredModeA, candIntraPredModeB) (8-11)

[0155] maxAB = Max(candIntraPredModeA, candIntraPredModeB) (8-12)

[0156] - If candIntraPredModeA and candIntraPredModeB are both greater than INTRA DC, candModeList[x] for x = 0..4 is derived as follows:

[0157] candModeList[0] = candIntraPredModeA (8-13)

[0158] candModeList[1] = candIntraPredModeB (8-14)

[0159] - If maxAB - minAB is equal to 1 (including 1), the following applies:

[0160] candModeList[2] = 2 + ((minAB + 61) % 64) (8-15)

[0161] candModeList[3] = 2 + ((maxAB - 1) % 64) (8-16)

[0162] candModeList[4] = 2 + ((minAB + 60) % 64) (8-17)

[0163] - Otherwise, if maxAB - minAB is greater than or equal to 62, the following applies:

[0164] candModeList[2] = 2 + ((minAB - 1) % 64) (8-18)

[0165] candModeList[3] = 2 + ((maxAB + 61) % 64) (8-19)

[0166] candModeList[4] = 2 + ((maxAB + 61) % 64) (8-20)

[0167] - Otherwise, if maxAB - minAB is equal to 2, the following applies:

[0168] candModeList[2] = 2 + ((minAB - 1) % 64) (8-21)

[0169] candModeList[3] = 2 + ((minAB + 61) % 64) (8-22)

[0170] candModeList[4] = 2 + ((maxAB - 1) % 64) (8-23)

[0171] - Otherwise, the following applies:

[0172] candModeList[2] = 2 + ((minAB + 61) % 64) (8-24)

[0173] candModeList[3] = 2 + ((minAB - 1) % 64) (8-25)

[0174] candModeList[4] = 2 + ((maxAB + 61) % 64) (8-26)

[0175] - Otherwise, (candIntraPredMode or candIntraPredModeB is greater than INTRA DC), candModeList[x] for x = 0..4 is derived as follows:

[0176] candModeList[0] = maxAB (8-27)

[0177] candModeList[1] = 2 + ((maxAB + 61) % 64) (8-28)

[0178] candModeList[2] = 2 + ((maxAB - 1) % 64) (8-29)

[0179] candModeList[3] = 2 + ((maxAB + 60) % 64) (8-30)

[0180] candModeList[4] = 2 + (maxAB % 64) (8-31)

[0181] - Otherwise, the following applies:

[0182] candModeList[ 0 ] = INTRA DC (8-32)

[0183] candModeList[ 1 ] = INTRA ANGULAR50 (8-33)

[0184] candModeList[ 2 ] = INTRA ANGULAR18 (8-34)

[0185] candModeList[ 3 ] = INTRA ANGULAR46 (8-35)

[0186] candModeList[ 4 ] = INTRA ANGULAR54 (8-36)

[0187] 4. IntraPredModeY[ xCb ][ yCb ] is derived by applying the following procedure:

[0188] - If intra_luma_mpm_flag[ xCb ][ yCb ] is equal to 1, IntraPredModeY[ xCb ][ yCb ] is set equal to candModeList[ intra_luma_mpm_idx[ xCb ][ yCb ] ].

[0189] - Otherwise, IntraPredModeY[ xCb ][ yCb ] can be derived by applying the following sequential steps:

[0190] 1. When i = 0..3 and for each i, j = (i+1)..4, candModeList[ i ] is greater than candModeList[ j ], the two values are swapped as follows:

[0191] ( candModeList[ i ], candModeList[ j ] ) = Swap( candModeList[ i ], candModeList[ j ] ) (8-37)

[0192] 2. IntraPredModeY[ xCb ][ yCb ] is derived in the following sequential steps:

[0193] i. IntraPredModeY[ xCb ][ yCb ] is set equal to intra_luma_mpm_remainder[ xCb ][ yCb ].

[0194] ii. The value of IntraPredModeY[ xCb ][ yCb ] is increased by 1.

[0195] iii. For i equal to 0 to 4, inclusive, when IntraPredModeY[ xCb ][ yCb ] is greater than or equal to candModeList[ i ], the value of IntraPredModeY[ xCb ][ yCb ] is increased by 1.

[0196] The variable IntraPredModeY[ x ][ y ] with x = xCb.. xCb + cbWidth - 1 and y = yCb.. yCb + cbHeight - 1 is set equal to IntraPredModeY[ xCb ][ yCb ].

[0197] 2.3 Chroma Intra Prediction Modes

[0198] For chroma intra mode coding, 8 or 5 intra modes are allowed in total depending on whether Cross Component Linear Model (CCLM) is enabled or not. These modes include five traditional intra modes and three cross component linear model modes (setting IntraPredModeC to 81, 82, and 83, respectively).

[0199] 2.3.1 DM Mode

[0200] In chroma direct mode or derived mode (DM), the prediction mode of the collocated luma block is used to derive the chroma intra prediction mode.

[0201] First, the intra prediction mode lumaIntraPredMode is derived:

[0202] • If the collocated luma block is coded in MIP mode, lumaIntraPredMode is set equal to the planar mode.

[0203] • Otherwise, if the collocated luma block is coded in IBC mode or palette mode, lumaIntraPredMode is set equal to the DC mode.

[0204] • Otherwise, lumaIntraPredMode is set equal to the intra prediction mode of the collocated luma block covering the corresponding luma sample at the center of the chroma block. Figure 10 An example is shown in FIG. 2.

[0205] Second, the chroma intra prediction mode (called IntraPredModeC) is derived according to the following table with lumaIntraPredMode highlighted in bold italic. Note that intra_chroma_pred_mode equal to 4 refers to the DM mode.

[0206] Table 8-2: Specification of IntraPredModeC[xCb][yCb] depending on cclm_mode_flag, cclm_mode_idx, intra_chroma_pred_mode and lumaIntraPredMode

[0207]

[0208] Finally, if the color format of the picture is 4:2:2, IntraPredModeC is further modified for the DM mode according to the following table.

[0209] Specification of the 4:2:2 mapping process from chroma intra prediction mode X to mode Y when chroma_format_idc is equal to 2

[0210]

[0211] The following detailed draft on derivation of chroma intra prediction mode is defined in VVC D:

[0212] 8.4.3 Derivation process of chroma intra prediction mode

[0213] The inputs of this process are:

[0214] - the luma position (xCb, yCb) specifying the top-left sample of the current chroma coding block with respect to the top-left luma sample of the current picture,

[0215] - the variable cbWidth specifying the width of the current coding block in luma samples,

[0216] - the variable cbHeight specifying the height of the current coding block in luma samples.

[0217] In this process, the chroma intra prediction mode IntraPredModeY[xCb][yCb] is derived.

[0218] The corresponding luma intra prediction mode lumaIntraPredMode is derived as follows:

[0219] - if intra_mip_flag[xCb][yCb] is equal to 1, lumaIntraPredMode is set equal to INTRA_PLANAR.

[0220] - Otherwise, if CuPredMode[0][xCb][yCb] is equal to MODE_IBC or MODE_PLT, lumaIntraPredMode is set equal to INTRA_DC.

[0221] - Otherwise, lumaIntraPredMode is set equal to IntraPredModeY[xCb + cbWidth / 2][yCb + cbHeight / 2].

[0222] The chroma intra prediction mode IntraPredModeC[xCb][yCb] is derived using cclm_mode_mode_flag, cclm_mode_idx, intra_chroma_pred_mode and lumaIntraPredMode as specified in Table 8-2.

[0223] Table 8-2 Specification of IntraPredModeC[xCb][yCb] in terms of cclm_mode_mode_flag, cclm_mode_idx, intra_chroma_pred_mode and lumaIntraPredMode

[0224]

[0225] When chroma_format_idc is equal to 2, the chroma intra prediction mode Y is derived using the chroma intra prediction mode X in Table 8-2 as specified in Table 8-3, and the chroma intra prediction mode X is then set equal to the chroma intra prediction mode Y.

[0226] Table 8-3 Specification of the 4:2:2 mapping process from chroma intra prediction mode X to mode Y when chroma_format_idc is equal to 2

[0227]

[0228] 2.4 Intra prediction in the Audio Video Coding Standard (AVS)

[0229] In addition to the angular intra prediction modes, AVS supports a planar mode and a variant of the bilinear mode.

[0230] In the planar mode of AVS, the prediction value is obtained by linear interpolation of the two samples in the left and top neighboring blocks using the following formula:

[0231] P[x, y] = (a + (x - (M » 1) + 1) * b + (y - (N » 1) + 1) * c + 16) » 5

[0232] where (x, y) denotes the coordinates relative to the top-left sample in the current block, and the variables a, b, c depend on the values of the block width M and block height N and the reconstructed neighboring samples.

[0233] In the bilinear mode of AVS, a multi-step interpolation process is needed. The sample to be predicted is denoted as “C”, the two neighboring samples on top are denoted as “A” and “H”, the two neighboring samples on the left in the same row are denoted as “B” and “G”, the sample on the bottom in the same column is denoted as “E”, and the sample on the right in the same row is denoted as “F”. The relationship of all samples is shown in Figure 1 . First, the prediction value of the bottom-right sample “D” in the current block is derived using A and B by linear interpolation. Then, based on the distance information, the prediction value of “E” is derived using sample “B” and the predicted sample “D”, and the prediction value of “F” is derived using sample “A” and the predicted sample “D”. Subsequently, the prediction value of “C” is obtained using samples “G”, “H”, and the prediction values of “E” and “F”.

[0234] Figure 11 An example of the bilinear intra prediction mode is depicted.

[0235] 3. Examples of technical problems overcome by the solution provided by this document

[0236] The existing technology design of intra mode coding still depends on the intra prediction mode (IPM) of the spatial neighboring block. However, for screen content, the IPM of the non-neighboring block has higher correlation. It is necessary to study how to better utilize this information.

[0237] 4. Enumerations of technologies and embodiments

[0238] The following enumerations will serve as examples to explain the general concepts. These inventions should not be interpreted in a narrow way. Furthermore, the inventions can be combined in any way.

[0239] In the following discussion, the conventional intra prediction method can represent the way of using the neighboring row / column for intra prediction, which can use an interpolation filter along the prediction direction. And the additional intra coding method can represent those newly introduced in VVC, which can be introduced in the future, and additional signaling is needed for the usage of this method. The additional method can be one or more of ALWIP, MRL, ISP, or QR-BDPCM / PCM, etc.

[0240] Let's assume that there are M allowed intra prediction modes (IPM) for a given block type. For example, the block type can be a given block dimension (whether 4x4 or larger); a given coding method (e.g., whether general intra coding or MIP).

[0241] The following example illustrates the use of the frequency table and the IPM table. For example, if during the conversion 100 blocks are coded in intra mode, 10 is DC (IPM with index 1), 20 is planar (IPM with index 0), 40 is Ver (IPM with index 50), 30 is IPM and index 7.

[0242] In this case, the frequency table can be like [40, 30, 20, 10...], while the IPM table is [50, 7, 0, 1]. Thus, in this embodiment, the value of IPMTable[k] is the intra prediction mode for an index equal to k, while the value of FrequenceTable[k] shows the occurrences of the mode IPMTable[k].

[0243] General concept of frequency table and sorted IPM table

[0244] 1. One or more frequency tables (also referred to as history table / list) are maintained during the encoding / decoding process of a video unit, wherein the frequency table records the frequency of intra modes. When a block (e.g., CU or PU) is finished encoding / decoding, the frequency table and the sorted IPM table can be updated.

[0245] a. In one example, the frequency table can be further associated with a sorted IPM table.

[0246] i. In addition, alternatively, when the frequency table is updated, the sorted IPM table can be updated accordingly.

[0247] ii. Alternatively, the entries in the frequency table can be associated with an intra prediction mode and the occurrences / frequency of that intra prediction mode. In this case, there is no need to further maintain a sorted IPM table corresponding to the frequency table.

[0248] iii. In one example, the kth entry of the sorted IPM table represents the IPM that is the kth most frequently used in the history.

[0249] b. In one example, the frequency table can be further associated with an IPM order mapping table.

[0250] i. In one example, the kth entry of the IPM order mapping table represents the index of the order of the IPM with index equal to k.

[0251] c. In one example, assume that the frequency table is denoted by FrequenceTable, the sorted IPM table is denoted by IPMTable, and the IPM order mapping table is denoted by orderT. Then, for an index equal to k, the value of IPMTable[k] is the intra prediction mode that indicates the sorting result, and the value of FrequenceTable[k] shows the occurrence of the mode IPMTable[k].

[0252] d. In one example, assume that the frequency table is denoted by freqT, the sorted IPM table is denoted by modeT, and the IPM order mapping table is denoted by orderT. Then, for an index equal to k, the value of modeT[k] is the intra prediction mode that indicates the sorting result, the value of orderT[k] is the mapping index after the mode with index equal to k is sorted, and the value of freqT[k] shows the occurrence of the mode with index equal to k.

[0253] i. Also, alternatively, orderT[modeT[k]] = k, where, for example, k can represent the order index k.

[0254] ii. Also, alternatively, modeT[orderT[m]] = m, where, for example, m can represent the intra prediction mode;

[0255] iii. Also, alternatively, freqT[modeT[k]] > freqT[modeT[k+1]].

[0256] e. In one example, the video unit is a sub-region of a CTU (e.g., VPDU) / CTU / CTB / multiple CTUs / multiple CUs / CTU row / tile / brick / slice / picture / sub-picture, etc.

[0257] f. In one example, the updated table can be further used to code subsequent blocks.

[0258] g. In one example, for each block type, a frequency table and / or a sorted IPM table can be maintained.

[0259] i. Alternatively, one frequency table and / or one sorted IPM table can be maintained for multiple blocks.

[0260] ii. One type (or one kind) of block can refer to blocks with the same width and / or the same height.

[0261] 2. The frequency table can be associated with M entries, and each entry is associated with the frequency of one of the allowed M intra prediction modes.

[0262] a. In one example, the IPM ordered table can have the same number of entries as the associated frequency table.

[0263] b. In one example, the M intra prediction modes can be grouped into N categories.

[0264] i. Additionally, alternatively, the frequency table can be associated with N entries, where N is less than M, and each entry is associated with the frequency of a category that can correspond to one or more prediction modes.

[0265] 1) Additionally, alternatively, the ordered IPM table associated with the frequency table can also have N entries, which show the ordered category indices.

[0266] c. In one example, the frequency table can be associated with N entries, where N is less than M. The N entries correspond to N intra prediction modes selected from the M intra prediction modes.

[0267] i. In one example, the M intra prediction modes can not include wide-angle intra prediction modes.

[0268] ii. In one example, the N intra prediction modes can include at least one of DC, planar, horizontal, vertical, and bilinear IPMs.

[0269] iii. In one example, the N selected intra prediction modes can be predefined or signaled or derived based on coding information (e.g., whether it is screen content).

[0270] d. In one example, the frequency table can be associated with N entries, where N is less than M. The N entries correspond to N intra prediction modes that are updated on-the-fly based on decoding information.

[0271] i. In one example, the N IPMs can be first initialized and then updated based on decoding information.

[0272] Use of sorted IPM table (also referred to as table-based intra mode coding)

[0273] 3. The frequency table / ordered IPM table can be used for intra mode coding in the MPM list construction process, assuming the MPM list size is N.

[0274] a. In one example, the entire MPM list is determined from one or more ordered IPM tables.

[0275] i. In one example, the top N modes showing the highest frequencies can be used as input to the MPM list.

[0276] 4. Intra mode coding can be based on both the frequency table / ordered IPM table and other non-table-based methods.

[0277] a. In one example, the entire MPM list is determined from one or more ordered IPM tables and other non-table-based methods.

[0278] i. In one example, the top M (M < N) modes showing the highest frequency can be used as input to the MPM list.

[0279] b. In one example, the modes selected from the ordered IPM tables can be combined with other intra prediction modes derived from non-table-based methods to form the final MPM list.

[0280] i. In one example, the other intra prediction modes derived from non-table-based methods can include using some default modes (e.g., planar / DC).

[0281] ii. Alternatively, the other intra prediction modes derived from non-table-based methods can be some modes derived from spatially neighboring (immediately or not immediately) blocks.

[0282] 1) In one example, the spatially neighboring blocks can be defined as the “above” block and the “left” block of the current block, as depicted in Figure 9

[0283] 2) In one example, the default mode order is {mode of left block, mode of above block, planar, DC}.

[0284] 3) In one example, for the case where the above or left block is not available (e.g., CTU or slice boundary), -1 is used instead of the missing mode index.

[0285] iii. In one example, the other intra prediction modes can be added to the MPM list before those selected from the ordered IPM tables.

[0286] iv. In one example, the other intra prediction modes can be added to the MPM list after those selected from the ordered IPM tables.

[0287] v. In one example, the other intra prediction modes can be added to the MPM list before and after those selected from the ordered IPM tables.

[0288] vi. In one example, pruning can be applied to avoid adding redundant modes to the MPM list.

[0289] vii. In one example, the order of adding the IPMs derived from the ordered IPM tables and non-table-based methods can be changed from block to block, one video unit to another.​

[0290] viii. In one example, the number of IPMs derived from the sorted IPM table and from non-table based methods can vary from block to block, from one video unit to another.

[0291] ix. In one example, the first L (e.g., L = 2 or 6) modes in the sorted table can be added to the MPM list.

[0292] x. In one example, the last L (e.g., L = 2 or 6) modes in the sorted table can be added to the MPM list.

[0293] xi. In one example, when adding modes (selected L (e.g., L = 2 or 6)) in the sorted table to the MPM list, the addition can be based on ascending / descending order of entry index.

[0294] xii. In one example, when adding modes (selected L (e.g., L = 2 or 6)) in the sorted table to the MPM list, the addition can be based on ascending / descending order of intra prediction mode index.

[0295] 1) In one example, the indices of the first L modes in the sorted table can be selected to be added to the MPM list, denoted as iPM0, iPM1.., iPM L-1 , where orderT[iPM m ] = m, m is [0, L-1]. If iPM i is less than iPM j , iPM i can be added before iPM j .

[0296] a. Alternatively, if iPM i is greater than iPM j , iPM i can be added before iPM j .

[0297] c. Whether to determine the entire MPM list from the sorted IPM table can depend on frequency.

[0298] d. Whether to determine the entire MPM list from the sorted IPM table can depend on decoding information, e.g., block dimension (e.g., whether the width and / or height of the block is not greater than a threshold) / video content type (e.g., whether it is screen content).

[0299] 5. The sorted IPM table can be used for intra mode coding of the rest of the modes other than MPM.

[0300] a. In one example, the indices of the rest of the modes with smaller signaling are corresponding to the modes with higher frequency in the sorted IPM table.

[0301] 6. Instead of coding MPM flag / MPM list index / rest of the intra prediction modes except MPM, one syntax element can be coded to indicate the selected IPM, which can depend on the history information (e.g. frequency information). In addition, alternatively, the mapping between the coded syntax element value to IPM is changed from block to block.

[0302] a. In one example, it is proposed to code the index directly to the sorted IPM table.

[0303] i. In addition, alternatively, the IPM to be used for encoding / decoding the block is derived from the index.

[0304] ii. In one example, assuming the sorted IPM table is [VER, HOR, DC, Planar, Mode 3,...], the index equal to 0 is interpreted as the vertical intra prediction mode.

[0305] b. In one example, it is proposed to code the index corresponding to the intra prediction mode directly based on the descending order of the frequency associated with the IPM.

[0306] i. In one example, assuming the frequency table is [F(Planar), F(DC), F(Mode 1),.. F(VER),..]. If F(VER) and F(DC) are the two largest values, then the index “0” can be coded to indicate the “VER” mode and the index “1” to indicate the “DC” mode.

[0307] c. In one example, the index can be coded with a binarization method, such as truncated unary / truncated binary / exp-golomb method, etc.

[0308] d. In one example, the index can be context coded for all bins or partial bins, such as the first few bins.

[0309] 7. The proposed method can be applied to chroma direct mode (DM) coding, where the DM is determined according to the frequency table / sorted IPM table / IPM order mapping table, such as the IPM associated with the highest frequency table.

[0310] 8. The proposed method can be applied to chroma direct mode (DM) coding, where the chroma mode candidate list is determined from a frequency table / sorted IPM table / IPM order mapping table (e.g., IPM associated with the highest frequency).

[0311] Reset / initialization of frequency table and / or sorted IPM table

[0312] 9. The frequency table and / or the sorted IPM table can be reset / initiated when a new video unit is being coded / decoded.

[0313] a. In one example, the video unit is a sub-region of a CTU (e.g., VPDU) / CTU / CTB / multiple CTUs / multiple CUs / CTU row / tile / brick / slice / picture / sub-picture, etc.

[0314] b. In one example, the sorted IPM table can be reset / initiated to be the same as the allowed intra prediction modes in ascending order of index, e.g., [Planar, DC, Mode 2, …, Mode 66].

[0315] c. In one example, the sorted IPM table can be reset / initiated to be the same as the allowed intra prediction modes in descending order of index, such as [Mode 66, Mode 65, …, Mode 2, DC, Planar].

[0316] d. In one example, the sorted IPM table can be reset / initiated to first include a plurality of default MPMs, followed by the remaining modes.

[0317] i. In one example, the default MPMs can include {VER, HOR, VER-4, VER+4, 2, DIA}

[0318] ii. In one example, the default MPMs can include {Planar, DC, VER, HOR, 2, DIA}

[0319] iii. In one example, the above DIA is the IPM with the largest index.

[0320] iv. Additionally, alternatively, these default modes can be put in the table in different orders.

[0321] v. Additionally, alternatively, how these default modes are defined and / or how these default modes are placed can depend on the block type / decoding information.

[0322] 10. The frequency table can be reset / initiated by a specific value of one or more entries in the table.

[0323] a. In one example, the specific value corresponding to the IPM can be set to an equal value.

[0324] i. In one example, the "equal value" can be set to N (e.g. N=0).

[0325] b. In one example, the specific value corresponding to the IPM can be set to a unique value, i.e. for any two IPMs, their initialization frequency values are different.

[0326] c. In one example, the specific value corresponding to the IPM can be set to the same or different values, i.e. for at least two IPMs, their initialization frequency values are different; for at least another two IPMs, their initialization frequency values are the same.

[0327] d. In one example, assuming the allowed MPMs are denoted by K, a default MPM list is defined.

[0328] i. In one example, the initialization value of the i-th IPM excluded in the MPM list is set to (M-1-i).

[0329] ii. In one example, the initialization value of the j-th IPM in the MPM list (where j is in the range [0, K-1]) is set to M+f(j), where f(j) returns a positive integer value.

[0330] 1) In one example, f(j) is set to (K-j).

[0331] e. In one example, multiple sets of specific values can be predefined and one set of values can be selected from them according to the coding information.

[0332] When to update frequency table and / or sorted IPM table

[0333] 11. The frequency table and / or the sorted IPM table can be updated after a block is encoded / decoded in the general intra prediction mode.

[0334] a. In one example, the frequency table and / or the sorted IPM table can be updated only when the prediction mode is intra mode.

[0335] i. In addition, alternatively, the frequency table and / or the sorted IPM table are not allowed to be updated when the prediction mode is not equal to intra mode.

[0336] b. In one example, the frequency table and / or the sorted IPM table can be updated or not updated after a block is encoded / decoded in the matrix-based intra prediction (MIP) mode.

[0337] i. Alternatively, the frequency table and / or the sorted IPM table are updated after a block encoded / decoded in the matrix-based intra prediction (MIP) mode is decoded.

[0338] ii. In one example, the conversion of MIP modes to general intra modes can be updated.

[0339] c. In one example, the frequency table and / or the sorted IPM table can not be updated after decoding a block coded in intra sub-partition (ISP) mode.

[0340] i. Alternatively, the frequency table and / or the sorted IPM table can be updated after decoding a block coded in intra sub-partition (ISP) mode.

[0341] 1) For example, the frequency table and / or the sorted IPM table can be updated once after decoding the whole block.

[0342] 2) For example, the frequency table and / or the sorted IPM table can be updated once after decoding one sub-partition.

[0343] d. In one example, the frequency table and / or the sorted IPM table can or can not be updated after encoding / decoding a block in BDPCM / RDPCM mode.

[0344] e. In one example, the frequency table and / or the sorted IPM table can not be updated after encoding / decoding a block in intra prediction mode that is not part of the selected intra prediction mode, e.g. mentioned in bullet 2.c.

[0345] 12. The frequency table and / or the sorted IPM table can be updated after encoding / decoding a block in a prediction mode that is not equal to intra mode, such as inter / IBC / palette mode.

[0346] a. In one example, the frequency table and / or the sorted IPM table can be updated if a block is coded in a prediction mode that is not equal to intra mode and an intra prediction signal is generated, such as using combined intra-inter prediction (CIIP) mode.

[0347] b. In one example, the frequency table and / or the sorted IPM table can not be allowed to be updated if a block is coded in a prediction mode that is not equal to intra mode and no intra prediction signal is generated.

[0348] i. Alternatively, the table can be updated using a default IPM.

[0349] How to update frequency table and / or sorted IPM table

[0350] 13. The frequency table can be updated based on the input IPM, e.g. the decoded IPM of the current block.

[0351] Let us assume that before the n-th update of the table (where n starts from 1), the input IPM is denoted as M i , and the frequency is denoted as F n (M i ).

[0352] a. In one example, the frequency associated with M i is updated to F n+1 (M i ), e.g., F n+1 (M i ) = F n (M i ) + K, where K is an integer.

[0353] i. In one example, K is set to 1.

[0354] ii. In one example, K is set to an integer value larger than the number of entries in the frequency table.

[0355] iii. In one example, K can depend on the initialization value, i.e., F0.

[0356] 1) In one example, K is set to a maximum value equal to P * the initialization value (i.e., the maximum value of F0(M i )), where P is a positive integer value.

[0357] iv. In one example, K depends on the intra prediction mode and / or the block type.

[0358] 1) In one example, K is set to an integer value larger than the number of allowed intra prediction modes.

[0359] 2) In one example, K is set to a value equal to P * the number of allowed intra prediction modes, where P is a positive integer value.

[0360] v. In one example, K depends on the number of table updates, e.g., based on the variable n.

[0361] vi. In one example, K can be updated on the fly based on decoder information (e.g., the intra prediction mode).

[0362] b. In one example, the frequencies associated with other IPMs than M i remain unchanged, i.e., F n+1 (M j ) = F n (M j ).

[0363] i. Alternatively, the frequencies associated with other IPMs than M iThe frequency associated with the other IPM part is updated, and for the rest of the IPMs, the frequency remains unchanged.

[0364] c. Alternatively, the frequency table can be updated based on the input category index (e.g., the mapped category index of the IPM with the current block decoded) and the associated frequency with the input category index.

[0365] d. In one example, the length of the frequency table can be less than the number of allowed IPMs (e.g., 6, 12).

[0366] i. In one example, the length of the frequency table can be set to the number of allowed MPMs.

[0367] ii. In one example, when the frequency table needs to be updated with an input mode, and if the frequency of the input mode is already included in the frequency table, the corresponding frequency can be updated accordingly.

[0368] iii. In one example, when the frequency table needs to be updated with an input mode, and if the frequency of the input mode is not included in the frequency table, the entry with the lowest frequency (the corresponding IPM is M n ) can be replaced with the frequency of the input mode.

[0369] 1) In addition, alternatively, the sorted IPM table can also be replaced with the input mode with the mode M n .

[0370] 14. The sorting of the frequencies associated with different IPMs can be applied to generate an updated sorted IPM table.

[0371] a. In one example, the frequency table is fully sorted in descending order according to the values of the updated mode frequencies.

[0372] b. The sorting process can be performed from the current entry to the last entry corresponding to the input mode.

[0373] i. Alternatively, the sorting process can be performed from the current entry to the first entry corresponding to the input mode.

[0374] c. Figure 12 An example is shown in Table 1. After the update, the frequency associated with M i is F n+1 (M i ). A forward search is applied until a mode M j is found that satisfies F n+1 (M j )>F n+1 (M i )> = F n+1 (M j-1 ).

[0375] d. Once a pattern is found that satisfies certain conditions, the sorting process can be terminated.

[0376] i. Figure 12 An example is shown in . After updating, with M i The associated frequency is F n+1 (M i Once we find a n+1 (M j )>F n+1 (M i )>=F n+1 (M j-1 ) a pattern M j , a forward search is applied.

[0377] e. In one example, a sequence table is used to record the display mode sequence frequency table instead of sorting the frequency table. i order, and the corresponding frequencies can be obtained.

[0378] f. In one example, only a few elements of the frequency table are taken into account in the sorting process.

[0379] i. In one example, when updating the frequency table for the current pattern, the comparison process involves the first L elements.

[0380] Figure 12 is an example diagram of an update frequency table and a sorted IPM table.

[0381] 15. In addition to the pattern occurrence information, the frequency table can also store other auxiliary information, such as the location where the pattern occurs.

[0382] a. In one example, the location may include starting coordinates (eg, relative to a video unit, CTU / slice / etc.) and / or a block size.

[0383] b. In one example, after encoding / decoding an intra block, both the pattern frequency and position information need to be updated.

[0384] i. In one example, the i The associated frequency is updated to F n+1 (M i )(F n+1 (M i )=F n (M i )+K), where K is an integer.

[0385] ii. In one example, the iThe associated position information is replaced by the latest coded / decoded block.

[0386] c. In one example, the frequency table can be sorted according to the position information before the new block is coded / decoded.

[0387] i. In one example, the frequency table can be sorted according to the Euclidean distance between the current position and the recorded positions.

[0388] Enabling the proposed method

[0389] 16. Whether to enable the proposed method and / or which item(s) to apply can be signaled in the video unit level.

[0390] a. In one example, the video unit can be a slice / tile / strip / picture / subpicture / sequence / view, etc.

[0391] b. In one example, whether to enable the proposed method and / or how to enable the proposed method can be signaled in the sequence parameter set / view parameter set / adaptation parameter set / picture parameter set / picture header / strip header / sequence header.

[0392] i. In one example, a syntax element can be signaled to indicate whether to enable determining IPM from the frequency table / / sorted IPM table / IPM order mapping table.

[0393] ii. In one example, a syntax element can be signaled to indicate how many IPM can be determined from the frequency table / / sorted IPM table / IPM order mapping table.

[0394] iii. In one example, a syntax element can be signaled to indicate how many MPM can be determined from the frequency table / sorted IPM table / IPM order mapping table.

[0395] c. In one example, whether to enable the proposed method and / or how to enable the proposed method can be controlled by other syntax elements, such as one indicating whether the video content is screen content.

[0396] d. In one example, whether to enable the proposed method and / or how to enable the proposed method can be controlled by some features derived from the reconstructed samples in previously coded blocks.

[0397] 17. Whether to enable the proposed method and / or how to enable the proposed method can depend on the coding information, such as block dimension, slice type / picture type / temporal layer index / video content, etc.

[0398] a. In one example, the proposed method can be applied for blocks with width no larger than T1 and height no larger than T2.

[0399] b. In one example, the proposed method can be applied for blocks with width no larger than T1 or height no larger than T2.

[0400] c. In one example, the proposed method can be applied for blocks with width times height no larger than T3.

[0401] d. In one example, the proposed method can be disabled for blocks with width no larger than T1 and height no larger than T2.

[0402] e. In one example, the proposed method can be disabled for blocks with width no larger than T1 or height no larger than T2.

[0403] f. In one example, the proposed method can be disabled for blocks with width times height no larger than T3.

[0404] 18. Whether to enable the proposed method and / or how to enable the proposed method can depend on the color component / color coding method (e.g., separate plane coding) / color format (e.g., 4:2 or 4:4:4) / split tree coding method (e.g., single tree or dual tree).

[0405] 2:2 or 4:4:4) / split tree coding method (e.g., single tree or dual tree).

[0406] a. In one example, the proposed method can be applied only to luma intra prediction mode coding.

[0407] For illustration purposes, some embodiments using the techniques listed above are described.

[0408] 5 Embodiments

[0409] An example of the decoding process is shown below. A table with intra mode frequencies is maintained and updated during the encoding / decoding process. More specifically, the table records the frequencies of intra modes, and the modes are ordered by the accumulated frequencies.

[0410] 5.1 Embodiment #1

[0411] Figure 13 and Figure 14 An example of the decoding flowchart using the proposed method is shown.

[0412] First, when a new slice or a new CTU is encountered, the table is reset with a series of fixed values (or an array composed of the modes of the neighboring blocks). After a block with intra information is decoded / encoded, the table can be updated accordingly. During the encoding and decoding of the subsequent blocks, the updated table can be further utilized.

[0413] 5.2 Example #2

[0414] First, when a new video unit (e.g., a stripe or a new CTU) is encountered, the table is reset with a series of fixed values ​​(or an array of patterns of adjacent blocks). In particular, the table can be reset or initialized based on the content of the stripe. If it is a screen content stripe, mode set0 is used for initialization. Otherwise, mode set1 can be used. Based on VVC, mode set0 includes six modes {VER, HOR, VER-4, VER+4, 2, DIA}, which can be assigned a higher initial frequency when a screen content stripe is encountered. Mode set1 also contains six modes. The first six modes in mode set1 can be {plane, DC, VER, HOR, 2, DIA}. Alternatively, when a natural scene stripe is encountered, the initialized mode set1 can be assigned a higher initial frequency. More specifically, if mode M i Not in set0 or set1, then mode M i Frequency F n (M i ) is reset to F n (M i )=(66-M i ). Otherwise, the mode M j Frequency F n (M j ) reset to F n (M j )=67+(6-idx(M j )), where idx(M j ) represents the associated index in set0 or set1. In VVC, 67 intra modes are supported. Therefore, i and j are in the range of 0 to 66.

[0415] Secondly, after decoding a block containing intra information, the table is updated by recalculating the frequencies of the associated intra modes. In particular, the intra information can be an intra mode in an intra-coded block or an intra mode in an intra-inter combined block. Figure 15 As shown, in particular, after the mode update, the frequency associated with the current intra mode is adjusted to F n+1 (M i )=F n (M i )+δ(δ=1).

[0416] Figure 15 An example of an update and reordering process is shown.

[0417] Third, after the mode update, the frequency table is re-ordered in descending order according to the updated mode frequency. In particular, both the comparison and position swapping of the updated mode and the previous mode's frequency are performed until the updated mode's frequency is lower than the previous mode's frequency.

[0418] To efficiently implement, a sequential table is employed to assist the update and ordering procedures. In particular, the sequential table records the associated mode order in the frequency table. The order of the M i s can be referenced from the sequential table, and the corresponding frequencies can be obtained. In this way, the search time can be greatly saved.

[0419] With the sequential table, only six candidates are involved in the ordering process. In particular, the current intra mode and the first six modes in the sequential table are compared.

[0420] Therefore, the table can be used in the most probable mode (MPM) list or the remaining mode (RM) list construction process.

[0421] For the construction of the MPM list, the modes near the front of the frequency table are first added to the MPM list after the modes of the left neighboring block, the modes of the above neighboring block, the planar and DC modes. Subsequently, a redundancy check is applied to skip the same modes. After the MPM list is determined, the modes not in the MPM can be used to construct the remaining modes, which are then ordered according to the mode frequency from the frequency table. In Figure 16 the overall determination procedure is shown.

[0422] Figure 16 An example of using the frequency table to construct the MPM list and the non-MPM list is shown.

[0423] However, as shown in Figure 17 , the MPM list can also be directly established using the first few modes in the frequency table.

[0424] Figure 17 An example of using the frequency table and the non-MPM list to construct all the MPM modes is shown.

[0425] 5.3 Embodiment #3

[0426] Two sub-lists are maintained, named local and global sub-lists. The local sub-list contains some modes derived from the neighboring blocks, while the global sub-list corresponds to the ordered IPM table. Depending on certain conditions, it will be decided how many MPMs come from the local sub-list and how many MPMs come from the global sub-list. An example is shown in Figure 18 .

[0427] Figure 18 is a diagram of the MPM construction procedure using the local and global sub-lists.

[0428] InFigure 19A An example encoder codec flow is shown in FIG. 1. The construction strategy of the MPM list is replaced by our proposed method, which exploits both short- and long-range correlations to further improve the intra coding efficiency. In particular, the local and global sub-lists will be maintained separately following predefined principles. The local sub-list is adopted to reserve modes that can represent short-range correlations. Typically, the DC, planar, and neighboring CU’s modes are included in the local sub-list in a certain order. In addition, the global sub-list is also established to capture modes with long-range similarities by means of a frequency table. In this way, a conditional random field (CRF) based merging is performed to construct the final MPM list. Subsequently, RDO is performed and the best intra mode of the current block is used for CU coding.

[0429] 5.4 Embodiment #4

[0430] First, when a new video unit (e.g., slice or new CTU) is encountered, the table is reset with a series of fixed values. The mode set0 is used for initialization. In one example, the mode set0 includes two modes {VER, HOR} that can be assigned a higher initial frequency. N (e.g., 34 or 66) intra prediction modes are supported, and L (e.g., 2) MPMs are supported. More specifically, if the intra prediction mode M i is not in set0, the frequency F i (M n ) of the mode M i is reset to F n (M i ) = (N - M i ). Otherwise, the frequency F j (M n ) of the mode M j is reset to F n (M j ) = N + (L - idx(M j )), where idx(M j ) denotes the associated index in set0 (e.g., idx(VER) = 0, idx(HOR) = 1). In this way, i and j are in the range of 0 to N-1.

[0431] • In one example, it is assumed that the frequency of 34 modes is used for checking and updating. In the initialization / reset process, the following applies:

[0432] F0[VER] = 33 + (2 - 0), F0[HOR] = 33 + (2 - 1);

[0433] orderT[VER] = 0, orderT[HOR] = 1;

[0434] modeT[0] = VER, modeT[l] = HOR;

[0435] For the rest of the IPMs (excluding VER and HOR), the following applies:

[0436] - Set currOrder = 2;

[0437] - For modes 2 to 33, the following order applies:

[0438] F0[mode] = (33 - mode). Alternatively, F0[mode] = L.

[0439] orderT[mode] = currOrder. Alternatively, orderT[mode] = 2.

[0440] modeT[currOrder] = mode

[0441] currOrder++.

[0442] where VER and HOR are the vertical / horizontal intra prediction modes, respectively; freqT is the frequency table, F n denotes the state after the n-th update procedure, modeT is the ordered table of intra prediction modes, and orderT is used to derive the mapping index for a given intra prediction mode.

[0443] Second, after decoding a block containing intra information, the table is updated by recalculating the frequency of the associated intra mode. In particular, the intra information can be the intra mode in an intra coded block. As shown in Figure 22 , in particular, the frequency associated with the current intra mode (denoted by M i ) is adjusted to F n+1 (M i ) = F n (M i ) + δ (e.g., δ = 70 or 1).

[0444] Third, after the mode update, the frequency table is reordered in descending order according to the updated mode frequencies. In particular, both a comparison and a position swap of the updated mode and the previous mode frequencies are performed until the updated mode frequency is lower than the previous mode frequency. Alternatively, only the updated frequency can be compared with the frequencies associated with the top L modes in the ordered table.

[0445] For efficient implementation, an order table is employed to assist the update and ordering procedures. In particular, the order table records the relative mode order in the frequency table at the same time. M iThe order of the modes in the list can be used to obtain the corresponding frequencies. This way, the search time can be greatly reduced.

[0446] For the order list, only L (e.g., 2) candidates are involved in the sorting process. In particular, the current intra mode is compared with the first L modes in the order list.

[0447] Therefore, the table can be used in the most probable mode (MPM) list or the remaining mode (RM) list construction process. For the construction of the MPM list, the first (e.g., 2) modes in the frequency table can also be directly used to establish the MPM list. In particular, the modes in the MPM list can be sorted in ascending order according to the mode index. Subsequently, a redundancy check is applied to remove the same modes.

[0448] 5.5 Embodiment #5

[0449] 3. Terminology and Description

[0450] The following terms and definitions apply throughout this document.

[0451] Intra mode frequency table:

[0452] It is used for intra prediction and consists of the frequencies of the intra prediction modes of the prediction unit.

[0453] 4. Abbreviations

[0454] FIMC Frequency information based intra mode coding (frequency based intra mode coding)

[0455] 7.1.2.2 Sequence header description

[0456] For a description of the sequence header, see Table 14.

[0457] Table 14 Sequence header description

[0458]

[0459]

[0460] 7.2.2.2 Sequence header

[0461] Frequency intra mode coding enabled flag fimc_enable_flag

[0462] Binary variable. The value "1" indicates that frequency based intra mode coding can be used; the value "0" indicates that frequency based intra mode coding should not be used. The value of FimcEnableFlag is equal to the value of fimc_enable_flag. If fimc_enable_flag is not present in the bitstream, the value of FimcEnableFlag is equal to 0.

[0463] 9.4 Decoding of the largest coding unit

[0464] The largest coding units are decoded in the order of the raster scan order within the slice, and the decoding process is as follows:

[0465] - If the current largest coding unit is the first largest coding unit of the current line in the slice, the value of the candidate number CntHmvp in the history motion information table is initialized to 0

[0466] - In the following discussion, the symbol IpdCnt represents the number of intra prediction modes. When EIPM is off, the value of IpdCnt is set to 34; when EIPM is on, the value of IpdCnt is set to 66. x0 and y0 represent the horizontal and vertical positions. LcuSizeInBit is set to 8 by default, which represents log2(LcuSize). When the condition "((x0%(l « LcuSizeInBit))==0)&&((y0%(l « (LcuSizeInBit-1))==0)" is encountered while decoding the current coding unit, the value of the high frequency mode candidate number CntFimc in the intra mode frequency table is initialized to 2, and the values of the high frequency modes ModeFimc[0] and ModeFimc[l] in the intra mode frequency table are initialized to Intra_Luma_Vertical(12) and Intra_Luma_Horizontal(24), respectively, and the intra mode frequency table FimcFrequencyList is initialized according to the following steps.

[0467]

[0468]

[0469] Decode the coding tree of the current largest coding unit, and decode each coding unit of the coding tree in turn (see 9.5). After the decoding of the current largest coding unit is completed, update LcuIndex according to the following steps. After the update, if the value of LcuIndex / pictureWidthInLcu is greater than or equal to PatchBelowInLcu, end the decoding of all the largest coding units in the current tile.

[0470] 9.5 Decoding of the coding unit

[0471] 9.5.6.2 General intra prediction mode

[0472] The general intra prediction mode of each prediction block of the current coding unit is determined using the following method:

[0473] a) If the current prediction block E is a luma block

[0474] 1) If the value of FimcEnableFlag is 1, the prediction value for the prediction mode of the current prediction block is computed according to the following steps:

[0475] predIntraPredMode0 is equal to Min(ModeFimc[0], ModeFimc[1])

[0476] predIntraPredMode1 is equal to Max(ModeFimc[0], ModeFimc[1])

[0477] 2) If the value of FimcEnableFlag is 0, the prediction value for the prediction mode of the current prediction block is computed according to the following steps:

[0478] If the left prediction block A "exists" and is a regular intra prediction block, assign the IntraLumaPredMode of A to intraPredModeA; otherwise, intraPredModeA is equal to 0.

[0479] If the top prediction block B "exists" and is a regular intra prediction block, assign the IntraLumaPredMode of B to intraPredModeB; otherwise, intraPredModeB is equal to 0.

[0480] If intraPredModeA is not equal to intraPredModeB, predIntraPredMode0 is equal to the minimum value (intraPredModeA, intraPredModeB) and predIntraPredMode1 is equal to the maximum value (intraPredModeA, intraPredModeB); otherwise:

[0481] 1. If intraPredModeA is equal to 0, predIntraPredMode0 is equal to 0 and predIntraPredMode1 is equal to 2.

[0482] 2. If intraPredModeA is not equal to 0, predIntraPredMode0 is equal to 0 and predIntraPredMode1 is equal to intraPredModeA.

[0483] 3) If the value of intra_luma_pred_mode is 0, IntraLumaPredMode is equal to predIntraPredModeO; else if the value of intra_luma_pred_mode is 1, IntraLumaPredMode is equal to predIntraPredMode1; else:

[0484] ♦ If the value of intra_luma_pred_mode minus 2 is less than predIntraPredModeO, IntraLumaPredMode is equal to intra_luma_pred_mode minus 2;

[0485] ♦ Else if the value of intra_luma_pred_mode minus 1 is greater than predIntraPredModeO and less than predIntraPredMode1, IntraLumaPredMode is equal to intra_luma_pred_mode minus 1;

[0486] ♦ Else, IntraLumaPredMode is equal to intra_luma_pred_mode.

[0487] 4) If the value of FimcEnableFlag is 1, the intra mode frequency table is updated using the luma intra prediction mode IntraLumaPredMode of the current prediction unit according to the method defined in 9.20.

[0488] b) If the current prediction block E is a chroma block:

[0489] 1) If the luma prediction mode IntraLumaPredMode of the prediction block with the value of 0 of PredBlockOrder in the current coding unit is equal to 0, 2, 12 or 24, isRedundant is equal to 1; otherwise, isRedundant is equal to 0.

[0490] 2) If the value of tscpm_enable_flag is equal to '1' and the value of intra_chroma_pred_mode is equal to 1, IntraChromaPredMode is equal to 5;

[0491] 3) Otherwise,

[0492] • If the value of tscpm_enable_flag is equal to '1' and the value of intra_chroma_pred_mode is not equal to 0, the value of intra_chroma_pred_mode is decremented by 1;

[0493] • If isRedundant is equal to 0, IntraChromaPredMode is equal to intra_chroma_pred_mode; otherwise, the following operations are performed in this order:

[0494] • If IntraLumaPredMode is equal to 0, predIntraChromaPredMode is equal to 1; if IntraLumaPredMode is equal to 2, predIntraChromaPredMode is equal to 4; if IntraLumaPredMode is equal to 12, predIntraChromaPredMode is equal to 3; if IntraLumaPredMode is equal to 24, predIntraChromaPredMode is equal to 2.

[0495] • If the value of intra_chroma_pred_mode is equal to 0, IntraChromaPredMode is equal to 0; otherwise, if the value of intra_chroma_pred_mode is less than predIntraChromaPredMode, IntraChromaPredMode is equal to intra_chroma_pred_mode; otherwise, IntraChromaPredMode is equal to intra_chroma_pred_mode plus 1.

[0496] c) Depending on the value of IntraLumaPredMode, look-up table 79 to obtain the intra prediction mode for the luma prediction block. Depending on the value of IntraChromaPredMode, look-up table 80 to obtain the intra prediction mode for the chroma prediction block.

[0497] 9.20 Updating the frequency table of intra modes

[0498] After the decoding of the current prediction unit is completed, if the current prediction unit is an intra prediction unit other than a block copy intra prediction unit, the intra mode frequency table FimcFrequencyList is updated according to the luma intra prediction mode IntraLumaPredMode of the current prediction block and the high frequency mode when FimcEnableFlag is equal to 1; otherwise, the operation defined in this document will not be performed.

[0499] a) add 70 in FimcFrequencyList[IntraLumaPredMode];

[0500] b) let FreqCurr be equal to FimcFrequencyList[IntraLumaPredMode];

[0501] c) let ModeO be equal to ModeFimc[0] and FreqO be equal to FimcFrequencyList[ModeO];

[0502] d) let Mode1 be equal to ModeFimc[1] and Freq1 be equal to FimcFrequencyList[Mode1];

[0503] e) if FreqCurr is greater than or equal to FreqO, then ModeFimc[0] is equal to IntraLumaPredMode;

[0504] 1) if ModeO is not equal to IntraLumaPredMode, then ModeFimc[1] is equal to ModeO.

[0505] 2) else, ModeFimc[1] is equal to Mode1.

[0506] f) else, if FreqCurr is greater than or equal to Freq1, then ModeFimc[1] is equal to IntraLumaPredMode.

[0507] 1) else, ModeFimc will not be updated.

[0508] Figure 20 is a block diagram of a video processing device 2000. The device 2000 can be used to implement one or more of the methods described herein. The device 2000 can be embodied as a smartphone, a tablet computer, a computer, an Internet of Things (IoT) receiver, etc. The device 2000 can include one or more processors 2002, one or more memories 2004, and video processing hardware 2006. The processor(s) 2002 can be configured to implement one or more methods described herein. The memory(ies) 2004 can be used for storing data and code used for implementing the methods and techniques described herein. The video processing hardware 2006 can be used to implement, in hardware circuitry, some of the techniques described herein. In some embodiments, the hardware 2006 can be at least partially in a processor, for example in a graphics co-processor.

[0509] In some embodiments, the following solutions can be implemented as preferred solutions.

[0510] The following solutions can be implemented together with the additional techniques described in the items listed in the previous section (e.g., items 1, 3, 4, and 5).

[0511] 1. A method of video processing (e.g., method 2100 shown in FIG. 21), comprising: performing (2102) a conversion between a video unit of a video and a coded representation of the video unit using one or more frequency tables, wherein the one or more frequency tables comprise information about frequencies of intra prediction modes used in the conversion for the video; and selectively updating (2104), due to the conversion, the one or more frequency tables based on a coding mode of the video unit. Figure 21

[0512] 2. The method of solution 1, further comprising determining one or more ordered intra prediction mode (IPM) tables that store intra prediction modes in order of frequencies in the one or more frequency tables.

[0513] 3. The method of any of solutions 1-2, wherein selectively updating comprises updating the one or more frequency tables if an intra prediction mode is used during the conversion of the video unit.

[0514] 4. The method of any of solutions 1-2, wherein selectively updating comprises refraining from updating the one or more frequency tables if an intra prediction mode is not used during the conversion of the video unit.

[0515] 5. The method of any of solutions 1-4, wherein the one or more frequency tables comprise tables for multiple types of video units.

[0516] 6. The method of any of solutions 1-5, wherein the video unit is a prediction unit (PU).

[0517] 7. The method of any of solutions 1-5, wherein the video unit is a coding unit (CU).

[0518] 8. The method of solution 1, wherein for the conversion, a most probable coding mode (MPM) list is constructed using the one or more frequency tables and / or the one or more IPM tables.

[0519] 9. The method of solution 8, wherein the MPM coding list can be constructed using N entries from the one or more ordered IPM tables with the highest frequencies.

[0520] ​10. The method of any of solutions 1-9, wherein the coding mode for the conversion of the video unit is derived based on one or more frequency tables, one or more IPM tables, and non-table based operations.

[0521] 11. The method of solution 10, wherein the coding mode is derived by first generating an MPM coding list.

[0522] 12. The method of solution 10, wherein the MPM coding list is constructed using one or more IPM tables and non-table based operations, and the coding mode is determined from the MPM list.

[0523] 13. The method of solution 8, wherein the coding mode for the conversion of the video unit is derived from one or more IPM tables and the remaining modes not in the MPM coding list.

[0524] The following solutions can be implemented with the additional techniques described in the items listed in the previous section (e.g., items 9 and 10).

[0525] 14. The method of solution 3, wherein the intra prediction mode is a general intra prediction mode.

[0526] 15. The method of solution 2, wherein one or more frequency tables and one or more IPM tables are selectively updated depending on whether an intra mode is used for the conversion.

[0527] 16. The method of solution 2, wherein one or more frequency tables and one or more IPM tables are selectively updated due to a conversion using a matrix-based intra prediction (MIP) mode.

[0528] 17. The method of solution 2, wherein one or more frequency tables and one or more IPM tables are not updated due to a conversion using a matrix-based intra prediction (MIP) mode.

[0529] 18. The method of solution 2, wherein one or more frequency tables and one or more IPM tables are not updated due to a conversion using an intra sub-partition (ISP) mode.

[0530] 19. The method of solution 2, wherein one or more frequency tables and one or more IPM tables are selectively updated due to a conversion using a non-intra mode.

[0531] 20. The method of solution 19, wherein the non-intra mode is an inter mode or an intra block copy mode or a palette mode.

[0532] The following solutions can be implemented together with the additional techniques described in the items listed in the previous section (e.g., item 11).

[0533] 21. The method of solution 3, wherein the intra prediction mode used during the conversion is M i , and wherein the corresponding frequency before the n-th update is denoted by F n (M i ), where n starts from 1 and then for M i , F n (M i ) is updated to F n+1 (M i ), where F n+1 (M i ) = F n (M i ) + K, where K is an integer.

[0534] 22. The method of solution 21, wherein K = 1.

[0535] 23. The method of solution 21, wherein K is a function of the type of coding unit or intra prediction mode.

[0536] The following solutions can be implemented together with the additional techniques described in the items listed in the previous section (e.g., item 12).

[0537] 24. The method of solution 2, wherein the one or more sorted IMP tables are sorted according to a sorting rule.

[0538] 25. The method of solution 24, wherein the sorting rule specifies to sort in descending order after the update.

[0539] 26. The method of solution 24, wherein the sorting rule specifies to terminate the sorting when an entry satisfying the condition is found.

[0540] The following solutions can be implemented together with the additional techniques described in the items listed in the previous section (e.g., item 2).

[0541] 27. A method of video processing, comprising: performing a conversion between a video unit of a video and a coded representation of the video unit using a frequency table, wherein the frequency table comprises a plurality of entries each representative of a frequency of occurrence of a corresponding intra coding mode in the conversion; and selectively updating the frequency table based on coded information of the video unit with the conversion.

[0542] 28. The method of solution 27, further comprising maintaining a sorted intra- prediction mode (IPM) table comprising M entries associated with corresponding M entries in the frequency table.

[0543] 29. The method of any of solutions 27-28, wherein the M entries are grouped into N categories, wherein N is less than M.

[0544] 30. The method of solution 27, wherein the frequency table comprises N entries, the method further comprising:

[0545] maintaining a sorted intra-prediction mode (IPM) table comprising M entries, wherein M is greater than N, and wherein the N entries of the frequency table are associated with corresponding N entries in the IPM table.

[0546] The following solutions can be implemented together with the additional techniques described in the items listed in the previous section (e.g., items 7 and 8).

[0547] 31. A video processing method comprising: performing a conversion between a current video unit and a next video unit of a video and a coded representation using one or more frequency tables and / or using one or more intra-prediction mode tables ordered according to frequencies indicated in the one or more frequency tables; wherein the one or more frequency tables comprise information about frequencies of intra-prediction modes used in the conversion; and wherein the one or more frequency tables and / or the one or more intra-prediction mode tables are reset or initialized between a use in the conversion of the current video unit and a use in the conversion of the next video unit.

[0548] 32. The method of solution 31, wherein the current video unit and / or the next video unit is a sub-region of a coding tree unit, or a coding tree unit, or a coding tree block, or a plurality of coding tree units, or a plurality of coding units of a line of coding tree units, or a slice, or a tile, or a slice, or a picture, or a sub-picture of the video.

[0549] 33. The method of any of solutions 31-32, wherein the order of frequencies is an ascending order of frequencies.

[0550] 34. The method of any of solutions 31-33, wherein the resetting or initializing of the one or more intra-prediction mode tables comprises switching between an ascending order and a descending order.

[0551] 35. The method of any of solutions 31-34, wherein the resetting or initializing of the one or more frequency tables comprises setting entries of the one or more frequency tables to a predetermined value.

[0552] 36. The method of solution 35, wherein the predetermined values are equal to each other.

[0553] 37. The method of solution 35, wherein the predetermined values are different from each other.

[0554] 38. The method of any of solutions 35-37, wherein the predetermined values are defined in multiple sets, and a particular set is selected based on the codec information used in the conversion.

[0555] The following solutions can be implemented together with the additional techniques described in the items listed in the previous section (e.g., item 13).

[0556] 39. A method of video processing, comprising: performing a conversion between a video unit of a video and a coded representation of the video unit using one or more frequency tables, wherein the one or more frequency tables comprise information on frequencies of intra prediction modes used in the conversion of the video and side information on occurrences of the intra prediction modes.

[0557] 40. The method of solution 39, wherein the side information comprises a starting position or a block size of the intra prediction mode occurrences.

[0558] 41. The method of any of solutions 39-40, wherein the one or more frequency tables and / or the side information are updated at the conversion of the video unit.

[0559] 42. The method of any of solutions 39-41, wherein the conversion is performed by first sorting the one or more frequency tables according to a starting position.

[0560] The following solutions can be implemented together with the additional techniques described in the items listed in the previous section (e.g., items 14 and 15).

[0561] 43. The method of any of solutions 1-42, wherein a field in the coded representation indicates that the method is used during the conversion.

[0562] 44. The method of any of solutions 1-42, wherein the video unit corresponds to a tile, a brick, a slice, a picture, a subpicture, a sequence, or a view of the video.

[0563] 45. The method of any of solutions 1-42, wherein the method is applied to the conversion because the video unit satisfies a criterion.

[0564] 46. The method of solution 3, wherein the criteria comprise a size of the video unit, or a slice type, or a picture type, or a temporal layer index, or a content of the video.

[0565] 47. The method of any of solutions 1 to 46, wherein the one or more frequencies are associated with a particular type of video unit.

[0566] 48. The method of any of solutions 1 to 46, wherein one frequency table is associated with a particular type of video unit; and different frequency tables are associated with different particular types of video units.

[0567] 49. The method of any of solutions 47-48, wherein the particular type corresponds to a particular block dimension.

[0568] 50. The method of any of solutions 47-48, wherein the particular type corresponds to a particular coding method used during conversion.

[0569] The following solutions can be implemented together with the additional techniques described in the items (e.g., item 6) listed in the previous section.

[0570] 51. A video processing method, comprising: for a conversion between a video unit of a video and a coded representation of the video, determining an intra prediction mode for the conversion; and performing the conversion based on the intra prediction mode; wherein the intra prediction mode is signaled in the coded representation with a syntax element.

[0571] 52. The method of solution 51, wherein a value of the syntax element is changeable for the video unit to another video unit.

[0572] 53. The method of any of solutions 51-52, wherein the intra prediction mode is based on a history or frequency information of previous intra prediction modes during the conversion.

[0573] 54. The method of any of solutions 51-53, wherein the syntax element codes an index of an intra prediction mode table.

[0574] 55. The method of solution 54, wherein the index is based on the intra prediction mode table arranged in descending order.

[0575] 56. The method of any of solutions 1-55, wherein the video unit corresponds to a video coding block.

[0576] 57. The method of any of solutions 1 to 56, wherein the conversion comprises encoding the video into the coded representation.

[0577] 58. The method of any of solutions 1 to 56, wherein the converting comprises decoding the coded representation to generate pixel values of the video.

[0578] 59. A video decoding device comprising a processor configured to implement a method recited in one or more of solutions 1 to 58.

[0579] 60. A video encoding device comprising a processor configured to implement a method recited in one or more of solutions 1 to 58.

[0580] 61. A computer program product having computer code stored thereon, the code, when executed by a processor, causing the processor to implement a method recited in any of solutions 1 to 58.

[0581] 62. The method, apparatus or system described in this document.

[0582] Figure 26 is a block diagram illustrating an example video processing system 2600 in which various techniques disclosed herein can be implemented. Various implementations can include some or all of the components of system 2600. System 2600 can include an input 2602 for receiving video content. The video content can be received in a raw or uncompressed format, e.g., 8-bit or 10-bit multi-component pixel values, or can be received in a compressed or coded format. Input 2602 can represent a network interface, a peripheral bus interface, or a storage interface. Examples of network interfaces include wired interfaces such as Ethernet, passive optical networks (PONs), etc., and wireless interfaces such as Wi-Fi or cellular interfaces.

[0583] System 2600 can include a coding component 2604, which can implement various encoding or decoding methods described herein. Coding component 2604 can reduce the average bitrate of video from input 2602 to the output of coding component 2604, to produce a coded representation of the video. Coding techniques are thus sometimes referred to as video compression or video transcoding techniques. The output of coding component 2604 can be stored, or transmitted via a communication connected, as shown by component 2606. The stored or communicated bitstream (or coded) representation of the video received at input 2602 can be used by component 2608 to generate pixel values or a displayable video sent to a display interface 2610. The process of generating a user-viewable video from a bitstream representation is sometimes referred to as video decompression. Moreover, although certain video processing operations are referred to as “coding” operations or tools, it will be appreciated that coding tools or operations are used at an encoder, and corresponding decoding tools or operations that reverse the results of the coding will be performed by a decoder.

[0584] Examples of peripheral bus interfaces or display interfaces may include Universal Serial Bus (USB), High-Definition Multimedia Interface (HDMI), or DisplayPort, etc. Examples of storage interfaces include SATA (Serial Advanced Technology Attachment), PCI, IDE interfaces, etc. The technology described herein may be implemented in various electronic devices, such as mobile phones, laptop computers, smartphones, or other devices capable of performing digital data processing and / or video display.

[0585] Some embodiments of the disclosed technology include making a decision or determination to enable a video processing tool or mode. In one example, when a video processing tool or mode is enabled, the encoder will use or implement the tool or mode in the processing of the video block, but will not necessarily modify the generated bitstream based on the use of the tool or mode. That is, the conversion from the video block to the bitstream representation of the video will use the video processing tool or mode when the video processing tool or mode is enabled based on the decision or determination. In another example, when the video processing tool or mode is enabled, the decoder processes the bitstream in the case of knowing that the bitstream is modified based on the video processing tool or mode. That is, the conversion from the bitstream representation of the video to the video block will be performed using the video processing tool or mode enabled based on the decision or determination.

[0586] Some embodiments of the disclosed technology include making a decision or determination to disable a video processing tool or mode. In one example, when the video processing tool or mode is disabled, the encoder does not use the tool or mode in converting video blocks to a bitstream representation of the video. In another example, when the video processing tool or mode is disabled, the decoder processes the bitstream knowing that the video processing tool or mode enabled based on the decision or determination has not been used to modify the bitstream.

[0587] Figure 27 is a block diagram illustrating an example video encoding and decoding system 100 that may utilize the techniques of the present invention. Figure 27 As shown, the video codec system 100 may include a source device 110 and a target device 120. The source device 110 generates encoded video data and may be referred to as a video encoding device. The target device 120 may decode the encoded video data generated by the source device 110 and may be referred to as a video decoding device. The source device 110 may include a video source 112, a video encoder 114, and an input / output (I / O) interface 116.

[0588] Video source 112 can include a source such as a video capture device, an interface to receive video data from a video content provider, and / or a computer graphics system for generating video data, or a combination thereof. The video data can comprise one or more pictures. Video encoder 114 encodes video data from video source 112 to generate a bitstream. The bitstream can include a sequence of bits that form a coded representation of the video data. The bitstream can include coded pictures and associated data. A coded picture is a coded representation of a picture. The associated data can include sequence parameter sets, picture parameter sets, and other syntax structures. I / O interface 116 can include a modulator / demodulator (modem) and / or a transmitter. The encoded video data can be transmitted directly to destination device 120 via I / O interface 116 by network 130a. The encoded video data can also be stored onto a storage medium / server 130b for access by target device 120.

[0589] Destination device 120 can include I / O interface 126, video decoder 124, and display device 122.

[0590] I / O interface 126 can include a receiver and / or a modem. I / O interface 126 can acquire encoded video data from source device 110 or storage medium / server 130b. Video decoder 124 can decode the encoded video data. Display device 122 can display the decoded video data to a user. Display device 122 can be integrated with destination device 120, or can be external to destination device 120 which is configured to interface with an external display device.

[0591] Video encoder 114 and video decoder 124 can operate according to a video compression standard, such as High Efficiency Video Coding (HEVC) standard, Versatile Video Coding (VVM) standard, and other current and / or further standards.

[0592] Figure 28 is a block diagram illustrating an example of a video encoder 200 that can be Figure 27 video encoder 114 in system 100 shown in FIG. 1.

[0593] Video encoder 200 can be configured to perform any or all of the techniques of this disclosure. In Figure 28 example, video encoder 200 includes a plurality of functional components. The techniques described in this disclosure can be shared between the components of video encoder 200. In some examples, a processor can be configured to perform any or all of the techniques described in this disclosure.

[0594] The functional components of video encoder 200 can include partition unit 201, prediction unit 202 which can include mode select unit 203, motion estimation unit 204, motion compensation unit 205, and intra-prediction unit 206, residual generation unit 207, transform unit 208, quantization unit 209, inverse quantization unit 210, inverse transform unit 211, reconstruction unit 212, buffer 213, and entropy encoding unit 214.

[0595] In other examples, video encoder 200 can include more, less, or different functional components. In examples, prediction unit 202 can include an intra-block copy (IBC) unit. The IBC unit can perform prediction in an IBC mode in which at least one reference picture is the picture in which the current video block is located.

[0596] Furthermore, some components, such as motion estimation unit 204 and motion compensation unit 205, can be highly integrated, but are shown separately for illustrative purposes. Figure 28

[0597] Partition unit 201 can partition a picture into one or more video blocks. Video encoder 200 and video decoder 300 can support various video block sizes.

[0598] Mode select unit 203 can select one of the coding modes, intra or inter, e.g., based on the error results, and provide the resulting intra or inter coded block to residual generation unit 207 to generate residual block data, and to reconstruction unit 212 for reconstructing the coded block for use as a reference picture. In some examples, mode select unit 203 can select a combination of intra prediction and inter prediction (CIIP) mode, where the prediction is based on both an inter prediction signal and an intra prediction signal. In the case of inter prediction, mode select unit 203 can also select a resolution for the motion vectors (e.g., sub-pixel or integer pixel precision) for the block.

[0599] To perform inter prediction for a current video block, motion estimation unit 204 can generate motion information for the current video block by comparing one or more reference frames from buffer 213 to the current video block. Motion compensation unit 205 can determine a predicted video block for the current video block based on the motion information and decoded samples of pictures from buffer 213, rather than based on the picture with which the current video block is associated.

[0600] Motion estimation unit 204 and motion compensation unit 205 can perform different operations, e.g., depending on whether the current video block is in an I slice, a P slice, or a B slice.

[0601] ​In some examples, the motion estimation unit 204 can perform uni-prediction for the current video block and the motion estimation unit 204 can search for a reference picture of List 0 or List 1 for a reference video block of the current video block. The motion estimation unit 204 can then generate a reference index indicating the reference picture containing the reference video block in List 0 or List 1 and a motion vector indicating a spatial displacement between the current video block and the reference video block. The motion estimation unit 204 can output the reference index, the prediction direction indicator, and the motion vector as the motion information for the current video block. The motion compensation unit 205 can generate a predicted video block for the current block based on the reference video block indicated by the motion information for the current video block.

[0602] In other examples, the motion estimation unit 204 can perform bi-prediction for the current video block, the motion estimation unit 204 can search for a reference picture of List 0 for a reference video block of the current video block and can also search for a reference picture of List 1 for another reference video block of the current video block. The motion estimation unit 204 can then generate a reference index indicating the reference picture containing the reference video block in List 0 or List 1 and a motion vector indicating a spatial displacement between the reference video block and the current video block. The motion estimation unit 204 can output the reference index and the motion vector for the current video block as the motion information for the current video block. The motion compensation unit 205 can generate a predicted video block for the current block based on the reference video block indicated by the motion information for the current video block.

[0603] In some examples, the motion estimation unit 204 can output a full set of motion information for decoding processing at the decoder.

[0604] In some examples, the motion estimation unit 204 can not output a full set of motion information for the current video. Instead, the motion estimation unit 204 can signal the motion information for the current video block with reference to the motion information of another video block. For example, the motion estimation unit 204 can determine that the motion information for the current video block is sufficiently similar to the motion information of a neighboring video block.

[0605] In one example, the motion estimation unit 204 can indicate a value in a syntax structure associated with the current video block that indicates to the video decoder 300 that the current video block has the same motion information as another video block.

[0606] In another example, the motion estimation unit 204 can identify another video block and a motion vector difference (MVD) in a syntax structure associated with the current video block. The motion vector difference indicates a difference between the motion vector of the current video block and the motion vector of the indicated video block. The video decoder 300 can use the motion vector of the indicated video block and the motion vector difference to determine the motion vector of the current video block.

[0607] As described above, video encoder 200 can predictively signal motion vectors. Two examples of prediction signaling techniques that can be implemented by video encoder 200 include advanced motion vector prediction (AMVP) and merge mode signaling.

[0608] Intra prediction unit 206 can perform intra prediction on a current video block. When intra prediction unit 206 performs intra prediction on the current video block, intra prediction unit 206 can generate prediction data for the current video block based on decoded samples of other video blocks in the same picture. The prediction data for the current video block can include a predicted video block and various syntax elements.

[0609] Residual generation unit 207 can generate residual data for the current video block by subtracting (e.g., represented by a negative sign) the predicted video block of the current video block from the current video block. The residual data for the current video block can include a residual video block that corresponds to different sample components of samples in the current video block.

[0610] In other examples, for a current video block, there can be no residual data for the current video block, e.g., in skip mode, and residual generation unit 207 can not perform the subtraction operation.

[0611] Transform processing unit 208 can generate one or more transform coefficient video blocks for the current video block by applying one or more transforms to the residual video block associated with the current video block.

[0612] After transform processing unit 208 generates a transform coefficient video block associated with the current video block, quantization unit 209 can quantize the transform coefficient video block associated with the current video block based on one or more quantization parameter (QP) values associated with the current video block.

[0613] Inverse quantization unit 210 and inverse transform unit 211 can apply inverse quantization and inverse transform, respectively, to a transform coefficient video block to reconstruct a residual video block from the transform coefficient video block. Reconstruction unit 212 can add the reconstructed residual video block to corresponding samples of one or more predicted video blocks generated by prediction unit 202 to produce a reconstructed video block associated with the current block for storage in buffer 213.

[0614] After reconstruction unit 212 reconstructs a video block, in-loop filtering operations can be performed to reduce video blocky artifacts in the video block.

[0615] Entropy encoding unit 214 can receive data from other functional components of video encoder 200. When entropy encoding unit 214 receives data, entropy encoding unit 214 can perform one or more entropy encoding operations to generate entropy encoded data and output a bitstream that includes the entropy encoded data.

[0616] Figure 29 is a block diagram illustrating an example of a video decoder 300 that can be Figure 27 the video decoder 114 in the system 100 shown.

[0617] The video decoder 300 can be configured to perform any or all of the techniques of this disclosure. In Figure 29 example, the video decoder 300 includes a plurality of functional components. The techniques described in this disclosure can be shared among the various components of the video decoder 300. In some examples, a processor can be configured to perform any or all of the techniques described in this disclosure.

[0618] In Figure 29 example, the video decoder 300 includes an entropy decoding unit 301, a motion compensation unit 302, an intra prediction unit 303, an inverse quantization unit 304, an inverse transformation unit 305, and a reconstruction unit 306 and a buffer 307. In some examples, the video decoder 300 can perform a decoding process generally reciprocal to the encoding process described with respect to the video encoder 200 Figure 28 ).

[0619] The entropy decoding unit 301 can obtain an encoded bitstream. The encoded bitstream can include entropy encoded video data (e.g., encoded blocks of video data). The entropy decoding unit 301 can decode the entropy encoded video data and the motion compensation unit 302 can determine motion information, including motion vectors, motion vector precision, reference picture list indices, and other motion information, from the entropy decoded video data. The motion compensation unit 302 can determine such information, for example, by performing AMVP and merge mode.

[0620] The motion compensation unit 302 can generate a motion compensated block, possibly performing interpolation based on an interpolation filter. An identifier of the interpolation filter used at sub-pixel precision can be included in the syntax elements.

[0621] The motion compensation unit 302 can use the interpolation filter used by the video encoder 200 during coding of the video block to calculate the interpolation of sub-integer pixels of the reference block. The motion compensation unit 302 can determine the interpolation filter used by the video encoder 200 from the received syntax information and use the interpolation filter to generate the prediction block.

[0622] Motion compensation unit 302 can use some of the syntax information to determine the size of the blocks used to encode the frame(s) and / or slice(s) of the coded video sequence, partitioning information that describes how each macroblock of a picture of the coded video sequence is partitioned, the mode that indicates how each partition is coded, one or more reference frames (and reference frame lists) for inter-coded blocks, and other information for decoding the coded video sequence.

[0623] Intra prediction unit 303 can use, for example, intra prediction modes received in the bitstream to form a predicted block from spatially neighboring blocks. Inverse quantization unit 303 inverse quantizes, i.e., de-quantizes, quantized video block coefficients provided in the bitstream and decoded by entropy decoding unit 301. Inverse transform unit 303 performs an inverse transform.

[0624] Reconstruction unit 306 can add the residual block to the corresponding predicted block, generated by motion compensation unit 202 or intra prediction unit 303, to form a decoded block. If desired, a deblocking filter can also be applied to the decoded block to remove blocking artifacts. The decoded video blocks are then stored in buffer 307, which provides reference blocks for subsequent motion compensation / intra prediction, and also produces decoded video for display on a display device.

[0625] Figure 23 An example method 2300 of video processing is shown. The method 2300 includes performing (2302) a conversion between a video comprising video units and a coded representation of the video, wherein, when processing a video unit in the conversion, one or more frequency tables are selectively updated to include information about frequencies of one or more intra prediction modes of the video unit used in the processing, wherein the frequencies indicate occurrences of the one or more intra prediction modes used for the conversion, and wherein, when processing the video unit, one or more ordered intra prediction mode (IPM) tables are selectively updated to indicate the one or more intra prediction modes used in the processing.

[0626] Figure 24An example method 2400 for video processing is shown. The method 2400 includes performing (2402) a conversion between a video unit of a video and a codec representation of the video using one or more frequency tables or one or more sorted intra-prediction mode (IPM) tables, wherein the one or more frequency tables include information about a frequency of one or more intra-prediction modes used in the video conversion, wherein the frequency indicates an occurrence of the one or more intra-prediction modes used for the conversion, wherein the one or more sorted IPM tables indicate the one or more intra-prediction modes in a sorted order, wherein the one or more frequency tables or the one or more sorted IPM tables are used for intra-mode encoding and decoding in constructing a most probable mode (MPM) list of size N, where N is an integer.

[0627] Figure 25 An example method 2500 of video processing is shown. The method 2500 includes performing (2502) a conversion between a video block of a video and a codec representation of the video, wherein the codec representation includes a syntax element indicating a selected intra-prediction mode for the conversion, wherein the codec representation does not include one or more syntax elements indicating a most probable mode (MPM), or an index into an MPM list, or intra-prediction modes other than the intra-prediction modes included in the MPM list, and wherein the selected intra-prediction mode is based on history information indicating a frequency of one or more intra-prediction modes used in another conversion between one or more video blocks of the video and the video prior to the conversion of the video block.

[0628] The following sections describe example video processing techniques:

[0629] Example 1. A video processing method, comprising:

[0630] Perform conversions between the video containing the video unit and a codec representation of that video,

[0631] wherein, when processing a video unit in the conversion, one or more frequency tables are selectively updated to include information regarding the frequency of one or more intra prediction modes for the video unit used in the processing,

[0632] wherein the frequency indicates the occurrence of one or more intra prediction modes for conversion, and

[0633] Wherein, when processing the video unit, one or more sorted intra-frame prediction mode (IPM) tables are selectively updated to indicate one or more intra-frame prediction modes used in the processing.

[0634] Example 2. The method of example 1, wherein the one or more sorted IPM tables are associated with one or more frequency tables.

[0635] Example 3. The method of any of examples 1-2, wherein when the one or more frequency tables are updated, the one or more ordered IPM tables are updated accordingly.

[0636] Example 4. The method of any of examples 1-2, wherein the one or more ordered IPM tables are not updated due to entries in the one or more frequency tables including intra prediction modes and occurrences or frequencies of the intra prediction modes.

[0637] Example 5. The method of any of examples 1-2, wherein a kth entry of the one or more ordered IPM tables represents an intra prediction mode that has historically been the kth most frequently used.

[0638] Example 6. The method of example 1, wherein the video unit comprises a sub-region of a coding tree unit (CTU).

[0639] Example 7. The method of example 6, wherein the sub-region of the CTU comprises a virtual pipeline data unit (VPDU), another CTU, a coding tree block (CTB), a plurality of CTUs, a plurality of coding units (CUs), a CTU row, a slice, a tile, a slice, a picture, or a sub-picture.

[0640] Example 8. The method of example 1, wherein the one or more frequency tables are associated with an IPM order mapping table.

[0641] Example 9. The method of example 8, wherein a kth entry of the IPM order mapping table represents an order index of an intra prediction mode having an index equal to k.

[0642] Example 10. The method of example 1, wherein the video unit is a prediction unit (PU).

[0643] Example 11. The method of example 1, wherein the video unit is a coding unit (CU).

[0644] Example 12. The method of example 1, wherein the one or more frequency tables and the one or more ordered IPM tables are used for another conversion between one or more additional video units of the video and the coded representation of the video, wherein the video unit precedes the one or more additional video units in time.

[0645] Example 13. The method of example 1, wherein values of intra prediction modes in the ordered IPM table are associated with index values that are the same as index values of frequencies of the intra prediction modes in the frequency table.

[0646] Example 14. The method of example 1, wherein one or more frequency tables and one or more ordered IPM tables are updated for each block type of the video.

[0647] Example 15. The method of example 1, wherein one frequency table from one or more frequency tables and one or more ordered IPM tables are updated for a plurality of block types of the video.

[0648] Example 16. The method of any of examples 14-15, wherein the block types comprise blocks having a same width and / or a same height.

[0649] Example 17. The method of example 1, wherein, for an index value equal to k:

[0650] a first value modeT[k] is an intra prediction mode associated with the index value equal to k in the ordered IPM table,

[0651] a second value orderT[k] is a mapping index of the intra prediction mode associated with the index value equal to k after ordering the intra prediction modes in the IPM order mapping table, and

[0652] a third value freqT[k] is a frequency of the intra prediction mode associated with the index value equal to k in the frequency table.

[0653] Example 18. The method of example 17, wherein orderT[modeT[k]] = k, where k represents the order index.

[0654] Example 19. The method of example 17, wherein modeT[orderT[m]] = m, where m represents the intra prediction mode.

[0655] Example 20. The method of example 17, wherein freqT[modeT[k]] > = freqT[modeT[k+1]].

[0656] Example 21. The method of example 1, wherein the frequency table from one or more frequency tables is associated with M entries, where M is an integer, and wherein each entry is associated with a frequency of one of the M allowed intra prediction modes.

[0657] Example 22. The method of example 21, wherein the ordered IPM table from one or more ordered IPM tables comprises the same number of entries as the number of entries in the frequency table associated with the ordered IPM table.

[0658] Example 23. The method of example 21, wherein the allowed M intra prediction modes are grouped into N categories, where N is an integer.

[0659] Example 24. The method of example 23, wherein the frequency table includes N entries, where N is an integer that is less than M, and wherein each entry in the frequency table is associated with a frequency of one category corresponding to the one or more intra prediction modes.

[0660] Example 25. The method of example 23, wherein one or more ordered IPM tables are associated with the frequency table, and wherein the one or more ordered IPM tables include N entries with ordered category indices.

[0661] Example 26. The method of example 21, wherein the frequency table includes N entries, where N is an integer that is less than M, and wherein the N entries correspond to N selected intra prediction modes from the allowed M intra prediction modes.

[0662] Example 27. The method of example 26, wherein the allowed M intra prediction modes do not include a wide-angle intra prediction mode.

[0663] Example 28. The method of example 26, wherein the N selected intra prediction modes include at least one of a direct current (DC) mode, a planar mode, a horizontal mode, a vertical mode, or a bilinear intra prediction mode.

[0664] Example 29. The method of example 26, wherein the N selected intra prediction modes are predefined or signaled or are derived from coding information of a video unit.

[0665] Example 30. The method of example 29, wherein the coding information indicates whether the video unit includes screen content.

[0666] Example 31. The method of example 21, wherein the frequency table includes N entries, where N is an integer that is less than M, wherein the N entries correspond to N intra prediction modes, and wherein the frequency table including the N entries is updated during a conversion based on decoding information of a video unit.

[0667] Example 32. The method of example 31, wherein the N intra prediction modes are first initialized and then updated based on the decoding information of the video unit.

[0668] Example 33. A video processing method, comprising:

[0669] using one or more frequency tables or one or more ordered intra prediction mode (IPM) tables to perform a conversion between a video unit of a video and a coded representation of the video,

[0670] wherein the one or more frequency tables comprise information about frequencies of one or more intra prediction modes used in the conversion of the video,

[0671] wherein the frequency indicates occurrences of the one or more intra prediction modes used for the conversion,

[0672] wherein the one or more ordered IPM tables indicate the one or more intra prediction modes in an order of the ordering,

[0673] wherein the one or more frequency tables or the one or more ordered IPM tables are used for intra mode coding in a process of building a most probable mode (MPM) list of size N, and

[0674] wherein N is an integer.

[0675] Example 34. The method according to example 33, wherein all MPM lists are determined from the one or more ordered IPM tables.

[0676] Example 35. The method according to example 34, wherein the one or more intra prediction modes comprise a top N intra prediction modes associated with highest frequencies, and wherein the top N intra prediction modes are used as inputs for the MPM lists.

[0677] Example 36. The method according to example 33, wherein the one or more frequency tables and the one or more ordered IPM tables are used for intra mode coding.

[0678] Example 37. The method according to example 36, wherein all MPM lists are determined from the one or more ordered IPM tables and other non-table based intra prediction methods.

[0679] Example 38. The method according to example 37, wherein a top M intra prediction modes associated with highest frequencies are used as inputs for the MPM lists, wherein M is an integer smaller than N.

[0680] Example 39. The method according to example 36, wherein one or more selected intra prediction modes from the one or more ordered IPM tables are combined with other intra prediction modes derived from non-table based intra prediction methods to form the MPM lists.

[0681] Example 40. The method according to example 39, wherein the other intra prediction modes are derived from non-table based intra prediction methods comprising one or more default intra prediction modes.

[0682] Example 41. The method of example 40, wherein the one or more default intra prediction modes include a planar mode or a direct current (DC) mode.

[0683] Example 42. The method of example 39, wherein the other intra prediction modes are derived from a non-table-based intra prediction method that includes one or more intra prediction modes derived from spatial neighboring blocks available to the video unit.

[0684] Example 43. The method of example 42, wherein the spatial neighboring blocks include a top-left neighboring block that is immediately adjacent to the video unit or a top-right neighboring block that is not immediately adjacent to the video unit.

[0685] Example 44. The method of example 42, wherein the spatial neighboring blocks include a top neighboring block that is above the video unit or a left neighboring block that is to the left of the video unit.

[0686] Example 45. The method of example 42, wherein the default intra prediction mode order is: an intra prediction mode from a left neighboring block that is to the left of the video unit, an intra prediction mode from a top neighboring block that is above the video unit, a planar mode, and a direct current (DC) mode.

[0687] Example 46. The method of example 39, wherein a value of -1 is used for a missing mode index in the replacement MPM list for a first intra prediction mode associated with a left-side neighboring block that is not available to the left of the video unit, or for a second intra prediction mode associated with a top neighboring block that is not available above the video unit.

[0688] Example 47. The method of example 46, wherein the video unit includes a coding tree unit or a slice boundary.

[0689] Example 48. The method of example 39, wherein the other intra prediction modes are added to the MPM list before the one or more selected intra prediction modes from the one or more ordered IPM tables are added to the MPM list.

[0690] Example 49. The method of example 39, wherein the other intra prediction modes are added to the MPM list after the one or more selected intra prediction modes from the one or more ordered IPM tables are added to the MPM list.

[0691] Example 50. The method of example 39, wherein the other intra prediction modes are added to the MPM list before and after the one or more selected intra prediction modes from the one or more ordered IPM tables are added to the MPM list.

[0692] Example 51. The method of example 39, wherein a pruning technique is applied to other intra prediction modes and the one or more selected intra prediction modes to avoid adding redundant intra prediction modes to the MPM list.

[0693] Example 52. The method of example 39, wherein the order of addition of intra prediction modes derived from one or more ordered IPM tables and from non-table based intra prediction methods is changed between one video unit of a video to another video unit of the video and between one video block of a video to another video block of the video.

[0694] Example 53. The method of example 39, wherein the number of intra prediction modes derived from one or more ordered IPM tables and from non-table based intra prediction methods is changed between one video unit of a video to another video unit of the video and between one video block of a video to another video block of the video.

[0695] Example 54. The method of example 39, wherein the first L intra prediction modes in the one or more ordered IPM tables are added to the MPM list.

[0696] Example 55. The method of example 39, wherein the last L intra prediction modes in the one or more ordered IPM tables are added to the MPM list.

[0697] Example 56. The method of example 39, wherein the one or more selected intra prediction modes from the ordered IPM tables are added to the MPM list based on ascending order of entry index of the ordered IPM tables.

[0698] Example 57. The method of example 39, wherein the one or more selected intra prediction modes from the ordered IPM tables are added to the MPM list based on descending order of entry index of the ordered IPM tables.

[0699] Example 58. The method of example 39, wherein the one or more selected intra prediction modes from the ordered IPM tables are added to the MPM list based on ascending order of intra prediction mode index.

[0700] Example 59. The method of example 39, wherein the one or more selected intra prediction modes from the ordered IPM tables are added to the MPM list based on descending order of intra prediction mode index.

[0701] Example 60. The method of any of examples 58-59, wherein one or more indices of the top L intra prediction modes in the ordered IPM table are selected to be added to the MPM list.

[0702] Example 61. The method of example 60, wherein, in response to a first value of a first intra prediction mode being less than a second value of a second intra prediction mode, the first intra prediction mode is added to the MPM list before the second intra prediction mode is added to the MPM list.

[0703] Example 62. The method of example 60, wherein, in response to a first value of a first intra prediction mode being greater than a second value of a second intra prediction mode, the first intra prediction mode is added to the MPM list before the second intra prediction mode is added to the MPM list.

[0704] Example 63. The method of example 28, wherein whether to determine the entire MPM list from the one or more ordered IPM tables depends on a frequency of one or more intra prediction modes.

[0705] Example 64. The method of example 36, wherein whether to determine the entire MPM list from the one or more ordered IPM tables depends on a rule associated with decoding information of the video unit.

[0706] Example 65. The method of example 64, wherein the decoding information comprises a block dimension or a video content type.

[0707] Example 66. The method of example 65, wherein the rule associated with the block dimension comprises whether a block width and / or a block height is greater than a threshold.

[0708] Example 67. The method of example 65, wherein the rule associated with the video content type comprises whether the video content type is screen content.

[0709] Example 68. The method of example 33, wherein the ordered IPM table from the one or more ordered IPM tables is for intra mode coding of remaining intra prediction modes included in the MPM list other than the most probable mode.

[0710] Example 69. The method of example 68,

[0711] wherein a first value of a first index of a first remaining intra prediction mode corresponds to a first intra prediction mode,

[0712] wherein a second value of a second index of a second remaining intra prediction mode corresponds to a second intra prediction mode,

[0713] wherein the first value is less than the second value in response to the first intra prediction mode having a higher frequency than the second intra prediction mode, and

[0714] wherein the first value is included in the coded representation.

[0715] Example 70. A method of video processing, comprising:

[0716] performing a conversion between a video block of a video and a coded representation of the video,

[0717] wherein the coded representation includes a syntax element indicating a selected intra prediction mode for the conversion,

[0718] wherein the coded representation does not include one or more syntax elements indicating a most probable mode (MPM), or an index of a list of MPMs, or remaining intra prediction modes other than the intra prediction modes included in the list of MPMs, and

[0719] wherein the selected intra prediction mode is based on history information indicating frequencies of one or more intra prediction modes used in another conversion between one or more video blocks of the video and the video prior to the conversion of the video block.

[0720] Example 71. The method of example 70, wherein a mapping between the syntax element and the selected intra prediction mode is changed for a conversion from one video block to another video block.

[0721] Example 72. The method of any of examples 70-71,

[0722] wherein the ordered intra prediction mode (IPM) table includes one or more indices associated with one or more intra prediction modes in the ordered IPM table,

[0723] wherein the selected intra prediction mode is from the one or more intra prediction modes, and

[0724] wherein the syntax element includes an index from the one or more indices.

[0725] Example 73. The method of example 72, wherein the one or more indices are used to select the selected intra prediction mode.

[0726] Example 74. The method of example 72, wherein the one or more indices include an index value of zero that corresponds to a first intra prediction mode listed in the ordered IPM table.

[0727] Example 75. The method of any of examples 70-71,

[0728] wherein the ordered intra prediction mode (IPM) table includes an index associated with an intra prediction mode in the ordered IPM table,

[0729] wherein the index is in a descending order based on a frequency of use associated with the intra prediction mode,

[0730] wherein the selected intra prediction mode is from the intra prediction mode, and

[0731] wherein the syntax element includes an index from the index.

[0732] Example 76. The method of example 75, wherein the index with the lowest value is associated with the intra prediction mode with the highest frequency of use.

[0733] Example 77. The method of any of examples 70-71,

[0734] wherein the ordered intra prediction mode (IPM) table includes one or more indices associated with one or more intra prediction modes in the ordered IPM table,

[0735] wherein the one or more indices are coded with a binarization technique,

[0736] wherein the selected intra prediction mode is from the one or more intra prediction modes, and

[0737] wherein the syntax element includes an index from the one or more indices.

[0738] Example 78. The method of example 77, wherein the binarization technique includes a truncated unary method, a truncated binary method, or an exponential Golomb method.

[0739] Example 79. The method of any of examples 70-71,

[0740] wherein the ordered intra prediction mode (IPM) table includes one or more indices associated with one or more intra prediction modes in the ordered IPM table,

[0741] wherein the one or more indices are context coded for all containers or partial containers,

[0742] wherein the selected intra prediction mode is from the one or more intra prediction modes, and

[0743] wherein the syntax element includes an index from the one or more indices.

[0744] Example 80. The method of example 79, wherein the partial containers include a first number of containers.

[0745] Example 81. The method of any of examples 1-80,

[0746] wherein chroma direct mode (DM) coding is applied to the video unit or video block, and

[0747] wherein the DM coding is determined based on a frequency table, or one or more frequency tables, or a sorted IPM table, or one or more sorted IPM tables, or an IPM order mapping table.

[0748] Example 82. The method of example 81, wherein the IPM order mapping table includes an intra prediction mode associated with a highest frequency table.

[0749] Example 83. The method of any of examples 1-80,

[0750] wherein chroma direct mode (DM) coding is applied to the video unit or video block, and

[0751] wherein the chroma DM candidate list is determined based on a frequency table, or one or more frequency tables, or a sorted IPM table, or one or more sorted IPM tables, or an IPM order mapping table.

[0752] Example 84. The method of any of examples 1-83, wherein an indication of whether to selectively update or use one or more frequency tables or one or more sorted IPM tables for a video unit is signaled in a coded representation at a video unit level.

[0753] Example 85. The method of example 84, wherein the video unit includes a tile, a brick, a slice, a picture, a subpicture, a sequence, or a view.

[0754] Example 86. The method of example 84, wherein the indication of whether to selectively update or use one or more frequency tables or one or more sorted IPM tables and / or another indication of a technique to selectively update one or more frequency tables or one or more sorted IPM tables is indicated in a sequence parameter set, a view parameter set, an adaptation parameter set, a picture parameter set, a picture header, a slice header, or a sequence header.

[0755] Example 87. The method of example 86, wherein a syntax element in the coded representation indicates whether to enable determination of one or more intra prediction modes from one or more frequency tables, or one or more sorted IPM tables, or an IPM order mapping table.

[0756] Example 88. The method of example 86, wherein the syntax element in the coded representation indicates a number of intra prediction modes determined from one or more frequency tables, or one or more ordered IPM tables, or an IPM order mapping table.

[0757] Example 89. The method of example 86, wherein the syntax element in the coded representation indicates a number of most probable modes determined from one or more intra prediction modes from one or more frequency tables, or one or more ordered IPM tables, or an IPM order mapping table.

[0758] Example 90. The method of example 84,

[0759] wherein a technique of selectively updating or using one or more frequency tables or one or more ordered IPM tables and / or with which one or more frequency tables or one or more ordered IPM tables are selectively updated and used is indicated in a syntax element in the coded representation, and

[0760] wherein the syntax element indicates whether video content of the video unit is screen content.

[0761] Example 91. The method of example 84,

[0762] wherein the technique of selectively updating or using one or more frequency tables or one or more ordered IPM tables and / or with which one or more frequency tables or one or more ordered IPM tables are selectively updated and used is based on a characteristic derived from reconstructed samples in a previously coded block that precedes in time the video block associated with the video unit.

[0763] Example 92. The method of any of examples 1-91,

[0764] wherein the technique of selectively updating or using one or more frequency tables or one or more ordered IPM tables and / or with which one or more frequency tables or one or more ordered IPM tables are selectively updated or used is based on coding information of the video block.

[0765] Example 93. The method of example 92, wherein the coding information comprises a block dimension, a slice type, a picture type, a temporal layer index, or video content of the video unit.

[0766] Example 94. The method of example 92, wherein the technique of selectively updating or using one or more frequency tables or one or more ordered IPM tables is selected when a width of the video unit is less than or equal to Tl and a height of the video unit is less than or equal to T2, wherein Tl and T2 are integers.

[0767] Example 95. The method of example 92, wherein the one or more frequency tables or the one or more sorted IPM tables are selectively updated or used when a width of the video unit is less than or equal to Tl or a height of the video unit is less than or equal to T2, wherein Tl and T2 are integers.

[0768] Example 96. The method of example 92, wherein the one or more frequency tables or the one or more sorted IPM tables are selectively updated or used when a width of the video unit multiplied by a height of the video unit is less than or equal to T3, wherein T3 is an integer.

[0769] Example 97. The method of example 92, wherein the one or more frequency tables or the one or more sorted IPM tables are prohibited from being selectively updated or used when a width of the video unit is less than or equal to Tl and a height of the video unit is less than or equal to T2, wherein Tl and T2 are integers.

[0770] Example 98. The method of example 92, wherein the one or more frequency tables or the one or more sorted IPM tables are prohibited from being selectively updated or used when a width of the video unit is less than or equal to Tl or a height of the video unit is less than or equal to T2, wherein Tl and T2 are integers.

[0771] Example 99. The method of example 92, wherein the one or more frequency tables or the one or more sorted IPM tables are prohibited from being selectively updated or used when a width of the video unit multiplied by a height of the video unit is less than or equal to T3, wherein T3 is an integer.

[0772] Example 100. The method of any of examples 1 to 99,

[0773] wherein whether the one or more frequency tables or the one or more sorted IPM tables are selectively updated and / or the technique with which the one or more frequency tables or the one or more sorted IPM tables are selectively updated is based on a color component, or a color coding method, or a color format, or a partition tree coding method of the video unit.

[0774] Example 101. The method of example 100, wherein the one or more frequency tables or the one or more sorted IPM tables are selectively updated and / or the technique with which the one or more frequency tables or the one or more sorted IPM tables are selectively updated is only for luma intra prediction mode coding.

[0775] Example 102. The method of any of examples 1 to 101, wherein the converting comprises encoding the video into a coded representation.

[0776] Example 103. The method of any of examples 1-101, wherein converting comprises decoding the coded representation to generate pixel values of the video.

[0777] Example 104. A video decoding apparatus comprising a processor configured to implement a method recited in one or more of examples 1-103.

[0778] Example 105. A video encoding apparatus comprising a processor configured to implement a method recited in one or more of examples 1-103.

[0779] Example 106. A computer program product having computer code stored thereon, the code, when executed by a processor, causing the processor to implement a method recited in any of examples 1-103.

[0780] From the foregoing, it will be appreciated that specific embodiments of the presently disclosed technology have been described herein for purposes of illustration, but well-known modifications can be made by persons skilled in the art. Accordingly, the presently disclosed technology is not limited except as by the appended claims.

[0781] The solutions, examples, embodiments, modules and functional operations disclosed herein and other described solutions can be implemented in digital electronic circuitry, or in computer software, firmware, or hardware, including the structures disclosed herein and their structural equivalents, or in combinations of one or more of them. The disclosed embodiments and other embodiments can be implemented as one or more computer program products, i.e., one or more modules of computer program instructions encoded on a computer readable medium for execution by, or to control the operation of, data processing apparatus. The computer readable medium can be a machine-readable storage device, a machine-readable storage substrate, a memory device, a composition of matter effecting a machine-readable propagated signal, or a combination of one or more of them. The term "data processing apparatus" encompasses all apparatus, devices, and machines for processing data, including by way of example a programmable processor, a computer, or multiple processors or computers. The apparatus can also include, in addition to a processor, code that creates an execution environment for computer programs, e.g., code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them. A propagated signal is an artificially generated signal, e.g., a machine-generated electrical, optical, or electromagnetic signal, that is generated to encode information for transmission to suitable receiver apparatus.

[0782] The solutions, examples, embodiments, modules and functional operations disclosed herein and other described solutions can be implemented in digital electronic circuitry, or in computer software, firmware, or hardware, including the structures disclosed herein and their structural equivalents, or in combinations of one or more of them. The disclosed embodiments and other embodiments can be implemented as one or more computer program products, i.e., one or more modules of computer program instructions encoded on a computer readable medium for execution by, or to control the operation of, data processing apparatus. The computer readable medium can be a machine-readable storage device, a machine-readable storage substrate, a memory device, a composition of matter effecting a machine- readable propagated signal, or a combination of one or more of them. The term "data processing apparatus" encompasses all apparatus, devices, and machines for processing data, including by way of example a programmable processor, a computer, or multiple processors or computers. The apparatus can also include, in addition to a hardware component, code that creates an execution environment for computer programs in the form of code, such as code that forms a firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them. A propagated signal is an artificially generated signal, e.g., a machine-generated electrical, optical, or electromagnetic signal, that is generated to encode information for transmission to suitable receiver apparatus.

[0783] A computer program (also known as a program, software, software application, script, or code) can be written in any form of programming language, including compiled or interpreted languages, and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. A program can be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files that store one or more modules, sub programs, or portions of code). A computer program can be deployed to be executed on one computer or on multiple computers that are located at one site or distributed across multiple sites and are interconnected by a communication network.

[0784] The processes and logic flows described herein can be performed by one or more programmable processors executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows can also be performed by special purpose logic circuitry, and that apparatus can also be implemented as special purpose logic circuitry, e.g., an FPGA (field programmable gate array) or an ASIC (application specific integrated circuit).

[0785] Processors suitable for the execution of a computer program include, by way of example, both general and special purpose microprocessors, and any one or more processors of any kind of digital computer. Generally, a processor will receive instructions and data from a read-only memory or a random access memory or both. The essential elements of a computer are a processor for executing instructions and one or more memory devices for storing instructions and data. Generally, a computer will also include, or be operatively coupled to receive data from or transfer data to, or both, one or more mass storage devices for storing data, e.g., magnetic, magneto-optical disks, or optical disks. However, a computer need not have such devices. Computer readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media and memory devices, including by way of example semiconductor memory devices, e.g., EPROM, EEPROM, and flash memory devices; magnetic disks, e.g., internal hard disks or removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks. The processor and the memory can be supplemented by, or incorporated in, special purpose logic circuitry.

[0786] Although this patent document contains many details, it should not be construed to limit the scope of any implementation or of any claim, but is intended to describe features of particular implementations. Some features described in the context of separate embodiments can also be implemented in combination, in a single embodiment. Conversely, various features described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable sub-combination. Moreover, although features can be described above as acting in particular combinations, one or more features from a combination can in some cases be removed from the combination, and the claim can then be directed to a sub-combination or variation of a sub-combination.

[0787] Similarly, while operations are depicted in the drawings in a particular order, this should not be understood as requiring such an order, nor that all illustrated operations be performed, to achieve desirable results. In certain circumstances, multitasking and parallel processing can be advantageous. Moreover, the separation of various system components in the embodiments described should not be understood as requiring such separation in all embodiments.

[0788] Only a few implementations and examples are described and other implementations, enhancements and variations can be made based on what is described and illustrated in this patent document.

Claims

1. A method of video processing, comprising: maintaining a frequency table for conversion between a video region containing one or more video blocks and a bitstream of the video region, for a current video block of the one or more video blocks, determining to apply a first intra prediction mode to derive prediction samples of the current video block based on the frequency table, and performing the conversion based on the frequency table, wherein the frequency table records frequencies of one or more intra prediction modes, wherein the one or more intra prediction modes include the first intra prediction mode, wherein each entry in the frequency table indicates a number of times a corresponding intra prediction mode is used to code video blocks processed before a current video block, wherein the frequency table is updated based on the first intra prediction mode, wherein the updated frequency table is used to derive an intra prediction mode for video blocks after the current video block, wherein an intra prediction mode (IPM) table is constructed for the current video block based on the frequency table, and the IPM table is updated after the frequency table is updated, the IPM table including X entries, X being an integer no smaller than 2, the X entries representing X modes with highest frequencies in the frequency table, wherein when frequency-based intra mode is enabled, a prediction mode list containing intra prediction modes is constructed for the current video block based on the IPM table, and wherein the first intra prediction mode is determined based on the prediction mode list, wherein a size of the prediction mode list is N, N intra prediction modes in the prediction mode list are derived from N modes with highest frequencies in the IPM table, wherein N is an integer, and wherein when the frequency table is updated, the ordered IPM table is updated.

2. The method of claim 1, wherein, one or more of the ordered IPM tables are associated with one or more of the frequency tables.

3. The method of claim 1, wherein, the current video block includes a luma coding block.

4. The method of claim 1, wherein, entries in the one or more frequency tables include intra prediction modes and occurrences or frequencies of the intra prediction modes without updating the one or more ordered IPM tables.

5. The method of claim 1, wherein, a k-th entry of the one or more ordered IPM tables represents an intra prediction mode that is historically used with a k-th highest frequency.

6. The method of claim 1, wherein, the video block includes a sub-region of a coding tree unit (CTU).

7. The method of claim 6, wherein, the sub-region of the CTU includes a virtual pipeline data unit (VPDU), another CTU, a coding tree block (CTB), multiple CTUs, multiple coding units (CUs), a CTU row, a tile, a brick, a slice, a picture, or a subpicture.

8. The method of claim 1, wherein, the one or more frequency tables are associated with an IPM order mapping table.

9. The method of claim 8, wherein, a k-th entry of the IPM order mapping table represents an order index of an intra prediction mode with an index equal to k.

10. The method of claim 1, wherein, the video block is a prediction unit (PU).

11. The method of claim 1, wherein, the video block is a coding unit (CU).

12. The method of claim 1, wherein, the one or more frequency tables and the one or more ordered IPM tables are used for another conversion between one or more additional video blocks of the video and the bitstream of the video, wherein the video block precedes the one or more additional video blocks in time.

13. The method of claim 1, wherein, The values of the intra prediction modes in the sorted IPM table are associated with index values that are the same as the index values of the frequencies of the intra prediction modes in the frequency table.

14. The method of claim 1, wherein, The one or more frequency tables and the one or more sorted IPM tables are updated for each block type of the video.

15. The method of claim 1, wherein, A frequency table from the one or more frequency tables and the one or more sorted IPM tables are updated for a plurality of block types of the video.

16. The method of claim 14, wherein, The block types include blocks having the same width and / or the same height.

17. The method of claim 1, wherein, For an index value equal to k: A first value modeT[k] is an intra prediction mode associated with an index value equal to k in a sorted IPM table, A second value orderT[k] is a mapping index of the intra prediction mode associated with an index value equal to k after sorting the intra prediction modes in an IPM order mapping table, and A third value freqT[k] is a frequency of the intra prediction mode associated with an index value equal to k in a frequency table.

18. The method of claim 17, wherein, orderT[modeT[k]] = k, where k represents an order index; and / or, modeT[orderT[m]] = m, where m represents an intra prediction mode; and / or, freqT[modeT[k]] >= freqT[modeT[k+1]].

19. The method of claim 1, wherein, N and X are both equal to 2.

20. The method of claim 1, wherein, The indication of the frequency-based intra mode is included in a sequence parameter set and a picture header of the current video block.

21. The method of claim 1, wherein, The frequency table is associated with M entries, where M is an integer, and where each entry is associated with a frequency of one of M intra prediction modes.

22. The method of claim 21, wherein, A sorted IPM table from one or more sorted IPM tables includes a number of entries in a frequency table associated with the sorted IPM table.

23. The method of claim 21, wherein, The M intra prediction modes are grouped into N categories, where N is an integer.

24. The method of claim 23, wherein, The frequency table includes N entries, where N is an integer smaller than M, and where each entry in the frequency table is associated with a frequency corresponding to one category of the one or more intra prediction modes.

25. The method of claim 23, wherein, One or more sorted IPM tables are associated with the frequency table, and where the one or more sorted IPM tables include N entries having sorted category indices.

26. The method of claim 21, wherein, The frequency table includes N entries, where N is an integer smaller than M, and where the N entries correspond to N selected intra prediction modes from the M intra prediction modes.

27. The method of claim 21, wherein, The M intra prediction modes do not include a wide-angle intra prediction mode.

28. The method of claim 21, wherein, The M intra prediction modes include at least one of a direct current (DC) mode, a planar mode, a horizontal mode, a vertical mode, or a bilinear intra prediction mode.

29. The method of claim 26, wherein, The N selected intra prediction modes are predefined or signaled or derived from coding information of the video block.

30. The method of claim 29, wherein, The coding information indicates whether the video block includes screen content.

31. The method of claim 21, wherein, The frequency table includes N entries, where N is an integer less than M, where the N entries correspond to N intra prediction modes, and where the frequency table including N entries is updated during the conversion based on decoded information of the video block.

32. The method of claim 31, wherein, The N intra prediction modes are first initialized and then updated based on the decoded information of the video block.

33. The method of claim 1, wherein, one or more of the frequency tables or one or more of the ordered IPM tables are used to perform a conversion between a video block of a video and a bitstream of the video, wherein the one or more frequency tables include information on frequencies of one or more intra prediction modes used in the conversion of the video, wherein the frequencies indicate occurrences of the one or more intra prediction modes for the conversion, wherein the one or more ordered IPM tables indicate the one or more intra prediction modes in an ordered sequence, wherein the one or more frequency tables or the one or more ordered IPM tables are used for intra mode coding in a process of constructing a most probable mode (MPM) list of size N, and wherein N is an integer.

34. The method of claim 33, wherein, All of the MPM lists are determined from the one or more ordered IPM tables.

35. The method of claim 34, wherein, The one or more intra prediction modes include top N intra prediction modes associated with highest frequencies, and wherein the top N intra prediction modes are used as inputs to the MPM lists.

36. The method of claim 33, wherein, The one or more frequency tables and the one or more ordered IPM tables are used for intra mode coding.

37. The method of claim 36, wherein, All of the MPM lists are determined from the one or more ordered IPM tables and other non-table-based intra prediction methods.

38. The method of claim 37, wherein, Top M intra prediction modes associated with highest frequencies are used as inputs to the MPM lists, where M is an integer less than N.

39. The method of claim 36, wherein, One or more selected intra prediction modes from one or more ordered IPM tables are combined with other intra prediction modes derived from non-table-based intra prediction methods to form MPM lists.

40. The method of claim 39, wherein, The other intra prediction modes are derived from the non-table-based intra prediction methods including one or more default intra prediction modes.

41. The method of claim 40, wherein, The one or more default intra prediction modes include a planar mode or a direct current (DC) mode.

42. The method of claim 39, wherein, The other intra prediction modes are derived from non-table-based intra prediction methods including one or more intra prediction modes derived from spatial neighboring blocks available to the video block.

43. The method of claim 42, wherein, The spatial neighboring blocks include neighboring blocks that are immediately adjacent to the video block or neighboring blocks that are not immediately adjacent to the video block.

44. The method of claim 42, wherein, The spatial neighboring blocks include an above neighboring block located above the video block or a left neighboring block located to the left of the video block.

45. The method of claim 42, wherein, A default intra prediction mode order is: an intra prediction mode from a left neighboring block located to the left of the video block, an intra prediction mode from an above neighboring block located above the video block, a planar mode, and a direct current (DC) mode.

46. The method of claim 39, wherein, For a first intra prediction mode associated with a left neighboring block that is not available to the left of the video block, or for a second intra prediction mode associated with an above neighboring block that is not available above the video block, a value of -1 is used to replace a missing mode index in the MPM list.

47. The method of claim 46, wherein, The video block comprises a coding tree unit or a slice boundary.

48. The method of claim 39, wherein, The other intra prediction modes are added to the MPM list before the one or more selected intra prediction modes from the one or more ordered IPM tables are added to the MPM list.

49. The method of claim 39, wherein, The other intra prediction modes are added to the MPM list after the one or more selected intra prediction modes from the one or more ordered IPM tables are added to the MPM list.

50. The method of claim 39, wherein, The other intra prediction modes are added to the MPM list before and after the one or more selected intra prediction modes from the one or more ordered IPM tables are added to the MPM list.

51. The method of claim 39, wherein, A pruning technique is applied to the other intra prediction modes and the one or more selected intra prediction modes to avoid adding redundant intra prediction modes to the MPM list.

52. The method of claim 39, wherein, The order of addition of intra prediction modes from the one or more ordered IPM tables and from non-table-based intra prediction methods is changed from one video block of the video to another video block of the video and from one video block of the video to another video block of the video.

53. The method of claim 39, wherein, The number of intra prediction modes from the one or more ordered IPM tables and from non-table-based intra prediction methods is changed from one video block of the video to another video block of the video and from one video block of the video to another video block of the video.

54. The method of claim 39, wherein, The first L intra prediction modes in the one or more ordered IPM tables are added to the MPM list.

55. The method of claim 39, wherein, The last L intra prediction modes in the one or more ordered IPM tables are added to the MPM list.

56. The method of claim 39, wherein, The one or more selected intra prediction modes from the ordered IPM tables are added to the MPM list based on ascending order of entry indices of the ordered IPM tables.

57. The method of claim 39, wherein, The one or more selected intra prediction modes from the ordered IPM tables are added to the MPM list based on descending order of entry indices of the ordered IPM tables.

58. The method of claim 39, wherein, The one or more selected intra prediction modes from the ordered IPM tables are added to the MPM list based on ascending order of intra prediction mode indices.

59. The method of claim 39, wherein, The one or more selected intra prediction modes from the ordered IPM tables are added to the MPM list based on descending order of intra prediction mode indices.

60. The method of claim 58, wherein, One or more indices of the first L intra prediction modes in the ordered IPM tables are selected to be added to the MPM list.

61. The method of claim 60, wherein, in response to a first value of a first intra prediction mode being less than a second value of a second intra prediction mode, adding the first intra prediction mode to the MPM list before adding the second intra prediction mode to the MPM list.

62. The method of claim 60, wherein, in response to a first value of a first intra prediction mode being greater than a second value of a second intra prediction mode, adding the first intra prediction mode to the MPM list before adding the second intra prediction mode to the MPM list.

63. The method of claim 28, wherein, whether to determine the entire MPM list from the one or more ordered IPM tables depends on a frequency of the one or more intra prediction modes.

64. The method of claim 36, wherein, whether to determine the entire MPM list from the one or more ordered IPM tables depends on a rule associated with decoding information of the video block.

65. The method of claim 64, wherein, the decoding information comprises a block dimension or a video content type.

66. The method of claim 65, wherein, the rule associated with the block dimension comprises whether a block width and / or a block height is greater than a threshold.

67. The method of claim 65, wherein, the rule associated with the video content type comprises whether the video content type is screen content.

68. The method of claim 33, wherein, an ordered IPM table from the one or more ordered IPM tables is for intra mode coding of remaining intra prediction modes included in the MPM list other than a most probable mode.

69. The method of claim 68, wherein, a first value of a first index of a first remaining intra prediction mode corresponds to a first intra prediction mode, wherein a second value of a second index of a second remaining intra prediction mode corresponds to a second intra prediction mode, wherein in response to the first intra prediction mode having a higher frequency than the second intra prediction mode, the first value is less than the second value, and wherein the first value is included in the bitstream.

70. The method of claim 1, wherein the bitstream comprises a syntax element indicating a selected intra prediction mode for the conversion, wherein the bitstream does not comprise one or more syntax elements indicating a most probable mode (MPM), or an index of an MPM list, or remaining intra prediction modes other than intra prediction modes included in the MPM list, and wherein the selected intra prediction mode is based on history information indicating a frequency of one or more intra prediction modes used in another conversion between one or more video blocks of the video and the video prior to the conversion of the video block.

71. The method of claim 70, wherein, a mapping between the syntax element and the selected intra prediction mode is changed from one video block to another video block.

72. The method of claim 70, wherein, an ordered intra prediction mode (IPM) table comprises one or more indices associated with one or more intra prediction modes in the ordered IPM table, wherein the selected intra prediction mode is from the one or more intra prediction modes, and wherein the syntax element comprises an index from the one or more indices.

73. The method of claim 72, wherein, the one or more indices are used to select the selected intra prediction mode.

74. The method of claim 72, wherein, The one or more indices include an index value of zero that corresponds to a first intra prediction mode listed in the sorted IPM table.

75. The method of claim 70, wherein, The sorted intra prediction mode (IPM) table includes indices associated with intra prediction modes in the sorted IPM table, wherein the indices are in descending order based on a frequency of use associated with the intra prediction modes, wherein the selected intra prediction mode is from the intra prediction modes, and wherein the syntax element includes an index from the indices.

76. The method of claim 75, wherein, The index with the lowest value is associated with the intra prediction mode with the highest frequency of use.

77. The method of claim 70, wherein The sorted intra prediction mode (IPM) table includes one or more indices associated with one or more intra prediction modes in the sorted IPM table, wherein the one or more indices are coded with a binarization technique, wherein the selected intra prediction mode is from the one or more intra prediction modes, and wherein the syntax element includes an index from the one or more indices.

78. The method of claim 77, wherein, The binarization technique includes a truncated unary method, a truncated binary method, or an exponential Golomb method.

79. The method of claim 70, wherein The sorted intra prediction mode (IPM) table includes one or more indices associated with one or more intra prediction modes in the sorted IPM table, wherein the one or more indices are context coded for all containers or partial containers, wherein the selected intra prediction mode is from the one or more intra prediction modes, and wherein the syntax element includes an index from the one or more indices.

80. The method of claim 79, wherein, The partial containers include a first number of containers.

81. The method of claim 1, wherein, Chroma direct mode (DM) coding is applied to the video block or the video block, and wherein the DM coding is determined based on the frequency table, or the one or more frequency tables, or a sorted IPM table, or the one or more sorted IPM tables, or the IPM order mapping table.

82. The method of claim 81, wherein, The IPM order mapping table includes an intra prediction mode associated with a highest frequency table.

83. The method of claim 1, wherein Chroma direct mode (DM) coding is applied to the video block, and wherein a chroma DM candidate list is determined based on the frequency table, or the one or more frequency tables, or a sorted IPM table, or the one or more sorted IPM tables, or the IPM order mapping table.

84. The method of claim 1, wherein, An indication is signaled in a bitstream at a video block level whether the one or more frequency tables or the one or more sorted IPM tables are selectively updated or used for the video block.

85. The method of claim 84, wherein, The video block includes a tile, a brick, a slice, a picture, a subpicture, a sequence, or a view.

86. The method of claim 84, wherein, an indication in a sequence parameter set, a view parameter set, an adaptation parameter set, a picture parameter set, a picture header, a slice header, or a sequence header whether to selectively update or use the one or more frequency tables or the one or more ordered IPM tables and / or another indication of a technique to selectively update the one or more frequency tables or the one or more ordered IPM tables with.

87. The method of claim 86, wherein, a syntax element in the bitstream indicates whether to enable determination of the one or more intra prediction modes from the one or more frequency tables, or the one or more ordered IPM tables, or the IPM order mapping table.

88. The method of claim 86, wherein, a syntax element in the bitstream indicates a number of the one or more intra prediction modes determined from the one or more frequency tables, or the one or more ordered IPM tables, or the IPM order mapping table.

89. The method of claim 86, wherein, a syntax element in the bitstream indicates a number of the most probable modes determined from the one or more intra prediction modes from the one or more frequency tables, or the one or more ordered IPM tables, or the IPM order mapping table.

90. The method of claim 84, wherein an indication in a syntax element in the bitstream whether to selectively update or use the one or more frequency tables or the one or more ordered IPM tables and / or a technique to selectively update and use the one or more frequency tables or the one or more ordered IPM tables with, and wherein the syntax element indicates whether video content of the video block is screen content.

91. The method of claim 84, wherein, whether to selectively update or use the one or more frequency tables or the one or more ordered IPM tables and / or a technique to selectively update and use the one or more frequency tables or the one or more ordered IPM tables with is based on a characteristic derived from reconstructed samples in a previously coded block that precede in time a video block associated with the video block.

92. The method of claim 1, wherein whether to selectively update or use the one or more frequency tables or the one or more ordered IPM tables and / or a technique to selectively update or use the one or more frequency tables or the one or more ordered IPM tables with is based on coding information of the video block.

93. The method of claim 92, wherein, the coding information includes a block dimension, a slice type, a picture type, a temporal layer index, or a video content of the video block.

94. The method of claim 92, wherein, selectively updating or using the one or more frequency tables or the one or more ordered IPM tables when a width of the video block is less than or equal to Tl and a height of the video block is less than or equal to T2, where Tl and T2 are integers.

95. The method of claim 92, wherein, selectively updating or using the one or more frequency tables or the one or more ordered IPM tables when a width of the video block is less than or equal to Tl or a height of the video block is less than or equal to T2, where Tl and T2 are integers.

96. The method of claim 92, wherein, selectively updating or using the one or more frequency tables or the one or more sorted IPM tables is prohibited when the width of the video block is less than or equal to T1 or the height of the video block is less than or equal to T2, where T1 and T2 are integers. where T3 is an integer.

97. The method of claim 92, wherein, selectively updating or using the one or more frequency tables or the one or more sorted IPM tables is prohibited when the width of the video block is less than or equal to T1 or the height of the video block is less than or equal to T2, where T1 and T2 are integers.

98. The method of claim 92, wherein, selectively updating or using the one or more frequency tables or the one or more sorted IPM tables is prohibited when the width of the video block is less than or equal to T1 or the height of the video block is less than or equal to T2, where T1 and T2 are integers.

99. The method of claim 92, wherein, selectively updating or using the one or more frequency tables or the one or more sorted IPM tables is prohibited when the width of the video block is less than or equal to T1 or the height of the video block is less than or equal to T2, where T1 and T2 are integers. where T3 is an integer.

100. The method of claim 1, wherein whether the one or more frequency tables or the one or more sorted IPM tables are selectively updated and / or techniques to selectively update the one or more frequency tables or the one or more sorted IPM tables are based on a color component, or a color coding method, or a color format, or a partition tree coding method of the video block.

101. The method of claim 100, wherein, the one or more frequency tables or the one or more sorted IPM tables are selectively updated and / or techniques to selectively update the one or more frequency tables or the one or more sorted IPM tables are only for luma intra prediction mode coding.

102. The method of any one of claims 1 to 101, wherein, the conversion includes encoding the video into the bitstream.

103. The method of any one of claims 1 to 101, wherein, the conversion includes decoding the bitstream to generate pixel values of the video.

104. A video decoding apparatus comprising: a processor configured to implement a method recited in any of claims 1-103.

105. A video coding device comprising: a processor configured to implement a method recited in any of claims 1-103.

106. A computer program product having computer code stored thereon, the code, when executed by a processor, causing the processor to implement a method recited in any of claims 1-103.

Images