Deriving the matrix in intra-coding mode

Abstract

To provide a device, a system, and a method for digital video coding, including a matrix-based intra prediction method for video coding.SOLUTION: In an exemplary aspect, a method for video processing includes performing a conversion between a current video block of a video and a bitstream representation of the current video block according to a rule, and determining a predictive block for the current video block when the rule specifies a sample of the current video block and a matrix or offset value to be applied in matrix weighted intra prediction (MIP) mode during the conversion, and the MIP mode performs a boundary downsampling operation, followed by a matrix-vector multiplication operation, and selectively followed by an upsampling operation on the previously encoded sample of the video.SELECTED DRAWING: Figure 11

Description

[Technical field]

[0001] This patent document is based on the video coding Related to techniques, devices, and systems.

[0002] CROSS-REFERENCE TO RELATED APPLICATIONS This application is a divisional application of patent application No. 2021-560955, filed on October 14, 2021, based on international patent application PCT / CN2020 / 085050, filed on April 16, 2020, which claims priority to and the benefit of international patent application PCT / CN2019 / 082813, filed on April 16, 2019. All of the above patent applications are incorporated herein by reference in their entireties. [Background technology]

[0003] Despite advances in video compression, digital video still accounts for the largest bandwidth usage in the Internet and other digital communications networks. As the number of connected user devices capable of receiving and displaying video increases, the bandwidth demands for digital video usage are expected to continue to increase. Summary of the Invention

[0004] Digital Video coding Apparatus, systems, and methods relating to digital video coding, and in particular to video coding A matrix-based intra prediction method for a video is described. coding Standard (e.g. High Efficiency Video coding (HEVC)) and future video coding Standard (e.g. Versatile video coding (VVC)) or both codecs.

[0005] A first exemplary method of video processing includes performing a conversion between a current video block of a video and a bitstream representation of the current video block according to a rule, where the rule specifies samples of the current video block and a matrix or offset value to be applied in a matrix weighted intra prediction (MIP) mode during the conversion, and where the MIP mode specifies samples of a previously encoded bitstream representation of the video. Coding The method includes determining a prediction block of the current video block by performing a boundary downsampling operation, followed by a matrix vector multiplication operation, and optionally followed by an upsampling operation, on the obtained samples.

[0006] A second exemplary method of video processing includes generating an intermediate prediction block for a current video block using a matrix weighted intra prediction (MIP) mode, where the intermediate prediction block for the current video block is a prediction block for a previously stored video block. Coding the intermediate prediction block being determined by performing a boundary downsampling operation, followed by a matrix-vector multiplication operation, and optionally followed by an upsampling operation, on the intermediate prediction block samples; generating a final prediction block based on the intermediate prediction block and based on an additional operation; and performing a conversion between the current video block and a bitstream representation of the current video block based on the final prediction signal.

[0007] A third exemplary method of video processing includes performing a conversion between a current video block of a video and a bitstream representation of the current video block, where the conversion includes predicting a plurality of samples of at least a portion of the current video block in a matrix weighted intra prediction (MIP) mode, where a previously encoded bitstream representation of the current video block includes a bitstream representation of the current video block. CodingA predictive block for a portion of the current video block is determined by performing a boundary downsampling operation, followed by a matrix vector multiplication operation, and optionally followed by an upsampling operation, on the sampled samples to generate a predictive block of the current video block.

[0008] A fourth exemplary method of video processing includes performing a conversion between a current video block of a video and a bitstream representation of the current video block, where the conversion is based on a rule indicating whether to filter neighboring samples of the current video block prior to applying a matrix weighted intra prediction (MIP) mode during the conversion, and where the MIP mode is based on a rule indicating whether to filter neighboring samples of the current video block prior to applying a matrix weighted intra prediction (MIP) mode to a previous bitstream representation of the video. Coding The method includes determining a prediction block of the current video block by performing a boundary downsampling operation, followed by a matrix vector multiplication operation, and optionally followed by an upsampling operation, on the obtained samples.

[0009] In yet another representative aspect, the disclosed techniques can be used to provide a method for video processing. This exemplary method includes: Coding determining that the ALWIP mode is to be selected based on at least a portion of the MPM list for the non-ALWIP intra mode based on the determination; and performing a conversion between the current video block and a bitstream representation of the current video block based on the MPM list for the ALWIP mode.

[0010] In yet another representative aspect, the disclosed techniques can be used to provide a method for video processing in which a luma component of a current video block is predicted using an affine linear weighted intra prediction (ALWIP) mode. CodingThe method includes determining whether a chrominance intra mode will be used for the current video block, estimating a chrominance intra mode based on the determination, and performing a conversion between the current video block and a bitstream representation of the current video block based on the chrominance intra mode.

[0011] In yet another representative aspect, the disclosed techniques can be used to provide a method for video processing. This exemplary method includes: Coding and performing a conversion between the current video block and a bitstream representation of the current video block based on the determination.

[0012] In yet another representative aspect, the disclosed techniques can be used to provide a method for video processing in which a current video block is predicted using a different Affine Linear Weighted Intra Prediction (ALWIP) mode. coding Using Mode Coding and performing a conversion between the current video block and a bitstream representation of the current video block based on the determination.

[0013] In yet another representative aspect, the disclosed techniques may be used to provide a method for video processing that includes generating a first prediction for a current video block using an affine linear weighted intra prediction (ALWIP) mode, generating a second prediction based on the first prediction using position-dependent intra prediction combining (PDPC), and performing a conversion between the current video block and a bitstream representation of the current video block based on the second prediction.

[0014] In yet another representative aspect, the disclosed techniques can be used to provide a method for video processing. This exemplary method includes: Codingthe ALWIP mode; predicting a plurality of sub-blocks of the current video block based on the ALWIP mode; and performing a conversion between the current video block and a bitstream representation of the current video block based on the prediction.

[0015] In yet another exemplary embodiment, the above-described methods are embodied in the form of processor executable code and stored on a computer readable program medium.

[0016] In yet another exemplary aspect, a device configured or operable to perform the above-described method is disclosed. The device may include a processor programmed to implement the method.

[0017] In yet another representative aspect, a video decoder device may implement the methods described herein.

[0018] The above aspects and features of the disclosed technology are explained in more detail in the drawings, specification, and claims. [Brief description of the drawings]

[0019] [Figure 1] FIG. 1 shows an example of one of the 33 intra prediction directions. [Diagram 2] FIG. 2 shows an example of one of the 67 intra-prediction modes. [Diagram 3] FIG. 3 shows one example of the sample locations used to derive the linear model weights. [Figure 4] FIG. 4 shows an example of one of four reference lines adjacent to a prediction block. [Figure 5A] FIG. 5A shows an example of sub-partitions according to block size. [Figure 5B] FIG. 5B shows an example of sub-partitions according to block size. [Figure 6]FIG. 6 shows an example of ALWIP for a 4×4 block. [Figure 7] FIG. 7 shows an example of ALWIP for an 8×8 block. [Figure 8] FIG. 8 shows an example of ALWIP for an 8×4 block. [Figure 9] FIG. 9 shows an example of ALWIP for a 16×16 block. [Figure 10] FIG. 10 shows an example of a contiguous block used in an MPM list configuration. [Figure 11] FIG. 11 illustrates a flowchart of an exemplary method for matrix-based intra prediction in accordance with the disclosed techniques. [Figure 12] FIG. 12 illustrates a flowchart of another exemplary method for matrix-based intra prediction in accordance with the disclosed techniques. [Figure 13] FIG. 13 illustrates a flowchart of yet another exemplary method for matrix-based intra prediction in accordance with the disclosed techniques. [Figure 14] FIG. 14 illustrates a flowchart of yet another exemplary method for matrix-based intra prediction in accordance with the disclosed techniques. [Figure 15] FIG. 15 is a block diagram of an example hardware platform for implementing the visual media decoding or encoding techniques described in this document. [Figure 16] FIG. 16 is a block diagram illustrating an example video processing system in which various techniques disclosed in this document can be implemented. [Figure 17] FIG. 17 is a block diagram illustrating an example video coding system that can utilize the techniques of this document. [Figure 18] FIG. 18 is a block diagram showing an example of a video encoder. [Figure 19] FIG. 19 is a block diagram showing an example of a video decoder. [Figure 20] FIG. 20 shows an example flow chart of an additional example method for matrix-based intra prediction in accordance with the disclosed techniques. [Figure 21] FIG. 21 shows an example flow chart of an additional example method for matrix-based intra prediction in accordance with the disclosed techniques. [Figure 22] FIG. 22 shows an example flow chart of an additional example method for matrix-based intra prediction in accordance with the disclosed techniques. [Figure 23] FIG. 23 shows an example flow chart of an additional example method for matrix-based intra prediction in accordance with the disclosed techniques. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0020] As the demand for higher resolution video increases, coding Methods and techniques are ubiquitous in modern technology. Video codecs typically include electronic circuits or software that compress or decompress digital video and are used to stream high quality, high quality, and high quality digital video. coding It is constantly being improved to provide efficiency. Video codecs convert uncompressed video into a compressed format. They determine the video quality, the amount of data used to represent the video (determined by the bit rate), coding and Decoding There is a complex relationship between algorithm complexity, sensitivity to data loss and errors, ease of editing, random access, and end-to-end delay (latency). Compression formats are often based on standard video compression specifications, e.g., High Efficiency Video Compression (HVDC). coding(High Efficiency Video Coding) standard (also known as H.265 or MPEG-H Part 2), is being finalized. Versatile video coding (VVC) standard or other current and / or future Video coding It is standards compliant.

[0021] Embodiments of the disclosed technology improve runtime performance by using existing video streams. coding This document uses section headings to improve the readability of the description and in no way limit the description or embodiments (and / or implementations) to only each section.

[0022] 1 A brief review of HEVC 1.1 Intra Prediction in HEVC / H.265 Intra prediction involves generating samples for a given TB (transform block) using previously reconstructed samples in the considered color channel. Intra prediction modes are signaled separately for the luma and chroma channels, with the chroma channel intra prediction mode being optionally dependent on the luma channel intra prediction mode via the "DM_CHROMA" mode. Although the intra prediction mode is signaled at the PB (prediction block) level, according to the remaining quad-tree hierarchy of the CU, the intra prediction process is applied at the TB level, thereby generating samples for one TB. coding is the next TB in the CU coding , and therefore the distance to the sample used as the reference value is shortened.

[0023] HEVC includes 35 intra-prediction modes: DC mode, planar mode, and 33 directional or "angular" intra-prediction modes. The 33 angular intra-prediction modes are shown in FIG. 1.

[0024] For PBs associated with chroma color channels, the intra prediction mode is specified as either planar, DC, horizontal, vertical, "DM_CHROMA" mode, or sometimes the diagonal mode "34".

[0025] For chroma formats 4:2:2 and 4:2:0, a chroma PB may overlap with two or four luma PBs (respectively), in which case the luma direction of DM_CHROMA is taken from the top-left of these luma PBs.

[0026] The DM_CHROMA mode indicates that the intra prediction mode of the luma color channel PB is applied to the chroma color channel PB. Since this is relatively common, the most probable mode coding scheme of the intra_croma_pred_mode is biased to indicate this mode being selected.

[0027] 2. Example of intra prediction in VVC 2.1 Intra mode with 67 intra prediction modes coding To capture any edge direction present in natural video, the number of directional intra modes, as used in HEVC, is expanded from 33 to 65. The additional directional modes are depicted as red dotted arrows in Figure 2, and the planar and DC modes remain the same. These denser directional intra prediction modes apply for all block sizes and for both luma and chroma intra prediction.

[0028] 2.2 Cross-Component Linear Model (CCLM) Example In some embodiments, and to reduce cross-component redundancy, a cross-component linear model (CCLM) prediction mode (also called LM) is used in JEM: Chroma samples are predicted based on the reconstructed luma samples of the same CU by using a linear model as follows:

[0029] pred c (i,j)=α·rec L ′(i,j)+β (1)

[0030] Here, pred c (i,j) represents the predicted chroma sample in a CU, and rec L ′(i,j) represents the downsampled reconstructed luma sample of the same CU. The linear model parameters α and β are derived from the relationship between the luma and chroma values ​​from two samples, which are the luma sample with the minimum and maximum sample values ​​in the set of downsampled adjacent luma samples, and the corresponding chroma sample. Figure 3 shows an example of the location of the top left sample and one of the samples of the current block involved in CCLM mode.

[0031] This parameter calculation is Decoding It is performed as part of the process and not simply as an encoder search operation, and as a result no syntax is used to communicate the values ​​of α and β to the decoder.

[0032] Chroma Intra Mode coding Now, Chroma Intra mode coding A total of eight intra modes are allowed for Chroma mode. These modes include five traditional intra modes and three cross-component linear model modes (CCLM, LM_A, and LM_L). codingis directly dependent on the intra prediction mode of the corresponding luma block. Since separate block partition structures are enabled for luma and chroma components in I slices, one chroma block may correspond to multiple luma blocks. Therefore, for the chroma DM mode, the intra prediction mode of the corresponding luma block covering the center position of the current chroma block is directly inherited.

[0033] 2.3 Multiple Baseline (MRL) Intra Prediction Multiple reference line (MRL) intra prediction uses more reference lines for intra prediction. In FIG. 4, an example of four reference lines is shown, where samples of segments A and F are padded with the nearest samples from segments B and E, respectively, instead of being fetched from reconstructed neighboring samples. HEVC intra-picture prediction uses the nearest reference line (i.e., reference line 0). In MRL, two additional lines (reference line 1 and reference line 3) are used. The index of the selected reference line (mrl_idx) is signaled and used to generate the intra predictor. Reference line idx greater than 0 contains only additional reference line modes in the MPM list, and signals mpm index without remaining modes.

[0034] 2.4 Intra-Subpartition (ISP) The Intra Sub Partitions (ISP) tool splits a luma intra prediction block into two or four subpartitions vertically or horizontally depending on the block size. For example, the minimum block size for ISP is 4x8 (or 8x4). If the block size is more than 4x8 (or 8x4), the corresponding block is split into four subpartitions. Figure 5 shows an example of two possibilities. All subpartitions satisfy the condition of having 16 samples or more.

[0035] For each subpartition, the reconstructed samples are obtained by adding the residual signal to the prediction signal, where the residual signal has the entropy Decoding , inverse quantization, and inverse transform. Thus, the reconstructed sample values ​​of each subpartition are available to generate a prediction for the next subpartition, and each subpartition is processed iteratively. In addition, the first subpartition processed contains the top-left samples of the CU, and then continues downward (horizontal division) or rightward (vertical division). As a result, the reference samples used to generate the subpartition prediction signal are located only on the left and top of the line. All subpartitions share the same intra mode.

[0036] 2.5 Affine Linear Weighted Intra Prediction (ALWIP or Matrix-Based Intra Prediction) Affine linear weighted intra prediction (ALWIP, also known as matrix-based intra prediction (MIP)) has been proposed in JVET-N0217.

[0037] In JVET-N0217, two tests are conducted. In test 1, ALWIP is designed with a memory limit of 8K bytes and a maximum of 4 multiplications per sample. Test 2 is similar to test 1, but further simplifies the design in terms of memory requirements and model architecture.

[0038] - A single set of matrices and offset vectors for all block shapes

[0039] - Reduced number of modes to 19 for all block shapes

[0040] - Memory requirements reduced to 5760 10-bit values, or 7.20 kilobytes

[0041] - Linear interpolation of the predicted samples, as in the first test, is performed in a single step per direction, replacing the iterative interpolation

[0042] 2.5.12.5.1 JVET-N0217 Study 1 To predict samples of a rectangular block of width W and height H, Affine Linear Weighted Intra Prediction (ALWIP) takes as input one line H of reconstructed adjacent boundary samples to the left of the block and one line W of reconstructed adjacent boundary samples above the block. If reconstructed samples are not available, they are generated similarly to conventional intra prediction. The generation of the prediction signal is based on three steps:

[0043] From the boundary samples, 4 samples are extracted by averaging if W=H=4, and 8 samples in all other cases.

[0044] Using the averaged samples as input, a matrix-vector multiplication is performed followed by the addition of an offset, resulting in a reduced prediction signal for a subsampled set of samples in the original block.

[0045] The predictions at the remaining positions are generated from the predictions for the subsampled set by linear interpolation, which is a single-step linear interpolation in each direction.

[0046] The matrices and offset vectors required to generate the prediction signal are taken from three sets of matrices S0, S1, and S2. Set S0 contains 18 matrices A, each with 16 rows and 4 columns. i 0, i∈{0,...,17} and 18 offset vectors b i0, i ∈ {0,...,17}. The matrices and offset vectors in the set are used for blocks of size 4 × 4. The set S1 consists of 10 matrices A, each with 16 rows and 8 columns. i 1, i∈{0,...,9} and 10 offset vectors b i 1, i ∈ {0,...,9}. The matrices and offset vectors in the set are used for blocks of size 4 × 8, 8 × 4, and 8 × 8. Finally, the set S2 consists of 6 matrices A, each with 64 rows and 8 columns. i 2, i∈{0,...,5} and 6 offset vectors b of size 16 i 2, i ∈ {0,...,5}. Matrices and offset vectors from that set, or portions of these matrices and offset vectors, are used for all other block shapes.

[0047] The total number of multiplications required to compute the matrix-vector product is always less than or equal to 4×W×H. In other words, for the ALWIP mode, a maximum of 4 multiplications are required per sample.

[0048] 2.5.2 Boundary Equalization In the first step, the input boundary bdry top and bdry left is smaller than the boundary bdry top red and bdry left red where bdry top red and bdry left red are both made up of 2 samples for 4x4 blocks, and are both made up of 4 samples for all other cases.

[0049] For 4x4 blocks, for 0≦i<2, define the following:

number

[0050] And bdry left red is defined analogously.

[0051] Otherwise, the block width W is W=4·2 k Then, for 0≦i<4, define the following:

number

[0052] And bdry left red is defined analogously.

[0053] 2 reduced borders bdry top red and bdry left red is the reduced boundary vector bdry red which is of size 4 for blocks of shape 4x4 and of size 8 for blocks of all other shapes. When referring to the ALWIP mode, this concatenation is defined as follows:

[0054]

number

[0055] Finally, for the interpolation of the subsampled prediction signal, in large blocks, a second version of the averaging boundary is needed: if min(W,H)>8 and W≧H, then W=8*2 l and for 0≦i<8, define the following:

number

[0056] If min(W,H)>8 and H>W, then bdry leftredII is defined analogously.

[0057] 2.5.3 Generating reduced prediction signals by matrix-vector multiplication Reduced input vector bdry red From the reduced prediction signal pred red The latter signal has a width W red and height H red where W red and H red is defined as follows:

[0058]

number

[0059]

number

[0060] Reduced prediction signal pred red is calculated by computing the matrix vector product and adding an offset.

[0061] pred red =A·bdry red +b

[0062] where A is a matrix and W red H red rows, and 4 columns if W=H=4, and in all other cases. b is the size W red H red is a vector of

[0063] The matrix A and vector b are obtained from one of the sets S0, S1, S2 as follows: Let us define the index idx=idx(W,H) as follows:

[0064]

number

[0065] Furthermore, let m be as follows:

[0066]

number

[0067] Then, if idx≦1 or idx=2 and min(W,H)>4, then A=A m idx And b=b m idx If idx=2 and min(W,H)=4, then A=A m idx And b=b m idx It is. If idx=2 and min(W,H)=4, A is m idx Let A be the matrix resulting from removing all rows of A. If W=4, then let A be the matrix corresponding to odd x-coordinates in the downsampled block, or if H=4, then let A be the matrix corresponding to odd y-coordinates in the downsampled block.

[0068] Finally, the reduced prediction signal is replaced by its transpose if:

[0069] - W=H=4 and mode≧18

[0070] - max(W,H)=8 and mode≧10

[0071] - max(W,H)>8 and mode>6

[0072] pred red The number of multiplications required to compute A is 4 if W=H=4, because in this case A has 4 columns and 16 rows. In all other cases, A has 8 columns and W red Hred rows, and in these cases, 8·W red H red We immediately verify that multiplication by ≦4·W·H is necessary. That is, even in these cases, pred red To compute, up to four multiplications are required per sample.

[0073] 2.5.4 Diagram of the overall ALWIP process The whole process of averaging, matrix vector multiplication, and linear interpolation is illustrated for different shapes in Figures 6-9. Note that the remaining shapes are treated similarly to the one of the illustrated cases.

[0074] 1. Given a 4x4 block, ALWIP takes two averages along each axis of the boundary. The resulting 4 input samples go into a matrix vector multiplication. The matrix is ​​taken from set S0. After adding an offset, this results in 16 final predicted samples. No linear interpolation is required to generate the predicted signal. Thus, a total of (4·16) / (4·4)=4 multiplications are performed per sample.

[0075] 2. Given an 8x8 block, ALWIP takes 4 averages along each axis of the boundary. The resulting 8 input samples go into a matrix-vector multiplication. The matrix is ​​taken from set S1. This results in 16 samples in odd positions of the prediction block. Thus, a total of (8·16) / (8·8)=2 multiplications are performed per sample. After adding the offset, the samples are vertically interpolated by using the reduced top boundary. Horizontal interpolation follows by using the original left boundary.

[0076] 3. Given an 8x4 block, ALWIP takes the four averages along the horizontal axis of the boundary, and the four original boundary values ​​for the left boundary. The resulting eight input samples go into a matrix-vector multiplication. The matrix is ​​taken from set S1. This results in 16 samples at odd horizontal positions and at each vertical position of the prediction block. Thus, a total of (8·16) / (8·4)=4 multiplications are performed per sample. After adding the offset, these samples are horizontally interpolated using the original left boundary.

[0077] 4. Given a 16x16 block, ALWIP takes 4 averages along each axis of the boundary. The resulting 8 input samples go into a matrix vector multiplication. The matrix is ​​taken from set S2. This results in 64 samples in odd positions of the prediction block. Thus, a total of (8·64) / (16·16)=2 multiplications are performed per sample. After adding the offset, these samples are vertically interpolated by using the 8 averages of the top boundary. Horizontal interpolation follows by using the original left boundary. The interpolation process does not add any multiplications in this case. Thus, overall, 2 multiplications per sample are needed to compute the ALWIP prediction.

[0078] For larger shapes the procedure is essentially the same and it is easy to check that the number of multiplications per sample is less than four.

[0079] For W×8 blocks where W>8, only horizontal interpolation is needed since samples are provided at odd horizontal positions and at every vertical position.

[0080] Finally, for W×4 blocks where W>8, let A_kbe be the matrix resulting from excluding each row that corresponds to odd entries along the horizontal of the downsampled block. Thus, the output size is 32, and again, only horizontal interpolation remains to be performed.

[0081] When it is transposed, it is handled accordingly.

[0082] 2.5.5 Single-step linear interpolation For a W×H block where max(W,H)≥8, the prediction signal is the reduced prediction signal pred red ×H red for W, which is generated from. Depending on the block shape, linear interpolation is performed vertically, horizontally, or in both directions. When linear interpolation is applied in both directions, if W<H, it is applied first horizontally, and otherwise it is applied first vertically. red Without loss of generality, consider a W×H block where max(W,H)≥8 and W≥H. Then, one-dimensional linear interpolation is performed as follows. Without loss of generality, it is sufficient to describe the linear interpolation in the vertical direction. First, the reduced prediction signal is extended to the top by the boundary signal. Define the vertical upsampling factor U

[0083] =H / H ver =H / H red and write U ver =2 Uver Next, define the extended reduced prediction signal as follows.

[0084]

Equation

[0085] Then, from this extended reduced prediction signal, the vertical linear interpolation prediction signal is generated as follows.

[0086] pred ups,ver red [x][U ver ·y + k] = ((U ver ―k - 1)·pred red [x][y - 1] + (k + 1)·pred red [x][y] + U ver / 2) >> u ver

[0087] where 0≦x <W red , 0≦y <H red , and 0≦k ver

[0088] 2.5.6 Proposed Signaling of Intra Prediction Modes In intra mode coding For each prediction unit (CU), a flag is transmitted in the bitstream indicating whether the ALWIP mode applies to the corresponding prediction unit (PU). The signaling of the latter index is in line with the MRL as well as JVET-M0043. If the ALWIP mode applies, the ALWIP mode index predmode is signaled in 3MPMS using the MPM-list.

[0089] Here, the derivation of MPM is performed using the intra modes of the top and left PUs as follows: Angular There are three fixed tables that assign ALWIP mode to: map_angular_to_alwip idx , idx∈{0,1,2}

[0090] Predmode ALWIP =map_angular_to_alwip idx [predmode Angular ]

[0091] For each PU of width W and height H, an index is defined.

[0092] idx(PU) = idx(W,H)∈{0,1,2}

[0093] As in section 2.5.3, indicates from which of the three sets the ALWIP-parameters are taken.

[0094] Upper prediction unit Unit above ​is available, currently belongs to the same CTU as the PU, and is in intra mode, then idx(PU) = idx(PU above ), and ALWIP is PU above ALWIP mode predmode above ALWIP When applied in

[0095] mode above ALWIP =predmode above ALWIP

[0096] If the upper PU is available, belongs to the same CTU as the current PU, and is in intra mode, then the conventional intra prediction mode predmode above Angular If applies to the above PU, then put:

[0097] mode above ALWIP =map_angular_to_alwip idx(PUabove) [predmode above Angular ]

[0098] In all other cases:

[0099] mode above ALWIP =-1

[0100] It means that this mode is not available. Similarly, but without the constraint that the left PU must belong to the same CTU as the current PU, mode left ALWIP Derive the mode.

[0101] Finally, there are three fixed default lists: idx , idx∈{0,1,2}, each containing three different ALWIP modes. idx(PU) , and mode above ALWIPand mode left ALWIP From the above, we construct three separate MPMs by replacing the default values ​​with -1 and omitting repetitions.

[0102] The left and top neighboring blocks used in the ALWIP MPM list construction are A1 and B1 as shown in FIG.

[0103] 2.5.7 Adaptive MPM List Derivation for Traditional Luma and Chroma Intra Prediction Modes The proposed ALWIP mode is an MPM-based alternative to conventional intra prediction modes. coding The luma and chroma MPM list derivation process for conventional intra prediction modes is reconciled with the fixed table map_alwip_to_angular idx , idx∈{0,1,2}, and the ALWIP mode premode on a given PU ALWIP to one of the conventional intra prediction modes.

[0104] Premode Angular =map_alwip_to_angular idx(PU) [premode ALWIP ]

[0105] In the derivation of the Luma MPM list, the ALWIP mode premode ALWIP Whenever a neighboring luma block using the conventional intra prediction mode premode is encountered, this block is Angular In the derivation of the chroma MPM list, the same mapping is used to convert ALWIP modes to conventional intra prediction modes whenever the luma block currently uses LWIP modes.

[0106] 2.5.8 Corresponding revised working draft In some implementations, portions relating to intra_lwip_flag, intra_lwip_mpm_flag, intra_lwip_mpm_idx, and intra_lwip_mpm_lester have been added to the working draft in accordance with embodiments of the disclosed technology as described in this section.

[0107] In some embodiments, as described in this section, a working draft may be used to indicate additions and modifications to the working draft based on embodiments of the disclosed technology. <begin>Tags and <end>Tags are used. [Table 1] TIFF0007612795000011.tif237169TIFF0007612795000012.tif246168TIFF0007612 795000013.tif245169TIFF0007612795000014.tif239163TIFF0007612795000015.t if247169TIFF0007612795000016.tif47168TIFF0007612795000017.tif254168TIFF 0007612795000018.tif241169TIFF0007612795000019.tif175169TIFF00076127950 00020.tif250167TIFF0007612795000021.tif246169TIFF0007612795000022.tif24 2168TIFF0007612795000023.tif237168TIFF0007612795000024.tif241168TIFF000 7612795000025.tif242169TIFF0007612795000026.tif249169TIFF00076127950000 27.tif245169TIFF0007612795000028.tif244167TIFF0007612795000029.tif156168

[0108] Summary of ALWIP To predict samples of a rectangular block of width W and height H, Affine Linear Weighted Intra Prediction (ALWIP) takes as input one line of H reconstructed adjacent boundary samples to the left of the block and one line of W reconstructed adjacent boundary samples above the block. If reconstructed samples are not available, they are generated similarly to regular intra prediction. ALWIP is only applied to luma intra blocks. For chroma intra blocks, traditional intra prediction is used. Coding mode Applies.

[0109] The generation of the prediction signal is based on three steps:

[0110] 1. Of the boundary samples, 4 samples are extracted by averaging when W=H=4, and 8 samples are extracted otherwise.

[0111] 2. Using the averaged samples as input, a matrix-vector multiplication followed by the addition of an offset is performed, resulting in a reduced prediction signal for the set of subsampled samples in the original block.

[0112] 3 Predictions at the remaining positions are generated from the predictions in the subsampled set by linear interpolation, which is a single-step linear interpolation in each direction.

[0113] If an ALWIP mode is applied, the index of the ALWIP mode is signaled using the MPM-list in 3MPMS, where the derivation of the MPM is performed using the intra modes of the top and left PUs as follows: 3 fixed tables map_angular_to_alwip idx , idx∈{0,1,2}, where predmode is the conventional intra prediction mode. Angular Assign ALWIP mode to.

[0114] Predmode ALWIP =map_angular_to_alwip idx [predmode Angular ]

[0115] For each PU of width W and height H, an index is defined.

[0116] idx(PU) = idx(W,H)∈{0,1,2}

[0117] It indicates from which of the three sets the ALWIP-parameters are taken.

[0118] Upper prediction unit Unit above is available, currently belongs to the same CTU as the PU, and is in intra mode, then idx(PU) = idx(PU above ), and ALWIP is PU above ALWIP mode predmode above ALWIP When applied in

[0119] mode above ALWIP =predmode above ALWIP

[0120] If the upper PU is available, belongs to the same CTU as the current PU, and is in intra mode, then the conventional intra prediction mode predmode above Angular If applies to the above PU, then put:

[0121] mode above ALWIP =map_angular_to_alwip idx(PUabove) [predmode above Angular ]

[0122] In all other cases:

[0123] mode above ALWIP =-1

[0124] It means that this mode is not available. Similarly, but without the constraint that the left PU must belong to the same CTU as the current PU, mode left ALWIP Derive the mode.

[0125] Finally, there are three fixed default lists: idx , idx∈{0,1,2}, each containing three different ALWIP modes. idx(PU) , and mode above ALWIP and mode left ALWIP From the above, we construct three separate MPMs by replacing the default values ​​with -1 and omitting repetitions.

[0126] In the derivation of the Luma MPM list, the ALWIP mode premode ALWIP Whenever a neighboring luma block using the conventional intra prediction mode premode is encountered, this block is Angular is treated as if it were using

[0127] Premode Angular =map_alwip_to_angular idx(PU) [premode ALWIP ]

[0128] 3 Conversion in VVC 3.1 Multiple Transform Selection (MTS) In addition to the DCT-II used in HEVC, the Multiple Transform Selection (MTS) method is used to select residuals for both inter-coded and intra-coded blocks. coding It uses several transforms selected from DCT8 / DST7. The newly introduced transform matrices are DST-VII and DCT-VIII.

[0129] 3.2 Reduced Quadratic Transform (RST) proposed in JVET-N0193 The reduced secondary transform (RST) applies 16x16 and 16x64 non-separable transforms to 4x4 and 8x8 blocks, respectively. The primary forward and inverse transforms are still performed in the same way as the two one-dimensional horizontal / vertical transform passes. The secondary forward and inverse transforms are separate process steps from the primary transform. For the encoder, the primary forward transform is performed first, followed by the secondary forward transform and quantization, and the CABAC bitmap. encoding For the decoder, the CABAC bits Decoding and inverse quantization, then a secondary inverse transform is performed first, and then a primary inverse transform is performed. RST is applied only to intra-coded TUs in both intra slices and inter slices.

[0130] 3.3 Intra mode in JVET-N0185 coding Unified MPM for Multiple Reference Line Listings (MRL) and Intra-Subpartition Listings (ISP) coding Regardless of whether the tool is applied or not, a unified 6MPM list is proposed for intra blocks. The MPM list is built based on the left and above intra modes of adjacent blocks, as in VTM4.0. If the left mode is indicated as Left and the above block mode is indicated as Above, the unified MPM list is It is constructed as follows: If no adjacent blocks are available, the Intra mode defaults to Planar. If both Left and Above modes are non-angular modes: a. MPM list → {Planar, C, V, H, V-4, V+4} If one of the Left and Above modes is an angle mode and the other is a non-angle mode: Set mode Max as the larger mode in Left and Above. b. MPM list → {Planar, Max, DC, Max-1, Max+1, Max-2} If both the Left mode and the Above mode are angle modes and they are different: Set mode Max as the larger mode in Left and Above. b. If the difference between the Left mode and the Above mode is in the range of 2-62, i. MPM list → {Planar,Left,Above,DC,Max-1,Max+1} c. Otherwise, i. MPM list → {Planar,Left,Above,DC,Max-2,Max+2} If both the Left and Above modes are angle modes and they are the same: a. MPM list → {Planar, Left, Left-1, Left+1, DC, Left-2}

[0131] In addition, the first bin of the MPM index codeword is CABAC context coded. A total of three contexts are used, corresponding to whether the current intra block is MRL enabled, ISP enabled, or a normal intra block.

[0132] The left and top neighboring blocks used in the unified MPM list construction are A2 and B2 as shown in FIG.

[0133] One MPM flag is coded first. If the block is coded with one of the modes in the MPM list, the MPM index is coded further. Otherwise, the indices of the remaining modes (excluding MPM) are coded.

[0134] 4 Examples of shortcomings in existing implementations The design of ALWIP in JVET-N0217 has the following problems.

[0135] 1) At the March 2019 JVET meeting, a unified 6-MPM list generation was adopted for MRL, ISP, and normal intra modes. However, the affine linear weighted prediction mode uses a different 3-MPM list configuration, which complicates the MPM list configuration. The complex MPM list configuration can compromise the decoder throughput, especially for small blocks such as 4x4 samples.

[0136] 2) ALWIP applies only to the luma component of a block. For the chroma components of an ALWIP coded block, the chroma mode index is Coding and then transmitted to the decoder. This can result in unnecessary signaling.

[0137] 3) The interaction of ALWIP with other coding tools should be considered.

[0138] 4) If upsBdryX is calculated as follows: upsBdryX[x]=(

number

[0139] 5) When upsampling the prediction samples, no rounding is applied.

[0140] 6) In the deblocking process, ALWIP coded blocks are treated as normal intra blocks.

[0141] 5. Matrix-based intranet coding Exemplary Methods for DETAILED DESCRIPTION OF THE DISCLOSURE The embodiments of the techniques of the present disclosure overcome the shortcomings of existing embodiments, thereby providing video coding with higher coding efficiency but with less computational complexity. coding Provide video. coding The matrix-based intra prediction method for, as described in this paper, coding The potential for improving both standards is elucidated in the following examples, in which various implementations are described. The examples of the disclosed technology provided below are intended to illustrate general concepts and are not intended to be construed as limiting. In one example, the various features described in these examples can be combined, unless expressly indicated to the contrary.

[0142] In the following description, intra prediction mode refers to angular intra prediction mode (including DC, planar, CCLM, and other possible intra prediction modes), and intra mode refers to normal intra mode, or MRL, or ISP or ALWIP.

[0143] In the following description, "other intra modes" refers to one or more intra modes other than ALWIP, such as normal intra mode, MRL, or ISP.

[0144] In the following description, SatShift(x,n) is defined as follows:

[0145]

number

[0146] Shift(x,n) is defined as Shift(x,n)=(x+offset0)>>n.

[0147] In one example, offset0 and / or offset1 are (1<<n)> >1 or (1<<(n-1)). In another example, offset0 and / or offset1 are set to 0.

[0148] In another example, offset0=offset1=((1<n)> >1)-1 or (1<(n-1))-1.

[0149] Clip3(min,max,x) is defined as follows:

number

[0150] ALWIP MPM list configuration 1. It is proposed that all or part of the MPM list for ALWIP be constructed following all or part of the procedures for constructing MPM lists for non-ALWIP intra-modes (normal intra-mode, MRL, ISP, etc.). In one example, the size of the MPM list for ALWIP can be the same as the size of the MPM list for non-ALWIP intra mode. i. For example, in both ALWIP and non-ALWIP intra modes, the size of the MPM list is 6. b. In one example, the MPM list for ALWIP can be derived from the MPM list for non-ALWIP intra modes. In one example, an MPM list for non-ALWIP intra modes can be built first, after which some or all of them can be converted to MPMs, which can then be added to the MPM list of the ALWIP coded block. 1) Alternatively, pruning may also be applied when adding transformed MPMs to the MPM list of an ALWIP coded block. 2) The default mode can be added to the MPM list of the ALWIP coded block. In one example, a default mode can be added from the MPM list of non-ALWIP intra modes before being converted. b. Alternatively, the default mode can be added after conversion from the MPM list of non-ALWIP intra modes. c. Alternatively, the default modes may be added in an interleaved manner with the modes converted from the MPM list of non-ALWIP intra modes. d. In one example, the default mode can be fixed to be the same for all types of blocks. e. Instead, the default mode will use the available neighboring blocks, the mode information of the neighboring blocks, block dimensions, etc. Coding It may be determined according to the information. ii. In one example, an intra-prediction mode in an MPM list of non-ALWIP intra modes may be converted to a corresponding ALWIP intra-prediction mode when it is put into an ALWIP MPM list. 1) Alternatively, all intra-prediction modes in the MPM list of non-ALWIP intra modes may be converted to corresponding ALWIP intra-prediction modes before being used to construct the ALWIP MPM list. 2) Alternatively, all candidate intra prediction modes (which may include intra prediction modes from neighboring blocks and default intra prediction modes such as planar and DC) may be converted to corresponding ALWIP intra prediction modes before being used to construct the MPM list of non-ALWIP intra modes, provided that the MPM list of non-ALWIP intra modes can be further used to derive an ALWIP MPM list. 3) In one example, two transformed ALWIP intra prediction modes can be compared. In one example, if they are the same, only one of them will be put into the ALWIP MPM list. b. In one example, if they are the same, only one of them can be in the MPM list in intra modes other than ALWIP. In one example, K of S intra prediction modes in the MPM list of non-ALWIP intra modes may be selected as the MPM list of ALWIP modes, for example, K is equal to 3 and S is equal to 6. 1) In one example, the first K intra prediction modes of the MPM list for non-ALWIP intra modes may be selected as the MPM list for the ALWIP mode. 2. It is proposed that one or more neighboring blocks used to derive an MPM list for ALWIP may be used to derive an MPM list for a non-ALWIP intra mode (such as normal intra mode, MRL, or ISP). In one example, the neighboring blocks remaining for the current block used to derive the ALWIP MPM list should be the same as those used to derive the non-ALWIP intra mode MPM list. i. Assuming the top-left corner of the current block is (xCb, yCb) and the width and height of the current block are W and H, in one example, the left neighboring block used to derive the MPM lists for both ALWIP and non-ALWIP intra modes may cover the position (xCb-1, yCb). In another example, the left neighboring block used to derive the MPM lists for both ALWIP and non-ALWIP intra modes may cover the position (xCb-1, yCb+H-1). ii. For example, the left neighboring block and the top neighboring block used in the unified MPM list construction are A2 and B2 as shown in FIG. b. In one example, the neighboring blocks above the current block used to derive the ALWIP MPM list should be the same as those used to derive the non-ALWIP intra mode MPM list. i. Assuming the top left corner of the current block is (xCb, yCb) and the width and height of the current block are W and H, in one example, the neighboring blocks used to derive MPM lists for both ALWIP and non-ALWIP intra modes may cover position (xCb, yCb-1). In another example, the neighboring blocks used to derive MPM lists for both ALWIP and non-ALWIP intra modes may cover position (xCb+W-1, yCb-1). ii. For example, the left neighboring block and the top neighboring block used in the unified MPM list construction are A1 and B1 as shown in FIG. 3. We propose that the MPM list for ALWIP can be constructed in different ways depending on the width and / or height of the current block. In one example, different adjacent blocks can be accessed due to different block sizes. 4. We propose that MPM lists for ALWIP and for non-ALWIP intra modes can be constructed using the same procedure but with different parameters. In one example, K of S intra prediction modes in the MPM list construction procedure for non-ALWIP intra modes can be derived for the MPM list used in the ALWIP mode, for example, K is equal to 3 and S is equal to 6. i. In one example, the initial K intra-prediction modes in the MPM list construction procedure may be derived for the MPM list used in the ALWIP mode. b. In one example, the first mode in the MPM list may be different. i. For example, the first mode in an MPM list for non-ALWIP intra modes may be planar, but it may be mode X0 in an MPM list for ALWIP. 1) In one example, X0 may be an ALWIP intra prediction mode converted from Planar. c. In one example, the stuffing modes within the MPM list may differ. i. For example, the first three stuffing modes in the MPM list of a non-ALWIP intra mode may be DC, Vertical and Horizontal, but the modes in the MPM list of an ALWIP mode may be X1, X2, X3. 1) In one example, X1, X2, X3 may be different for different size Id. ii. In one example, the number of stuffing modes may vary. d. In one example, neighboring modes in the MPM list may be different. For example, the normal intra prediction modes of neighboring blocks are used to build MPM lists for non-ALWIP intra prediction modes, then convert them to ALWIP intra prediction modes and build MPM lists for ALWIP modes. e. In one example, the shifted modes in the MPM list may be different. For example, X+K0, where X is a normal intra prediction mode and K0 is an integer, can be placed in the MPM list for non-ALWIP intra modes, and Y+K1, where Y is an ALWIP intra prediction mode and K1 is an integer, can be placed in the MPM list for ALWIP, where K0 may be different from K1. 1) In one example, K1 may depend on the width and height. 5. When constructing the MPM list for the current block in non-ALWIP intra mode, if the neighboring block is ALWIP, Coding We propose that if so, the adjacent block be treated as unavailable. a. Instead, if a neighboring block is coded in ALWIP, it is treated as being coded in a predefined intra prediction mode (e.g., Planar) when building an MPM list for the current block in a non-ALWIP intra mode. 6. When constructing the MPM list for the current block in ALWIP mode, if the adjacent block is in non-ALWIP intra mode, Coding We propose that if so, the adjacent block be treated as unavailable. a. Instead, if a neighboring block is coded in a non-ALWIP intra mode when constructing the MPM list for the current block in ALWIP mode, it is treated as being coded in a predefined ALWIP intra prediction mode X. i. In one example, X may depend on block dimensions such as width and / or height. 7. Proposed removing storage of the ALWIP flag from the line buffer. In one example, if the second block to be accessed is in a different LCU / CTU row / region than the current block, the conditional check whether the second block is ALWIP coded is skipped. b. In one example, if the second block to be accessed is located in a different LCU / CTU row / region than the current block, the second block is treated the same as in non-ALWIP mode, such that it is treated as a normal intra-coded block. 8. ALWIP flag encoding When doing this, up to K contexts can be used (K>=0). In one example, K=1. 9. Instead of directly storing the mode index associated with the ALWIP mode, we proposed to store the transformed intra-prediction mode of the ALWIP coded block. In one example, one ALWIP coding block is associated with Decoding The given mode index is mapped to a regular intra mode, such as according to map_alwip_to_angular, as described in section 2.5.7. b. Alternatively, further storage of the ALWIP flag may be removed entirely. c. Alternatively, further, ALWIP mode storage is eliminated entirely. d. Alternatively, one of the adjacent / current blocks is also flagged as ALWIP. Coding The state check as to whether the setting is enabled may be omitted. e. Alternatively, one may further skip the mode conversion assigned to the ALWIP coded block and the normal intra prediction associated with one accessed block. ALWIP for different color components 10. If the corresponding luma block is in ALWIP mode Coding It is proposed that if so, an estimated chroma intra mode (eg, DM mode) may always be applied. In one example, a chroma intra mode is inferred to be a DM mode without a signal if the corresponding luma block is coded in ALWIP mode. b. In one example, the corresponding luma block may be one that covers the corresponding samples of chroma samples located at a given position (e.g., the top left of the current chroma block, the center of the current chroma block). c. In one example, the DM mode may be derived according to the intra prediction mode of the corresponding luma block, for example, by mapping the (ALWIP) mode to one of the regular intra modes. 11. The corresponding luma block of a chroma block is in ALWIP mode. Coding If so, several DM modes can be derived. 12. One corresponding luma block is in ALWIP mode Coding It was proposed that special modes be assigned to chroma blocks when In one example, a special mode is defined to be a given normal intra-prediction mode regardless of the intra-prediction mode associated with the ALWIP coded block. b. In one example, intra prediction can be assigned to this special mode in a different way. 13. We proposed that ALWIP can also be applied to chroma components. In one example, the matrices and / or bias vectors may be different for different color components. b. In one embodiment, a matrix and / or bias vector may be defined jointly for Cb and Cr. i. In one example, the Cb and Cr components can be linked. ii. In one example, the Cb and Cr components may be interleaved. c. In one example, a chroma component may share the same ALWIP intra prediction mode as the corresponding luma block. i. In one example, the same ALWIP intra prediction mode is applied to a chroma component if the corresponding luma block applies the ALWIP mode and the chroma block is coded in DM mode. ii. In one example, the same ALWIP intra prediction mode is applied to the chroma components, and subsequent linear interpolation can be skipped. iii. In one example, the same ALWIP intra prediction mode is applied to chroma components with subsampled matrices and / or bias vectors. d. In one example, the number of ALWIP intra prediction modes for different components may be different. i. For example, the number of ALWIP intra prediction modes for a chroma component may be less than the number for a luma component of the same block width and height. Applicability of ALWIP 14. We proposed to signal whether ALWIP is applicable or not. a. For example, it can be signaled at the sequence level (e.g., in the SPS), picture level (e.g., in the PPS or picture header), slice level (e.g., in the slice header), tile group level (e.g., in the tile group header), tile level, CTU row level, or CTU level. b. For example, if ALWIP is not applicable, intra_lwip_flag is not signaled and can be inferred to be 0. 15. It is proposed that whether ALWIP can be applied depends on the block width (W) and / or height (H). c. For example, when W >= T1 (or W > T1) and H >= T2 (or H > T2), ALWIP may not be applied. Example: T1 = T2 = 32; i. For example, when W <= T1 (or W < T1) and H <= T2 (or H < T2), ALWIP may not be applied. Example: T1 = T2 = 32; d. For example, when W >= T1 (or W > T1) or H >= T2 (or H > T2), ALWIP may not be applied. Example: T1 = T2 = 32; i. For example, when W <= T1 (or W < T1) or H <= T2 (or H < T2), ALWIP may not be applied. Example: T1 = T2 = 32; e. For example, when W + H >= T (or W * H > T), ALWIP is not applied. For example, T = 256; i. For example, when W + H <= T (or W + H < T), ALWIP is not applied. For example, T = 256; f. For example, when W * H >= T (or W * H > T), ALWIP is not applied. For example, T = 256; i. For example, when W * H <= T (or W * H < T), ALWIP is not applied. For example, T = 256; g. For example, if ALWIP cannot be applied, intra_lwip_flag may not be signaled and is presumed to be 0. Computational issues in ALWIP 16. It is proposed that for the shift operation involved in ALWIP, only a left shift or right shift of the number by S is possible if S must be greater than or equal to 0. a. In one example, when S is 0 or greater than 0, the right shift operation may be different. i. In one example, upsBdryX[x] is calculated as follows. When uDwn > 1, upsBdryX[x] = (

number

number

number

[0151] The above examples may be incorporated into the context of the methods described below, eg, methods 1100-1400 and 2000-2300, and may be implemented in a video encoder and / or decoder.

[0152] 11 illustrates a flowchart of an example method for video processing. The method 1100 includes, at step 1110, determining whether a current video block is predicted in an affine linear weighted intra-prediction mode. (ALWIP) Using Coding "Determining that the

[0153] The method 1100 includes, at step 1120, constructing at least a portion of a Most Probable Mode (MPM) list for the ALWIP mode based at least in part on the MPM list for the non-ALWIP intra mode based on the determination.

[0154] The method 1100 includes, at step 1130, performing a conversion between the current video block and a bitstream representation of the current video block based on the MPM list for the ALWIP mode.

[0155] In some embodiments, the size of the MPM list in the ALWIP mode is the same as the size of the MPM list in the non-ALWIP intra mode. In one example, the size of the MPM list in the ALWIP mode is 6.

[0156] In some embodiments, the method 1100 further includes inserting a default mode into the MPM list for the ALWIP mode. In one example, the default mode is inserted before the portion of the MPM list for the ALWIP mode that is based on the MPM list for the non-ALWIP intra mode. In another example, the default mode is inserted after the portion of the MPM list for the ALWIP mode that is based on the MPM list for the non-ALWIP intra mode. In yet another example, the default mode is inserted in an interleaved manner with the portion of the MPM list for the ALWIP mode that is based on the MPM list for the non-ALWIP intra mode.

[0157] In some embodiments, constructing the MPM list for the ALWIP mode and the MPM list for the non-ALWIP intra mode is based on one or more adjacent blocks.

[0158] In some embodiments, constructing the MPM list for the ALWIP mode and the MPM list for the non-ALWIP intra mode is based on the height or width of the current video block.

[0159] In some embodiments, constructing the MPM list for the ALWIP mode is based on a first set of parameters that differ from a second set of parameters used to construct the MPM list for the non-ALWIP intra mode.

[0160] In some embodiments, the method 1100 further includes determining that a neighboring block of the current video block is coded in ALWIP mode and designating the neighboring block as unavailable when constructing an MPM list for non-ALWIP intra modes.

[0161] In some embodiments, the method 1100 further includes determining whether a neighboring block of the current video block is in non-ALWIP intra mode. Coding and designating the adjacent block as unavailable when building an MPM list for the ALWIP mode.

[0162] In some implementations, the non-ALWIP intra mode is based on a normal intra mode, a multi-baseline intra prediction mode (MRL), or an intra sub-partition (ISP) tool.

[0163] 12 illustrates a flowchart of an example method for video processing. The method 1200 includes, at step 1210, predicting that a luma component of a current video block is predicted using an affine linear weighted intra prediction mode. Coding "Determining that the

[0164] The method 1200 includes, at step 1220, inferring a chroma intra mode based on the determination.

[0165] The method 1200 includes, at step 1230, performing a conversion between the current video block and a bitstream representation of the current video block based on the chroma intra mode.

[0166] In some implementations, the luma component covers a predefined chroma sample of the chroma component, hi one embodiment, the predefined chroma sample is the top-left sample or the center sample of the chroma component.

[0167] In some embodiments, the assumed chroma intra mode is a DM mode.

[0168] In some embodiments, the assumed chroma intra mode is an ALWIP mode.

[0169] In some embodiments, the ALWIP mode is applied to one or more chroma components of the current video block.

[0170] In some embodiments, different matrices or bias vectors in the ALWIP mode are applied to different color components of the current video block. In one example, different matrices or bias vectors are defined for the Cb and Cr components jointly. In another example, the Cb and Cr components are concatenated. In yet another example, the Cb and Cr components are interleaved.

[0171] 13 illustrates a flowchart of an example method for video processing. The method 1300 includes, at step 1310, determining whether a current video block is predicted using an affine linear weighted intra prediction (ALWIP) mode. Coding "Determining that the

[0172] The method 1300 includes, at step 1320, performing a conversion between the current video block and the bitstream representation of the current video block based on the determination.

[0173] In some embodiments, the decision is based on a signal in a sequence parameter set (SPS), a picture parameter set (PPS), a slice header, a tile group header, a tile header, a coding tree unit (CTU) row, or a CTU domain.

[0174] In some embodiments, the determination is based on the height (H) or width (W) of the current video block. In one example, W > T1 or H > T2. In another example, W ≥ T1 or H ≥ T2. In yet another example, W < T1 or H < T2. In yet another example, W ≤ T1 or H ≤ T2. In yet another example, T1 = 32 and T2 = 32.

[0175] In some embodiments, the determination is based on the height (H) or width (W) of the current video block. In one example, W + H ≤ T. In another embodiment, W + H ≥ T. In yet another embodiment, W × H ≤ T. In yet another embodiment, W × H ≥ T. In yet another embodiment, T = 256.

[0176] FIG. 14 shows a flowchart of an exemplary method for video processing. This method 1400 includes, at step 1410, determining that the current video block is to be coding used in a mode different from the affine linear weighted intra prediction (ALWIP) mode. Coding

[0177] Method 1400 includes, at step 1420, performing a conversion between the current video block and the bitstream representation of the current video block based on the determination.

[0178] In some embodiments, coding the mode is the combined intra and inter prediction (CIIP) mode, and method 1400 further includes the step of performing a selection between the ALWIP mode and the normal intra prediction mode. In one example, performing the selection is based on an explicit signaling in the bitstream representation of the current video block. In another example, performing the selection is based on a default rule. In yet another example, the default rule is that when the current video block is to be used in the CIIP mode Coding In yet another example, the default rule is that the video block currently uses CIIP mode. Coding When the prediction mode is changed, the normal intra prediction mode is always selected.

[0179] In some embodiments, coding The mode is a cross-component linear model (CCLM) prediction mode. In one example, the downsampling procedure of the ALWIP mode is based on the downsampling procedure of the CCLM prediction mode. In another example, the downsampling procedure of the ALWIP mode is based on a first set of parameters, and the downsampling procedure of the CCLM prediction mode is based on a second set of parameters different from the first set of parameters. In yet another example, the downsampling procedure for the ALWIP mode or the CCLM prediction mode includes at least one of a selection of a downsampling position, a selection of a downsampling filter, a rounding operation, or a clipping operation.

[0180] In some embodiments, the method 1400 further includes applying one or more of a shrinking quadratic transform (RST), a quadratic transform, a rotation transform, or a non-separable quadratic transform (NSST).

[0181] In some embodiments, the method 1400 further comprises: Coding applying digitally coded multi-level modulation (DPCM) or residual digital multi-level modulation (DPCM).

[0182] 6 Exemplary implementations of the disclosed technology FIG. 15 is a block diagram of a video processing device 1500. The device 1500 may be used to implement one or more methods described herein. The device 1500 may be embodied in a smartphone, a tablet, a computer, an Internet of Things (IoT) receiver, etc. The device 1500 may include one or more processors 1502, one or more memories 1504, and video processing hardware 1506. The processor 1502 may be configured to implement one or more methods described herein (including, but not limited to, methods 1100 to 1400 and 2000 to 2300). The memory(s) 1504 may be used to store data and codes used to implement the methods and techniques described herein. The video processing hardware 1506 may be used to implement some techniques described herein in hardware circuits.

[0183] In some embodiments, the video coding The method may be implemented using an apparatus implemented on a hardware platform such as that described with respect to FIG.

[0184] 16 is a block diagram illustrating an example video processing system 1600 in which various techniques disclosed herein may be implemented. Various implementations may include some or all of the components of system 1600. System 1600 may include an input 1602 for receiving video content. The video content may be received in a raw or uncompressed format, e.g., 8 or 10 bit multi-component pixel values, or may be compressed or encoding The input 1602 may be received in any format. The input 1602 may represent a network interface, a peripheral bus interface, or a storage interface. Examples of network interfaces include wired interfaces such as Ethernet, passive optical networks (PON), and wireless interfaces such as Wi-Fi or cellular interfaces.

[0185] The system 1600 may include various coding or encoding The method may be implemented coding It may include component 1604. coding The component 1604 receives the coding Component 1604 output can reduce the average bit rate of the video Coding Generate the expression: coding The techniques are sometimes referred to as video compression or video transcoding techniques. coding The output of component 1604 may be stored or transmitted over a communication link, as represented by component 1606. A stored or communicated bitstream (or code) associated with the video received at input 1602 may be transformation The resulting (decompressed) representation may be used by component 1608 to generate pixel values ​​or displayable video, which is sent to display interface 1610. The process of generating user-viewable video from the bitstream representation is sometimes called video decompression. Furthermore, certain video processing operations are referred to as "video decompression." coding " These are called operations or tools, coding The tool or operation is used with the encoder coding Invert the result of the corresponding Decoding It will be appreciated that the tools or operations are performed at the decoder.

[0186] Examples of peripheral bus interfaces or display interfaces may include Universal Serial Bus (USB) or High Definition Multimedia Interface (HDMI), or DisplayPort, etc. Examples of storage interfaces may include SATA (serial advanced technology attachment), PCI, IDE interfaces, etc. The techniques described herein may be embodied in a variety of electronic devices, such as mobile phones, laptops, smartphones, or other devices capable of performing digital data processing and / or video display.

[0187] Some embodiments of the disclosed techniques include making a decision or determination to enable a video processing tool or mode. In one example, if a video processing tool or mode is enabled, the encoder uses or implements the tool or mode in processing blocks of video, but may not necessarily modify the resulting bitstream based on the use of the tool or mode. That is, the conversion of blocks of video to a bitstream representation of video uses the video processing tool or mode if enabled based on the decision or determination. In another example, if a video processing tool or mode is enabled, the decoder processes the bitstream knowing that the bitstream has been modified based on the video processing tool or mode. That is, the conversion of a bitstream representation of video to blocks of video is performed using the video processing tool or mode enabled based on the decision or determination.

[0188] Some embodiments of the disclosed techniques include making a decision or determination to disable a video processing tool or mode. In one example, when a video processing tool or mode is disabled, an encoder does not use the tool or mode in converting blocks of video into a bitstream representation of the video. In another example, when a video processing tool or mode is disabled, a decoder processes the bitstream knowing that the bitstream has not been modified using the disabled video processing tool or mode based on the decision or determination.

[0189] FIG. 17 is an example video that can utilize the techniques of this disclosure. coding FIG. 17 is a block diagram showing a system 100. coding The system 100 may include a source device 110 and a destination device 120. The source device 110 may Encoded Generate video data and video encoding The destination device 120 may be referred to as a device. , encoded Video data Decoding Can be video Decoding The source device 110 may include a video source 112, a video encoder 114, and an input / output (I / O) interface 116.

[0190] The video source 112 may include sources such as a video capture device, an interface for receiving video data from a video content provider, and / or a computer graphics system for generating video data, or a combination of such sources. The video data may include one or more images. The video encoder 114 converts the video data from the video source 112 into encoding The bitstream is generated by Coding A bitstream may include a sequence of bits forming a representation. Coding The image may include a captured image and associated data. Coding The image is Coding The I / O interface 116 may include a modulator / demodulator (modem) and / or a transmitter. encoding The rendered video data may be transmitted directly to destination device 120 via I / O interface 116 over network 130a. encoding The rendered video data may also be stored on storage medium / server 130b for access by destination device 120.

[0191] The destination device 120 may include an I / O interface 126 , a video decoder 124 , and a display device 122 .

[0192] The I / O interface 126 may include a receiver and / or a modem. The I / O interface 126 receives data from the source device 110 or the storage medium / server 130b. Encoded The video decoder 124 can obtain video data. encoding The video data Deencoding The display device 122 can display the decoded video data to a user. The display device 122 can be integrated with the destination device 120 or can be external to the destination device 120 configured to interface with an external display device.

[0193] The video encoder 114 and the video decoder 124 are based on the High Efficiency Video Coding (HEVC) standard. Versatile It may operate in accordance with video compression standards, such as the Video Coding (VVM) standard, and other current and / or future standards.

[0194] FIG. 18 is a block diagram illustrating an example of a video encoder 200, which may be the video encoder 114 in the system 100 shown in FIG.

[0195] Video encoder 200 may be configured to perform any or all of the techniques of this disclosure. In the example of FIG. 18, video encoder 200 includes multiple functional components. The techniques described in this disclosure may be shared among various components of video encoder 200. In some examples, a processor may be configured to perform any or all of the techniques described in this disclosure.

[0196] Functional components of the video encoder 200 may include a partition unit 201, a mode selection unit 203, a motion estimation unit 204, a motion compensation unit 205, an intra prediction unit 206, a residual generation unit 207, a transform unit 208, a quantization unit 209, an inverse quantization unit 210, an inverse transform unit 211, a reconstruction unit 212, a buffer 213, and an entropy coding unit 214.

[0197] In other examples, the video encoder 200 may include more, fewer, or different functional components. In one example, the prediction unit 202 may include an intra block copy (IBC) unit. The IBC unit may perform prediction in an IBC mode, where at least one reference image is the image in which the current video block is located.

[0198] Furthermore, some components, such as the motion estimation unit 204 and the motion compensation unit 205, may be highly integrated, but are shown separately in the example of FIG. 18 for illustrative purposes.

[0199] The partition unit 201 may divide an image into one or more video blocks. The video encoder 200 and the video decoder 300 may support a variety of video block sizes.

[0200] The mode selection unit 203 may, for example, select based on the error result: coding One of the modes, intra or inter, can be selected, and the resulting intra- or inter-coded block is sent to the residual generation unit 207 for generating residual block data and to the residual generation unit 208 for use as a reference image. Encoded The mode selection unit 203 may provide the motion vector to the reconstruction unit 212 to reconstruct the block. In some examples, the mode selection unit 203 may select a combined intra- and inter-prediction (CIIP) mode, where prediction is based on an inter-prediction signal and an intra-prediction signal. The mode selection unit 203 may also select a resolution of the motion vector for the block (e.g., sub-pixel or integer pixel accuracy) in the case of inter-prediction.

[0201] To perform inter prediction on the current video block, motion estimation unit 204 may generate motion information for the current video block by comparing one or more reference frames from buffer 213 to the current video block. Motion compensation unit 205 may generate motion information for the current video block by comparing the motion information of pictures from buffer 213 other than the picture associated with the current video block and Decoding Based on the determined samples, a predicted video block for the current video block may be determined.

[0202] Motion estimation unit 204 and motion compensation unit 205 may perform different operations on the current video block depending on whether the current video block is in an I slice, a P slice, or a B slice, for example.

[0203] In some examples, motion estimation unit 204 may perform unidirectional prediction on the current video block, and motion estimation unit 204 may search reference pictures in list 0 or list 1 for reference video blocks for the current video block. Motion estimation unit 204 may then generate a reference index indicating a reference picture in list 0 or list 1, including the reference video block and a motion vector indicating a spatial displacement between the current video block and the reference video block. Motion estimation unit 204 may output the reference index, the prediction direction indicator, and the motion vector as motion information for the current video block. Motion compensation unit 205 may generate a prediction video block for the current block based on the reference video block indicated by the motion information of the current video block.

[0204] In another example, motion estimation unit 204 may perform bidirectional prediction for the current video block, and motion estimation unit 204 may search reference pictures in list 0 for a reference video block for the current video block and may also search reference pictures in list 1 for another reference video block for the current video block. Motion estimation unit 204 may then generate a reference index indicating the reference pictures in list 0 and list 1, which includes the reference video block and a motion vector indicating a spatial displacement between the current video block and the reference video block. Motion estimation unit 204 may output the reference index and the motion vector of the current video block as motion information of the current video block. Motion compensation unit 205 may generate a predictive video block for the current video block based on the reference video block indicated by the motion information of the current video block.

[0205] In some examples, the motion estimation unit 204 may include Decoding A full set of operational information can be output for processing.

[0206] In some examples, motion estimation unit 204 may not output a complete set of motion information for the current video. Rather, motion estimation unit 204 may signal the motion information of the current video block with reference to motion information of other video blocks. For example, motion estimation unit 204 may determine that the motion information of the current video block is sufficiently similar to the motion information of a neighboring video block.

[0207] In one example, motion estimation unit 204 may indicate, in a syntax structure associated with the current video block, a value that indicates to video decoder 300 that the current video block has the same motion information as another video block.

[0208] In another example, motion estimation unit 204 may identify another video block and a motion vector difference (MVD) in a syntax structure associated with the current video block. The motion vector difference indicates a difference between the motion vector of the current video block and the motion vector of the indicated video block. Video decoder 300 may determine the motion vector of the current video block using the motion vector of the indicated video block and the motion vector difference.

[0209] As mentioned above, the video encoder 200 can predictively signal motion vectors. Two examples of predictive signaling techniques that can be implemented by the video encoder 200 include advanced motion vector prediction (AMVP) and merge mode signaling.

[0210] Intra prediction unit 206 may perform intra prediction on the current video block. When intra prediction unit 206 performs intra prediction on the current video block, intra prediction unit 206 may also perform intra prediction on other video blocks in the same picture. Decoding Prediction data for the current video block may be generated based on the generated samples. The prediction data for the current video block may include a predicted video block and various syntax elements.

[0211] Residual generation unit 207 may generate residual data for the current video block by subtracting (e.g., indicated by a minus sign) the prediction video block for the current video block from the current video block. The residual data for the current video block may include residual video blocks that correspond to different sample components of the samples in the current video block.

[0212] In other examples, for example, in skip mode, residual data for the current video block may not exist and residual generation unit 207 may not perform a subtraction operation.

[0213] Transform processing unit 208 may generate one or more transform coefficient video blocks for the current video block by applying one or more transforms to the residual video block associated with the current video block.

[0214] After the transform processing unit 208 generates a transform coefficient video block associated with the current video block, the quantization unit 209 may quantize the transform coefficient video block associated with the current video block based on one or more quantization parameter (QP) values ​​associated with the current video block.

[0215] Inverse quantization unit 210 and inverse transform unit 211 may apply inverse quantization and inverse transform, respectively, to the transform coefficient video block to reconstruct a residual video block from the transform coefficient video block. Reconstruction unit 212 may add the reconstructed residual video block to corresponding samples from one or more prediction video blocks generated by prediction unit 202 to generate a reconstructed video block related to the current block for storage in buffer 213.

[0216] After reconstruction unit 212 reconstructs the video blocks, a loop filtering operation may be performed to reduce video blocking artifacts in the video blocks.

[0217] The entropy encoding unit 214 may receive data from other functional components of the video encoder 200. Once the entropy encoding unit 214 receives the data, the entropy encoding unit 214 may perform one or more entropy encoding operations to generate entropy encoded data and output a bitstream that includes the entropy encoded data.

[0218] FIG. 19 is a block diagram illustrating an example of a video decoder 300, which may be the video decoder 114 in the system 100 shown in FIG.

[0219] The video decoder 300 may be configured to perform any or all of the techniques of this disclosure. In the example of FIG. 19, the video decoder 300 includes multiple functional components. The techniques described in this disclosure may be shared among various components of the video decoder 300. In some examples, a processor may be configured to perform any or all of the techniques described in this disclosure.

[0220] In the example of Figure 19, the video decoder 300 includes an entropy decoding unit 301, a motion compensation unit 302, an intra prediction unit 303, an inverse quantization unit 304, an inverse transform unit 305, a reconstruction unit 306, and a buffer 307. The video decoder 300 may, in some examples, be implemented using the same techniques as those described with respect to the video encoder 200 (Figure 18). coding Reciprocal to the path Decoding The path can be executed.

[0221] The entropy decoding unit 301 may retrieve an encoded bitstream. The encoded bitstream may include entropy encoded video data (e.g., coded blocks of video data). The entropy decoding unit 301 may decode the entropy encoded video data, and from the entropy decoded video data, the motion compensation unit 302 may determine motion information including motion vectors, motion vector precision, reference picture list index, and other motion information. The motion compensation unit 302 may determine such information, for example, by implementing AMVP and merge mode.

[0222] The motion compensation unit 302 may generate motion compensation blocks, possibly performing interpolation based on a compensation filter. An identifier for the interpolation filter used with sub-pixel accuracy may be included in the syntax element.

[0223] The motion compensation unit 302 performs the motion compensation on the video block. encoding The motion compensation unit 302 may use the interpolation filters used by video encoder 200 during the calculation of the prediction values ​​for the sub-integer pixels of the reference block. The motion compensation unit 302 may determine the interpolation filters used by video encoder 200 according to received syntax information and use the interpolation filters to generate the prediction blocks.

[0224] The motion compensation unit 302 includes: encoding frames and / or slices of the video sequence encoding The size of the block used to encoding partition information describing how each macroblock of an image of a video sequence is partitioned; encoding a mode indicating whether the inter-coded block is to be encoded, one or more reference frames (and a reference frame list) for each inter-coded block, and encoding The resulting video sequence Decoding The system may use some of the syntax information to determine other information to be used.

[0225] The intra prediction unit 303 can form a prediction block from spatially adjacent blocks, for example, using an intra prediction mode received in the bitstream. The inverse quantization unit 303 can generate a prediction block from spatially adjacent blocks, for example, using an intra prediction mode received in the bitstream. No. The inverse transform unit 303 applies an inverse transform.

[0226] The reconstruction unit 306 sums the residual block with the corresponding prediction block generated by the motion compensation unit 202 or the intra prediction unit 303 to obtain: Decoding If desired, to remove blockiness artifacts, Decoding A deblocking filter may also be applied to filter the decoded blocks. Decoding The extracted video blocks are then stored in a buffer 307, which provides reference blocks for subsequent motion compensation / intra prediction and for presentation on a display device. Decoding It also generates a video of the video.

[0227] In some embodiments, in ALWIP or MIP mode, a prediction block for a current video block is determined by row and column wise averaging, followed by matrix multiplication, followed by interpolation to determine the prediction block.

[0228] 20 shows an example flow chart for an example method 2000 for matrix-based intra prediction. Operation 2002 includes performing a conversion between a current video block of a video and a bitstream representation of the current video block according to a rule, where the rule specifies a relationship between samples of the current video block and a matrix or offset value applied in a matrix weighted intra prediction (MIP) mode during the conversion, and where the MIP mode includes determining a prediction block of the current video block by performing a boundary downsampling operation, followed by a matrix vector multiplication operation, and optionally followed by an upsampling operation, on previously coded samples of the video.

[0229] In an embodiment of the method 2000, the rule specifies that the elements of the matrix applied in the MIP mode depend on the bit depth of the samples. In an embodiment of the method 2000, the rule specifies that the offset values ​​applied in the MIP mode depend on the bit depth of the samples. In an embodiment of the method 2000, the rule specifies that the elements of the matrix and the offset values ​​have M-bit precision for samples having N-bit precision, where M is less than or equal to N. In an embodiment of the method 2000, M is 8 and N is 10. In an embodiment of the method 2000, the bit depth of the samples is the same as the second bit depth of the input array for the color components. In an embodiment of the method 2000, the bit depth of the samples is the same as the second bit depth of the internal array or reconstructed samples for the color components. In some implementations of the method 2000, the color components include a luma component. In some embodiments of method 2000, the first set of parameters for the matrix and / or offset values ​​of the current video block are derived from a second set of parameters for the second set of matrices and / or the second set of offset values ​​of another video block. In some embodiments of method 2000, the current video block comprises an 8x8 video block, the other video block comprises a 4x4 video block, and the first set of parameters for the 16x8 matrix are derived from the second set of parameters for the 16x4 matrix.

[0230] 21 illustrates an example flow chart for an example method 2100 for matrix-based intra prediction. Operation 2102 includes generating an intermediate prediction block for a current video block using a matrix weighted intra prediction (MIP) mode, where the intermediate prediction block for the current video block is based on a previously selected prediction block of the video. Coding The intermediate prediction block may be determined by performing a boundary downsampling operation on the intermediate prediction block, followed by a matrix vector multiplication operation, and optionally followed by an upsampling operation. Operation 2104 includes generating a final prediction block based on the additional operations based on the intermediate prediction block. Operation 2106 includes performing a conversion between the current video block and a bitstream representation of the current video block based on the final prediction signal.

[0231] In some embodiments of the method 2100, the additional operation is a position dependent intra prediction combination (PDPC). In some embodiments of the method 2100, the first operation including generating a final prediction signal using PDPC is identical to the second operation including applying PDPC to a prediction signal generated using an intra prediction mode. In some embodiments of the method 2100, the intra prediction mode includes a planar mode or a DC mode. In some embodiments of the method 2100, the first operation including generating a final prediction signal using PDPC is identical to the second operation including applying PDPC to a prediction signal generated using an intra prediction mode, and the intra prediction mode is converted from a MIP mode.

[0232] In some embodiments of method 2100, the PDPC is applied to the intermediate predicted block of the current video block based on a rule. In an embodiment of method 2100, the rule indicates that the PDPC is applied to the intermediate predicted block of the current video block in response to a PDPC applied to a predicted signal generated by an intra-prediction mode converted from a MIP mode. In some embodiments of method 2100, the additional operation is a filtering operation in which boundary samples of the current video block are filtered with neighboring samples of the current video block. In some embodiments of method 2100, in MIP mode, Coding The filtering operation for filtering the boundary samples of the determined current video block is identical to another filtering operation for filtering the boundary samples using an intra prediction mode.

[0233] In some embodiments of the method 2100, the intra-prediction mode includes a planar mode or a direct current (DC) mode. In some embodiments of the method 2100, a filtering operation for filtering boundary samples of a current video block encoded in an MIP mode is identical to another filtering operation for filtering boundary samples using an intra-prediction mode, where the intra-prediction mode is converted from the MIP mode. In some embodiments of the method 2100, the filtering operation is applied based on a rule. In some embodiments of the method 2100, the rule indicates that a filtering operation is applied to filter the boundary samples in response to the boundary samples being filtered in an intra-prediction mode being converted from the MIP mode.

[0234] 22 illustrates an example flowchart for an example method 2200 for matrix-based intra prediction. Operation 2202 includes performing a conversion between a current video block of a video and a bitstream representation of the current video block, the conversion including predicting a plurality of samples of at least a portion of the current video block in a matrix weighted intra prediction (MIP) mode, where the prediction blocks of the portion of the current video block are based on previously-predicted samples of the current video block. Coding The boundary is determined by performing a downsampling operation on the sampled samples, followed by a matrix vector multiplication operation, and optionally followed by an upsampling operation.

[0235] In some embodiments of method 2200, the plurality of samples belong to a sub-block of a current video block, the current video block having a width (W) and a height (H), the sub-block having a width (sW) and a height (sH), and the plurality of samples of the sub-block are predicted by applying MIP to the sub-block. In some embodiments of method 2200, the plurality of samples of the sub-block having a width (sW) and a height (sH) include left neighboring samples of the current video block or top neighboring samples of the current video block. In some embodiments of method 2200, the plurality of samples belong to a sub-block of a current video block, the current video block having a width (W) and a height (H), the sub-block is a W / 2*H / 2 block to the top left of the current video block, and the plurality of samples of the sub-block are predicted by applying MIP to the sub-block. In some embodiments of method 2200, the multiple samples belong to a sub-block of a current video block, the current video block has a width (W) and a height (H), the sub-block is a left W / 2*H block of the current video block, and the multiple samples of the sub-block are predicted by applying MIP to the sub-block.

[0236] In some embodiments of method 2200, the plurality of samples belong to a sub-block of a current video block, the current video block having a width (W) and a height (H), the sub-block being W*H / 2 blocks above the current video block, and the plurality of samples of the sub-block are predicted by applying MIP to the sub-block. In some embodiments of method 2200, the plurality of samples belong to a sub-block of a current video block, the current video block having a width (W) and a height (H), the sub-block having a width (sW) and a height (sH), and the plurality of samples of the sub-block are predicted by applying MIP to the sub-block by using a left neighboring sample of the current video block or by using a top neighboring sample of the current video block.

[0237] In some embodiments of method 2200, the plurality of samples belong to a sub-block of a current video block, and the position of the sub-block is based on a relationship between a width (W) and a height (H) of the current video block. In some embodiments of method 2200, the sub-block is a left W / 2*H block of the current video block, where W≧H, and the plurality of samples of the sub-block are predicted by applying MIP to the sub-block. In some embodiments of method 2200, the sub-block is a top W*H / 2 block of the current video block, where H≧W, and the plurality of samples of the sub-block are predicted by applying MIP to the sub-block.

[0238] In some embodiments of method 2200, the sub-block is an upper-left W / 2*H / 2 block of the current video block, responsive to W=H, and the plurality of samples of the sub-block are predicted by applying MIP to the sub-block. In some embodiments of method 2200, the plurality of samples belong to a sub-block of the current video block, and the method further includes predicting a second set of samples of the current video block, where the second set of samples are located outside the sub-block, and where the second set of samples are predicted by applying MIP to the current video block.

[0239] In some embodiments of method 2200, the plurality of samples belong to a sub-block of the current video block, and where the method further includes predicting a second set of samples of the current video block, where the second set of samples are located outside the sub-block, and where the second set of samples are predicted by applying an intra-prediction mode to the current video block, and where the intra-prediction mode is converted from a MIP mode. In some embodiments of method 2200, the plurality of samples belong to a sub-block of the current video block, and where the method further includes predicting the second set of samples of the current video block, where the second set of samples are located outside the sub-block, and where the second set of samples are predicted by applying MIP to an area of ​​the current video block that excludes the sub-block.

[0240] In some embodiments of the method 2200, the plurality of samples belong to at least one sub-block of the current video block. In some embodiments of the method 2200, for each sub-block, the plurality of samples are predicted by applying MIP to the sub-block, and for each sub-block, MIP is applied to the sub-block by using neighboring reconstructed samples for the sub-block and / or by using neighboring predicted samples for the sub-block. In some embodiments of the method 2200, the neighboring reconstructed samples are used for sub-blocks located at boundaries of the current video block. In some embodiments of the method 2200, the neighboring reconstructed samples are used for sub-blocks located within the current video block such that a portion of a boundary of the sub-block is not coextensive with a portion of a boundary of the current video block. In some embodiments of the method 2200, the plurality of sub-blocks are predicted in raster-scan order. In some embodiments of the method 2200, the plurality of sub-blocks are predicted in zig-zag order.

[0241] In some embodiments of method 2200, a width and height of at least one sub-block are less than or equal to a maximum width and maximum height, respectively. In an embodiment, the method 2200 further includes dividing the current video block into a plurality of sub-blocks in response to any one or more of the widths and heights of the current video block being greater than or equal to a threshold. In some embodiments of method 2200, the threshold is predefined. In some embodiments of method 2200, the threshold is signaled in a sequence parameter set (SPS), a picture parameter set (PPS), a picture header, a slice header, a tile group header, or a tile header. In some embodiments of method 2200, the threshold is signaled in a sequence parameter set (SPS), a picture parameter set (PPS), a picture header, a slice header, a tile group header, or a tile header associated with the current video block. Coding In some embodiments of method 2200, Coding The information includes a block size of the current video block, a picture type of the current video block, or a temporal layer index of the current video block.

[0242] 23 shows an example flow chart for an example method 2300 for matrix-based intra prediction. Operation 2302 includes performing a conversion between a current video block of a video and a bitstream representation of the current video block, where the conversion is based on a rule indicating whether to filter neighboring samples of the current video block prior to applying a matrix weighted intra prediction (MIP) mode during the conversion, and where the MIP mode includes determining a predictive block of the current video block by performing a boundary downsampling operation, followed by a matrix vector multiplication operation, and optionally followed by an upsampling operation, on previously coded samples of the video.

[0243] In some embodiments of method 2300, the rule indicates that adjacent samples are filtered before being used in the MIP mode. In some embodiments of method 2300, the rule indicates that adjacent samples are not filtered before being used in the MIP mode. In some embodiments of method 2300, the rule indicates that adjacent samples are filtered before being used in the MIP mode in response to the MIP mode being equal to a particular value.

[0244] In this document, the terms "video processing" or "conversion" refer to video encoding ,video Decoding , video compression, or video decompression. For example, a video compression algorithm may be applied during the conversion of a pixel representation of a video to a corresponding bitstream representation, or vice versa. The bitstream representation of a current video block may correspond to bits being co-located or spread across different locations in the bitstream, e.g., as defined by a syntax. For example, a macroblock may be transformed and Coding in terms of the error residual values ​​calculated using the bits in the header and other fields in the bitstream. encoding Furthermore, during the conversion, the decoder may parse the bitstream with the knowledge that some fields may or may not be present, based on the decision, as described in the above solution. Similarly, the encoder may decide whether a particular syntax field is included or not, and accordingly Coding By including or excluding syntax fields in an expression, Coding A representation can be generated.

[0245] From the foregoing, it will be understood that, although specific embodiments of the disclosed technology have been described herein for purposes of illustration, various modifications can be made without departing from the scope of the invention. Accordingly, the presently disclosed technology is not to be limited except as by the appended claims.

[0246] The implementation and functional operation of the subject matter described in this patent document can be embodied in various systems, digital electronic circuitry, or computer software, firmware, or hardware, including the structures disclosed herein and their structural equivalents, or in one or more combinations thereof. The implementation of the subject matter described herein can be implemented in one or more computer program products, i.e., on a tangible and non-transitory computer-readable medium for performing by or controlling the operation of a data processing device. encoding The computer-readable medium may be implemented as one or more modules of computer program instructions written in a programmable logic circuit. The computer-readable medium may be a machine-readable storage device, a machine-readable storage substrate, a memory device, a machine-readable composition of matter affecting a propagated signal, or a combination of one or more thereof. The term "data processing unit" or "data processing apparatus" encompasses all apparatus, devices, and machines for processing data, including, for example, a programmable processor, a computer, or multiple processors or computers. In addition to hardware, an apparatus may include code that creates an execution environment for the computer program in question, such as code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or one or more combinations thereof.

[0247] Computer programs (also known as programs, software, software applications, scripts, or code) can be written in any form of programming language, including compiled or interpreted languages, and may be implemented as stand-alone programs or as modules, components, subroutines, or other units suitable for use in a computing environment. 、 A computer program can be deployed in any form, including a file in a file system. A computer program does not necessarily correspond to a file in a file system. A program can be stored as part of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in several coordinated files (e.g., a file that stores one or more modules, subprograms, or portions of code). A computer program can be deployed to be executed on several computers that are located on one computer or one site, or that are distributed across several sites and interconnected by a communications network.

[0248] The processes and logic flows described herein may be performed by one or more programmable processors executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows may also be performed by special purpose logic circuitry, such as an FPGA (Field Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit), or an apparatus.

[0249] Processors suitable for executing a computer program include, for example, both general purpose and special purpose microprocessors, and any one or more processors of any kind of digital computer. Typically, a processor receives instructions and data from a read-only memory or a random access memory, or both. The essential elements of a computer are a processor for executing instructions and one or more memory devices for storing instructions and data. Typically, a computer is also operatively coupled to receive data from, or transfer data to, one or more mass storage devices, e.g., magnetic disks, magneto-optical disks, or optical disks, for storing data. However, a computer need not have such devices. Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including, for example, semiconductor memory devices such as EPROM, EEPROM, and flash memory devices. The processor and memory may be supplemented by, or incorporated in, special purpose logic circuitry.

[0250] The specification, together with the drawings, are considered to be merely exemplary, where exemplary means an example. As used herein, the use of "or" is intended to include "and / or" unless the context clearly indicates otherwise.

[0251] Although this patent document contains many details, these should not be construed as limiting the scope of any invention or what may be claimed, but rather as descriptions of features specific to particular embodiments of a particular invention. Certain features described in this patent document in the context of separate embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented in multiple embodiments separately or in any suitable subcombination. Furthermore, although features may be described above as acting in a particular combination and initially claimed as such, one or more features from a claimed combination may, in some cases, be carved out of the combination, and the claimed combination may be directed to a subcombination or a variation of the subcombination.

[0252] Similarly, although the figures depict acts in a particular order, this should not be understood as requiring such acts to be performed in a particular order or sequential order, or to perform all of the acts illustrated, to achieve desired results. Furthermore, the separation of various system components in the embodiments described in this patent document should not be understood as requiring such separation in all embodiments.

[0253] Only a few embodiments and examples have been described; other embodiments, extensions and variations may be made based on what is described and explained in this patent document.< / end> < / begin>

Claims

1. 1. A method for processing video data, comprising the steps of: determining that a first intra mode is applied for a first video block of a video for a first conversion between the first video block of the video and a bitstream of the video; deriving reference samples for the first video block; performing a boundary downsampling operation, followed by a matrix vector multiplication operation, and optionally a subsequent upsampling operation, on the reference samples of the first video block based on a size of the first video block to generate prediction samples for the first video block; performing the first transform based on the predictive samples for the first video block; The matrix elements or offset values ​​applied in the matrix vector multiplication operation have a fixed precision, and The fixed precision is 8-bit precision, The prediction samples are determined in the upsampling operation according to the following formula: predSamples[xHor+dX][yHor]=((upHor−dX)*predSamples[xHor][yHor]+dX*predSamples[xHor+upHor][yHor]+offsetHor) / upHor, and predSamples[xVer][yVer+dY]=((upVer−dY)*predSamples[xVer][yVer]+dY*predSamples[xVer][yVer+upVer]+offsetVer) / upVer, where offsetHor and offsetVer are integers, upHor is a function of a predefined value based on the width of the current video block and the size of the current video block, upVer is a function of a predefined value based on the height of the current video block and the current video block size, dX runs from 1 to upHor-1, and dY runs from 1 to upVer-1, xHor is a position based on upHor, and yHor is a position based on upVer. method.

2. the precision of the first video block and the reference samples is 10 bits deep; The method of claim 1.

3. At least two blocks having different sizes share a matrix having the same size in the first intra mode. The method according to claim 1 or 2.

4. the reference samples are derived without invoking a reference sample filtering operation.

4. The method according to claim 1 .

5. the reference sample filtering operation is applied in a second intra mode different from the first intra mode. The method according to claim 4.

6. Whether the first intra mode is applied or not is specified by a first syntax element presented at a sequence level and a second syntax element presented at a coding unit level.

6. The method according to any one of claims 1 to 5.

7. At least one bin of the second syntax element is coded on a context basis. The method according to claim 6.

8. In response to a width-to-height ratio of the first video block being greater than 2, a context having an index of 3 is used for a first bin of the second syntax element. The method of claim 7.

9. and in response to a width-to-height ratio of the first video block being less than or equal to 2, a single context selected from contexts having an index of 0, 1, or 2 is used for a first bin of the second syntax element.

9. The method according to claim 7 or 8.

10. the boundary downsampling operation includes: deriving a boundary sample according to a rule in response to a number of shifted bits of the at least one reference sample being greater than zero by applying a right bit-shift operation to a sum of the at least one reference sample; 10. The method according to any one of claims 1 to 9.

11. The boundary sample y[x] is calculated using one of the following equations: If m>1, y[x]=( [0010] +(1<<(Log2(m)-1)))>>Log2(m) , or When m=1, y[x]=p[x] , where m is determined by the size of the first video block and a border size; where p[x] denotes the number of reference samples x, where >> denotes a right bit shift operation, where << denotes a left bit shift operation. The method of claim 10.

12. the first conversion includes encoding the first video block into the bitstream.

12. The method according to any one of claims 1 to 11.

13. the first conversion includes decoding the first video block from the bitstream.

13. The method according to any one of claims 1 to 12.

14. 1. An apparatus for processing video data, comprising: a processor; and a non-transitory memory containing instructions, the instructions, when executed by the processor, causing the processor to: determining that a first intra mode is to be applied for a first video block of a video for a first conversion between the first video block of the video and a bitstream of the video; Deriving a reference sample for the first video block; performing a boundary downsampling operation, followed by a matrix vector multiplication operation, and optionally a subsequent upsampling operation, on the reference samples of the first video block based on a size of the first video block to generate prediction samples for the first video block; and performing the first transformation based on the prediction samples for the first video block. Let them do so, the matrix elements or offset values ​​applied in said matrix vector multiplication operation have a fixed precision; The fixed precision is 8-bit precision, The prediction samples are determined in the upsampling operation according to the following formula: predSamples[xHor+dX][yHor]=((upHor−dX)*predSamples[xHor][yHor]+dX*predSamples[xHor+upHor][yHor]+offsetHor) / upHor, and predSamples[xVer][yVer+dY]=((upVer−dY)*predSamples[xVer][yVer]+dY*predSamples[xVer][yVer+upVer]+offsetVer) / upVer, where offsetHor and offsetVer are integers, upHor is a function of a predefined value based on the width of the current video block and the size of the current video block, upVer is a function of a predefined value based on the height of the current video block and the current video block size, dX runs from 1 to upHor-1, and dY runs from 1 to upVer-1, xHor is a position based on upHor, and yHor is a position based on upVer. Device.

15. A non-transitory computer-readable storage medium having instructions stored thereon, the instructions causing a processor to: determining that a first intra mode is to be applied for a first video block of a video for a first conversion between the first video block of the video and a bitstream of the video; Deriving a reference sample for the first video block; performing a boundary downsampling operation, followed by a matrix vector multiplication operation, and optionally a subsequent upsampling operation, on the reference samples of the first video block based on a size of the first video block to generate prediction samples for the first video block; and performing the first transformation based on the prediction samples for the first video block. the matrix elements or offset values ​​applied in said matrix vector multiplication operation have a fixed precision; The fixed precision is 8-bit precision, The prediction samples are determined in the upsampling operation according to the following formula: predSamples[xHor+dX][yHor]=((upHor−dX)*predSamples[xHor][yHor]+dX*predSamples[xHor+upHor][yHor]+offsetHor) / upHor, and predSamples[xVer][yVer+dY]=((upVer−dY)*predSamples[xVer][yVer]+dY*predSamples[xVer][yVer+upVer]+offsetVer) / upVer, where offsetHor and offsetVer are integers, upHor is a function of a predefined value based on the width of the current video block and the size of the current video block, upVer is a function of a predefined value based on the height of the current video block and the current video block size, dX runs from 1 to upHor-1, and dY runs from 1 to upVer-1, xHor is a position based on upHor, and yHor is a position based on upVer. A non-transitory computer-readable storage medium.

16. 1. A method for archiving a video bitstream, the method comprising: determining that a first intra mode is applied for a first video block of the video; deriving reference samples for the first video block; performing a boundary downsampling operation, followed by a matrix vector multiplication operation, and optionally a subsequent upsampling operation, on the reference samples of the first video block based on a size of the first video block to generate prediction samples for the first video block; generating the bitstream based on the predicted samples for the first video block; storing the bitstream on a non-transitory computer readable recording medium; Including, the matrix elements or offset values ​​applied in said matrix vector multiplication operation have a fixed precision; The fixed precision is 8-bit precision, The prediction samples are determined in the upsampling operation according to the following formula: predSamples[xHor+dX][yHor]=((upHor−dX)*predSamples[xHor][yHor]+dX*predSamples[xHor+upHor][yHor]+offsetHor) / upHor, and predSamples[xVer][yVer+dY]=((upVer−dY)*predSamples[xVer][yVer]+dY*predSamples[xVer][yVer+upVer]+offsetVer) / upVer, where offsetHor and offsetVer are integers, upHor is a function of a predefined value based on the width of the current video block and the size of the current video block, upVer is a function of a predefined value based on the height of the current video block and the current video block size, dX runs from 1 to upHor-1, and dY runs from 1 to upVer-1, xHor is a position based on upHor, and yHor is a position based on upVer. method.

Images