Constrained upsampling process in matrix-based intra prediction

Abstract

In one exemplary aspect, a method for digital video coding is described, including a matrix-based intra-prediction method for video coding. The method includes: performing a conversion between a current video block of a video and a bitstream representation of the current video block using a matrix-based intra-prediction (MIP) mode; the conversion includes performing an upsampling operation in which a final prediction block is determined by using a reduced prediction block of the current video block and by using reconstructed neighboring samples of the current video block according to a rule; and the reduced prediction block is obtained by performing a matrix-vector multiplication operation on reduced boundary samples of the current video block.

Description

[Technical Field]

[0001] [CROSS-REFERENCE TO RELATED APPLICATIONS] This application is the national phase stage of International Patent Application No. PCT / CN2020 / 092906, filed on May 28, 2020, which claims priority to and the benefit of International Patent Application No. PCT / CN2019 / 089590, filed on May 31, 2019. The entire disclosure of the above application is incorporated by reference into the disclosure of this application for all purposes under law.

[0002] [Technical field] This patent document coding Related to technologies, devices and systems. [Background technology]

[0003] Despite advances in video compression, digital video still accounts for the largest bandwidth usage on the Internet and other digital communication networks. As the number of connected user devices capable of receiving and displaying video increases, the bandwidth demands for digital video usage are expected to continue to increase. Summary of the Invention

[0004] Digital Video coding , and specifically, video coding The present invention relates to an apparatus, system, and method relating to a matrix-based intra prediction method for an existing video. coding Standards (e.g., High Efficiency Video Coding (HEVC) and future video coding It can be applied to both standards (eg, VVC (Versatile Video Coding)) or codecs.

[0005] In one exemplary aspect, the disclosed techniques may be used to provide a method for video processing. This exemplary method includes converting between a current video block of a video and a bitstream representation of the current video block using a matrix-based intra prediction (MIP) mode, in which a prediction block of the current video block is a prediction block of a previous video block. coding The transformation is determined by performing a boundary downsampling operation on the reduced boundary samples, followed by a matrix-vector multiplication operation, and optionally followed by an upsampling operation, and the transformation includes performing the boundary downsampling operation in a single stage in which the reduced boundary samples of the current video block are generated according to a rule based at least on reference boundary samples of the current video block, and the transformation includes performing a matrix-vector multiplication operation using the reduced boundary samples of the current video block.

[0006] In another exemplary aspect, the disclosed techniques may be used to provide a method for video processing. This exemplary method includes converting between a current video block of a video and a bitstream representation of the current video block using a matrix-based intra-prediction (MIP) mode, in which a final prediction block of the current video block is a prediction block of a previous bitstream representation of the video. coding The transformation includes performing an upsampling operation in which the final prediction block is determined by using a reduced prediction block of the current video block and by using reconstructed neighboring samples of the current video block according to a rule, and the reduced prediction block is obtained by performing a matrix-vector multiplication operation on the reduced boundary samples of the current video block.

[0007] In another exemplary aspect, the disclosed techniques may be used to provide a method for video processing, where a current video block is predicted using affine linear weighted intra prediction (ALWIP) mode. coding determining that the current video block will be a most probable mode (MPM) list for the ALWIP mode based on at least a portion of the MPM list for the non-ALWIP intra mode based on the determination; and performing a conversion between the current video block and a bitstream representation of the current video block based on the MPM list for the ALWIP mode.

[0008] In another exemplary aspect, the disclosed techniques can be used to provide a method for video processing. This exemplary method involves predicting the luma component of a current video block using an affine linear weighted intra prediction (ALWIP) mode. coding determining a chroma intra mode to be used; estimating a chroma intra mode based on the determination; and performing a conversion between the current video block and a bitstream representation of the current video block based on the chroma intra mode.

[0009] In yet another exemplary aspect, the disclosed techniques may be used to provide a method for video processing. This exemplary method includes: a current video block is predicted using affine linear weighted intra prediction (ALWIP) mode; coding and, based on the determination, performing a conversion between the current video block and a bitstream representation of the current video block.

[0010] In yet another exemplary aspect, the disclosed techniques can be used to provide a method for video processing, where a current video block is predicted in a different mode from an affine linear weighted intra prediction (ALWIP) mode. coding Using Mode coding and, based on the determination, performing a conversion between the current video block and a bitstream representation of the current video block.

[0011] In yet another representative aspect, the disclosed techniques may be used to provide a method for video processing. This example method includes generating a first prediction for a current video block using an affine linear weighted intra prediction (ALWIP) mode, generating a second prediction based on the first prediction using position dependent intra prediction combination (PDPC), and performing a conversion between the current video block and a bitstream representation of the current video block based on the second prediction.

[0012] In yet another exemplary aspect, the disclosed techniques may be used to provide a method for video processing. This exemplary method includes: a current video block is predicted using affine linear weighted intra prediction (ALWIP) mode; coding determining that a plurality of sub-blocks of the current video block will be coded based on the ALWIP mode; predicting a plurality of sub-blocks of the current video block based on the ALWIP mode; and performing a conversion between the current video block and a bitstream representation of the current video block based on the prediction.

[0013] In yet another representative aspect, a method of video processing is disclosed that includes determining a context of a flag indicating use of an affine linear weighted intra prediction (ALWIP) mode based on a rule for a current video block during conversion between a current video block and a bitstream representation of the current video block, predicting a plurality of sub-blocks of the current video block based on the ALWIP mode, and performing a conversion between the current video block and the bitstream representation of the current video block based on the prediction.

[0014] In yet another representative aspect, a method of video processing is disclosed, the method including: a current video block being predicted using an affine linear weighted intra prediction (ALWIP) mode; coding and performing at least two filtering stages on samples of the current video block in an upsampling process associated with the ALWIP mode during conversion between the current video block and a bitstream representation of the current video block, wherein a first precision of the samples in a first filtering stage of the at least two filtering stages is different from a second precision of the samples in a second filtering stage of the at least two filtering stages.

[0015] In yet another aspect, a method of video processing is disclosed, the method including: a current video block is predicted using an affine linear weighted intra prediction (ALWIP) mode; coding and performing at least two filtering stages on samples of the current video block in an upsampling process associated with the ALWIP mode during conversion between the current video block and a bitstream representation of the current video block, the upsampling process being performed in a fixed order when both vertical and horizontal upsampling are performed.

[0016] In yet another aspect, a method of video processing is disclosed, the method including: a current video block is predicted using an affine linear weighted intra prediction (ALWIP) mode; coding and performing at least two filtering stages on samples of the current video block in an upsampling process associated with the ALWIP mode during conversion between the current video block and a bitstream representation of the current video block, wherein the conversion includes performing a transposition operation before the upsampling process.

[0017] In yet another exemplary aspect, the above-described method is embodied in the form of processor-executable code and stored on a computer-readable program medium.

[0018] In yet another exemplary aspect, an apparatus configured or operable to perform the above-described method is disclosed. The apparatus may include a processor programmed to implement the method.

[0019] In yet another exemplary aspect, a video decoder device may implement the methods described herein.

[0020] These and other aspects and features of the disclosed technology are described in further detail in the drawings, specification, and claims. [Brief explanation of the drawings]

[0021] [Figure 1] An example of 33 intra prediction directions is shown. [Figure 2] An example of 67 intra prediction modes is shown. [Figure 3] 1 shows an example of sample locations used to derive the weights of a linear model. [Figure 4] 1 shows an example of four reference lines adjacent to a prediction block. [Figure 5A] 5A and 5B show examples of sub-partitions according to block sizes. [Figure 5B] 5A and 5B show examples of sub-partitions according to block sizes. [Figure 6] An example of ALWIP for a 4x4 block is shown. [Figure 7] An example of ALWIP for an 8x8 block is shown. [Figure 8] An example of ALWIP for an 8x4 block is shown. [Figure 9] An example of ALWIP for a 16x16 block is shown. [Figure 10] 1 shows an example of neighboring blocks used to build an MPM list. [Figure 11] 1 illustrates a flowchart of an example method for matrix-based intra prediction in accordance with the disclosed techniques. [Figure 12] 10 shows a flowchart of another example method for matrix-based intra prediction in accordance with the disclosed techniques. [Figure 13] 10 shows a flowchart of yet another example method for matrix-based intra prediction in accordance with the disclosed techniques. [Figure 14] 10 shows a flowchart of yet another example method for matrix-based intra prediction in accordance with the disclosed techniques. [Figure 15] FIG. 1 is a block diagram of an example hardware platform for implementing the visual media decoding or visual media encoding techniques described in this document. [Figure 16] 1 shows an example of adjacent blocks. [Figure 17] 1 is an example of the proposed reduced boundary sample generation. [Figure 18] 1 shows an example of the proposed upsampling using original reconstructed neighboring samples. [Figure 19] FIG. 2 is a block diagram illustrating an example of a video decoder. [Figure 20] FIG. 1 is a block diagram illustrating an example of a video processing system in which various techniques disclosed herein may be implemented. [Figure 21] FIG. 1 is a block diagram illustrating an example video coding system that can utilize techniques of this disclosure. [Figure 22] FIG. 1 is a block diagram illustrating an example of a video encoder. [Figure 23] 10 shows a flowchart of a further example method for matrix-based intra prediction in accordance with the disclosed techniques. [Figure 24] 10 shows a flowchart of a further example method for matrix-based intra prediction in accordance with the disclosed techniques. DETAILED DESCRIPTION OF THE INVENTION

[0022] With the increasing demand for higher resolution images, coding The methods and techniques have become ubiquitous in modern technology. Video codecs typically include electronic circuits or software that compress or decompress digital video, and are used to coding Video codecs are constantly being improved to provide greater efficiency. Video codecs convert uncompressed video into compressed formats and vice versa. There is a complex relationship between video quality, the amount of data used to represent the video (determined by the bitrate), the complexity of the encoding and decoding algorithms, sensitivity to data loss and errors, ease of editing, random access, and end-to-end delay (latency). Compressed formats are usually based on a standard, such as the High Efficiency Video Coding (HEVC) standard (also known as H.265 or MPEG-H Part 2), the upcoming Versatile Video Coding (VVC) standard, or other current and / or future video coding standards. coding It complies with standard video compression specifications, such as standards.

[0023] Embodiments of the disclosed technology utilize existing video streams to improve runtime performance. coding This document uses section headings to improve the readability of the description, but these section headings do not limit the description or embodiments (and / or implementations) to only those sections.

[0024] 1. A brief review of HEVC 1.1 Intra Prediction in HEVC / H.265 Intra prediction involves generating samples for a given TB (transform block) using previously reconstructed samples in the considered color channel. Intra prediction modes are signaled separately for the luma and chroma channels, and the chroma channel intra prediction mode optionally depends on the luma channel intra prediction mode via the 'DM_CHROMA' mode. Although the intra prediction mode is signaled at the PB (prediction block) level, the intra prediction process is applied at the TB level according to the remaining quadtree hierarchy of the CU, thereby generating samples for one TB. coding is the next TB in the CU coding , and therefore allows to shorten the distance to the sample used as reference.

[0025] HEVC includes 35 intra-prediction modes: DC mode, planar mode, and 33 directional or 'angular' intra-prediction modes. The 33 angular intra-prediction modes are shown in Figure 1.

[0026] For PBs associated with chroma color channels, intra prediction modes are defined as either planar mode, DC mode, horizontal mode, vertical mode, 'DM_CHROMA' mode, or sometimes diagonal mode '34'.

[0027] Note that in chroma formats 4:2:2 and 4:2:0, a chroma PB can overlap with two or four luma PBs (respectively), in which case the luma direction in DM_CHROMA is taken from the top-left of these luma PBs.

[0028] The DM_CHROMA mode indicates that the intra prediction mode of the luma color channel PB is applied to the chroma color channel PB. Since this is relatively common, the most probable mode coding scheme of intra_chroma_pred_mode is biased to favor this mode being selected.

[0029] 2. Example of intra prediction in VVC 2.1 Intra-mode coding with 67 intra-prediction modes To capture any edge direction present in natural video, the number of directional intra modes is expanded from 33 as used in HEVC to 65. The additional directional modes are shown as red dotted arrows in Figure 2, while the planar and DC modes remain the same. These denser directional intra prediction modes apply to all block sizes and to both luma and chroma intra prediction.

[0030] 2.2 Cross-Component Linear Model (CCLM) Example In some embodiments, and to reduce cross-component redundancy, a cross-component linear model (CCLM) prediction mode (also referred to as LM) is used in JEM, where chroma samples are calculated as follows:

number

[0031] where pred C (i,j) represents the predicted chroma sample in a CU, and recL'(i,j) represents the downsampled reconstructed luma sample of the same CU. The linear model parameters α and β are derived from the relationship between the luma and chroma values from two samples, the luma sample with the smallest sample value and the luma sample with the largest sample value and their corresponding chroma samples in the set of downsampled neighboring luma samples. Figure 3 shows an example of the positions of the samples of the current block and the left and top samples involved in CCLM mode.

[0032] This parameter calculation is performed as part of the decoding process, and not simply as an encoder search process. As a result, there is no syntax available to communicate the α and β values to the decoder.

[0033] In chroma intra mode coding, a total of eight intra modes are enabled for chroma intra mode coding. These modes include five traditional intra modes and three cross-component linear model modes (CCLM, LM_A, and LM_L). Chroma mode coding directly depends on the intra prediction mode of the corresponding luma block. In an I slice, separate block partition structures are possible for luma components and chroma components, so one chroma block may correspond to multiple luma blocks. Therefore, the chroma DM mode directly inherits the intra prediction mode of the corresponding luma block that covers the center position of the current chroma block.

[0034] 2.3 Multiple Reference Line (MRL) Intra Prediction Multiple reference line (MRL) intra prediction uses more reference lines for intra prediction. Figure 4 shows an example of four reference lines, where samples from segments A and F are padded with the nearest samples from segments B and E, respectively, rather than fetched from reconstructed neighboring samples. HEVC intra picture prediction uses the nearest reference line (i.e., reference line 0). In MRL, two additional lines (reference line 1 and reference line 3) are used. The index (mrl_idx) of the selected reference line is signaled and used to generate the intra predictor. For reference line idx greater than 0, only the additional reference line modes in the MPM list are included, and only the mpm index is signaled without the remaining modes.

[0035] 2.4 Intra-Subpartition (ISP) The Intra Sub-Partitions (ISP) tool divides a luma intra-prediction block into two or four sub-partitions vertically or horizontally depending on the block size. For example, the minimum block size in ISP is 4x8 (or 8x4). If the block size is larger than 4x8 (or 8x4), the corresponding block is divided into four sub-partitions. Figure 5 shows two possible examples. All sub-partitions satisfy the condition that they have at least 16 samples.

[0036] For each subpartition, a reconstructed sample is obtained by adding a residual signal to a prediction signal. Here, the residual signal is generated by processes such as entropy decoding, inverse quantization, and inverse transform. Therefore, the reconstructed sample values of each subpartition can be used to generate a prediction for the next subpartition, and each subpartition is processed iteratively. Furthermore, the first subpartition processed is the one containing the top-left sample of the CU, followed by the ones going downward (horizontal split) or right (vertical split). As a result, the reference samples used to generate the subpartition prediction signal are located only on the left and top of the line. All subpartitions share the same intra mode.

[0037] 2.5 Affine Linear Weighted Intra Prediction (ALWIP or Matrix-Based Intra Prediction) Affine linear weighted intra prediction (ALWIP, also known as matrix-based intra prediction (MIP)) is proposed in JVET-N0217.

[0038] Two tests are conducted in JVET-N0217. In test 1, ALWIP is designed with a memory constraint of 8K bytes and a maximum of 4 multiplications per sample. Test 2 is similar to test 1, but further simplifies the design in terms of memory requirements and model architecture: A single set of matrices and offset vectors for all block shapes. - Reduced the number of modes to 19 for all block shapes. · Memory requirements reduced to 5760 10-bit values or 7.20 kilobytes. A linear interpolation of the predicted samples is performed in a single step per direction, replacing the iterative interpolation as in the first test.

[0039] 2.5.1 Test 1 of JVET-N0217 To predict samples of a rectangular block of width W and height H, Affine Linear Weighted Intra Prediction (ALWIP) takes as input H reconstructed adjacent boundary samples of a line to the left of the block and W reconstructed adjacent boundary samples of a line above the block. If these reconstructed samples are not available, they are generated as is done in conventional intra prediction. The generation of the prediction signal is based on the following three steps:

[0040] From the boundary samples, four samples are extracted in the case where W=H=4, and eight samples in all other cases, by averaging.

[0041] Taking these averaged samples as input, a matrix-vector multiplication followed by the addition of an offset is performed. The result is a reduced prediction signal for a subsampled set of samples in the original block.

[0042] From the predicted signals for the subsampled set, the predicted signals for the remaining positions are generated by linear interpolation, which is a single-step linear interpolation in each direction.

[0043] The matrices and offset vectors required to generate the prediction signal are taken from three sets of matrices S0, S1, and S2. Set S0 contains 18 matrices A0, each with 16 rows and 4 columns. i ,i∈{0,…,17} and 18 offset vectors b0 each of size 16 i,i∈{0,…,17}. The matrices and offset vectors in this set are used for blocks of size 4x4. Set S1 consists of 10 matrices A1, each with 16 rows and 8 columns. i ,i∈{0,…,9} and 10 offset vectors b1, each of size 16 i ,i∈{0,…,9}. The matrices and offset vectors in this set are used for blocks of size 4×8, 8×4, and 8×8. Finally, the set S2 consists of six matrices A2, each with 64 rows and 8 columns. i , i∈{0,…,5} and 6 offset vectors b2 each of size 64 i , i∈{0,...,5}. This set of matrices and offset vectors, or a subset of these matrices and offset vectors, are used for all other block shapes.

[0044] The total number of multiplications required to compute a matrix-vector product is always less than or equal to 4 × W × H. In other words, at most 4 multiplications per sample are required in ALWIP mode.

[0045] 2.5.2 Boundary Equalization In the first step, the input boundary bdry top and bdry left but smaller boundary bdry red top and bdry red left where bdry red top and bdry red left In the 4x4 block case, both consist of 2 samples, and in all other cases, both consist of 4 samples.

[0046] In the case of 4x4 blocks, 0≦i<2,

number

[0047] Otherwise, if the block width W is given as W=4·2k, then 0≦i<4,

number

[0048] These two reduced boundaries bdry red top and bdry red left The boundary vector bdry decreases red which is therefore of size 4 for blocks of shape 4x4, and of size 8 for blocks of all other shapes. If mode refers to ALWIP mode, this concatenation is defined as follows:

number

[0049] Finally, for the interpolation of the subsampled prediction signal, a second version of the average boundary is needed for large blocks: if min(W,H)>8 and W≥H, then we write W=8×2l, with 0≤i<8,

number

[0050] Similarly, if min(W,H)>8 and H>W, then bdry redII left Define

[0051] 2.5.3 Generating a Reduced Prediction Signal by Matrix-Vector Multiplication The reduced input vector bdry red from the reduced predicted signal pred red The latter signal has a width W red and height H redwhere W red and H red teeth:

number

[0052] Decreased prediction signal pred red is calculated by taking the matrix vector product and adding the correction: pred red =A·bdry red +b

[0053] where A is W red H red is a matrix with 4 rows and 4 columns when W=H=4, and 8 columns in all other cases, and b is a matrix of size W red H red is a vector of

[0054] The matrix A and vector b are taken from one of the sets S0, S1, S2 as follows: Define the index idx=idx(W,H) as follows:

number

[0055] Also, let m be:

number

[0056] Then, if idx≦1 or idx=2 and min(W,H)>4, then A=A idx m and b=b idx m If idx=2 and min(W,H)=4, then in the case of W=4, the A corresponding to odd x coordinates in the downsampled block idx mor in the case of H=4, the rows of A corresponding to odd y coordinates in the downsampled block. idx m Let A be the matrix that results from excluding all rows of

[0057] Finally, the reduced prediction signal is replaced by its transpose if: W=H=4 and mode ≥ 18 max(W,H)=8 and mode≧10 max(W,H)>8 and mode≧6.

[0058] pred red The number of multiplications required to compute A is 4 in the case where W=H=4, because in this case A has 4 columns and 16 rows. In all other cases, A has 8 columns and W red H red In these cases, the 8 W red H red ≦4W H multiplications are required, i.e., even in these cases, pred red The number of multiplications required to compute σ is at most 4 per sample.

[0059] 2.5.4 Example of the entire ALWIP process The entire process of averaging, matrix-vector multiplication, and linear interpolation is illustrated for different shapes in Figures 6-9, where the remaining shapes are treated as in one of the cases shown.

[0060] 1. Given a 4x4 block, ALWIP takes two averages along each axis of the boundary. The resulting four input samples go into a matrix-vector multiplication. The matrix is taken from set S0. After adding an offset, this produces 16 final predicted samples. No linear interpolation is required to generate this predicted signal. Therefore, a total of (4 16) / (4 4) = 4 multiplications are performed per sample.

[0061] 2. Given an 8x8 block, ALWIP takes four averages along each axis of the boundary. The resulting eight input samples go into a matrix-vector multiplication. A matrix is taken from set S1. This produces 16 samples in the odd positions of the predicted block. Therefore, a total of (8 16) / (8 8) = 2 multiplications per sample are performed. After adding the offset, these samples are interpolated vertically using the reduced top boundary. This is followed by horizontal interpolation using the original left boundary.

[0062] 3. Given an 8x4 block, ALWIP takes the four averages along the horizontal axis of the boundary and the four original boundary values on the left boundary. The resulting eight input samples go into a matrix-vector multiplication. A matrix is taken from set S1. This produces 16 samples at odd horizontal positions and at each vertical position of the predicted block. Therefore, a total of (8 16) / (8 4) = 4 multiplications are performed per sample. After adding the offset, these samples are horizontally interpolated using the original left boundary.

[0063] 4. Given a 16x16 block, ALWIP takes four averages along each axis of the boundary. The resulting eight input samples enter matrix-vector multiplication. To generate the four averages, a two-stage downsampling operation is used. First, two consecutive samples are used to derive one downsampled value, thus obtaining eight averages of the top / left neighbors. Next, the eight averages per edge can be further downsampled to generate four averages per edge. The four averages are used to derive the reduced prediction signal. Furthermore, the eight averages per edge are further used to generate the final predicted sample via subsampling of the reduced prediction signal. A matrix is taken from set S2. This produces 64 samples in the odd positions of the prediction block. Therefore, a total of (8 64) / (16 16) = 2 multiplications per sample are performed. After applying the offset, these samples are vertically interpolated using the eight averages from the top boundary. Horizontal interpolation follows using the original left boundary. The interpolation process does not add any multiplications in this case, so in total, two multiplications per sample are required to compute the ALWIP prediction.

[0064] For larger shapes, the procedure is essentially the same, and it is easy to ensure that the number of multiplications per sample is less than four.

[0065] For Wx8 blocks where W>8, only horizontal interpolation is required since samples are provided at odd horizontal positions and at every vertical position.

[0066] Finally, for W × 4 blocks with W > 8, let A_k be the matrix resulting from excluding all rows corresponding to odd entries along the horizontal axis in the downsampled block. Thus, the output size is 32, and again only horizontal interpolation remains performed.

[0067] Transposed cases are also treated accordingly.

[0068] 2.5.5 Single-step linear interpolation For a W×H block where max(W,H)≥8, the prediction signal is the reduced prediction signal pred red for W red ×H red and is generated by linear interpolation from it. Depending on the block shape, linear interpolation is performed vertically, horizontally, or in both directions. When linear interpolation is applied in both directions, it is first applied horizontally if W < H, and vertically otherwise.

[0069] Without loss of generality, consider a W×H block where max(W,H)≥8 and W≥H. Then, one-dimensional linear interpolation is performed as follows. Without loss of generality, it is sufficient to describe the linear interpolation in the vertical direction. First, the reduced prediction signal is extended upward by the boundary signal. The vertical upsampling factor U ver =H / H red is defined, and U ver =2 uver >1 is described. Then, the extended reduced prediction signal is

Number

[0070] And from this extended reduced prediction signal, the vertically linearly interpolated prediction signal is for 0≤x<W red , 0≤y<H red , and 0≤k<U ver and is

Number

[0071] 2.5.6 Signaling of the proposed intra prediction modes For each in the intra mode codingFor each unit (CU), a flag is sent in the bitstream indicating whether the ALWIP mode is applied to the corresponding prediction unit (PU). The signaling of the latter index is harmonized with the MRL as in JVET-M0043. If the ALWIP mode is applied, the ALWIP mode index predmode is signaled using an MPM list with three MPMS.

[0072] Here, the MPM is derived using the intra modes of the top and left PUs as follows: Angular Three fixed tables map_angular_to_alwip that assign ALWIP mode to each of idx ,idx∈{0,1,2}exists: predmode ALWIP =map_angular_to_alwip idx [predmode Angular ]

[0073] For each PU of width W and height H, an index indicating from which of the three sets the ALWIP parameters are taken as in section 2.5.3: idx(PU)=idx(W,H)∈{0,1,2} Define

[0074] Upper prediction unit PU above is available, currently belongs to the same CTU as the PU, and is in intra mode, and idx(PU)=idx(PU above ) and ALWIP is in ALWIP mode predmode ALWIP above PU above When applied to

number

[0075] If the upper PU is available, belongs to the same CTU as the current PU, and is in intra mode, the conventional intra prediction mode predmode Angular above If applied to the upper PU,

number

[0076] In all other cases,

number

[0077] Finally, there are three fixed default lists, each containing three different ALWIP modes: idx , idx∈{0,1,2} is provided. Default list list idx(PU) and mode ALWIP above and mode ALWIP left From the above, we construct three different MPMs by replacing -1 with a default value and eliminating repetitions.

[0078] The left and top neighboring blocks used to construct the ALWIP MPM list are A1 and B1 shown in FIG.

[0079] 2.5.7 Adaptive MPM List Derivation for Traditional Luma and Chroma Intra Prediction Modes The proposed ALWIP mode is an alternative to the conventional MPM-based intra prediction mode. coding The luma and chroma MPM list derivation process for the traditional intra prediction modes is reconciled with the ALWIP mode predmode for a given PU. ALWIPA fixed table, map_alwip_to_angular, that maps idx , idx∈{0,1,2}: predmode Angular =map_alwip_to_angular idx(PU) [predmode ALWIP ]

[0080] In Luma MPM list derivation, ALWIP mode predmode ALWIP Whenever a neighboring luma block using the traditional intra prediction mode predmode is encountered, this block is treated as if it were a Angular In chroma MPM list derivation, whenever the current luma block uses LWIP mode, the same mapping is used to convert ALWIP mode to traditional intra prediction mode.

[0081] 2.5.8 Corresponding Change Working Draft In some embodiments, based on embodiments of the disclosed technology, as described in this section, portions relating to intra_lwip_flag, intra_lwip_mpm_flag, intra_lwip_mpm_idx, and intra_lwip_mpm_remainder have been added to the working draft.

[0082] In some embodiments, to represent additions and modifications to a working draft based on embodiments of the disclosed technology, as described in this section: <begin>Tags and <end>Tags are used. Syntax Table coding Unit Syntax [Table 1] TIFF0007722771000015.tif18166 semantic <begin>intra_lwip_flag[x0][y0] equal to 1 specifies that the intra prediction type for the luma sample is affine linear weighted intra prediction. intra_lwip_flag[x0][y0] equal to 0 specifies that the intra prediction type for the luma sample is not affine linear weighted intra prediction. If intra_lwip_flag[x0][y0] is not present, it is inferred to be equal to 0. The syntax elements intra_lwip_mpm_flag[x0][y0], intra_lwip_mpm_idx[x0][y0], and intra_lwip_mpm_remainder[x0][y0] specify the affine linear weighted intra prediction mode for the luma sample. The array indexes x0,y0 indicate the affine linear weighted intra prediction mode for the top-left luma sample of the picture. coding Specifies the location (x0,y0) of the top-left luma sample of the block. If intra_lwip_mpm_flag[x0][y0] is equal to 1, the neighboring intra-predicted sample is coding The affine linear weighted intra prediction mode is estimated from the unit. If intra_lwip_mpm_flag[x0][y0] is not present, it is inferred to be equal to 1. <end> intra_subpartitions_split_flag[x0][y0] specifies whether the intra subpartition split type is horizontal or vertical. If intra_subpartitions_split_flag[x0][y0] is not present, it is inferred as follows: - If intra_lwip_flag[x0][y0] is equal to 1, intra_subpartitions_split_flag[x0][y0] is inferred to be equal to 0. - Otherwise, the following applies: - if cbHeight is greater than MaxTbSizeY, intra_subpartitions_split_flag[x0][y0] is inferred to be equal to 0; - Otherwise (cbWidth is greater than MaxTbSizeY), intra_subpartitions_split_flag[x0][y0] is inferred to be equal to 1. Decryption Process 8.4.1 Intra prediction mode coding will be coding General decoding process for a unit The inputs to this process are: - the current coding the luma position (xCb, yCb) that defines the top left sample of the block relative to the top left luma sample of the current picture; - Current situation in Luma Sample coding The variable cbWidth, which specifies the width of the block, - Current situation in Luma Sample coding The variable cbHeight specifies the height of the block. - A variable treeType that specifies whether a single or a dual tree is used, and if a dual tree is used, whether the current tree corresponds to the luma or chroma component. The output of this process is the modified reconstructed picture before in-loop filtering. The quantization parameter derivation process specified in Section 8.7.1 determines the luma position (xCb, yCb), the current value in the luma sample, coding Block width cbWidth, current in luma samples coding It is called with the block height cbHeight and the variable treeType as input. If treeType is equal to SINGLE_TREE, or if treeType is equal to DUAL_TREE_LUMA, the decoding process for luma samples is specified as follows: - If pcm_flag[xCb][yCb] is equal to 1, the reconstructed picture is modified as follows: SL[xCb+i][yCb+j]= pcm_sample_luma[(cbHeight*j)+i]<<(BitDepthY-PcmBitDepthY), (8-6) with i=0..cbWidth-1,j=0..cbHeight-1 - Otherwise, the following applies: 1. The luma intra prediction mode is derived as follows: - if intra_lwip_flag[xCb][yCb] is equal to 1, the derivation process of the affine linear weighted intra prediction mode specified in Section 8.4.X is performed on the current luma position (xCb, yCb), luma sample coding The block width cbWidth and the current value in luma samples coding Called with the block height cbHeight as input; - Otherwise, the luma intra prediction mode derivation process specified in Section 8.4.2 is performed based on the luma position (xCb, yCb), the current coding The block width cbWidth and the current value in luma samples coding Called with the block height cbHeight as input; 2. The general decoding process for intra blocks specified in section 8.4.4.1 is called with inputs the luma position (xCb, yCb), the tree type treeType, the variable nTbW set equal to cbWidth, the variable nTbH set equal to cbHeight, the variable predModeIntra set equal to IntraPredModeY[xCb][yCb], and the variable cIdx set equal to 0, and the output is the modified reconstructed picture before in-loop filtering. … <begin> 8.4.X Affine Linear Weighted Intra Prediction Mode Derivation Process The inputs to this process are: - Currently Ruma coding the luma position (xCb, yCb) that defines the top left sample of the block relative to the top left luma sample of the current picture; - Current situation in Luma Sample coding The variable cbWidth, which specifies the width of the block, - Current situation in Luma Sample coding The variable cbHeight specifies the height of the block. In this process, an affine linear weighted intra prediction mode IntraPredModeY[xCb][yCb] is derived. IntraPredModeY[xCb][yCb] is derived by the following ordered steps: 1. The neighbor positions (xNbA, yNbA) and (xNbB, yNbB) are set equal to (xCb-1, yCb) and (xCb, yCb-1), respectively; 2. For X to be replaced by either A or B, the variable candLwipModeX is derived as follows: - The block availability derivation process [Ed.(BB): Neighbor Block Availability Check Process tbd] specified in Section 6.4.X is invoked with inputs location (xCurr, yCurr) set equal to (xCb, yCb) and neighboring location (xNbY, yNbY) set equal to (xNbX, yNbX), and its output is assigned to availableX; - A candidate affine linear weighted intra prediction mode candLwipModeX is derived as follows: - candLwipModeX is set equal to -1 if one or more of the following conditions are true; - variable availableX is equal to FALSE; - CuPredMode[xNbX][yNbX] is not equal to MODE_INTRA and mh_intra_flag[xNbX][yNbX] is not equal to 1; - pcm_flag[xNbX][yNbX] is equal to 1; - X is equal to B and yCb-1 is less than ((yCb >> CtbLog2SizeY) << CtbLog2SizeY); - Otherwise, the following applies: - The block size type derivation process defined in Section 8.4.X.1 is called with the current coding block width cbWidth in luma samples and the current coding block height cbHeight in luma samples as inputs, and its output is assigned to the variable sizeId; - If intra_lwip_flag[xNbX][yNbX] is equal to 1, the block size type derivation process defined in Section 8.4.X.1 is called with the width nbWidthX of the adjacent coding block and the height nbHeightX of the adjacent coding block in luma samples as inputs, and its output is assigned to the variable sizeIdX; - If sizeId is equal to sizeIdX, candLwipModeX is set equal to IntraPredModeY[xNbX][yNbX]; - Otherwise, candLwipModeX is set equal to -1; - Otherwise, candLwipModeX is derived using IntraPredModeY[xNbX][yNbX] and sizeId as defined in Table 8-X1; 3. candLwipModeList[x] for x = 0..2 is derived as follows using lwipMpmCand[sizeId] as defined in Table 8-X2: - If both candLwipModeA and candLwipModeB are equal to -1, the following applies: candLwipModeList[0]=lwipMpmCand[sizeId][0] (8-X1) candLwipModeList[1]=lwipMpmCand[sizeId][1] (8-X2) candLwipModeList[2]=lwipMpmCand[sizeId][2] (8-X3) - Otherwise, the following applies: - If candLwipModeA is equal to candLwipModeB, or if either candLwipModeA or candLwipModeB is equal to -1, the following applies: candLwipModeList[0]=(candLwipModeA!=-1)? candLwipModeA:candLwipModeB (8-X4) - If candLwipModeList[0] is equal to lwipMpmCand[sizeId][0], the following applies: candLwipModeList[1]=lwipMpmCand[sizeId][1] (8-X5) candLwipModeList[2]=lwipMpmCand[sizeId][2] (8-X6) - Otherwise, the following applies: candLwipModeList[1]=lwipMpmCand[sizeId][0] (8-X7) candLwipModeList[2]=(candLwipModeList[0]!=lwipMpmCand[sizeId][1])? lwipMpmCand[sizeId][1]:lwipMpmCand[sizeId][2] (8-X8) - Otherwise, the following applies: candLwipModeList[0]=candLwipModeA (8-X9) candLwipModeList[1]=candLwipModeB (8-X10) - If candLwipModeA and candLwipModeB are not both equal to lwipMpmCand[sizeId][0], the following applies: candLwipModeList[2]=lwipMpmCand[sizeId][0] (8-X11) - Otherwise, the following applies: - If candLwipModeA and candLwipModeB are not both equal to lwipMpmCand[sizeId][1], the following applies: candLwipModeList[2]=lwipMpmCand[sizeId][1] (8-X12) - Otherwise, the following applies: candLwipModeList[2]=lwipMpmCand[sizeId][2] (8-X13) 4. IntraPredModeY[xCb][yCb] is derived by applying the following steps: - if intra_lwip_mpm_flag[xCb][yCb] is equal to 1, IntraPredModeY[xCb][yCb] is set equal to candLwipModeList[intra_lwip_mpm_idx[xCb][yCb]; - Otherwise, IntraPredModeY[xCb][yCb] is derived by applying the following ordered steps: 1. For i=0..1, and for each i, j=(i+1)..2, if candLwipModeList[i] is greater than candLwipModeList[j], then both values are swapped as follows: (candLwipModeList[i],candLwipModeList[j])= Swap(candLwipModeList[i],candLwipModeList[j]) (8-X14) 2. IntraPredModeY[xCb][yCb] is derived by the following ordered steps: i. IntraPredModeY[xCb][yCb] is set equal to intra_lwip_mpm_remainder[xCb][yCb]; ii. For i equal to 0 to 2, inclusive, if IntraPredModeY[xCb][yCb] is greater than or equal to candLwipModeList[i], then the value of IntraPredModeY[xCb][yCb] is incremented by 1. The variable IntraPredModeY[x][y], where x=xCb..xCb+cbWidth-1 and y=yCb..yCb+cbHeight-1, is set equal to IntraPredModeY[xCb][yCb]. 8.4.X.1 Prediction Block Size Type Derivation Process The inputs to this process are: - Current situation in Luma Sample coding The variable cbWidth, which specifies the width of the block, - Current situation in Luma Sample coding The variable cbHeight specifies the height of the block. The output of this process is the variable sizeId. The variable sizeId is derived as follows: - if both cbWidth and cbHeight are equal to 4, sizeId is set equal to 0; - Otherwise, if both cbWidth and cbHeight are less than or equal to 8, then sizeId is set equal to 1; - Otherwise, sizeId is set equal to 2. [Table 2] [Table 3] <end> 8.4.2 Luma Intra Prediction Mode Derivation Process The inputs to this process are: - Currently Ruma coding the luma position (xCb, yCb) that defines the top left sample of the block relative to the top left luma sample of the current picture; - Current situation in Luma Sample coding The variable cbWidth, which specifies the width of the block, - Current situation in Luma Sample coding The variable cbHeight specifies the height of the block. In this process, the luma intra prediction mode IntraPredModeY[xCb][yCb] is derived. Table 8-1 defines the values and associated names for the intra prediction modes IntraPredModeY[xCb][yCb]. [Table 4] IntraPredModeY[xCb][yCb] is derived by the following ordered steps: 1. The neighbor positions (xNbA, yNbA) and (xNbB, yNbB) are set equal to (xCb-1, yCb+cbHeight-1) and (xCb+cbWidth-1, yCb-1), respectively; 2. For X to be replaced by either A or B, the variable candIntraPredModeX is derived as follows: - <begin>Block availability derivation process as specified in Section 6.4.X [Ed.(BB): Neighbor Block Availability Check Process tbd] <end>is called with the position (xCurr, yCurr) set equal to (xCb, yCb) and the neighboring position (xNbY, yNbY) set equal to (xNbX, yNbX), and its output is assigned to availableX; - The candidate intra prediction mode candIntraPredModeX is derived as follows: - If one or more of the following conditions are true, candIntraPredModeX is set equal to INTRA_PLANAR; - The variable availableX is equal to FALSE; - CuPredMode[xNbX][yNbX] is not equal to MODE_INTRA and clip_flag[xNbX][yNbX] is not equal to 1; - pcm_flag[xNbX][yNbX] is equal to 1; - X is equal to B and yCb - 1 is less than ((yCb >> CtbLog2SizeY) << CtbLog2SizeY); - Otherwise, candIntraPredModeX is derived as follows: - If intra_lwip_flag[xCb][yCb] is equal to 1, candIntraPredModeX is derived by the following ordered steps: i. The block size type derivation process defined in Section 8.4.X.1 is called with the width cbWidth of the current block in luma samples coding and the height cbHeight of the current block in luma samples coding as input, and its output is assigned to the variable sizeId; ii. candIntraPredModeX is derived using IntraPredModeY[xNbX][yNbX] and sizeId as defined in Table 8 - X3; - Otherwise, candIntraPredModeX is set equal to IntraPredModeY[xNbX][yNbX]; 3. The variables ispDefaultMode1 and ispDefaultMode2 are defined as follows: - if IntraSubPartitionsSplitType is equal to ISP_HOR_SPLIT, ispDefaultMode1 is set equal to INTRA_ANGULAR18 and ispDefaultMode2 is set equal to INTRA_ANGULAR5; - Otherwise, ispDefaultMode1 is set equal to INTRA_ANGULAR50 and ispDefaultMode2 is set equal to INTRA_ANGULAR63. … [Table 5] 8.4.3 Chroma Intra Prediction Mode Derivation Process The inputs to this process are: - a luma position (xCb, yCb) that defines the top left sample of the current chroma coding block relative to the top left luma sample of the current picture, - Current situation in Luma Sample coding The variable cbWidth, which specifies the width of the block, - Current situation in Luma Sample coding The variable cbHeight specifies the height of the block. In this process, the chrominance intra-prediction mode IntraPredModeC[xCb][yCb] is derived. The corresponding luma intra prediction mode lumaIntraPredMode is derived as follows: - If intra_lwip_flag[xCb][yCb] is equal to 1, lumaIntraPredMode is derived by the following ordered steps: i. The block size type derivation process specified in Section 8.4.X.1 determines the current size of the luma samples. coding Width of the block cbWidth and current in luma samples coding It is called with the block height cbHeight as input and its output is assigned to the variable sizeId; ii. The luma intra prediction mode is derived using IntraPredModeY[xCb+cbWidth / 2][yCb+cbHeight / 2] and sizeId as specified in Table 8-X3 and assigning the value of candIntraPredModeX to lumaIntraPredMode; Otherwise, lumaIntraPredMode is set equal to IntraPredModeY[xCb+cbWidth / 2][yCb+cbHeight / 2]. The chrominance intra prediction mode IntraPredModeC[xCb][yCb] is derived using intra_chroma_pred_mode[xCb][yCb] and lumaIntraPredMode as specified in Tables 8-2 and 8-3. … xxx. Intra-sample prediction <begin> The inputs to this process are: a sample position (xTbCmp, yTbCmp) that defines the top left sample of the current transform block relative to the top left sample of the current picture, - a variable predModeIntra that specifies the intra prediction mode; - a variable nTbW that defines the transform block width, - variable nTbH, which specifies the transformation block height; - coding The variable nCbW that specifies the block width, - coding The variable nCbH that defines the block height, - The variable cIdx that specifies the color component of the current block. The output of this process is the predicted samples predSamples[x][y] at x=0..nTbW-1, y=0..nTbH-1. The predicted samples predSamples[x][y] are derived as follows: - if intra_lwip_flag[xTbCmp][yTbCmp] is equal to 1 and cIdx is equal to 0, the affine linear weighted intra sample prediction process specified in clause 8.4.4.2.X1 is called with the position (xTbCmp, yTbCmp), the intra prediction mode predModeIntra, the transform block width nTbW and height nTbH as input, and its output is preSamples; - otherwise, the general intra sample prediction process specified in subclause 8.4.4.2.X1 is performed using the position (xTbCmp, yTbCmp), the intra prediction mode predModeIntra, the transform block width nTbW and height nTbH, coding It is called with the block width nCbW and height nCbH, and the variable cIdx as input, and its output is preSamples. 8.4.4.2.X1 Affine Linear Weighted Intra-Sample Prediction The inputs to this process are: a sample position (xTbCmp, yTbCmp) that defines the top left sample of the current transform block relative to the top left sample of the current picture, - a variable predModeIntra that specifies the intra prediction mode; - variable nTbW, which specifies the transform block width; - Variable nTbH that specifies the transformation block height. The output of this process is the predicted samples predSamples[x][y] at x=0..nTbW-1, y=0..nTbH-1. The block size type derivation process specified in section 8.4.X.1 is called with the transform block width nTbW and transform block height nTbH as input, and its output is assigned to the variable sizeId. The variables numModes, boundarySize, predW, predH, and predC are derived using sizeId as specified in Table 8-X4. [Table 6] The flag isTransposed is derived as follows: isTransposed=(predModeIntra>(numModes / 2))?1:0 (8-X15) The flags needUpsBdryHor and needUpsBdryVer are derived as follows: needUpsBdryHor=(nTbW>predW)?TRUE:FALSE (8X16) needUpsBdryVer=(nTbH>predH)?TRUE:FALSE (8X17) The variables upsBdryW and upsBdryH are derived as follows: upsBdryW=(nTbH>nTbW)?nTbW:predW (8-X18) upsBdryH=(nTbH>nTbW)?predH:nTbH (8-X19) The variables lwipW and lwipH are derived as follows: lwipW=(isTransposed==1)?predH:predW (8-X20) lwipH=(isTransposed==1)?predW:predH (8-X21) To generate the reference samples refT[x] at x=0..nTbW-1 and refL[y] at y=0..nTbH-1, the reference sample derivation process specified in Section 8.4.4.2.X2 is called with the sample position (xTbCmp, yTbCmp), the transform block width nTbW, and the transform block height nTbH as input, and the top reference samples refT[x] at x=0..nTbW-1 and the left reference samples refL[y] at y=0..nTbH-1 as output. For generation of boundary samples p[x] at x=0..2*boundarySize-1, the following applies: - The boundary reduction process specified in clause 8.4.4.2.X3 is called for the upper reference samples with inputs the block size nTbW, the reference sample refT, the boundary size boundarySize, the upsampling boundary flag needUpsBdryVer, and the upsampling boundary size upsBdryW, and with outputs the reduced boundary samples redT[x] at x = 0..boundarySize-1 and the upsampling boundary samples upsBdryT[x] at x = 0..upsBdryW-1. - The boundary reduction process specified in 8.4.4.2.X3 is called for the left reference sample with the block size nTbH, the reference sample refL, the boundary size boundarySize, the upsampling boundary flag needUpsBdryHor, and the upsampling boundary size upsBdryH as inputs, and with the reduced boundary samples redL[x] at x = 0..boundarySize-1 and the upsampling boundary samples upsBdryL[x] at x = 0..upsBdryH-1 as outputs. The reduced top and left boundary samples redT and redL are assigned to the boundary sample array p as follows: - if isTransposed equals 1, p[x] is set equal to redL[x] for x=0..boundarySize-1 and p[x+boundarySize] is set equal to redL[x] for x=0..boundarySize-1; - Otherwise, p[x] is set equal to redT[x] for x=0..boundarySize-1, and p[x+boundarySize] is set equal to redL[x] for x=0..boundarySize-1. For the intra sample prediction process according to predModeIntra, the following ordered steps are applied: 1. The affine linear weighted sample predLwip[x][y] at x=0..lwipW-1, y=0..lwipH-1 is derived as follows: - The variable modeId is derived as follows: modeId=predModeIntra-(isTransposed==1)?(numModes / 2):0 (8-X22) - The weight matrix mWeight[x][y], where x=0..2*boundarySize-1, y=0..predC*predC-1, is derived using sizeId and modoId as specified in Table 8-XX [TBD: Weight Matrix Addition]; - The bias vector vBias[y], where y=0..predC*predC-1, is derived using sizeId and modeId as specified in Table 8-XX [TBD: Bias Vector Addition]; - The variable sW is derived using sizeId and modeId as specified in Table 8-X5; - The affine linear weighted sample predLwip[x][y] at x=0..lwipW-1, y=0..lwipH-1 is derived as follows:

number

number

number

Table 8

Table 9

Table 10

[0083] Overview of ALWIP To predict samples of a rectangular block of width W and height H, affine linear weighted intra prediction (ALWIP) takes as input H reconstructed adjacent boundary samples of one line to the left of the block and W reconstructed adjacent boundary samples of one line above the block. If these reconstructed samples are not available, they are generated as is done in conventional intra prediction. ALWIP applies only to luma intra blocks. For chroma intra blocks, conventional intra prediction is used. coding The mode is applied.

[0084] The generation of the prediction signal is based on three steps:

[0085] 1. From the boundary samples, four samples are extracted in the case where W=H=4, and eight samples in all other cases, by averaging.

[0086] 2. Taking the averaged samples as input, a matrix-vector multiplication followed by the addition of an offset is performed. The result is a reduced prediction signal for a subsampled set of samples in the original block.

[0087] 3. From the predicted signals for the subsampled set, predicted signals for the remaining positions are generated by linear interpolation, which is a single-step linear interpolation in each direction.

[0088] When the ALWIP mode is applied, the index predmode of the ALWIP mode is signaled using an MPM list with three MPMS, where the derivation of the MPM is performed using the intra modes of the top and left PUs as follows: Conventional intra prediction mode predmode Angular Three fixed tables map_angular_to_alwip that assign ALWIP mode to each of idx ,idx∈{0,1,2}exists: predmode ALWIP =map_angular_to_alwip idx [predmode Angular ]

[0089] For each PU of width W and height H, an index indicating from which of the three sets the ALWIP parameters are taken: idx(PU)=idx(W,H)∈{0,1,2} Define

[0090] Upper prediction unit PU above is available, currently belongs to the same CTU as the PU, and is in intra mode, and idx(PU)=idx(PU above ) and ALWIP is in ALWIP mode predmode ALWIP above PU above When applied to

number

[0091] If the upper PU is available, belongs to the same CTU as the current PU, and is in intra mode, the conventional intra prediction mode predmode Angular above If applied to the upper PU,

number

[0092] In all other cases,

number

[0093] Finally, there are three fixed default lists, each containing three different ALWIP modes: idx , idx∈{0,1,2} is provided. Default list list idx(PU) and mode ALWIP above and mode ALWIP left From the above, we construct three different MPMs by replacing -1 with a default value and eliminating repetitions.

[0094] In Luma MPM list derivation, ALWIP mode predmode ALWIP Whenever a neighboring luma block using the traditional intra prediction mode predmode is encountered, this block is treated as if it were a Angular is treated as if it were using

number

[0095] 3. Conversion in VVC 3.1 MTS(Multiple Transform Selection) In addition to the DCT-II adopted in HEVC, a multi-transform selection (MTS) scheme is implemented for both inter and intra coding. coding Block Residual coding It uses several transforms selected from DCT8 / DST7. The newly introduced transform matrices are DST-VII and DCT-VIII.

[0096] 3.2 RST (Reduced Secondary Transform) proposed in JVET-N0193 RST applies 16x16 and 16x64 non-separable transforms to 4x4 and 8x8 blocks, respectively. The primary forward and inverse transforms are still performed in the same way as the two 1D horizontal / vertical transform passes. The secondary forward and inverse transforms are separate process steps from the primary transform. In the encoder, the primary forward transform is performed first, followed by the secondary forward transform and quantization, and CABAC bit encoding. In the decoder, the secondary inverse transform is performed first, followed by CABAC bit decoding and inverse quantization, and then the primary inverse transform. RST applies intra-slice and inverse transforms to both intra-slice and inter-slice. coding Applies to TU only.

[0097] 3.3 Unified MPM List for Intra-mode Coding in JVET-N0185 Multiple Reference Lines (MRL) and Intra-Subpartitions (ISP) coding Regardless of whether the tool is applied or not, a unified 6-MPM list is proposed for intra blocks. This MPM list is constructed based on the intra modes of the left and above neighboring blocks, as in VTM4.0. If we denote the left mode as Left and the above block mode as Above, the unified MPM list is as follows: When no adjacent blocks are available, the intra mode defaults to Planar. ·If modes Left and Above are both non-angle modes: MPM list → {Planar, DC, V, H, V-4, V+4} If one of the modes Left and Above is an angle mode and the other is a non-angle mode: a. Set the mode Max as the larger of Left and Above modes b. MPM list → {Planar, Max, DC, Max-1, Max+1, Max-2} ·If Left and Above are both in angle mode and are different: a. Set the mode Max as the larger mode between Left and Above; b. If the difference between modes Left and Above is between 2 and 62 inclusive: i. MPM list → {Planar,Left,Above,DC,Max-1,Max+1} c. Otherwise: i. MPM list → {Planar,Left,Above,DC,Max-2,Max+2} ·If Left and Above are both in angle mode and they are the same: a. MPM list → {Planar, Left, Left-1, Left+1, DC, Left-2}

[0098] In addition, the first bin of the MPM index codeword is coding A total of three contexts are used, depending on whether the current intra block is MRL enabled, ISP enabled, or a normal intra block.

[0099] The left and top neighboring blocks used to construct the unified MPM list are A2 and B2 shown in FIG.

[0100] First, one MPM flag is coding If the block is in one of the modes in the MPM list, coding If the MPM index is coding Otherwise, the indexes to the remaining modes (excluding MPM) are coding will be done.

[0101] 4 Examples of shortcomings in existing implementations The design of ALWIP in JVET-N0217 has the following problems: 1) At the JVET meeting in March 2019, a unified 6-MPM list generation was adopted for MRL mode, ISP mode, and normal intra mode. However, the affine linear weighted prediction mode uses a different 3-MPM list construction, which complicates the MPM list construction. The complex MPM list construction may impair decoder throughput, especially for small blocks such as 4x4 samples; 2) ALWIP applies only to the luma component of the block. coding For the chroma components of a block, the chroma mode index is coding and transmitted to the decoder, which may result in unnecessary signaling; 3) Other coding ALWIP and interaction with tools should be considered; 4) The following formula:

number

[0102] 5. Matrix-based intranet coding Exemplary methods for Embodiments of the technology disclosed herein address shortcomings of existing implementations, thereby enabling higher coding Efficiency and lower computational complexity of images coding Provide. coding The matrix-based intra prediction method for existing and future video sequences, as described in this paper, coding Both standards can be enhanced as will become apparent in the following examples, which are described with respect to various implementations. The examples of the disclosed technology provided below are intended to illustrate general concepts and are not meant to be construed as limiting. In an example, various features described in these examples can be combined, unless expressly stated otherwise.

[0103] In the following description, intra prediction mode refers to angular intra prediction mode (including DC, planar, CCLM, and other possible intra prediction modes), and intra mode refers to normal intra mode, or MRL, or ISP, or ALWIP.

[0104] In the following description, "other intra modes" may refer to one or more intra modes other than ALWIP, such as normal intra mode, or MRL, or ISP, for example.

[0105] In the following description, SatShift(x,n) is

number

[0106] In one example, offset0 and / or offset1 are (1<<n)> >1 or (1<<(n-1)). In another example, offset0 and / or offset1 are set to 0.

[0107] In another example, offset0=offset1=((1<<n)> >1)-1 or ((1<<(n-1)))-1.

[0108] Clip3(min,max,x) is

number

[0109] Building an MPM list for ALWIP 1. It is proposed that the MPM list for ALWIP may be constructed in whole or in part according to the procedure for constructing the MPM list for non-ALWIP intra-mode (e.g., normal intra-mode, MRL, or ISP); a. In one example, the size of the MPM list for ALWIP may be the same as the size of the MPM list for non-ALWIP intra mode; i. For example, in both ALWIP and non-ALWIP intra modes, the size of the MPM list is 6; b. In one example, the MPM list for ALWIP may be derived from the MPM list for non-ALWIP intra mode; i. In one example, first, an MPM list for non-ALWIP intra-mode may be constructed. Then, some or all of them may be added to the list for ALWIP intra-mode. coding It can be converted into an MPM that can be further added to the MPM list for the block; 1) Alternatively, you can use the converted MPM in ALWIP mode. coding Pruning may be applied when adding to the MPM list for a block; 2) Set the default mode to ALWIP coding May be added to the MPM list for blocking; a. In one example, a default mode may be added before the one converted from the MPM list of non-ALWIP intra modes; b. Alternatively, the default mode may be added after the one converted from the MPM list of non-ALWIP intra modes; c. Alternatively, a default mode may be added to be interleaved with those converted from the MPM list of non-ALWIP intra modes; d. In one example, the default mode may be fixed to be the same for all types of blocks; e. Instead, the default mode is determined based on, for example, the availability of neighboring blocks, the mode information of the neighboring blocks, the block dimensions, etc. coding may be determined in accordance with the information provided; ii. In one example, an intra-prediction mode in an MPM list for a non-ALWIP intra mode may be converted to a corresponding ALWIP intra-prediction mode when it is placed in an MPM list for ALWIP; 1) Alternatively, all intra prediction modes in the MPM list for non-ALWIP intra modes may be converted to corresponding ALWIP intra prediction modes before being used to construct the MPM list for ALWIP; 2) Alternatively, if the MPM list for a non-ALWIP intra mode can be further used to derive an MPM list for ALWIP, all candidate intra prediction modes (which may include intra prediction modes from neighboring blocks and default intra prediction modes such as planar and DC) may be converted to corresponding ALWIP intra prediction modes before being used to construct the MPM list for the non-ALWIP intra mode; 3) In one example, two transformed ALWIP intra prediction modes may be compared; In one example, if they are the same, only one of them may be put into the MPM list for ALWIP; b. In one example, if they are the same, only one of them may be put into the MPM list for non-ALWIP intra mode; iii. In one example, K of the S intra prediction modes in the MPM list for the non-ALWIP intra modes may be selected as the MPM list for the ALWIP mode, where K is equal to 3 and S is equal to 6; 1) In one example, the first K intra prediction modes in the MPM list for the non-ALWIP intra mode may be selected as the MPM list for the ALWIP mode; 2. It is proposed that one or more neighboring blocks used to derive an MPM list for ALWIP may also be used to derive an MPM list for non-ALWIP intra modes (e.g., normal intra mode, MRL, or ISP); a. In one example, the left neighboring block of the current block used to derive the MPM list for ALWIP should be the same as that used to derive the MPM list for non-ALWIP intra mode; i. Given that the top-left corner of the current block is (xCb, yCb) and the width and height of the current block are W and H, in one example, the left-neighboring block used to derive the MPM list for both ALWIP and non-ALWIP intra modes may cover the position (xCb-1, yCb). In an alternative example, the left-neighboring block used to derive the MPM list for both ALWIP and non-ALWIP intra modes may cover the position (xCb-1, yCb+H-1); ii. For example, the left adjacent block and the upper adjacent block used for constructing the unified MPM list are A2 and B2 shown in FIG. 10; b. In one example, the neighboring blocks above the current block used to derive the MPM list for ALWIP should be the same as those used to derive the MPM list for non-ALWIP intra mode; i. Given that the top left corner of the current block is (xCb, yCb) and the width and height of the current block are W and H, in one example, the upper neighboring block used to derive the MPM list for both ALWIP and non-ALWIP intra modes may cover the position (xCb, yCb-1). In an alternative example, the upper neighboring block used to derive the MPM list for both ALWIP and non-ALWIP intra modes may cover the position (xCb+W-1, yCb-1); ii. For example, the left adjacent block and the upper adjacent block used for constructing the unified MPM list are A1 and B1 shown in FIG. 10; 3. It is proposed that the MPM list for ALWIP can be constructed in different ways depending on the width and / or height of the current block; In one example, different adjacent blocks may be accessed for different block sizes; 4. It is proposed that MPM lists for ALWIP and MPM lists for non-ALWIP intra-mode can be constructed using the same procedure but different parameters; In one example, K of the S intra prediction modes in the MPM list construction procedure for non-ALWIP intra modes may be derived for the MPM list used in the ALWIP mode, for example, K is equal to 3 and S is equal to 6; i. In one example, the first K intra-prediction modes in the MPM list construction procedure may be derived for the MPM list used in the ALWIP mode; b. In one example, the first mode in the MPM list may be different; i. For example, the first mode in the MPM list for non-ALWIP intra modes may be Planar, but in the MPM list for ALWIP it may be Mode X0; 1) In one example, X0 may be an ALWIP intra prediction mode converted from Planar; c. In one example, the stuffing modes in the MPM list may be different; i. For example, the first three stuffing modes in the MPM list for non-ALWIP intra mode may be DC, Vertical, and Horizontal, but they may be Mode X1, X2, X3 in the MPM list for ALWIP; 1) In one example, X1, X2, X3 may be different for different sizeId; ii. In one example, the number of stuffing modes may be different; d. In one example, neighboring modes in the MPM list may be different; For example, the normal intra prediction modes of neighboring blocks are used to construct an MPM list for non-ALWIP intra modes, and then they are converted to ALWIP intra prediction modes to construct an MPM list for ALWIP modes; e. In one example, the shifted modes in the MPM list may be different; i. For example, X+K0 may be placed in the MPM list for a non-ALWIP intra mode, where X is a normal intra prediction mode and K0 is an integer, and Y+K1 may be placed in the MPM list for ALWIP, where Y is an ALWIP intra prediction mode and K1 is an integer, and K0 may be different from K1; 1) In one example, K1 may depend on width and height; 5. When building an MPM list for the current block in non-ALWIP intra mode, if a neighboring block is coding It is proposed that if a a. Alternatively, when building an MPM list for the current block in non-ALWIP intra mode, if a neighboring block is coding When the image is encoded, it is encoded in a predetermined intra prediction mode (e.g., Planar). coding be treated as being; 6. When building an MPM list for the current block in ALWIP intra mode, if a neighboring block is coding It is proposed that if a a. Alternatively, when building an MPM list for the current block in ALWIP intra mode, if a neighboring block is coding In case of a predetermined ALWIP intra mode X coding be treated as being; i. In one example, X may depend on the block dimensions, e.g., width and / or height; 7. It is proposed to remove the storage of the ALWIP flag from the line buffer; In one example, if the second block to be accessed is located in a different LCU / CTU row / region than the current block, the second block is ALWIP. coding The condition check is skipped; b. In one example, if the second block to be accessed is located in a different LCU / CTU row / region than the current block, the second block may be accessed, for example, in a normal intra- coding Treated the same as in non-ALWIP mode, i.e. treated as a block; 8. When encoding the ALWIP flag, up to K (K>=0) contexts may be used; a. In one example, K=1; 9. Instead of directly storing the mode index associated with the ALWIP mode, coding It is proposed to store the transformed intra prediction mode of the block; a. In one example, one ALWIP coding The decoded mode index associated with the block is mapped to a regular intra mode, e.g., according to map_alwip_to_angular as described in section 2.5.7; b. Alternatively or additionally, storage of the ALWIP flag is completely eliminated; c. Alternatively or additionally, ALWIP mode storage is completely eliminated; d. Or, in addition, one adjacent / current block has the ALWIP flag set. coding condition checks can be skipped; e. Alternatively, or in addition, ALWIP coding The mode assigned to the block and the transformation of the normal intra prediction associated with one block accessed may be skipped; ALWIP for different color components 10. The corresponding luma block is in ALWIP mode. coding It is proposed that when the estimated chroma intra mode (e.g., DM mode) can always be applied; In one example, the corresponding luma block is in ALWIP mode. coding If so, without signaling, the chromo-intra mode is presumed to be DM mode; b. In one example, the corresponding luma block may be the one that covers the corresponding sample of the chroma sample at a given position (e.g., top left of the current chroma block, center of the current chroma block); c. In one example, the DM mode may be derived according to the intra prediction mode of the corresponding luma block, such as by mapping that (ALWIP) mode to one of the normal intra modes; 11. The chroma block's corresponding luma block is in ALWIP mode. coding When this is done, several DM modes can be derived; 12. One corresponding luma block is in ALWIP mode coding It is proposed that a special mode be assigned to the chroma block if In one example, the special mode is ALWIP coding is determined to be a given normal intra-prediction mode regardless of the intra-prediction mode associated with the block; b. In one example, multiple different ways of intra prediction can be assigned to this particular mode; 13. It is proposed that ALWIP can also be applied to chroma components; In one example, the matrix and / or bias vector may be different for different color components; b. In one example, matrices and / or bias vectors may be jointly defined for Cb and Cr; i. In one example, the Cb and Cr components may be concatenated; ii. In one example, the Cb and Cr components may be interleaved; c. In one example, a chroma component may share the same ALWIP intra prediction mode as the corresponding luma block; i. In one example, the corresponding luma block is in ALWIP mode and the chroma block is in DM mode. coding the same ALWIP intra prediction mode is applied to the chroma components when ii. In one example, the same ALWIP intra prediction mode may be applied to the chroma components, and subsequent linear interpolation may be skipped; iii. In one example, the same ALWIP intra prediction mode is applied to the chroma components using subsampled matrices and / or bias vectors; d. In one example, the number of ALWIP intra prediction modes may be different for different components; i. For example, the number of ALWIP intra prediction modes for a chroma component may be less than that for a luma component of the same block width and height; Applicability of ALWIP 14. It is proposed that the applicability of ALWIP can be signalled; a. For example, it may be signaled at the sequence level (e.g., in an SPS), at the picture level (e.g., in a PPS or picture header), at the slice level (e.g., in a slice header), at the tile group level (e.g., in a tile group header), at the tile level, at the CTU row level, or at the CTU level; b. For example, if ALWIP cannot be applied, intra_lwip_flag may be presumed to be 0 without being signaled; 15. It is proposed that whether ALWIP can be applied may depend on the block width (W) and / or height (H); c. For example, when W >= T1 (or W > T1) and H >= T2 (or H > T2) (e.g., T1 = T2 = 32), it may be considered that ALWIP cannot be applied; i. For example, when W <= T1 (or W < T1) and H <= T2 (or H < T2) (e.g., T1 = T2 = 32), it may be considered that ALWIP cannot be applied; d. For example, when W >= T1 (or W > T1) or H >= T2 (or H > T2) (e.g., T1 = T2 = 32), it may be considered that ALWIP cannot be applied; i. For example, when W <= T1 (or W < T1) or H <= T2 (or H < T2) (e.g., T1 = T2 = 32), it may be considered that ALWIP cannot be applied; e. For example, when W + H >= T (or W + H > T) (e.g., T = 256), it may be considered that ALWIP cannot be applied; i. For example, when W + H <= T (or W + H < T) (e.g., T = 256), it may be considered that ALWIP cannot be applied; f. For example, when W * H >= T (or W * H > T) (e.g., T = 256), it may be considered that ALWIP cannot be applied; i. For example, when W * H <= T (or W * H < T) (e.g., T = 256), it may be considered that ALWIP cannot be applied; g. For example, if ALWIP cannot be applied, intra_lwip_flag may be presumed to be 0 without being signaled; ALWIP computational problems 16. It is proposed that for the shift operation involved in ALWIP, assuming S must be non - negative, it can only shift the number left or right by S only; In one example, the right shift operation may be different when S is 0 or greater than 0; i. In one example, upsBdryX[x] is When uDwn>1,

number

number

number

number

number

number

number

number

number

number

number

number

number

[0110] The above examples may be incorporated in the context of the methods described below, eg, methods 1100-1400 and 2300-2400, which may be implemented in, for example, a video encoder and / or decoder.

[0111] 11 shows a flowchart of an exemplary method for video processing. Method 1100 begins at step 1110 with determining whether a current video block is predicted using affine linear weighted intra prediction (ALWIP) mode. coding This includes determining whether the

[0112] Based on the determination, the method 1100 includes, at step 1120, constructing at least a portion of a most probable mode (MPM) list for the ALWIP mode based at least in part on the MPM list for the non-ALWIP intra mode.

[0113] Method 1100 includes, at step 1130, performing a conversion between the current video block and a bitstream representation of the current video block based on the MPM list for the ALWIP mode.

[0114] In some embodiments, the size of the MPM list for ALWIP mode is the same as the size of the MPM list for non-ALWIP intra mode. In one example, the size of the MPM list for ALWIP mode is 6.

[0115] In some embodiments, the method 1100 further includes inserting a default mode into the MPM list for the ALWIP mode. In one example, the default mode is inserted before the portion of the MPM list for the ALWIP mode that is based on the MPM list for the non-ALWIP intra mode. In another example, the default mode is inserted after the portion of the MPM list for the ALWIP mode that is based on the MPM list for the non-ALWIP intra mode. In yet another example, the default mode is inserted interleaved with the portion of the MPM list for the ALWIP mode that is based on the MPM list for the non-ALWIP intra mode.

[0116] In some embodiments, constructing the MPM list for the ALWIP mode and the MPM list for the non-ALWIP intra mode is based on one or more neighboring blocks.

[0117] In some embodiments, building the MPM list for ALWIP mode and the MPM list for non-ALWIP intra mode is based on the height or width of the current video block.

[0118] In some embodiments, constructing the MPM list for the ALWIP mode is based on a first parameter set that is different from a second parameter set used to construct the MPM list for the non-ALWIP intra mode.

[0119] In some embodiments, the method 1100 further includes determining that a neighboring block of the current video block is coded in ALWIP mode and designating the neighboring block as unavailable when constructing an MPM list for non-ALWIP intra mode.

[0120] In some embodiments, the method 1100 further includes determining that a neighboring block of the current video block is coded in a non-ALWIP mode and designating the neighboring block as unavailable when constructing an MPM list for the ALWIP mode.

[0121] In some embodiments, the non-ALWIP intra mode is based on a normal intra mode, a multiple reference line (MRL) intra prediction mode, or an intra sub-partition (ISP) tool.

[0122] 12 shows a flowchart of an example method for video processing. Method 1200 includes, at step 1210, predicting that the luma component of a current video block is predicted using affine linear weighted intra prediction (ALWIP) mode. coding This includes determining whether the

[0123] The method 1200 includes, at step 1220, estimating a chroma intra mode based on the determination.

[0124] Method 1200 includes, at step 1230, performing a conversion between the current video block and a bitstream representation of the current video block based on a chroma intra mode.

[0125] In some embodiments, the luma component covers a predetermined chroma sample of the chroma component, hi one example, the predetermined chroma sample is the top-left or center sample of the chroma component.

[0126] In some embodiments, the estimated chroma intra mode is a DM mode.

[0127] In some embodiments, the estimated chroma intra mode is an ALWIP mode.

[0128] In some embodiments, the ALWIP mode is applied to one or more chroma components of the current video block.

[0129] In some embodiments, different matrices or bias vectors of the ALWIP mode are applied to different color components of the current video block. In one example, the different matrices or bias vectors are jointly determined for the Cb and Cr components. In another example, the Cb component and the Cr component are concatenated. In yet another example, the Cb component and the Cr component are interleaved.

[0130] FIG. 13 shows a flowchart of an exemplary method for video processing. Method 1300 includes, at step 1310, determining that the current video block is to be coding performed using an Affine Linear Weighted Intra Prediction (ALWIP) mode.

[0131] Method 1300 includes, at step 1320, based on the determination, performing a conversion between the current video block and the bitstream representation of the current video block.

[0132] In some embodiments, the determination is based on signaling in a Sequence Parameter Set (SPS), a Picture Parameter Set (PPS), a slice header, a tile group header, a tile header, coding a Coding Tree Unit (CTU) row, or a CTU region.

[0133] In some embodiments, the determination is based on the height (H) or width (W) of the current video block. In one example, W>T1 or H>T2. In another example, W≧T1 or H≧T2. In yet another example, W<T1 or H<T2. In yet another example, W≦T1 or H≦T2. In yet another example, T1 = 32 and T2 = 32.

[0134] In some embodiments, the determination is based on the height (H) or width (W) of the current video block. In one example, W+H≦T. In another example, W+H≧T. In yet another example, W×H≦T. In yet another example, W×H≧T. In yet another example, T=256.

[0135] 14 shows a flowchart of an example method for video processing. The method 1400 begins at step 1410 with determining whether the current video block is in a different mode from the affine linear weighted intra prediction (ALWIP) mode. coding Using Mode coding This includes determining whether the

[0136] Method 1400 includes, at step 1420, performing a conversion between the current video block and a bitstream representation of the current video block based on the determination.

[0137] In some embodiments, coding The mode is a combined intra-inter prediction (CIIP) mode, and method 1400 further includes selecting between the ALWIP mode and the normal intra prediction mode. In one example, performing the selection is based on explicit signaling in a bitstream representation of the current video block. In another example, performing the selection is based on a predetermined rule. In yet another example, the predetermined rule is based on whether the current video block is predicted using the CIIP mode. coding In yet another example, the predetermined rule is to select the ALWIP mode whenever the current video block is coding When the normal intra prediction mode is selected, the normal intra prediction mode is always selected.

[0138] In some embodiments, coding The mode is a cross-component linear model (CCLM) prediction mode. In one example, the downsampling procedure for the ALWIP mode is based on the downsampling procedure for the CCLM prediction mode. In another example, the downsampling procedure for the ALWIP mode is based on a first parameter set, and the downsampling procedure for the CCLM prediction mode is based on a second parameter set different from the first parameter set. In yet another example, the downsampling procedure for the ALWIP mode or the CCLM prediction mode includes at least one of selecting a downsampling position, selecting a downsampling filter, a rounding operation, or a clipping operation.

[0139] In some embodiments, the method 1400 further includes applying one or more of a Reduced Secondary Transform (RST), a secondary transform, a rotation transform, or a non-separable secondary transform (NSST).

[0140] In some embodiments, the method 1400 further includes applying block-based differential pulse coded modulation (DPCM) or residual DPCM.

[0141] In some embodiments, a video processing method includes determining a context of a flag indicating use of an affine linear weighted intra prediction (ALWIP) mode based on a rule for a current video block during conversion between the current video block and a bitstream representation of the current video block, predicting a plurality of sub-blocks of the current video block based on the ALWIP mode, and performing a conversion between the current video block and the bitstream representation of the current video block based on the prediction. The rule may be specified implicitly using a priori techniques or coding It may be signaled within the bitstream. Other examples and aspects of this method are further described in Section 4, items 37 and 38.

[0142] In some embodiments, a method for video processing includes: a current video block is predicted using an affine linear weighted intra prediction (ALWIP) mode; coding and performing at least two filtering stages on samples of the current video block in an upsampling process associated with the ALWIP mode during conversion between the current video block and the bitstream representation of the current video block, wherein a first precision of the samples in a first filtering stage of the at least two filtering stages is different from a second precision of the samples in a second filtering stage of the at least two filtering stages.

[0143] In one example, the samples of the current video block are predicted samples, intermediate samples before the upsampling process, or intermediate samples after the upsampling process. In another example, the samples are upsampled in a first dimension horizontally in a first filtering stage, and the samples are upsampled in a second dimension vertically in a second filtering stage. In yet another example, the samples are upsampled in a first dimension vertically in a first filtering stage, and the samples are upsampled in a second dimension horizontally in a second filtering stage.

[0144] In one example, the output of the first filtering stage is right-shifted or divided to generate a processed output, which is the input to the second filtering stage. In another example, the output of the first filtering stage is left-shifted or multiplied to generate a processed output, which is the input to the second filtering stage. Other examples and aspects of this method are further described in Section 4, Item 40.

[0145] As further described in items 41 to 43 of Section 4, the video processing method may further include: coding and performing at least two filtering stages on samples of the current video block in an upsampling process associated with an ALWIP mode during conversion between the current video block and the bitstream representation of the current video block, the upsampling process being performed in a fixed order when both vertical upsampling and horizontal upsampling are performed. As further described in items 41 through 43 of Section 4, another method is to perform the upsampling process associated with an ALWIP mode when the current video block is converted using an affine linear weighted intra prediction (ALWIP) mode. coding and performing at least two filtering stages on samples of the current video block in an upsampling process associated with the ALWIP mode during conversion between the current video block and a bitstream representation of the current video block, wherein the conversion includes performing a transposition operation before the upsampling process.

[0146] Further features of the above method are described in Section 4, items 41 to 43.

[0147] 6. Implementation examples of the disclosed technology FIG. 15 is a block diagram of a video processing device 1500. The device 1500 may be used to implement one or more of the methods described herein. The device 1500 may be embodied in a smartphone, tablet, computer, or Internet of Things (IoT) receiver. The device 1500 may include one or more processors 1502, one or more memories 1504, and video processing hardware 1506. The processor(s) 1502 may be configured to execute one or more of the methods described herein (including, but not limited to, methods 1100-1400 and 2100-2300). The memory(s) 1504 may be used to store data and code used to execute the methods and techniques described herein. The video processing hardware 1506 may be used to implement some of the techniques described herein in hardware circuitry.

[0148] In some embodiments, the video coding The method may be implemented using an apparatus implemented on a hardware platform such as that described with respect to FIG.

[0149] Some embodiments of the disclosed technology include making a decision to enable a video processing tool or mode. In one example, when a video processing tool or mode is enabled, an encoder will use or implement that tool or mode in processing blocks of video, but will not necessarily modify the resulting bitstream based on the use of that tool or mode. That is, conversion from blocks of video to a bitstream representation of video will use the video processing tool or mode when it is enabled based on the decision. In another example, when a video processing tool or mode is enabled, a decoder will process the bitstream knowing that the bitstream has been modified based on the video processing tool or mode. That is, conversion from a bitstream representation of video to blocks of video will be performed using the video processing tool or mode that was enabled based on the decision.

[0150] Some embodiments of the disclosed technology include making a decision to disable a video processing tool or mode. In one example, when a video processing tool or mode is disabled, an encoder will not use that tool or mode in converting blocks of video into a video bitstream. In another example, when a video processing tool or mode is disabled, a decoder will process the bitstream knowing that the bitstream has not been modified using the video processing tool or mode that was disabled based on the decision.

[0151] FIG. 21 shows an example of an image that can utilize the techniques of this disclosure. coding 21 is a block diagram illustrating an example of a system 100. As shown in FIG. coding System 100 may include a source device 110 and a destination device 120. Source device 110 generates encoded video data and may be referred to as a video encoder. Destination device 120 can decode the encoded video data generated by source device 110 and may be referred to as a video decoder. Source device 110 may include a video source 112, a video encoder 114, and an input / output (I / O) interface 116.

[0152] Video source 112 may include sources such as, for example, a video capture device, an interface for receiving video data from a video content provider, and / or a computer graphics system for generating video data, or a combination of such sources. The video data may comprise one or more pictures. Video encoder 114 encodes the video data from video source 112 to generate a bitstream. The bitstream is a representation of the video data. coding A bitstream may include a sequence of bits that form a representation. A bitstream may include coded pictures and associated data. coding Picture is Picture coding The I / O interface 116 may include a modulator / demodulator (modem) and / or a transmitter. The encoded video data may be transmitted via the I / O interface 116 directly over the network 130a to the destination device 120. The encoded video data may also be stored on a storage medium / server 130b for access by the destination device 120.

[0153] The destination device 120 may include an I / O interface 126 , a video decoder 124 , and a display device 122 .

[0154] I / O interface 126 may include a receiver and / or a modem. I / O interface 126 may obtain encoded video data from source device 110 or storage medium / server 130b. Video decoder 124 may decode the encoded video data. Display device 122 may display the decoded video data to a user. Display device 122 may be integrated with destination device 120 or may be external to destination device 120 configured to interface with an external display device.

[0155] Video encoder 114 and video decoder 124 may operate according to a video compression standard, such as the High Efficiency Video Coding (HEVC) standard, the Versatile Video Coding (VVC) standard, and other current and / or future standards.

[0156] FIG. 22 is a block diagram illustrating an example of a video encoder 200, which may be video encoder 114 in system 100 shown in FIG.

[0157] Video encoder 200 may be configured to perform any or all of the techniques described in this disclosure. In the example of FIG. 22, video encoder 200 includes multiple functional components. The techniques described in this disclosure may be shared among various components of video encoder 200. In some examples, a processor may be configured to perform any or all of the techniques described in this disclosure.

[0158] The functional components of the video encoder 200 may include a division unit 201, a prediction unit 202, which may include a mode selection unit 203, a motion estimation unit 204, a motion compensation unit 205, and an intra prediction unit 206, a residual generation unit 207, a transform unit 208, a quantization unit 209, an inverse quantization unit 210, an inverse transform unit 211, a reconstruction unit 212, a buffer 213, and an entropy coding unit 214.

[0159] In other examples, video encoder 200 may include more, fewer, or different functional components. In one example, prediction unit 202 may include an intra block copy (IBC) unit. The IBC unit may perform prediction in an IBC mode, where at least one reference picture is the picture in which the current video block is located.

[0160] Also, some components, such as the motion estimation unit 204 and the motion compensation unit 205, although shown separately in the example of FIG. 22 for illustrative purposes, may be highly integrated.

[0161] Division unit 201 may divide a picture into one or more video blocks. Video encoder 200 and video decoder 300 may support a variety of video block sizes.

[0162] The mode selection unit 203 selects a plurality of modes, which may be intra or inter, based on, for example, the error result. coding Select one of the modes and choose the resulting Intra or Inter coding The block may be provided to a residual generation unit 207, which generates residual block data, and to a reconstruction unit 212, which reconstructs a coding block for use as a reference picture. In some examples, the mode selection unit 203 may select a combination of intra and inter predication (CIIP) mode, in which prediction is based on an inter prediction signal and an intra prediction signal. The mode selection unit 203 may also select the resolution of the motion vector for the block (e.g., sub-pixel or integer pixel precision) in the case of inter prediction.

[0163] To perform inter prediction on a current video block, motion estimation unit 204 may generate motion information for the current video block by comparing one or more reference frames from buffer 213 with the current video block. Motion compensation unit 205 may determine a prediction video block for the current video block based on the motion information and decoded samples of pictures from buffer 213 other than the picture associated with the current video block.

[0164] Motion estimation unit 204 and motion compensation unit 205 may perform different operations on the current video block depending on whether the current video block is in an I slice, a P slice, or a B slice, for example.

[0165] In some examples, motion estimation unit 204 may perform unidirectional prediction on the current video block, and motion estimation unit 204 may search reference pictures in list 0 or list 1 for reference video blocks for the current video block. Motion estimation unit 204 may then generate a reference index that points to the reference picture in list 0 or list 1 that contains the reference video block, and a motion vector that indicates the spatial displacement between the current video block and the reference video block. Motion estimation unit 204 may output the reference index, a prediction direction indicator, and the motion vector as motion information for the current video block. Based on the reference video block indicated by the motion information of the current video block, motion compensation unit 205 may generate a prediction video block for the current block.

[0166] In another example, motion estimation unit 204 may perform bidirectional prediction on the current video block, where motion estimation unit 204 may search reference pictures in list 0 for a reference video block for the current video block and may also search reference pictures in list 1 for another reference video block for the current video block. Motion estimation unit 204 may then generate reference indices that point to the reference pictures in lists 0 and 1 that contain the reference video blocks, and motion vectors that indicate spatial displacements between the reference video blocks and the current video block. Motion estimation unit 204 may output the reference indices and the motion vector for the current video block as motion information for the current video block. Based on the reference video blocks indicated by the motion information for the current video block, motion compensation unit 205 may generate a prediction video block for the current block.

[0167] In some examples, the motion estimation unit 204 may output a complete set of motion information for the decoding process of the decoder.

[0168] In some examples, motion estimation unit 204 may not output a complete set of motion information for the current video block. Rather, motion estimation unit 204 may signal motion information for the current video block by reference to motion information of another video block. For example, motion estimation unit 204 may determine that the motion information of the current video block is sufficiently similar to the motion information of a neighboring video block.

[0169] In one example, motion estimation unit 204 may point to a value within a syntax structure associated with the current video block that indicates to video decoder 300 that the current video block has the same motion information as another video block.

[0170] In another example, motion estimation unit 204 may identify another video block and a motion vector difference (MVD) within a syntax structure associated with the current video block. The motion vector difference indicates the difference between the motion vector of the current video block and the motion vector of the pointed-to video block. Video decoder 300 may use the motion vector of the pointed-to video block and the motion vector difference to determine the motion vector of the current video block.

[0171] As mentioned above, video encoder 200 may predictively signal motion vectors. Two examples of predictive signaling techniques that may be implemented by video encoder 200 include advanced motion vector prediction (AMVP) and merge mode signaling.

[0172] Intra prediction unit 206 may perform intra prediction on the current video block. When intra prediction unit 206 performs intra prediction on the current video block, intra prediction unit 206 may generate predictive data for the current video block based on decoded samples of other video blocks in the same picture. The predictive data for the current video block may include a predictive video block and various syntax elements.

[0173] Residual generation unit 207 may generate residual data for the current video block by subtracting (e.g., indicated by a minus sign) the prediction video block(s) of the current video block from the current video block. The residual data for the current video block may include residual video blocks that correspond to different sample components of the samples in the current video block.

[0174] In other examples, for example, in skip mode, residual data for the current video block may not exist and residual generation unit 207 may not perform a subtraction operation.

[0175] Transform processing unit 208 may generate one or more transform coefficient video blocks for the current video block by applying one or more transforms to the residual video block associated with the current video block.

[0176] After transform processing unit 208 generates the transform coefficient video block for the current video block, quantization unit 209 may quantize the transform coefficient video block for the current video block based on one or more quantization parameter (QP) values for the current video block.

[0177] Inverse quantization unit 210 and inverse transform unit 211 may apply inverse quantization and inverse transform, respectively, to the transform coefficient video block to reconstruct a residual video block from the transform coefficient video block. Reconstruction unit 212 may add the reconstructed residual video block to corresponding samples from one or more prediction video blocks generated by prediction unit 202 to generate a reconstructed video block for the current block that is stored in buffer 213.

[0178] After reconstruction unit 212 reconstructs the video blocks, a loop filtering operation may be performed to reduce video blocking artifacts in the video blocks.

[0179] An entropy encoding unit 214 may receive data from other functional components of the video encoder 200. Once the entropy encoding unit 214 receives the data, the entropy encoding unit 214 may perform one or more entropy encoding operations to generate entropy-encoded data and output a bitstream including the entropy-encoded data.

[0180] FIG. 19 is a block diagram illustrating an example of a video decoder 300, which may be video decoder 124 in system 100 shown in FIG.

[0181] Video decoder 300 may be configured to perform any or all of the techniques described in this disclosure. In the example of FIG. 19, video decoder 300 includes multiple functional components. The techniques described in this disclosure may be shared among various components of video decoder 300. In some examples, a processor may be configured to perform any or all of the techniques described in this disclosure.

[0182] 19, video decoder 300 includes an entropy decoding unit 301, a motion compensation unit 302, an intra prediction unit 303, an inverse quantization unit 304, an inverse transform unit 305, a reconstruction unit 306, and a buffer 307. Video decoder 300 may, in some examples, perform a decoding pass that is generally inverse to the encoding pass described with respect to video encoder 200 (FIG. 18).

[0183] The entropy decoding unit 301 may extract the coded bitstream. coding The entropy decoding unit 301 may include encoded video data (e.g., encoded blocks of video data). coding The video data may be decoded, and from the entropy decoded video data, the motion compensation unit 302 may determine motion information including motion vectors, motion vector precision, reference picture list indexes, and other motion information. The motion compensation unit 302 may determine such information by, for example, implementing AMVP and merge mode.

[0184] The motion compensation unit 302 may optionally perform interpolation based on an interpolation filter to generate the motion-compensated blocks. An identifier for the interpolation filter used with sub-pixel precision may be included in the syntax element.

[0185] Motion compensation unit 302 may calculate interpolated values for sub-integer pixels of the reference block using the interpolation filters used by video encoder 200 during encoding of the video block. Motion compensation unit 302 may determine the interpolation filters used by video encoder 200 according to the received syntax information and generate the predictive block using the interpolation filters.

[0186] The motion compensation unit 302 may use some of the syntax information to determine the size of the blocks used to encode the frames and / or slices of the encoded video sequence, partition information describing how each macroblock of a picture of the encoded video sequence is divided, a mode indicating how each partition is encoded, one or more reference frames (and reference frame lists) for each inter-coded block, and other information for decoding the encoded video sequence.

[0187] The intra prediction unit 303 may form a prediction block from spatially neighboring blocks using an intra prediction mode, e.g., received in the bitstream. The inverse quantization unit 303 inverse quantizes, or dequantizes, the quantized video block coefficients provided in the bitstream and decoded by the entropy decoding unit 301. The inverse transform unit 303 applies an inverse transform.

[0188] A reconstruction unit 306 may add the residual block with a corresponding prediction block generated by the motion compensation unit 302 or the intra prediction unit 303 to form a decoded block. If desired, a deblocking filter may also be applied to filter the decoded block to remove blocking artifacts. The decoded video block is then stored in a buffer 307, which provides a reference block for later motion compensation / intra prediction and also generates decoded video for presentation on a display device.

[0189] In this document, the term "video processing" may refer to video encoding, video decoding, video compression, or video decompression. For example, a video compression algorithm may be applied during the conversion of a pixel representation of a video to a corresponding bitstream representation, or vice versa. For example, a bitstream representation of a current video block or coding A representation may correspond to bits located together or spread across different positions in the bitstream, as defined by the syntax. For example, a video block may be represented by a transform and coding The error residual value may also be coded using bits in the header or other fields in the bitstream. Furthermore, during conversion, the decoder may parse the bitstream knowing that some fields may or may not be present based on decisions such as those described in the solutions above. Similarly, the encoder may also decide that certain syntax fields are or are not included and may code accordingly. coding By including or excluding syntactic fields from the expression, coding A representation may be generated.

[0190] 20 is a block diagram illustrating an example of a video processing system 2000 in which various techniques disclosed herein may be implemented. Various implementations may include some or all of the components of system 2000. System 2000 may include an input 2002 that receives video content. The video content may be received in a raw or uncompressed format, such as 8-bit or 10-bit multi-component pixel values, or in a compressed or encoded format. Input 2002 may represent a network interface, a peripheral bus interface, or a storage interface. Examples of network interfaces include wired interfaces such as Ethernet, passive optical networks (PONs), and wireless interfaces such as Wi-Fi or cellular interfaces.

[0191] The system 2000 may implement various coding or encoding methods described in this document. coding Component 2004 may be included. coding Component 2004 receives the input 2002 coding Reduce the average bit rate of the video to Component 2004 output, coding A representation may be generated. coding The technology is therefore sometimes called video compression technology or video transcoding technology. coding The output of component 2004 may be stored or transmitted via a communication connection as represented by component 2006. The stored or communicated bitstream (or coding ) representation can be used by component 2008 to generate pixel values or displayable video that are sent to display interface 2010. The process of generating a user-viewable video from a bitstream representation is sometimes called video decompression. Also, certain video processing operations are sometimes called "video decompression." coding Although sometimes referred to as a "manipulation or tool," it is understood that coding The tool or operation is used in the encoder, coding A corresponding decoding tool or operation that reverses the result of will be performed at the decoder.

[0192] Examples of peripheral bus interfaces or display interfaces may include Universal Serial Bus (USB) or High-Definition Multimedia Interface (HDMI) or DisplayPort, etc. Examples of storage interfaces include Serial Advanced Technology Attachment (SATA), PCI, IDE interfaces, etc. The techniques described herein may be embodied in a variety of electronic devices, such as, for example, mobile phones, laptops, smartphones, or other devices capable of performing digital data processing and / or video display.

[0193] In some embodiments, the ALWIP mode or the MIP mode may be used to coding The resulting samples are used to calculate a prediction block for the current image block by performing a boundary downsampling operation (or averaging operation), followed by a matrix-vector multiplication operation, and optionally, followed by an upsampling operation (or linear interpolation operation). In some embodiments, the ALWIP mode or MIP mode is used to calculate a prediction block for the current image block by performing a boundary downsampling operation (or averaging operation), followed by a matrix-vector multiplication operation, followed by, optionally, an upsampling operation (or linear interpolation operation). coding The resulting samples are used to calculate a prediction block for the current video block by performing a boundary downsampling operation (or averaging operation) on the resulting samples, followed by a matrix-vector multiplication operation. In some embodiments, the ALWIP mode or MIP mode can also perform an upsampling operation (or linear interpolation operation) after performing the matrix-vector multiplication operation.

[0194] 23 shows a flowchart of an example method 2300 for video processing. The method 2300 includes a step 302 of performing a conversion between a current video block of a video and a bitstream representation of the current video block using a matrix-based intra-prediction (MIP) mode, in which a prediction block of the current video block is a prediction block of a previous video block. coding The transformation is determined by performing a boundary downsampling operation on the reduced boundary samples, followed by a matrix-vector multiplication operation, and optionally followed by an upsampling operation, and the transformation includes performing the boundary downsampling operation in a single stage in which the reduced boundary samples of the current video block are generated according to a rule based at least on reference boundary samples of the current video block, and the transformation includes performing a matrix-vector multiplication operation using the reduced boundary samples of the current video block.

[0195] In some embodiments of method 2300, the reference boundary samples are reconstructed neighboring samples of the current video block. In some embodiments of method 2300, the rules specify that the reduced boundary samples are generated from reconstructed neighboring samples of the current video block. In some embodiments of method 2300, the reconstructed neighboring samples are decoded neighboring samples without a reference filtering process, where the reconstructed neighboring samples are not filtered before being used for the current video block. In some embodiments of method 2300, the reconstructed neighboring samples are decoded neighboring samples with a reference filtering process, where the reconstructed neighboring samples are filtered before being used for the current video block.

[0196] In some embodiments of method 2300, angular intra-predicted samples are generated from reconstructed neighboring samples. In some embodiments of method 2300, non-angular intra-predicted samples are generated from reconstructed neighboring samples. In some embodiments of method 2300, the rule specifies that the reduced boundary samples are generated from reconstructed neighboring samples located in a row above the current video block and / or reconstructed neighboring samples located in a column to the left of the current video block. In some embodiments of method 2300, a first number (N) of reduced boundary samples are generated from a second number (M) of reconstructed neighboring samples, and the reduced boundary samples are generated using a third number (K) of consecutive reconstructed neighboring samples. In some embodiments of method 2300, K = M / N. In some embodiments of method 2300, K = (M + N / 2)N. In some embodiments of method 2300, the reduced boundary samples are generated based on an average of consecutive reconstructed neighboring samples in the third number (K) of consecutive reconstructed neighboring samples.

[0197] In some embodiments of method 2300, the reduced boundary samples are generated based on a weighted average of consecutive reconstructed adjacent samples in a third number (K) of consecutive reconstructed adjacent samples. In some embodiments of method 2300, the rules specify that the reduced boundary samples located to the left of the current video block are generated from the reconstructed adjacent samples located in the adjacent column to the left of the current video block, and that the reduced boundary samples located above the current video block are generated from the reconstructed adjacent samples located in the adjacent row above the current video block. In some embodiments of method 2300, the current video block is a 16x16 video block, and the four reduced boundary samples located to the left of the 16x16 video block are generated from the reconstructed adjacent samples located in the adjacent column to the left of the 16x16 video block, and the four reduced boundary samples located above the 16x16 video block are generated from the reconstructed adjacent samples located in the adjacent row above the 16x16 video block.

[0198] In some embodiments of method 2300, the rules specify that the technique by which the reduced boundary samples are generated by the boundary downsampling operation is based on a dimension of the current video block. coding In some embodiments of method 2300, coding The information includes a width of the current video block, a height of the current video block, and an indication of an intra-prediction mode or a transform mode associated with the current video block. In some embodiments of method 2300, the rules specify that the reduced boundary samples are generated regardless of the size of the current video block. In some embodiments of method 2300, the rules specify that the reduced boundary samples located to the left of the current video block are generated by a different process than the reduced boundary samples located above the current video block. In some embodiments of method 2300, the current video block has a width (M) and a height (N), a first predetermined number of reduced boundary samples located above the current video block is the minimum of M, N, or M and N, a second predetermined number of reduced boundary samples located to the left of the current video block is the minimum of M, N, or M and N, the first number of reduced boundary samples are generated based on neighboring samples or by copying neighboring samples located in a row above the current video block, and the second number of reduced boundary samples are generated by copying neighboring samples or by copying neighboring samples located in a column to the left of the current video block.

[0199] 24 shows a flowchart of an example method 2400 for video processing. The method 2400 includes a step 2402 of performing a conversion between a current video block of a video and a bitstream representation of the current video block using a matrix-based intra-prediction (MIP) mode, in which a final prediction block of the current video block is a prediction block of a previous bitstream representation of the video. coding The transformation includes performing an upsampling operation in which the final prediction block is determined by using a reduced prediction block of the current video block and by using reconstructed neighboring samples of the current video block according to a rule, and the reduced prediction block is obtained by performing a matrix-vector multiplication operation on the reduced boundary samples of the current video block.

[0200] In some embodiments of method 2400, the rules specify that the final predicted block is determined by using all of the reconstructed neighboring samples. In some embodiments of method 2400, the rules specify that the final predicted block is determined by using a portion of the reconstructed neighboring samples. In some embodiments of method 2400, the reconstructed neighboring samples neighbor the current video block. In some embodiments of method 2400, the reconstructed neighboring samples do not neighbor the current video block. In some embodiments of method 2400, the reconstructed neighboring samples are neighboring rows above the current video block and / or are neighboring columns to the left of the current video block.

[0201] In some embodiments of method 2400, the rules specify that reduced boundary samples are excluded from the upsampling operation when determining the final predicted block. In some embodiments of method 2400, the rules specify that the final predicted block is determined by using a set of reconstructed neighboring samples selected from the reconstructed neighboring samples. In some embodiments of method 2400, the set of reconstructed neighboring samples selected from the reconstructed neighboring samples includes a selection of all of the reconstructed neighboring samples located to the left of the current video block. In some embodiments of method 2400, the set of reconstructed neighboring samples selected from the reconstructed neighboring samples includes a selection of all of the reconstructed neighboring samples located above the current video block.

[0202] In some embodiments of method 2400, the set of reconstructed neighboring samples selected from the reconstructed neighboring samples includes a selection of K of M contiguous reconstructed neighboring samples located to the left of the current video block. In some embodiments of method 2400, K is equal to 1 and M is equal to 2, 4, or 8. In some embodiments of method 2400, the K of M contiguous reconstructed neighboring samples include the last K of M contiguous reconstructed neighboring samples. In some embodiments of method 2400, the K of M contiguous reconstructed neighboring samples include the first K of M contiguous reconstructed neighboring samples. In some embodiments of method 2400, the set of reconstructed neighboring samples selected from the reconstructed neighboring samples includes a selection of K of M contiguous reconstructed neighboring samples located above the current video block. In some embodiments of method 2400, K is equal to 1 and M is equal to 2, 4, or 8. In some embodiments of method 2400, the K of M contiguous reconstructed neighboring samples include the last K of M contiguous reconstructed neighboring samples.

[0203] In some embodiments of method 2400, K of each of the M consecutive reconstructed neighboring samples include the first K reconstructed neighboring samples of each of the M consecutive ones. In some embodiments of method 2400, the set of reconstructed neighboring samples is selected from the reconstructed neighboring samples based on a width of the current video block and / or a height of the current video block. In some embodiments of method 2400, in response to the width of the current video block being greater than or equal to the height of the current video block, the set of reconstructed neighboring samples is selected to include all of the reconstructed neighboring samples located to the left of the current video block, and / or the set of reconstructed neighboring samples is selected to include a number of reconstructed neighboring samples located above the current video block, where the number of reconstructed neighboring samples depends on the width of the current video block.

[0204] In some embodiments of method 2400, the kth selected reconstructed neighboring sample located above the current video block is located at a position described by (blkX+(k+1)*blkW / M-1,blkY-1), where (blkX,blkY) represents a top-left position of the current video block, M is the number of reconstructed neighboring samples, and k is between 0 and (M-1), inclusive. In some embodiments of method 2400, in response to the width being less than or equal to 8, the number of reconstructed neighboring samples is equal to 4. In some embodiments of method 2400, in response to the width being greater than 8, the number of reconstructed neighboring samples is equal to 8. In some embodiments of method 2400, the set of reconstructed neighboring samples is selected to include all of the reconstructed neighboring samples located to the left of the current video block, and / or the set of reconstructed neighboring samples is selected to include a number of reconstructed neighboring samples located above the current video block, the number of reconstructed neighboring samples depending on the width of the current video block.

[0205] In some embodiments of method 2400, in response to the width being less than or equal to 8, the number of reconstructed neighboring samples is equal to 4. In some embodiments of method 2400, in response to the width being greater than 8, the number of reconstructed neighboring samples is equal to 8. In some embodiments of method 2400, in response to the width of the current video block being less than the height of the current video block, the set of reconstructed neighboring samples is selected to include all of the reconstructed neighboring samples located above the current video block, and / or the set of reconstructed neighboring samples is selected to include a number of reconstructed neighboring samples located to the left of the current video block, where the number of reconstructed neighboring samples depends on the height of the current video block.

[0206] In some embodiments of method 2400, the kth selected reconstructed neighboring sample located to the left of the current video block is located at a position described by (blkX-1,blkY+(k+1)*blkH / M-1), where (blkX,blkY) represents an upper-left position of the current video block, M is the number of reconstructed neighboring samples, and k is greater than or equal to (M-1). In some embodiments of method 2400, in response to the height being less than or equal to 8, the number of reconstructed neighboring samples is equal to 4. In some embodiments of method 2400, in response to the height being greater than 8, the number of reconstructed neighboring samples is equal to 8. In some embodiments of method 2400, the set of reconstructed neighboring samples is selected to include all of the reconstructed neighboring samples located above the current video block, and / or the set of reconstructed neighboring samples is selected to include a number of reconstructed neighboring samples located to the left of the current video block, the number of reconstructed neighboring samples depending on the height of the current video block.

[0207] In some embodiments of method 2400, in response to the height being less than or equal to 8, the number of reconstructed neighboring samples is equal to 4. In some embodiments of method 2400, in response to the height being greater than 8, the number of reconstructed neighboring samples is equal to 8. In some embodiments of method 2400, the rule specifies that the final predicted block is determined by using a set of modified reconstructed neighboring samples obtained by modifying the reconstructed neighboring samples. In some embodiments of method 2400, the rule specifies that the set of modified reconstructed neighboring samples is obtained by performing a filtering operation on the reconstructed neighboring samples. In some embodiments of method 2400, the filtering operation uses an N-tap filter. In some embodiments of method 2400, N is equal to 2 or 3.

[0208] In some embodiments of method 2400, the rules specify that the filtering operation is adaptively applied according to a MIP mode in which a final predicted block for the current video block is determined. In some embodiments of method 2400, the technique in which the final predicted block is determined by the upsampling operation is based on a dimension of the current video block. In some embodiments of method 2400, the technique in which the final predicted block is determined by the upsampling operation is based on a dimension of the current video block. coding In some embodiments of method 2400, coding The information includes an indication of the intra-prediction direction or transform mode associated with the current video block.

[0209] In some embodiments of the method of this patent document, performing the transformation includes generating a bitstream representation from the current video block. In some embodiments of the method of this patent document, performing the transformation includes generating a bitstream representation from the current video block.

[0210] From the foregoing, it will be understood that, although specific embodiments of the technology disclosed herein have been described herein for purposes of illustration, various modifications may be made without departing from the scope of the invention. Accordingly, the technology disclosed herein is not to be limited except as by the appended claims.

[0211] Implementations of the subject matter and functional operations described in this patent document may be embodied in various systems, digital electronic circuits, or computer software, firmware, or hardware, including the structures disclosed herein and their structural equivalents, or in one or more combinations of these. Implementations of the subject matter described herein may be implemented as one or more computer program products, i.e., as one or more modules of computer program instructions encoded on a tangible, non-transitory computer-readable medium for execution by or for controlling the operation of a data processing apparatus. A computer-readable medium may be a machine-readable storage device, a machine-readable storage substrate, a memory device, a composition of matter producing a machine-readable propagated signal, or a combination of one or more of these. The terms "data processing unit" or "data processing apparatus" encompass any apparatus, device, and machine that processes data, including, by way of example, a programmable processor, a computer, or multiple processors or computers. An apparatus may include, in addition to hardware, code that creates an execution environment for the computer program at issue, such as code comprising processor firmware, a protocol stack, a database management system, an operating system, or one or more combinations of these.

[0212] A computer program (also known as a program, software, software application, script, or code) may be written in any form of programming language, including compiled or interpreted languages, and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. A program may be stored as part of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program at hand, or in multiple cooperating files (e.g., files that store one or more modules, subprograms, or portions of code). A computer program may be deployed to be executed on one computer or on multiple computers, either collocated or distributed across multiple locations and interconnected by a communications network.

[0213] The processes and logic flows described herein may be performed by one or more programmable processors executing one or more computer programs to perform functions by operating on input data and generating output. These processes and logic flows may also be performed by, and apparatus may be implemented as, special purpose logic circuitry, such as an FPGA (Field Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit).

[0214] Processors suitable for the execution of a computer program include, by way of example, both general-purpose and special-purpose microprocessors, and any one or more processors of any kind of digital computer. Typically, a processor receives instructions and data from a read-only memory or a random-access memory, or both. The essential elements of a computer are a processor for executing instructions and one or more memory devices for storing instructions and data. Typically, a computer also includes one or more mass storage devices for storing data, such as magnetic, magneto-optical, or optical disks, or is operatively coupled to receive data from or transfer data to the mass storage devices. However, a computer need not have such devices. Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including, by way of example, semiconductor memory devices such as EPROM, EEPROM, and flash memory devices. The processor and memory may be supplemented by, or incorporated in, special-purpose logic circuitry.

[0215] It is intended that the specification, together with the drawings, be considered merely exemplary, and by exemplary I mean by example. As used herein, the use of "or" is intended to include "and / or" unless the context clearly indicates otherwise.

[0216] While this patent document contains numerous details, these should not be construed as limitations on the scope of any invention or what may be claimed, but rather as descriptions of features that may be specific to particular embodiments of a particular invention. Certain features described in this patent document in the context of separate embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented in multiple embodiments separately or in any suitable subcombination. Furthermore, while features may be described above as operating in a particular combination, and even initially claimed as such, in some cases one or more features from a claimed combination may be removed from the combination, or the claimed combination may be subject to subcombinations or variations of the subcombination.

[0217] Similarly, although the figures may depict operations in a particular order, this should not be understood as requiring that those operations be performed in the particular order or sequence shown, or that all of the operations shown be performed, to achieve desired results. Additionally, the separation of various system components in the embodiments described in this patent document should not be understood as requiring such separation in all embodiments.

[0218] Only a few implementations and examples have been described, and other implementations, extensions and variations may be made based on what is described and illustrated in this patent document.< / end> < / end> < / begin> < / end> < / begin> < / end> < / begin> < / end> < / begin> < / end> < / begin>

Claims

1. 1. A method for processing video data, comprising: determining a first intra mode to be applied to a video block of a video for conversion between a video block of the video and a bitstream of the video, wherein a process in the first intra mode includes a matrix vector multiplication operation followed by an upsampling operation to generate prediction samples for the video block of the video; performing the transformation based on the predicted samples; Including, inputs to the upsampling operation include neighboring reference samples of the video block of the video; The input to the upsampling operation is: a set of left-neighboring reference samples selected from left-neighboring reference samples of the video block; a set of upper-neighboring reference samples selected from upper-neighboring reference samples of the video block; Including, a method in which the upsampling operation includes at least one of a horizontal upsampling operation and a vertical upsampling operation, and the order of the horizontal and vertical upsampling operations in the upsampling operation for the video block having a height greater than its width is the same as that for the video block having a width greater than its height.

2. The method of claim 1 , wherein the set of left-neighboring reference samples and the set of above-neighboring reference samples are derived without an intra-reference filtering process.

3. 3. The method of claim 1, wherein the video block has a width W and a height H, the set of left-neighboring reference samples includes a selection of H left-neighboring reference samples of the video block, and the set of above-neighboring reference samples includes a selection of W above-neighboring reference samples, where W and H are integers.

4. the upsampling operation includes at least one of a horizontal upsampling operation and a vertical upsampling operation; 4. The method of claim 1, wherein the horizontal upsampling operation comprises interpolating samples by using the left-neighboring reference samples, and the vertical upsampling operation comprises interpolating samples by using the above-neighboring reference samples.

5. The method of claim 1 , wherein the process in the first intra mode further comprises a downsampling operation before the matrix vector multiplication operation based on a size of the video block.

6. the downsampling operation is performed on the neighboring reference samples of the video block to generate reduced samples; The method of claim 5 , wherein the reduced samples are input to the matrix vector multiplication operation and excluded from the upsampling operation.

7. N reduced samples are derived from M adjacent reference samples without deriving intermediate samples, and each of the N reduced samples is determined based on K consecutive samples of the M adjacent reference samples; The method of claim 6 , wherein M and N are integers and K is determined based on M / N.

8. The method of claim 7 , wherein each of the N reduced boundary samples is determined based on an average of the K consecutive samples of the M adjacent reference samples.

9. 8. The method of claim 6 or 7, further comprising deriving a one-dimensional vector array based on concatenating the reduced samples, and using the one-dimensional vector array as an input of the matrix-vector multiplication operation to generate a two-dimensional array.

10. 10. The method of claim 9, wherein the two-dimensional array having a width of first values and a height of second values is transposed into an array having a width of second values and a height of first values before performing the upsampling operation according to a syntax element.

11. The method of claim 1 , wherein the transforming comprises encoding the video blocks into the bitstream.

12. The method of claim 1 , wherein the converting comprises decoding the video block from the bitstream.

13. 1. An apparatus for processing video data, comprising a processor and a non-transitory memory having instructions, The instructions, when executed by the processor, cause the processor to: determining a first intra mode to be applied to a video block of a video for conversion between a video block of the video and a bitstream of the video, wherein a process in the first intra mode includes a matrix vector multiplication operation followed by an upsampling operation to generate prediction samples for the video block of the video; performing the transformation based on the predicted samples; Execute inputs to the upsampling operation include neighboring reference samples of the video block of the video; The input to the upsampling operation is: a set of left-neighboring reference samples selected from left-neighboring reference samples of the video block; a set of upper-neighboring reference samples selected from upper-neighboring reference samples of the video block; Including, the upsampling operation includes at least one of a horizontal upsampling operation and a vertical upsampling operation, and the order of the horizontal and vertical upsampling operations in the upsampling operation for the video block having a height greater than its width is the same as that for the video block having a width greater than its height.

14. The processor determining a first intra mode to be applied to a video block of a video for conversion between a video block of the video and a bitstream of the video, wherein a process in the first intra mode includes a matrix vector multiplication operation followed by an upsampling operation to generate prediction samples for the video block of the video; performing the transformation based on the predicted samples; and storing an instruction to execute the inputs to the upsampling operation include neighboring reference samples of the video block of the video; The input to the upsampling operation is: a set of left-neighboring reference samples selected from left-neighboring reference samples of the video block; a set of upper-neighboring reference samples selected from upper-neighboring reference samples of the video block; Including, a non-transitory computer-readable storage medium, wherein the upsampling operation includes at least one of a horizontal upsampling operation and a vertical upsampling operation, and an order of the horizontal upsampling operation and the vertical upsampling operation in the upsampling operation for the video block having a height greater than its width is the same as that for the video block having a width greater than its height.

15. 1. A method for storing a video bitstream, comprising: determining that a first intra mode is to be applied to a video block of the picture, wherein a process in the first intra mode includes a matrix-vector multiplication operation followed by an upsampling operation to generate prediction samples for the video block of the picture; generating the bitstream based on the determination; storing the bitstream on a non-transitory computer-readable recording medium; Including, inputs to the upsampling operation include neighboring reference samples of the video block of the video; The input to the upsampling operation is: a set of left-neighboring reference samples selected from left-neighboring reference samples of the video block; a set of upper-neighboring reference samples selected from upper-neighboring reference samples of the video block; Including, a method in which the upsampling operation includes at least one of a horizontal upsampling operation and a vertical upsampling operation, and the order of the horizontal and vertical upsampling operations in the upsampling operation for the video block having a height greater than its width is the same as that for the video block having a width greater than its height.

Images