Coding method, decoding method, code stream, coder, decoder, and storage medium

Abstract

Disclosed in the present application are a coding method, a decoding method, a code stream, a coder, a decoder, and a storage medium. The method comprises: decoding a code stream, so as to determine the value of a first syntax element (S701); when the first syntax element indicates that a current block uses a matrix-based intra prediction mode, determining a first prediction block of the current block, and performing up-sampling processing on the first prediction block, so as to determine a second prediction block of the current block (S702); and, on the basis of dimensional information of the current block, using the first prediction block or the second prediction block to determine a derived intra prediction mode for the current block, and determining a transform set for the current block according to the derived intra prediction mode (S703). Thus, the accuracy of transform prediction can be improved, thereby improving the compression efficiency, and further improving the coding and decoding performance.

Description

Coding method, code stream, encoder, decoder and storage medium TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of video coding, and particularly relate to a coding method, a code stream, an encoder, a decoder and a storage medium. BACKGROUND

[0002] In a video coding standard, a matrix-based intra prediction (MIP) technology can be used to predict a current coding unit. Specifically, a reference sample adjacent to the current coding unit is used to determine an input vector, the input vector is multiplied by a weight matrix to obtain an output vector, and the output vector is further arranged and up-sampled to obtain a prediction block of the current coding unit.

[0003] In the related art, the MIP technology does not comprehensively consider the transformation process, which may result in an inaccurate transformation set and reduce the coding performance.

[0004] SUMMARY

[0005] Embodiments of the present application provide a coding method, a code stream, an encoder, a decoder and a storage medium, which can improve compression efficiency and further improve coding performance.

[0006] The technical solution of the embodiments of the present application can be implemented as follows:

[0007] In a first aspect, the embodiments of the present application provide a decoding method applied to a decoder, and the method comprises:

[0008] decoding a code stream to determine a value of a first syntax element;

[0009] when the first syntax element indicates that a matrix-based intra prediction mode is used for the current block, determining a first prediction block of the current block, and performing up-sampling processing on the first prediction block to determine a second prediction block of the current block;

[0010] determining a derived intra prediction mode of the current block based on size information of the current block and using the first prediction block or the second prediction block, and determining a transformation set of the current block according to the derived intra prediction mode.

[0011] In a second aspect, the embodiments of the present application provide an encoding method applied to an encoder, and the method comprises:

[0012] when a matrix-based intra prediction mode is used for the current block, determining a first prediction block of the current block, and performing up-sampling processing on the first prediction block to determine a second prediction block of the current block;

[0013] Based on the size information of the current block, the first prediction block or the second prediction block is used to determine a derived intra prediction mode of the current block, and a transform set of the current block is determined according to the derived intra prediction mode.

[0014] In a third aspect, an embodiment of the present application provides a bitstream, which is generated by bit encoding of to-be-encoded information; wherein the to-be-encoded information comprises at least one of the following: quantization coefficients of a current block, a transform kernel index of the current block, prediction parameters of the current block, a value of a first syntax element, a value of a second syntax element and a value of a third syntax element.

[0015] The prediction parameters comprise transposition identification information and weight matrix index information, the second syntax element is used to indicate whether the current sequence allows to use the matrix-based intra prediction mode, the third syntax element is used to indicate a prediction type of the current block, the first syntax element is used to indicate whether the current block uses the matrix-based intra prediction mode, and the current sequence comprises the current block.

[0016] In a fourth aspect, an embodiment of the present application provides an encoder, which comprises a first prediction unit and a first determination unit, wherein:

[0017] The first prediction unit is configured to, when the current block uses the matrix-based intra prediction mode, determine a first prediction block of the current block, and perform up-sampling processing on the first prediction block to determine a second prediction block of the current block.

[0018] The first determination unit is configured to, based on size information of the current block, use the first prediction block or the second prediction block to determine a derived intra prediction mode of the current block, and determine a transform set of the current block according to the derived intra prediction mode.

[0019] In a fifth aspect, an embodiment of the present application provides an encoder, which comprises a first memory and a first processor; wherein:

[0020] The first memory is configured to store a computer program capable of running on the first processor.

[0021] The first processor is configured to, when running the computer program, execute the method in the second aspect.

[0022] In a sixth aspect, an embodiment of the present application provides a decoder, which comprises a decoding unit, a second prediction unit and a second determination unit, wherein:

[0023] The decoding unit is configured to decode the bitstream to determine a value of the first syntax element.

[0024] The second prediction unit is configured to determine a first prediction block of the current block when the first syntax element indicates that the current block uses the matrix-based intra prediction mode, and to perform up-sampling processing on the first prediction block to determine a second prediction block of the current block.

[0025] The second determination unit is configured to determine a derived intra prediction mode of the current block by using the first prediction block or the second prediction block based on size information of the current block, and to determine a transform set of the current block according to the derived intra prediction mode.

[0026] In a seventh aspect, an embodiment of the present application provides a decoder, which comprises a second memory and a second processor; wherein,

[0027] The second memory is configured to store a computer program capable of running on the second processor.

[0028] The second processor is configured to execute the method according to the first aspect when running the computer program.

[0029] In an eighth aspect, an embodiment of the present application provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the method according to the first aspect or the method according to the second aspect.

[0030] In a ninth aspect, an embodiment of the present application provides a computer program product, which comprises a computer program or instructions, and the computer program or instructions are executed by a processor to implement the method according to the first aspect or the method according to the second aspect.

[0031] The embodiments of the present application provide a coding method, a bitstream, an encoder, a decoder and a storage medium. When the current block uses the matrix-based intra prediction mode, the first prediction block of the current block is determined, and the up-sampling processing is performed on the first prediction block to determine the second prediction block of the current block. The derived intra prediction mode of the current block is determined by using the first prediction block or the second prediction block based on the size information of the current block, and the transform set of the current block is determined according to the derived intra prediction mode. In this way, when the current block uses the MIP mode, the first prediction block and the second prediction block of the current block are first determined. Then, the derived intra prediction mode of the current block can be determined by selecting the first prediction block or the second prediction block according to the size information of the current block, and then the transform set suitable for the current block can be determined, and a suitable transform kernel is selected for transformation. In this way, the accuracy of the transform prediction can be improved, the compression efficiency is improved, and the coding performance is improved. BRIEF DESCRIPTION OF DRAWINGS

[0032] FIG. 1 is a flow block diagram of a hybrid coding framework;

[0033] FIG. 2 is a schematic diagram of a prediction process of the MIP technology;

[0034] FIG. 3 is a schematic diagram of weighted fusion of three intra prediction modes;

[0035] FIG. 4 is a schematic diagram of a network architecture of a video coding provided by an embodiment of the present application;

[0036] FIG. 5 is a schematic diagram of a system composition block diagram of an encoder provided by an embodiment of the present application;

[0037] FIG. 6 is a schematic diagram of a composition structure of a decoder provided by an embodiment of the present application;

[0038] FIG. 7 is a schematic diagram of a flow of a decoding method provided by an embodiment of the present application;

[0039] FIG. 8 is a schematic diagram of a flow of a decoding method provided by an embodiment of the present application;

[0040] FIG. 9 is a schematic diagram of a flow of a decoding method provided by an embodiment of the present application;

[0041] FIG. 10 is a schematic diagram of a flow of a decoding method provided by an embodiment of the present application;

[0042] FIG. 11 is a schematic diagram of a flow of a decoding method provided by an embodiment of the present application;

[0043] FIG. 12 is a schematic diagram of a flow of an encoding method provided by an embodiment of the present application;

[0044] FIG. 13 is a schematic diagram of a flow of an encoding method provided by an embodiment of the present application;

[0045] FIG. 14 is a schematic diagram of a composition structure of an encoder provided by an embodiment of the present application;

[0046] FIG. 15 is a schematic diagram of a specific hardware structure of an encoder provided by an embodiment of the present application;

[0047] FIG. 16 is a schematic diagram of a composition structure of a decoder provided by an embodiment of the present application;

[0048] FIG. 17 is a schematic diagram of a specific hardware structure of a decoder provided by an embodiment of the present application;

[0049] FIG. 18 is a schematic diagram of a composition structure of a coding system provided by an embodiment of the present application. DETAILED DESCRIPTION

[0050] In order to be able to understand the features and technical contents of the embodiments of the present application more thoroughly, the implementation of the embodiments of the present application will be described in detail below with reference to the accompanying drawings, which are only used for reference and do not limit the embodiments of the present application.

[0051] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in the description herein is for describing particular embodiments only and is not intended to be limiting of the application.

[0052] In the following description, reference is made to the "some embodiments", which describe a subset of all possible embodiments, but it is understood that "some embodiments" can be the same subset or a different subset of all possible embodiments, and can be combined with each other as long as there is no conflict.

[0053] It should also be noted that the terms "first\second\third" involved in the embodiments of the present application are only used to distinguish similar objects, and do not represent the specific order of the objects. It can be understood that "first\second\third" can be interchanged with specific order or sequence as long as it is allowed, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0054] In a video image, a coding block (CB) is generally represented by a first color component, a second color component and a third color component. Among them, the three color components are a luminance component, a blue chroma component and a red chroma component, respectively. Specifically, the luminance component is usually represented by the symbol Y, the blue chroma component is usually represented by the symbol Cb or U, and the red chroma component is usually represented by the symbol Cr or V. In this way, the video image can be represented in YCbCr format or YUV format.

[0055] Before the embodiments of the present application are further described in detail, the terms and terms involved in the embodiments of the present application are explained, which are applicable to the following explanations:

[0056] H.265 / High Efficiency Video Coding (HEVC);

[0057] H.266 / Versatile Video Coding (VVC);

[0058] VVC Test Model (VTM) of the reference software test platform of VVC;

[0059] Enhanced Compression Model (ECM);

[0060] Joint Video Experts Team (JVET);

[0061] coding unit (CU);

[0062] coding tree unit (CTU);

[0063] largest coding unit (LCU);

[0064] prediction unit (PU);

[0065] transform unit (TU);

[0066] matrix-based intra prediction (MIP);

[0067] discrete cosine transform (DCT);

[0068] discrete sine transform (DST);

[0069] low-frequency non-separable secondary transform (LFNST);

[0070] non-separable primary transform (NSPT);

[0071] decoder-side intra mode derivation (DIMD);

[0072] template-based intra mode derivation (TIMD).

[0073] Currently, the general video coding standards (such as H.266 / VVC) all adopt a block-based hybrid coding framework. Each picture or sub-picture or frame in a video is divided into square maximum coding units of the same size (such as 128x128, 64x64, etc.). Each maximum coding unit can be divided into rectangular coding units according to rules. The coding units can also be divided into prediction units, transform units, etc. Specifically, as shown in FIG. 1, the hybrid coding framework includes modules such as prediction, transform, quantization, entropy coding, inv. quantization, inv. transform, in loop filter, etc. Among them, the prediction module can include intra prediction and inter prediction, and the inter prediction can include motion estimation and motion compensation. Since there is a strong correlation between adjacent pixels in a picture of a video, the intra prediction method is used in video coding technology to eliminate the spatial redundancy between adjacent pixels. Since there is a strong similarity between adjacent pictures in a video, the inter prediction method is used in video coding technology to eliminate the temporal redundancy between adjacent pictures, thereby improving the coding efficiency.

[0074] The basic process of a video codec is as follows: at the encoding end, a frame of image is divided into blocks, a prediction block of a current block is generated using intra prediction or inter prediction, a residual block is obtained by subtracting the prediction block from the original image block (i.e., original block) of the current block, a quantized coefficient matrix is obtained by transforming and quantizing the residual block, and the quantized coefficient matrix is entropy encoded and output to a bitstream. At the decoding end, a prediction block of a current block is generated using intra prediction or inter prediction, and on the other hand, a quantized coefficient matrix is obtained by parsing the bitstream, the quantized coefficient matrix is dequantized and inverse transformed to obtain a residual block, and the prediction block and the residual block are added to obtain a reconstructed block. The reconstructed blocks constitute a reconstructed image, and the reconstructed image is loop filtered based on the image or based on the block to obtain a decoded image. The encoding end also needs to perform similar operations as the decoding end to obtain a decoded image. The decoded image can be used as a reference image for subsequent frames for inter prediction. If necessary, the block division information, prediction, transformation, quantization, entropy encoding, loop filtering, and other mode information or parameter information determined by the encoding end need to be output to the bitstream. The decoding end determines the same block division information, prediction, transformation, quantization, entropy encoding, loop filtering, and other mode information or parameter information by parsing the bitstream and analyzing the existing information, so as to ensure that the decoded image obtained by the encoding end is the same as the decoded image obtained by the decoding end. The decoded image obtained by the encoding end is usually also called a reconstructed image. When predicting, the current block can be divided into prediction units, and when transforming, the current block can be divided into transformation units. The division of the prediction units and the transformation units can be different. The above is the basic process of a video codec under a block-based hybrid coding framework. With the development of technology, some modules or steps of the framework or process can be optimized. The embodiments of the present application are applicable to the basic process of a video codec under the block-based hybrid coding framework, but are not limited to the framework and process.

[0075] In the embodiments of the present application, a current block (CB) can be a current coding unit, a current prediction unit, or a current transformation unit, etc. Due to the need for parallel processing, an image can be divided into slices, etc., and the slices in the same image can be processed in parallel, that is, there is no data dependency between them. "Frame" is a commonly used term, which can be generally understood as a frame being an image. In the embodiments of the present application, the frame can also be replaced by an image or a slice, etc.

[0076] The international video coding standard organization JVET has established a team to study a coding model beyond H.266 / VVC, and named the model, i.e., platform test software, as ECM. ECM starts to receive updated and more efficient compression algorithms on the basis of VTM, and currently has a coding performance of about 13% beyond VVC. ECM not only expands the size of a coding unit of a specific resolution, but also integrates many modules of improvement technology, and has changed a lot in algorithm compared with VVC.

[0077] The related schemes of several prediction technologies are described in detail below.

[0078] (I) Matrix-based intra prediction technology.

[0079] Traditional intra prediction modes and wide-angle modes calculate the prediction value in a sample mapping or linear gradient manner, which cannot effectively predict irregular textures. In order to adapt to different sample distributions and image textures, H.266 / VVC uses MIP technology. For YUV 4:2:0 sampling format, MIP technology is mainly used on the luminance component; for YUV 4:4:4 sampling format, MIP technology can also be used on the chrominance component.

[0080] MIP technology adopts the method of offline training neural network to obtain a plurality of fixed weight matrices (or called "coefficient matrices"), and then uses the weight matrices to calculate the prediction value. The prediction process of MIP is shown in FIG. 2. In order to predict a block with a width of W and a height of H, MIP needs the left column of H reconstructed samples of the current block and the top row of W reconstructed samples of the current block as input. MIP generates the prediction block in the following three steps: (a) reference sample averaging, (b) matrix vector multiplication, and (c) interpolation. Here, the core of MIP technology is considered to be matrix vector multiplication. It can be considered as a process of generating a prediction block using input samples (reference samples) in a matrix multiplication manner. MIP provides a plurality of weight matrices, and the difference of prediction methods is reflected in the difference of weight matrices. The same input sample will get different results using different weight matrices. The processes of reference sample averaging and interpolation are a design of performance and complexity compromise. For a block with a larger size, the reference sample averaging can realize an approximate down-sampling effect, so that the input can be adapted to a smaller matrix, and the interpolation realizes an up-sampling effect.

[0081] That is, in the prediction process of MIP, the reference samples are processed to obtain an input vector, the input vector is multiplied by the weight matrix to obtain an output vector, and the prediction value of the CU to be encoded is obtained through further arrangement and up-sampling.

[0082] Exemplarily, taking the CU as an example, the MIP technology is closely related to the size of the CU. According to the size of the CU, the CU can be divided into three categories, which are identified by the class index classIdx. Table 1 shows the CU size, the number of weight matrices and the size of the weight matrix corresponding to classIdx. It can be seen from this that the larger the classIdx is, the larger the CU size is.

[0083] Table 1

[0084] where input vector bdry red is obtained by downsampling the reference samples bdry left and bdry top . When classIdx is equal to 0, 1 and 2, the number of elements of the input vector is 4, 8 and 7 respectively; the number of elements of the output vector is 16, 16 and 64 respectively, and the output vector is upsampled to get the prediction value of each sample.

[0085] The classIdx of MIP technique is determined according to the size of CU, at this time the size of weight matrix is determined, and the number of elements of input vector is also determined. Therefore, it is necessary to first downsample the reference samples to the number of elements of input vector, when classIdx is 0, each of the upper reference row and the left reference column is downsampled to 2 sample points; when classIdx is 1 or 2, each of the upper reference row and the left reference column is downsampled to 4 sample points.

[0086] Taking a CU block with width W and height H as an example, assuming that the CU block belongs to the category of classIdx being 2, then the input vectors all need to be downsampled to 4 sample points. MIP technique adopts the method of taking average to compose the downsampled input vector, that is, taking average every W / 4 sample points, and finally obtaining 4 downsampled upper boundary input vectors bdry RedTop . Similarly, taking average every H / 4 sample points is also done to the left reference column, and 4 downsampled left boundary input vectors bdry RedLeft are obtained.

[0087] MIP technique supports different orders of bdry RedTop and bdry RedLeft splicing as input, that is, if bdry RedTop is in front of bdry RedLeft , then it is normal input; otherwise, if bdry RedLeft is in front of bdry RedTop , then it is transposed input. In addition, it is also necessary to process the input vector, for the case of classIdx being 2, the current element is the value of the next element minus the first element; for the other two cases, the first element is the maximum value of the sample minus the first element, and the remaining elements are the values minus the first element.

[0088] The prediction process of MIP technique is in the form of matrix multiplication in FIG. 2.

[0089] The output of the aforementioned MIP technology has a fixed size of 16 or 64 sample points, so the MIP output vector often needs to be upsampled to the same size as the current coding unit. Upsampling the MIP output vector requires arranging the elements output by the matrix operation in the positions shown in FIG. 2. First, the horizontal direction is upsampled, and each missing sample position is linearly interpolated using the left and right adjacent sample values. Then, the vertical direction is upsampled, and the method is the same as the horizontal direction, that is, each missing sample position is linearly interpolated using the upper and lower adjacent samples. After completing the horizontal and vertical interpolation, a prediction block with the same size as the current coding unit is obtained. It should be noted that if the input vector is not transposed, each element of the output vector is arranged in the grid filling position of the CU in FIG. 2 from left to right in row units; if the input vector is transposed, each element of the output vector is arranged in the grid filling position of the CU in FIG. 2 from top to bottom in column units.

[0090] The MIP technology needs a coding unit level flag to indicate whether the current coding unit uses the MIP technology. If the current coding unit uses the MIP technology, the MIP prediction mode and the MIP transpose flag need to be parsed, and the MIP prediction mode uses truncated binary coding.

[0091] (II) Intra mode derivation technology at the decoding end.

[0092] The DIMD technology is an intra prediction technology of ECM, and VVC does not have this technology. The main core point of this technology is that the intra prediction mode is derived at the decoding end using the same method as at the encoding end to avoid transmitting the intra prediction mode index of the current coding unit in the code stream, thereby saving bit overhead. The specific method includes two main steps:

[0093] The first step, i.e., deriving the prediction mode, uses the same prediction mode strength calculation method at the encoding and decoding ends. The encoding end uses the Sobel operator to calculate the gradient histogram under each prediction mode, and the action area is the upper three rows of adjacent reconstructed samples, the left three columns of adjacent reconstructed samples, and the corresponding adjacent reconstructed sample at the top left of the current block. By calculating the gradient histogram of the aforementioned L-shaped area, the first prediction mode (mode1, M1) corresponding to the maximum amplitude and the second prediction mode (mode2, M2) corresponding to the second largest amplitude can be obtained. The decoding end derives the first and second prediction modes in the same way.

[0094] The second step is to derive the prediction block. The same prediction block derivation method is used at the encoding end to derive the current prediction block. The encoding end determines the following two conditions: ① the amplitude of the second prediction mode is not 0; and ② the first prediction mode and the second prediction mode are not both PLANAR or DC prediction modes. If the above two conditions are not simultaneously established, the current prediction block only uses the first prediction mode to calculate the prediction sample value of the current block, that is, the ordinary prediction prediction process is applied to the first prediction mode; otherwise, that is, the above two conditions are both established, the current prediction block will use the weighted average method to derive the current prediction block. The specific method is that the PLANAR mode occupies 1 / 3 of the weighted weight, and the remaining 2 / 3 is the gradient strength (i.e. amplitude) of the first prediction mode as the weighted weight according to the gradient strength of the first prediction mode, and the gradient strength of the second prediction mode as the weighted weight according to the gradient strength of the second prediction mode. The above three prediction modes, that is, the PLANAR mode, the first prediction mode and the second prediction mode, are weighted and averaged to obtain the prediction block of the current coding unit. The decoding end derives the prediction block in the same steps. The specific operation process is shown in FIG. 3. The three intra prediction modes include the first prediction mode M1, the second prediction mode M2 and the PLANAR mode. The prediction values obtained by the three intra prediction modes are set as Pred1, Pred2 and Pred3 respectively. The weight values of the three intra prediction modes are set as w1, w2 and w3 respectively. Here, the specific weight calculation is as follows:

[0095] w3 = Weight (PLANAR) = 1 / 3;

[0096] w1 = Weight (mode1) = 2 / 3 * (amp1 / (amp1+amp2));

[0097] w2 = Weight (mode2) = 1 - Weight (PLANAR) - Weight (mode1);

[0098] Where mode1 and mode2 represent the first prediction mode and the second prediction mode respectively, and amp1 and amp2 represent the gradient strength value of the first prediction mode and the gradient strength value of the second prediction mode respectively. DIMD needs to transmit a flag to the decoding end to indicate whether the DIMD technology is used for the current coding unit.

[0099] The prediction block obtained by the weighted average can be as follows:

[0100] In summary, DIMD utilizes gradient analysis of reconstructed samples to screen intra prediction modes, and can further weight two intra prediction modes plus the PLANAR mode according to the analysis result. If the DIMD method is selected, it does not need to indicate which intra prediction mode is used, but the decoder itself derives it through the above process, which saves some overhead.

[0101] Further, the low-frequency non-separable transform technology is introduced below.

[0102] LFNST is applied between the forward primary transform and quantization at the encoding end, and between the inverse quantization and inverse primary transform at the decoding end. After the residual of the current coding unit is subjected to the primary transform, LFNST further converts part of the coefficients in the frequency domain, transforms part of the frequency domain coefficients, obtains coefficients in another domain, and then performs quantization, entropy coding and other operations. LFNST further removes statistical redundancy and has good performance on the existing VVC reference software VTM.

[0103] LFNST mainly performs secondary transform on the 4x4 or 8x8 region at the top left corner of the transform block. In addition, the transform kernel of LFNST is mainly classified into 4 transform sets in VVC, and each transform set has 2 candidate transform kernels. In ECM, the transform kernel of LFNST is expanded from the original 4 transform sets to 35 transform sets, and from the original 2 candidate transform kernels of each transform set to 3 candidate transform kernels of each transform set.

[0104] LFNST allows to act on intra prediction and inter prediction. This proposal mainly improves the intra prediction technology, so only the intra part of LFNST is introduced. LFNST uses the method of selecting a transform set corresponding to an intra prediction mode to save bit overhead. Since intra prediction usually has corresponding intra prediction modes, i.e. DC mode, PLANAR mode or angle prediction mode, these intra prediction modes are bound to the transform set of LFNST. For example, in VVC, the DC mode and the PLANAR mode correspond to the first transform set, as shown in Table 2, which provides the correspondence between LFNST and intra prediction modes in VVC.

[0105] Table 2

[0106] Correspondingly, after the transform kernel of LFNST is expanded in ECM, the corresponding LFNST transform set of different intra prediction modes will be more fine-grained, as shown in Table 3, which provides the correspondence between LFNST and intra prediction modes in ECM.

[0107] Table 3

[0108] In summary, the MIP technology predicts the current coding unit, and predicts the prediction block of the current coding unit by means of the surrounding neighboring reconstructed samples through the above steps. The DIMD technology is used on the prediction block before the MIP upsampling to derive the optimal intra prediction mode, and the LFNST transform set index of the current coding unit is obtained by looking up table 2 or table 3. The residual block of the current coding unit is obtained by subtracting the prediction sample from the original sample of the current coding unit. The primary transform is performed on the residual block to obtain the transform coefficient block of the current coding unit. If the use condition of the LFNST is met, the secondary transform is performed on part of the transform coefficient block, the intra prediction mode derived by the DIMD technology is mapped to the LFNST transform set, and the optimal LFNST transform kernel of the current coding unit is selected by traversing the transform set.

[0109] In the related art, the DIMD technology is applied to the MIP prediction block before upsampling, that is, the MIP output vector element. However, because the sizes of the coding units are different, the MIP technology may be different in sampling coefficients for upsampling the width or height of the output vector. For example, for some rectangular blocks, a plurality of sample points may be linearly interpolated on one side. Thus, the output vector before upsampling and the prediction block after upsampling may deviate in the texture direction, and the LFNST transform set obtained by this method may not be the most suitable for the current coding unit, thereby reducing the coding and decoding performance.

[0110] Based on this, the embodiment of the present application provides an encoding method. When a matrix-based intra prediction mode is used for a current block, a first prediction block of the current block is determined, and the first prediction block is upsampled to determine a second prediction block of the current block. Based on size information of the current block, a derived intra prediction mode of the current block is determined by using the first prediction block or the second prediction block, and a transform set of the current block is determined according to the derived intra prediction mode. The embodiment of the present application also provides a decoding method. A code stream is decoded, and a value of a first syntax element is determined.

[0111] When the first syntax element indicates that a matrix-based intra prediction mode is used for the current block, a first prediction block of the current block is determined, and the first prediction block is upsampled to determine a second prediction block of the current block. Based on size information of the current block, a derived intra prediction mode of the current block is determined by using the first prediction block or the second prediction block, and a transform set of the current block is determined according to the derived intra prediction mode.

[0112] In this way, at the encoding end or the decoding end, when the MIP mode is used for the current block, first, the first prediction block and the second prediction block of the current block are determined; then, according to the size information of the current block, the first prediction block or the second prediction block can be selected to determine the derived intra prediction mode of the current block, and then the transform set suitable for the current block can be determined, and a suitable transform kernel is selected for transformation, so that the accuracy of the transform prediction can be improved, the compression efficiency is improved, and the coding performance is improved.

[0113] The embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0114] FIG. 4 is a schematic diagram of a network architecture of a video coding provided by an embodiment of the present application. As shown in FIG. 4, the network architecture includes one or more electronic devices 13 to 1N and a communication network 01, wherein the electronic devices 13 to 1N can perform video interaction through the communication network 01. The electronic devices in the implementation can be various types of devices with video coding functions, for example, the electronic devices can include a mobile phone, a tablet computer, a personal computer, a personal digital assistant, a navigation device, a digital telephone, a video telephone, a television, a sensing device, a server, etc., and the embodiments of the present application are not limited thereto.

[0115] In the embodiments of the present application, a network architecture of a video coding system including a decoding method and an encoding method is provided. The decoder or the encoder in the embodiments of the present application can be the electronic device described above. That is, the electronic device in the embodiments of the present application has a video coding function, and generally includes a video / image encoder (referred to as an encoder) and a video / image decoder (referred to as a decoder).

[0116] FIG. 5 is a schematic diagram of a system composition of an encoder according to an embodiment of the present application. As shown in FIG. 5, the encoder 100 can include a transform and quantization unit 101, an intra estimation unit 102, an intra prediction unit 103, a motion compensation unit 104, a motion estimation unit 105, an inverse transform and inverse quantization unit 106, a filter control analysis unit 107, a filter unit 108, an encoding unit 109, and a decoded picture buffer unit 110, etc. The filter unit 108 can implement a deblocking filter and a sample adaptive offset (SAO) filter, and the encoding unit 109 can implement header information encoding and context-based adaptive binary arithmetic coding (CABAC).For the input raw video signal, a video coding block can be obtained by the division of a coding tree unit (CTU), and then the residual pixel information obtained after the intra- or inter-frame prediction is transformed by the transform and quantization unit 101 on the video coding block, including transforming the residual information from the pixel domain to the transform domain, and quantizing the obtained transform coefficients to further reduce the bit rate; the intra-estimation unit 102 and the intra-prediction unit 103 are used for intra-frame prediction of the video coding block; in particular, the intra-estimation unit 102 and the intra-prediction unit 103 are used to determine the intra-prediction mode to be used to encode the video coding block; the motion compensation unit 104 and the motion estimation unit 105 are used to perform inter-frame prediction encoding of the received video coding block relative to one or more blocks in one or more reference frames to provide temporal prediction information; the motion estimation performed by the motion estimation unit 105 is a process of generating a motion vector that can estimate the motion of the video coding block, and then the motion compensation is performed by the motion compensation unit 104 based on the motion vector determined by the motion estimation unit 105; after the intra-prediction mode is determined, the intra-prediction unit 103 is also used to provide the selected intra-prediction data to the encoding unit 109, and the motion estimation unit 105 also sends the calculated determined motion vector data to the encoding unit 109; in addition, the inverse transform and inverse quantization unit 106 is used for the reconstruction of the video coding block, reconstructing the residual block in the pixel domain, which removes the blocking artifact through the filter control analysis unit 107 and the filtering unit 108, and then adds the reconstructed residual block to a predictive block in one of the frames of the decoded image buffer unit 110 to generate a reconstructed video coding block; the encoding unit 109 is used to encode various encoding parameters and quantized transform coefficients, and in the CABAC-based encoding algorithm, the context content can be based on the adjacent coding block, which can be used to encode the information indicating the determined intra-prediction mode, and output the bitstream of the video signal; and the decoded image buffer unit 110 is used to store the reconstructed video coding block for prediction reference. As the video image encoding proceeds, new reconstructed video coding blocks are continuously generated, which are all stored in the decoded image buffer unit 110.

[0117] FIG. 6 is a schematic diagram of a structure of a decoder according to an embodiment of the present application. As shown in FIG. 6, the decoder 200 includes a decoding unit 201, an inverse transform and inverse quantization unit 202, an intra prediction unit 203, a motion compensation unit 204, a filtering unit 205, and a decoded picture buffer unit 206, etc. The decoding unit 201 can implement header information decoding and CABAC decoding, and the filtering unit 205 can implement deblocking filtering and SAO filtering. After the input video signal is processed by the encoder of FIG. 5, a bitstream of the video signal is output. The bitstream is input into the decoder 200, and first passes through the decoding unit 201 to obtain decoded transform coefficients. The inverse transform and inverse quantization unit 202 processes the transform coefficients to generate a residual block in the pixel domain. The intra prediction unit 203 can be used to generate prediction data of a current video decoding block based on a determined intra prediction mode and data from previously decoded blocks of the current frame or picture. The motion compensation unit 204 determines prediction information for the video decoding block by parsing motion vectors and other associated syntax elements, and uses the prediction information to generate a predictive block of the video decoding block being decoded. The decoded video block is formed by summing the residual block from the inverse transform and inverse quantization unit 202 and the corresponding predictive block generated by the intra prediction unit 203 or the motion compensation unit 204. The decoded video signal passes through the filtering unit 205 to remove blockiness artifacts and improve video quality. The decoded video block is then stored in the decoded picture buffer unit 206, which stores reference pictures for subsequent intra prediction or motion compensation, and also outputs the video signal, i.e., the original video signal is recovered.

[0118] It should be noted that the method according to the embodiments of the present application is mainly applied to the intra estimation unit 102 and the intra prediction unit 103 as shown in FIG. 5, and the intra prediction unit 203 as shown in FIG. 6. That is, the embodiments of the present application can be applied to an encoder, a decoder, or both, but are not limited here.

[0119] It should be further noted that when the embodiments of the present application are applied to the encoder 100, the "current block" specifically refers to a current block to be encoded (also referred to as an "encoding block") in a video image. When the embodiments of the present application are applied to the decoder 200, the "current block" specifically refers to a current block to be decoded (also referred to as a "decoding block") in a video image.

[0120] In an embodiment of the present application, FIG. 7 is a flowchart of a decoding method according to an embodiment of the present application. As shown in FIG. 7, the method can include:

[0121] S701, decoding a bitstream to determine a value of a first syntax element.

[0122] It should be noted that in the embodiments of the present application, the method is applied to a decoder. Specifically, based on the component structure of the decoder 200 shown in FIG. 6, the decoding method of the embodiments of the present application is mainly applied to the blocks in the intra prediction mode. Wherein, when the current block adopts the intra prediction mode, the optimization scheme based on the derivation of the transform set of the prediction block in the intra prediction mode is mainly proposed here to improve the compression efficiency.

[0123] It should be further noted that in the embodiments of the present application, considering that the size of the current block is different, the sampling coefficients of the MIP mode on the width or height of the up-sampling of the output vector can be different, so that there can be a deviation in the texture directionality between the prediction block before the up-sampling and the prediction block after the up-sampling. Therefore, in the process of deriving the transform set, the decoding method of the embodiments of the present application is specifically the transform set derivation method based on the prediction block in the MIP mode.

[0124] It should be further noted that in the embodiments of the present application, some mode indication information (or mode flag) in the form of syntax elements can be written in the code stream. In this way, by analyzing the value of the syntax element in the code stream, the prediction mode used by the current block can be determined. Exemplarily, the first syntax element can be used to indicate whether the matrix-based intra prediction mode is used for the current block, the second syntax element can be used to indicate whether the matrix-based intra prediction mode is allowed to be used for the current sequence, and the third syntax element can be used to indicate the prediction type of the current block. Wherein, the prediction type includes intra prediction or inter prediction.

[0125] In some embodiments, the method can include: decoding the code stream to determine the value of the second syntax element; when the second syntax element indicates that the matrix-based intra prediction mode is allowed to be used for the current sequence, decoding the code stream to determine the value of the third syntax element; and when the third syntax element indicates that the prediction type of the current block is intra prediction, performing the step of decoding the code stream to determine the value of the first syntax element.

[0126] In the embodiments of the present application, the current sequence can include the current block. Wherein, the second syntax element is a sequence-level syntax element, and the first syntax element and the third syntax element are CU-level syntax elements.

[0127] In some embodiments, if the value of the second syntax element is a first value, it is determined that the second syntax element indicates that the matrix-based intra prediction mode is allowed to be used for the current sequence; and if the value of the second syntax element is a second value, it is determined that the second syntax element indicates that the matrix-based intra prediction mode is not allowed to be used for the current sequence.

[0128] In some embodiments, if the third syntax element has the first value, it is determined that the third syntax element indicates that the prediction type of the current block is intra prediction; if the third syntax element has the second value, it is determined that the third syntax element indicates that the prediction type of the current block is inter prediction.

[0129] In some embodiments, if the first syntax element has the first value, it is determined that the first syntax element indicates that the current block uses the matrix-based intra prediction mode; if the first syntax element has the second value, it is determined that the first syntax element indicates that the current block does not use the matrix-based intra prediction mode.

[0130] In the embodiments of the present application, the first value is different from the second value. The first value can be set as 1, and the second value can be set as 0; or the first value can be set as 0, and the second value can be set as 1; or the first value can be set as true, and the second value can be set as false; or the first value can be set as false, and the second value can be set as true.

[0131] In the embodiments of the present application, the second syntax element can be referred to as a MIP allowed use flag, which is a sequence level flag, denoted as sps_mip_enable_flag. The first syntax element can be referred to as a MIP use flag, which is a CU level flag.

[0132] For example, taking true as the first value and false as the second value, the decoding end first decodes the value of the second syntax element to determine whether the MIP mode is allowed to be used in the current sequence; if the value of the second syntax element is true, it indicates that the MIP mode is allowed to be used in the current sequence, then the decoding end continues to decode the value of the third syntax element; if the value of the third syntax element is true, it indicates that the prediction type of the current block is intra prediction, then the decoding end continues to decode the value of the first syntax element, i.e., the MIP use flag of the current block, to determine whether the MIP mode is used in the current block.

[0133] S702, when the first syntax element indicates that the current block uses the matrix-based intra prediction mode, determining a first prediction block of the current block, and performing up-sampling processing on the first prediction block to determine a second prediction block of the current block.

[0134] In the embodiments of the present application, if the value of the first syntax element is true, it indicates that the MIP mode is used in the current block, at this time, the MIP mode can be used to determine the first prediction block of the current block. Since the size of the first prediction block is different from the size of the current block, by performing filtering processing on the first prediction block, the second prediction block of the current block can be determined, and the size of the second prediction block is the same as the size of the current block.

[0135] In the embodiments of the present application, the first prediction block of the current block is determined, and specifically, a prediction sample in the first prediction block of the current block is determined by using the MIP mode. In addition, the first prediction block is up-sampled to determine a prediction sample in the second prediction block of the current block.

[0136] In a possible implementation, if the size of the first prediction block is smaller than the size of the current block, the filtering process can be up-sampling, that is, the first prediction block is up-sampled to determine the second prediction block of the current block. In another possible implementation, if the size of the first prediction block is larger than the size of the current block, the filtering process can be down-sampling, that is, the first prediction block is down-sampled to determine the second prediction block of the current block.

[0137] In the embodiments of the present application, the first prediction block obtained by the MIP mode is of a fixed size, for example, 16 or 64 sample points, and therefore, the first prediction block usually needs to be up-sampled to obtain a second prediction block of the same size as the current block.

[0138] In a specific embodiment, for step S702, referring to FIG. 8, the step can include:

[0139] S801, when the first syntax element indicates that the current block uses the matrix-based intra prediction mode, decoding the code stream to determine the prediction parameter of the current block, the prediction parameter including the transposition identification information and the weight matrix index information.

[0140] S802, performing down-sampling on the neighboring reference sample of the current block to determine an initial input vector of the current block.

[0141] S803, determining a target input vector of the current block according to the transposition identification information and the initial input vector.

[0142] S804, determining the weight matrix of the current block according to the weight matrix index information, and performing multiplication operation on the target input vector and the weight matrix to determine an output vector of the current block.

[0143] S805, arranging and placing the output vector of the current block according to the transposition identification information to obtain the first prediction block of the current block.

[0144] S806, performing up-sampling on the first prediction block to determine the second prediction block of the current block.

[0145] In the embodiments of the present application, if the current block uses the MIP mode, the prediction parameter of the current block needs to be further decoded, specifically including the transposition identification information and the weight matrix index information. The weight matrix index information can also be referred to as the MIP mode index.

[0146] In the embodiments of the present application, in combination with the foregoing Table 1, the class index value of the current block can be determined according to the size information of the current block, denoted as classIdx (or sizeId). Then the input vector element number and the output vector element number of the current block can be determined according to the classIdx.

[0147] For example, if the classIdx is equal to 0, the input vector element number is 4 and the output vector element number is 16; if the classIdx is equal to 1, the input vector element number is 8 and the output vector element number is 16; if the classIdx is equal to 2, the input vector element number is 7 and the output vector element number is 64.

[0148] In the embodiments of the present application, the adjacent reference samples of the current block are down-sampled to determine the initial input vector of the current block, and the element number in the initial input vector is equal to the input vector element number.

[0149] In the embodiments of the present application, the target input vector of the current block is determined according to the transpose identification information and the initial input vector. The target input vector of the current block can be determined by splicing the initial input vector according to the transpose identification information and processing the spliced input vector. Assuming that the initial input vector of the current block includes the left boundary input vector bdry RedLeft and the upper boundary input vector bdry RedTop , when splicing, if the transpose identification information indicates normal input, bdry RedTop is in front of bdry RedLeft ; if the transpose identification information indicates transposed input, bdry RedLeft is in front of bdry RedTop .

[0150] In the embodiments of the present application, after the output vector element number of the current block is determined according to the size information of the current block, the multiplication operation is performed on the target input vector and the weight matrix to determine the output vector of the current block, and the element number in the output vector is equal to the output vector element number.

[0151] In the embodiments of the present application, the output vector of the current block is arranged according to the transposition identification information, to obtain the prediction samples in the first prediction block of the current block; and then the prediction samples in the first prediction block are up-sampled to determine the prediction samples in the second prediction block of the current block. Here, the up-sampling can be interpolation processing. Specifically, each element in the output vector is placed in the corresponding position according to the arrangement order shown in FIG. 2 according to the MIP transposition identification bit information. The interpolation order follows horizontal linear interpolation first, and then vertical linear interpolation. The method for horizontal linear interpolation is similar to that for vertical linear interpolation. For the missing sample points, new sample points are obtained by linear interpolation with the left and right adjacent sample points. After interpolation up-sampling, the second prediction block with the same size as the current block is obtained. It should be noted that when each element in the output vector is placed in the corresponding position according to the arrangement order shown in FIG. 2 according to the MIP transposition identification bit information, if the transposition identification information indicates normal input, i.e., the input vector is not transposed, each element in the output vector is arranged in a horizontal line from left to right and row by row; if the transposition identification information indicates transposed input, i.e., the input vector is transposed, each element in the output vector is arranged in a column from top to bottom and column by column.

[0152] That is, when the current block uses the MIP mode, according to the output vector of the current block, the first prediction block obtained is the prediction block before up-sampling, and the second prediction block is the prediction block after up-sampling, and the size of the second prediction block is the same as that of the current block.

[0153] S703, determining the derived intra prediction mode of the current block based on the size information of the current block by using the first prediction block or the second prediction block, and determining the transform set of the current block according to the derived intra prediction mode.

[0154] In the embodiments of the present application, for the derived intra prediction mode, the DIMD method can be used for intra prediction mode derivation. For example, the first prediction block can be used to determine the derived intra prediction mode of the current block, or the second prediction block can be used to determine the derived intra prediction mode of the current block. That is, in the embodiments of the present application, for the derived intra prediction mode, the prediction samples in the first prediction block can be used to determine the derived intra prediction mode of the current block, or the prediction samples in the second prediction block can be used to determine the derived intra prediction mode of the current block; and then the transform set of the current block is determined according to the derived intra prediction mode.

[0155] In some embodiments, determining the derived intra prediction mode of the current block based on the size information of the current block using the first prediction block or the second prediction block can comprise: determining the derived intra prediction mode using the second prediction block when the size information of the current block satisfies a first condition; and determining the derived intra prediction mode using the first prediction block when the size information of the current block satisfies a second condition.

[0156] In a possible implementation, the size information of the current block satisfying the first condition can comprise: the current block being a non-square block; and the size information of the current block satisfying the second condition can comprise: the current block being a square block.

[0157] That is, in the embodiments of the present application, different current blocks are distinguished. Specifically, if the current block is a non-square block, or the current block is a rectangular block, the second prediction block is used to determine the derived intra prediction mode of the current block, and then the transform set of the current block is determined; if the current block is a square block, or the width of the current block is equal to the height of the current block, the first prediction block is used to determine the derived intra prediction mode of the current block, and then the transform set of the current block is determined.

[0158] In another possible implementation, the size information of the current block satisfying the first condition can comprise: the class index value of the current block being equal to 0 or 1; and the size information of the current block satisfying the second condition can comprise: the class index value of the current block being equal to 2.

[0159] That is, in the embodiments of the present application, different current blocks are distinguished according to the classIdx. Specifically, if the classIdx is equal to 0 or 1, the second prediction block is used to determine the derived intra prediction mode of the current block, and then the transform set of the current block is determined; if the classIdx is equal to 2, the first prediction block is used to determine the derived intra prediction mode of the current block, and then the transform set of the current block is determined.

[0160] In yet another possible implementation, the size information of the current block satisfying the first condition can comprise: the current block being a non-square block and the class index value of the current block not being equal to 2; and the size information of the current block satisfying the second condition can comprise: the current block being a square block or the class index value of the current block being equal to 2.

[0161] That is, in the embodiments of the present application, the current block can be distinguished as follows: if the current block is a non-square block and the classIdx is not equal to 2, the second prediction block is used to determine the derived intra prediction mode of the current block, and then the transform set of the current block is determined; otherwise, the first prediction block is used to determine the derived intra prediction mode of the current block, and then the transform set of the current block is determined.

[0162] It should be noted that the first condition and the second condition are not limited to the above cases, for example, other limited cases of the index value can also be used, such as the first condition includes that the category index value of the current block is equal to 0, and the second condition includes that the category index value of the current block is equal to 1; or the first condition includes that the category index value of the current block is equal to 1, and the second condition includes that the category index value of the current block is equal to 0; and the like. Even the area size of the current block can be used for limitation, for example, the first condition includes that the area of the current block is less than 256 sample sizes, and the second condition includes that the area of the current block is greater than 256 sample sizes, and the like, which are not limited herein.

[0163] In another possible implementation, a preset intra prediction mode can also be directly used as the derived intra prediction mode of the current block, and then the transform set of the current block is determined according to the derived intra prediction mode.

[0164] In some embodiments, the method can further include: when the size information of the current block satisfies the first condition, determining the preset intra prediction mode as the derived intra prediction mode of the current block, and determining the transform set of the current block according to the derived intra prediction mode.

[0165] In some embodiments, the method can further include: when the size information of the current block satisfies the second condition, determining the preset intra prediction mode as the derived intra prediction mode of the current block, and determining the transform set of the current block according to the derived intra prediction mode.

[0166] In the embodiments of the present application, the preset intra prediction mode can be a PLANAR mode, a DC mode or other modes, instead of using DIMD to derive the intra prediction mode of the current block. For example, for determining the derived intra prediction mode of the current block by using the second prediction block, a preset intra prediction mode such as a PLANAR mode, a DC mode or other modes can be used instead; for determining the derived intra prediction mode of the current block by using the first prediction block, a preset intra prediction mode such as a PLANAR mode, a DC mode or other modes can also be used instead.

[0167] In a possible implementation, DIMD method can be used herein to determine the derived intra prediction mode of the current block by using the first prediction block or the second prediction block.

[0168] In some embodiments, determining the derived intra prediction mode of the current block using the first prediction block can include: determining candidate samples for deriving the intra prediction mode based on the first prediction block; determining at least one candidate intra prediction mode and a gradient strength accumulation value corresponding to the at least one candidate intra prediction mode according to horizontal gradient values and vertical gradient values of the candidate samples; determining a maximum gradient strength accumulation value in the gradient strength accumulation values corresponding to the at least one candidate intra prediction mode, and determining a candidate intra prediction mode corresponding to the maximum gradient strength accumulation value as the derived intra prediction mode of the current block.

[0169] In some embodiments, determining the derived intra prediction mode of the current block using the second prediction block can include: determining candidate samples for deriving the intra prediction mode based on the second prediction block; determining at least one candidate intra prediction mode and a gradient strength accumulation value corresponding to the at least one candidate intra prediction mode according to horizontal gradient values and vertical gradient values of the candidate samples; determining a maximum gradient strength accumulation value in the gradient strength accumulation values corresponding to the at least one candidate intra prediction mode, and determining a candidate intra prediction mode corresponding to the maximum gradient strength accumulation value as the derived intra prediction mode of the current block.

[0170] It should be noted that, in the embodiments of the present application, determining the at least one candidate intra prediction mode and the gradient strength accumulation value corresponding to the at least one candidate intra prediction mode according to the horizontal gradient values and the vertical gradient values of the candidate samples can include: determining a reference intra prediction mode and a gradient strength value corresponding to each of the candidate samples according to the horizontal gradient values and the vertical gradient values of the candidate samples; and determining the at least one candidate intra prediction mode and the gradient strength accumulation value corresponding to the at least one candidate intra prediction mode according to the reference intra prediction mode and the gradient strength value corresponding to each of the candidate samples.

[0171] It should be further noted that, in the embodiments of the present application, the horizontal gradient value of the candidate sample can be represented by grad x , and the vertical gradient value of the candidate sample can be represented by grad y . In this way, the reference intra prediction mode can be derived according to grad x and grad y . In addition, the gradient strength value corresponding to the candidate sample can be denoted as amp, amp = abs(grad x ) + abs(grad y ).

[0172] In the embodiments of the present application, the horizontal gradient value and the vertical gradient value of the candidate sample can be calculated using a Sobel operator. Exemplarily, for the Sobel operator, the following is shown:

[0173] Operator of the horizontal gradient value:

[0174] Operator of vertical gradient value:

[0175] Thus, assuming the sample value of sample position (x, y) is P x,y , the horizontal gradient value grad x and the vertical gradient value grad y are calculated as shown below: grad x = P x+1,y-1 + 2*P x+1,y + P x+1,y+1 - P x-1,y-1 - 2*P x-1,y - P x-1,y+1 (2) grad y = P x-1,y+1 + 2*P x,y+1 + P x+1,y+1 - P x-1,y-1 - 2*P x,y-1 - P x+1,y-1 (3)

[0176] It should be further noted that, in the embodiments of the present application, the candidate samples used for deriving the intra prediction mode can be the prediction block of the current block, can be the reconstructed area on the left and on the top of the current block, or can be both the prediction block of the current block and the reconstructed area on the left and on the top of the current block. In this way, more samples are used for deriving the intra prediction mode, and the derived intra prediction mode is more accurate. In addition, in the embodiments of the present application, the number of candidate samples used for deriving the intra prediction mode can be at least one, for example, 1, 2, 3 or more.

[0177] In some embodiments, the determining of the at least one candidate intra prediction mode and the gradient strength accumulation value corresponding to the at least one candidate intra prediction mode according to the reference intra prediction mode corresponding to each of the candidate samples and the gradient strength value can include: determining the at least one candidate intra prediction mode with different characteristics according to the reference intra prediction mode corresponding to each of the candidate samples, and accumulating the gradient strength values belonging to the same candidate texture feature index according to the gradient strength value corresponding to each of the candidate samples to determine the gradient strength accumulation value corresponding to the at least one candidate intra prediction mode.

[0178] Thus, after determining the gradient strength accumulated values corresponding to the at least one candidate intra prediction mode, the maximum gradient strength accumulated value is determined from the gradient strength accumulated values corresponding to the at least one candidate intra prediction mode, and the candidate intra prediction mode corresponding to the maximum gradient strength accumulated value is determined as the derived intra prediction mode of the current block.

[0179] In another possible implementation, the TIMD method can be used here to determine the derived intra prediction mode of the current block by using the first prediction block or the second prediction block.

[0180] In some embodiments, determining the derived intra prediction mode of the current block by using the first prediction block can include: determining a template region corresponding to the first prediction block; predicting the template region by using at least one candidate intra prediction mode to determine predicted samples of the template region corresponding to the at least one candidate intra prediction mode; performing cost calculation on the predicted samples of the template region and the reconstructed samples of the template region respectively according to the at least one candidate intra prediction mode to determine cost results corresponding to the at least one candidate intra prediction mode; determining a minimum cost result from the cost results corresponding to the at least one candidate intra prediction mode, and determining the candidate intra prediction mode corresponding to the minimum cost result as the derived intra prediction mode of the current block.

[0181] In some embodiments, determining the derived intra prediction mode of the current block by using the second prediction block can include: determining a template region corresponding to the second prediction block; predicting the template region by using at least one candidate intra prediction mode to determine predicted samples of the template region corresponding to the at least one candidate intra prediction mode; performing cost calculation on the predicted samples of the template region and the reconstructed samples of the template region respectively according to the at least one candidate intra prediction mode to determine cost results corresponding to the at least one candidate intra prediction mode; determining a minimum cost result from the cost results corresponding to the at least one candidate intra prediction mode, and determining the candidate intra prediction mode corresponding to the minimum cost result as the derived intra prediction mode of the current block.

[0182] It should be noted that in the embodiments of the present application, the cost calculation here can be determined according to the cost result of rate distortion optimization (RDO), or according to the cost result of sum of absolute difference (SAD), or even according to the cost result of sum of absolute transformed difference (SATD), but here is not limited in any way.

[0183] That is, the process of using DIMD to derive the intra prediction mode in the embodiments of the present application can be replaced by the method of using TIMD. The TIMD method is a template matching technology. The template region can be set as the neighboring region of the current block or the size of the current block. By calculating the SAD or SATD of the prediction sample and the reconstructed sample, the cost result is obtained. Different intra prediction modes correspond to different prediction samples, so the optimal intra prediction mode can be obtained according to the principle of minimum cost, which is used as the derived intra prediction mode of the current block.

[0184] In some embodiments, determining the transform set of the current block according to the derived intra prediction mode can include: determining a transform set index of the current block according to the first lookup table and the derived intra prediction mode; and determining the transform set of the current block according to the transform set index.

[0185] In the embodiments of the present application, the first lookup table is used to record the mapping relationship between the intra prediction mode and the transform set index. For example, in combination with Table 3 described above, after the derived intra prediction mode is obtained, the corresponding transform set index can be determined by looking up the table, and then the transform set of the current block is determined. The transform set of the current block can include at least one transform kernel.

[0186] It can be understood that in the embodiments of the present application, after the transform set of the current block is determined, the residual sample of the current block can be further determined. Referring to FIG. 9, the method can further include:

[0187] S901, determining the transform kernel of the current block according to the transform set.

[0188] S902, determining the transform coefficient of the current block.

[0189] S903, transforming the transform coefficient of the current block according to the transform kernel to determine the residual sample of the current block.

[0190] In the embodiments of the present application, the execution order of steps S901 and S902 has no sequence. For example, step S901 can be executed first, and then step S902 can be executed. Or step S902 can be executed first, and then step S901 can be executed. Or step S901 and step S902 can be executed in parallel, which is not limited here.

[0191] In some embodiments, determining the transform kernel of the current block according to the transform set can include: decoding the code stream to determine the transform kernel index of the current block; and determining the transform kernel of the current block according to the transform set and the transform kernel index.

[0192] It should be noted that in the embodiments of the present application, when the current block uses the MIP mode, in addition to the prediction parameters such as the transposition identification information and the weight matrix index information, the transform kernel index information is also included in the code stream; so that the corresponding transform kernel can be obtained from the transform set and the transform kernel is applied to the decoding obtained transform coefficient.

[0193] It should be further noted that in the embodiments of the present application, determining the transform coefficient of the current block can include: decoding the code stream to determine the quantized coefficient of the current block; and dequantizing the quantized coefficient of the current block to determine the transform coefficient of the current block. That is, the transform kernel here is applied to the dequantized transform coefficient.

[0194] It should be further noted that in the embodiments of the present application, the transform set here can include the NSPT transform set and the LFNST transform set. The transform kernel for NSPT can be determined according to the NSPT transform set; and the transform kernel for LFNST can be determined according to the LFNST transform set.

[0195] In a possible implementation, referring to FIG. 10, after step S902, the method can further include:

[0196] S1001, performing non-separable primary transform on the transform coefficient of the current block according to the transform kernel to determine the residual sample of the current block.

[0197] In the embodiments of the present application, if the size parameter of the current block meets the first condition, for example, the size parameter of the current block is less than a certain threshold, that is, the size parameter of the current block is small, the transform kernel of NSPT can be used at this time. That is, for a block with a smaller size, the transform kernel of NSPT is used here, that is, the NSPT inverse transform is performed on the transform coefficient of the current block according to the transform kernel to determine the residual sample of the current block.

[0198] In another possible implementation, referring to FIG. 11, after step S902, the method can further include:

[0199] S1101, performing low-frequency non-separable transform on the transform coefficient of the current block according to the transform kernel to determine the transform sample of the current block.

[0200] S1102, performing discrete cosine transform on the transform sample of the current block to determine the residual sample of the current block.

[0201] In the embodiments of the present application, if the size parameter of the current block satisfies a second condition, for example, the size parameter of the current block is greater than a certain threshold, that is, the size parameter of the current block is large, the transform kernel of LFNST can be used at this time. That is, for a block with a large size, the transform kernel of LFNST is used here, that is, the inverse LFNST of the transform kernel is performed on the transform coefficients of the current block to determine the transform samples of the current block; and the inverse DCT2 is performed on the transform samples of the current block to determine the residual samples of the current block.

[0202] It should also be noted that in the embodiments of the present application, the "inverse transform" of the transform coefficients at the decoding end can also be referred to as "transform" in the standard text. "Transform" and "inverse transform" in this article correspond to two opposite processes, such as "transform" converts the values in the spatial domain to the coefficients in the frequency domain, and then "inverse transform" converts the coefficients in the frequency domain to the values in the spatial domain. "Inverse" is relative to "positive", and they are essentially both transforms. It should be noted that if the standard only specifies decoding, then "transform" in the standard text is the part of decoding, specifically referring to "inverse transform" in this article. The "inverse transform" of the transform coefficients at the decoding end can also be referred to as "transform" in the standard text.

[0203] It can be understood that in the embodiments of the present application, after the residual samples of the current block are determined, the method can further include: determining the reconstructed block of the current block according to the prediction samples of the second prediction block and the residual samples of the current block. Specifically, the prediction samples of the second prediction block and the residual samples of the current block can be added to determine the reconstructed block of the current block.

[0204] It can also be understood that in the embodiments of the present application, when the current block does not use the MIP mode, the decoding end needs to continue to decode the code stream to determine the other prediction mode used by the current block, and then determine the prediction samples of the current block by using the other prediction mode; then the residual samples obtained by decoding are added to the prediction samples of the current block to finally determine the reconstructed block of the current block.

[0205] It can also be understood that in the embodiments of the present application, after all the reconstructed blocks in the current image are filtered through the loop filtering technology and the like, the final reconstructed image is obtained, which can be simultaneously output as a video image and used as a subsequent decoding reference.

[0206] The embodiment of the present application provides a decoding method, which comprises the following steps: decoding a code stream, determining the value of a first syntax element; when the first syntax element indicates that a current block uses a matrix-based intra prediction mode, determining a first prediction block of the current block, and performing up-sampling processing on the first prediction block to determine a second prediction block of the current block; determining a derived intra prediction mode of the current block by using the first prediction block or the second prediction block based on size information of the current block, and determining a transform set of the current block according to the derived intra prediction mode. In this way, when the current block uses the MIP mode, the first prediction block and the second prediction block of the current block are determined; then, according to the size information of the current block, the first prediction block or the second prediction block can be selected to determine the derived intra prediction mode of the current block, and then the transform set suitable for the current block can be determined, the transform coefficients of the current block are transformed according to the transform kernel in the transform set, and therefore the accuracy of the transform prediction can be improved, the compression efficiency is improved, and the coding and decoding performance is improved.

[0207] In another embodiment of the present application, Fig. 12 is a flowchart of an encoding method provided by the embodiment of the present application. As shown in Fig. 12, the method can comprise the following steps:

[0208] S1201, when the current block uses the matrix-based intra prediction mode, determining a first prediction block of the current block, and performing up-sampling processing on the first prediction block to determine a second prediction block of the current block.

[0209] It should be noted that in the embodiment of the present application, the method is applied to an encoder. Specifically, based on the component structure of the encoder 100 shown in Fig. 5, the encoding method of the embodiment of the present application is mainly applied to the block of intra prediction. When the current block uses the intra prediction mode, the optimization scheme based on the prediction block for deriving the transform set in the intra prediction mode is mainly proposed herein to improve the compression efficiency.

[0210] It should be further noted that in the embodiment of the present application, considering that the size of the current block is different, the sampling coefficients of the MIP mode on the width or height of the up-sampling of the output vector can be different, so that there can be a deviation in the texture directionality between the prediction block before the up-sampling and the prediction block after the up-sampling. Therefore, in the process of deriving the transform set, the encoding method of the embodiment of the present application is a transform set derivation method based on the prediction block in the MIP mode.

[0211] It should be further noted that, in the embodiments of the present application, the encoding end can write some mode indication information (or mode flag) in the form of syntax elements in the code stream. In this way, the subsequent decoding end can determine the prediction mode used by the current block by parsing the values of the syntax elements in the code stream. For example, a first syntax element can be used to indicate whether the current block uses the matrix-based intra prediction mode, a second syntax element can be used to indicate whether the current sequence allows the use of the matrix-based intra prediction mode, and a third syntax element can be used to indicate the prediction type of the current block. The prediction type includes intra prediction or inter prediction.

[0212] In some embodiments, the method can include: determining the value of the second syntax element; wherein the second syntax element is used to indicate whether the current sequence allows the use of the matrix-based intra prediction mode; and encoding the value of the second syntax element and writing the obtained coded bits into the code stream.

[0213] In some embodiments, the method can include: when the current sequence allows the use of the matrix-based intra prediction mode, determining the value of the third syntax element; wherein the third syntax element is used to indicate the prediction type of the current block; and encoding the value of the third syntax element and writing the obtained coded bits into the code stream.

[0214] In some embodiments, the method can include: when the prediction type of the current block is intra prediction, determining the value of the first syntax element; wherein the first syntax element is used to indicate whether the current block uses the matrix-based intra prediction mode; and encoding the value of the first syntax element and writing the obtained coded bits into the code stream.

[0215] In the embodiments of the present application, the current sequence can include the current block. The second syntax element is a sequence-level syntax element, and the first syntax element and the third syntax element are CU-level syntax elements.

[0216] In some embodiments, if the current sequence allows the use of the matrix-based intra prediction mode, the value of the second syntax element is determined to be a first value; and if the current sequence does not allow the use of the matrix-based intra prediction mode, the value of the second syntax element is determined to be a second value.

[0217] In some embodiments, if the prediction type of the current block is intra prediction, the value of the third syntax element is determined to be a first value; and if the prediction type of the current block is inter prediction, the value of the third syntax element is determined to be a second value.

[0218] In some embodiments, if the current block uses the matrix-based intra prediction mode, the value of the first syntax element is determined to be a first value; and if the current block does not use the matrix-based intra prediction mode, the value of the first syntax element is determined to be a second value.

[0219] In the embodiments of the present application, the first value is different from the second value. The first value can be set as 1, and the second value can be set as 0; or the first value can be set as 0, and the second value can be set as 1; or the first value can be set as true, and the second value can be set as false; or the first value can be set as false, and the second value can be set as true.

[0220] In the embodiments of the present application, the second syntax element can be referred to as a MIP enabling flag, which is a sequence level flag, denoted as sps_mip_enable_flag. The first syntax element can be referred to as a MIP using flag, which is a CU level flag.

[0221] Exemplarily, taking the first value as true and the second value as false as an example, the encoding end first determines whether the MIP mode is allowed to be used for the current sequence, and writes the value of the second syntax element into the code stream; if the value of the second syntax element is true, it indicates that the MIP mode is allowed to be used for the current sequence, then the prediction type of the current block is determined, and the value of the third syntax element is written into the code stream; if the value of the third syntax element is true, it indicates that the prediction type of the current block is intra prediction, then it is determined whether the MIP mode is used for the current block, and the value of the first syntax element is written into the code stream.

[0222] It can be understood that in the embodiments of the present application, whether the MIP mode is used for the current block can be determined based on the cost results of a plurality of candidate prediction modes. In some embodiments, the method can include: determining at least one candidate prediction mode of the current block; performing cost calculation on the current block by using the at least one candidate prediction mode to determine the cost result of the at least one candidate prediction mode; and if the cost result of the matrix-based intra prediction mode in the at least one candidate prediction mode is the smallest, determining that the matrix-based intra prediction mode is used for the current block.

[0223] In the embodiments of the present application, the cost calculation here can be determined according to the cost result of rate distortion optimization (RDO), or according to the cost result of sum of absolute difference (SAD), or even according to the cost result of sum of absolute transformed difference (SATD), but here is not limited in any way.

[0224] Taking the RDO cost as an example, a current block is predicted by using at least one candidate prediction mode to determine a candidate prediction block corresponding to the at least one candidate prediction mode; then, a transform coefficient corresponding to the at least one candidate prediction mode is determined according to the candidate prediction block corresponding to the at least one candidate prediction mode; after quantization, dequantization and inverse transformation are performed on the transform coefficient corresponding to the at least one candidate prediction mode, the reconstructed sample corresponding to the at least one candidate prediction mode can be obtained; the rate-distortion cost calculation is performed on the reconstructed sample corresponding to the at least one candidate prediction mode and the original sample respectively, and the cost result corresponding to the at least one candidate prediction mode is determined. If the cost result corresponding to the matrix-based intra prediction mode is the minimum, it is determined that the current block uses the matrix-based intra prediction mode, that is, the value of the first syntax element is true; if the cost result corresponding to the matrix-based intra prediction mode is not the minimum, it is determined that the current block does not use the matrix-based intra prediction mode, that is, the value of the first syntax element is false.

[0225] In the embodiment of the present application, if the value of the first syntax element is true, it indicates that the current block uses the MIP mode, at this time, the first prediction block of the current block can be determined by using the MIP mode. Since the size of the first prediction block is different from the size of the current block, the second prediction block of the current block can be determined by filtering the first prediction block, and the size of the second prediction block is the same as the size of the current block.

[0226] In a possible implementation, if the size of the first prediction block is smaller than the size of the current block, the filtering at this time can be up-sampling processing, that is, the second prediction block of the current block can be determined by up-sampling the first prediction block. In another possible implementation, if the size of the first prediction block is greater than the size of the current block, the filtering at this time can be down-sampling processing, that is, the second prediction block of the current block can be determined by down-sampling the first prediction block.

[0227] In the embodiment of the present application, since the first prediction block obtained by the MIP mode is of a fixed size, for example, 16 or 64 sample points, it is usually necessary to up-sample the first prediction block to obtain a second prediction block of the same size as the current block.

[0228] It can be understood that, in the embodiment of the present application, the first prediction block of the current block is determined, specifically, the prediction samples in the first prediction block of the current block are determined by using the MIP mode. In addition, the first prediction block is up-sampled, specifically, the prediction samples in the second prediction block of the current block are up-sampled.

[0229] In some embodiments, when the current block uses the matrix-based intra prediction mode, the method can further include: determining a prediction parameter of the current block, the prediction parameter including transpose identification information and weight matrix index information; and performing encoding processing on the prediction parameter of the current block, and writing the obtained encoded bits into the bitstream. The weight matrix index information can also be referred to as MIP mode index.

[0230] It should be noted that in the embodiments of the present application, the prediction parameter of the current block is used for matrix-based intra prediction of the current block to determine the first prediction block of the current block. In some embodiments, for determining the first prediction block of the current block, the following steps can be included: when the current block uses the matrix-based intra prediction mode, determining a prediction parameter of the current block, the prediction parameter including transpose identification information and weight matrix index information; performing down-sampling processing on the neighboring reference samples of the current block to determine an initial input vector of the current block; determining a target input vector of the current block according to the transpose identification information and the initial input vector; determining a weight matrix of the current block according to the weight matrix index information, and performing multiplication operation on the target input vector and the weight matrix to determine an output vector of the current block; and arranging and placing the output vector of the current block according to the transpose identification information to obtain the first prediction block of the current block. Then the first prediction block can be further up-sampled to determine the second prediction block of the current block.

[0231] In the embodiments of the present application, according to the size information of the current block, the class index value of the current block can be determined, which is denoted as classIdx (or sizeId). Then according to the classIdx, the input vector element number and the output vector element number of the current block can be determined.

[0232] For example, if the classIdx is equal to 0, the input vector element number is 4 and the output vector element number is 16; if the classIdx is equal to 1, the input vector element number is 8 and the output vector element number is 16; if the classIdx is equal to 2, the input vector element number is 7 and the output vector element number is 64.

[0233] In the embodiments of the present application, the initial input vector of the current block is determined by down-sampling the neighboring reference samples of the current block, and the number of elements in the initial input vector is equal to the input vector element number.

[0234] In the embodiments of the present application, the target input vector of the current block can be determined according to the transpose identification information and the initial input vector, which can be obtained by splicing the initial input vector according to the transpose identification information, and processing the spliced input vector to determine the target input vector of the current block. Assuming that the initial input vector of the current block includes a left boundary input vector bdry RedLeft and an upper boundary input vector bdry RedTopThen, when performing the splicing, if the transposition identification information indicates normal input, the bdry RedTop In the bdry RedLeft front; if the transposition identification information indicates transposed input, the bdry RedLeft In the bdry RedTop front.

[0235] In the embodiment of the present application, after the number of output vector elements of the current block is determined according to the size information of the current block, the multiplication operation is performed on the target input vector and the weight matrix to determine the output vector of the current block, and the number of elements in the output vector is equal to the number of output vector elements.

[0236] In the embodiment of the present application, the output vector of the current block is arranged according to the transposition identification information to obtain the prediction samples in the first prediction block of the current block; and then the prediction samples in the first prediction block are up-sampled to determine the prediction samples in the second prediction block of the current block. Here, the up-sampling can be interpolation processing. Specifically, each element in the output vector is placed in the corresponding position according to the arrangement order shown in FIG. 2 according to the MIP transposition identification bit information. The interpolation order follows the horizontal linear interpolation first, and then the vertical linear interpolation. The method for the horizontal linear interpolation is similar to that for the vertical linear interpolation. The missing sample points are obtained by linear interpolation with the left and right adjacent sample points, and then the new sample points are obtained by linear interpolation with the upper and lower adjacent sample points. After the interpolation up-sampling, the second prediction block with the same size as the current block is obtained. It should be noted that when each element in the output vector is placed in the corresponding position according to the arrangement order shown in FIG. 2 according to the MIP transposition identification bit information, if the transposition identification information indicates normal input, i.e., the input vector is not transposed, each element in the output vector is arranged in a horizontal line from left to right and row by row; if the transposition identification information indicates transposed input, i.e., the input vector is transposed, each element in the output vector is arranged in a column from top to bottom and column by column.

[0237] That is, when the current block uses the MIP mode, according to the output vector of the current block, the first prediction block obtained is the prediction block before up-sampling, and the second prediction block is the prediction block after up-sampling, and the size of the second prediction block is the same as that of the current block.

[0238] In S1202, the derived intra prediction mode of the current block is determined by using the first prediction block or the second prediction block based on the size information of the current block, and the transform set of the current block is determined according to the derived intra prediction mode.

[0239] In the embodiments of the present application, for deriving the intra prediction mode, the DIMD method can be used to derive the intra prediction mode. For example, the first prediction block can be used to determine the derived intra prediction mode of the current block, or the second prediction block can be used to determine the derived intra prediction mode of the current block. That is, in the embodiments of the present application, for deriving the intra prediction mode, the prediction samples in the first prediction block can be used to determine the derived intra prediction mode of the current block, or the prediction samples in the second prediction block can be used to determine the derived intra prediction mode of the current block; and then the transform set of the current block is determined according to the derived intra prediction mode.

[0240] In some embodiments, based on the size information of the current block, the first prediction block or the second prediction block is used to determine the derived intra prediction mode of the current block, which can include: when the size information of the current block satisfies a first condition, the second prediction block is used to determine the derived intra prediction mode; and when the size information of the current block satisfies a second condition, the first prediction block is used to determine the derived intra prediction mode.

[0241] In a possible implementation, the size information of the current block satisfying the first condition can include that the current block is a non-square block; and the size information of the current block satisfying the second condition can include that the current block is a square block.

[0242] That is, in the embodiments of the present application, different current blocks are distinguished. Specifically, if the current block is a non-square block, or the current block is a rectangular block, the second prediction block is used to determine the derived intra prediction mode of the current block, and then the transform set of the current block is determined; if the current block is a square block, or the width of the current block is equal to the height of the current block, the first prediction block is used to determine the derived intra prediction mode of the current block, and then the transform set of the current block is determined.

[0243] In another possible implementation, the size information of the current block satisfying the first condition can include that the class index value of the current block is equal to 0 or 1; and the size information of the current block satisfying the second condition can include that the class index value of the current block is equal to 2.

[0244] That is, in the embodiments of the present application, different current blocks are distinguished according to the classIdx. Specifically, if the classIdx is equal to 0 or 1, the second prediction block is used to determine the derived intra prediction mode of the current block, and then the transform set of the current block is determined; if the classIdx is equal to 2, the first prediction block is used to determine the derived intra prediction mode of the current block, and then the transform set of the current block is determined.

[0245] In yet another possible implementation, the size information of the current block satisfying the first condition can include: the current block being a non-square block and the class index value of the current block not equaling 2; and the size information of the current block satisfying the second condition can include: the current block being a square block or the class index value of the current block equaling 2.

[0246] That is, in the embodiments of the present application, the current block can be distinguished as follows: if the current block is a non-square block and the class index value is not equal to 2, the second prediction block is used to determine the derived intra prediction mode of the current block, and then the transform set of the current block is determined; otherwise, the first prediction block is used to determine the derived intra prediction mode of the current block, and then the transform set of the current block is determined.

[0247] It should be noted that the first condition and the second condition are not limited to these cases, for example, other limited cases of index values can also be used, such as the first condition including the class index value of the current block equaling 0, and the second condition including the class index value of the current block equaling 1; or the first condition including the class index value of the current block equaling 1, and the second condition including the class index value of the current block equaling 0, and so on. Even according to the area size of the current block, for example, the first condition including the area of the current block being less than 256 sample sizes, and the second condition including the area of the current block being greater than 256 sample sizes, and so on, which are not limited here.

[0248] In yet another possible implementation, the preset intra prediction mode can also be directly used as the derived intra prediction mode of the current block, and then the transform set of the current block is determined according to the derived intra prediction mode.

[0249] In some embodiments, the method can further include: when the size information of the current block satisfies the first condition, determining the preset intra prediction mode as the derived intra prediction mode of the current block, and determining the transform set of the current block according to the derived intra prediction mode.

[0250] In some embodiments, the method can further include: when the size information of the current block satisfies the second condition, determining the preset intra prediction mode as the derived intra prediction mode of the current block, and determining the transform set of the current block according to the derived intra prediction mode.

[0251] In the embodiments of the present application, the preset intra prediction mode can be a PLANAR mode, a DC mode or other modes, instead of using DIMD to derive the intra prediction mode of the current block. For example, the preset intra prediction mode such as the PLANAR mode, the DC mode or other modes can be used instead of using the second prediction block to determine the derived intra prediction mode of the current block; and the preset intra prediction mode such as the PLANAR mode, the DC mode or other modes can also be used instead of using the first prediction block to determine the derived intra prediction mode of the current block.

[0252] In a possible implementation, the DIMD method can be used here to determine the derived intra prediction mode of the current block by using the first prediction block or the second prediction block.

[0253] In some embodiments, determining the derived intra prediction mode of the current block by using the first prediction block can include: determining candidate samples for deriving the intra prediction mode based on the first prediction block; determining at least one candidate intra prediction mode and gradient strength accumulation values corresponding to the at least one candidate intra prediction mode according to horizontal gradient values and vertical gradient values of the candidate samples; determining a maximum gradient strength accumulation value in the gradient strength accumulation values corresponding to the at least one candidate intra prediction mode, and determining the candidate intra prediction mode corresponding to the maximum gradient strength accumulation value as the derived intra prediction mode of the current block.

[0254] In some embodiments, determining the derived intra prediction mode of the current block by using the second prediction block can include: determining candidate samples for deriving the intra prediction mode based on the second prediction block; determining at least one candidate intra prediction mode and gradient strength accumulation values corresponding to the at least one candidate intra prediction mode according to horizontal gradient values and vertical gradient values of the candidate samples; determining a maximum gradient strength accumulation value in the gradient strength accumulation values corresponding to the at least one candidate intra prediction mode, and determining the candidate intra prediction mode corresponding to the maximum gradient strength accumulation value as the derived intra prediction mode of the current block.

[0255] It should be noted that, in the embodiments of the present application, determining the at least one candidate intra prediction mode and the gradient strength accumulation values corresponding to the at least one candidate intra prediction mode according to the horizontal gradient values and the vertical gradient values of the candidate samples can include: determining the reference intra prediction mode and the gradient strength value corresponding to each candidate sample according to the horizontal gradient values and the vertical gradient values of the candidate samples; and determining the at least one candidate intra prediction mode and the gradient strength accumulation values corresponding to the at least one candidate intra prediction mode according to the reference intra prediction mode and the gradient strength value corresponding to each candidate sample.

[0256] It should also be noted that, in the embodiments of the present application, the horizontal gradient value of the candidate sample can be grad xThe vertical gradient value of the candidate sample can be represented as grad y . In this way, according to grad x and grad y , the reference intra prediction mode can be derived. In addition, the gradient strength value corresponding to the candidate sample can be denoted as amp, amp = abs(grad x ) + abs(grad y ).

[0257] In the embodiments of the present application, the horizontal gradient value and the vertical gradient value of the candidate sample can be calculated using the Sobel operator. Exemplarily, for the Sobel operator, the following is shown:

[0258] Operator of the horizontal gradient value:

[0259] Operator of the vertical gradient value:

[0260] Thus, assuming that the sample value of the sample at position (x, y) is P x,y , the calculation of the horizontal gradient value grad x and the vertical gradient value grad y is shown as follows: grad x = P x+1,y-1 + 2*P x+1,y + P x+1,y+1 - P x-1,y-1 - 2*P x-1,y - P x-1,y+1 (4) grad y = P x-1,y+1 + 2*P x,y+1 + P x+1,y+1 - P x-1,y-1 - 2*P x,y-1 - P x+1,y-1 (5)

[0261] It should be further noted that, in the embodiments of the present application, the candidate sample used for deriving the intra prediction mode can be the prediction block of the current block, can be the reconstructed region on the left and on the top of the current block, or can be both the prediction block of the current block and the reconstructed region on the left and on the top of the current block. In this way, more samples are used for deriving the intra prediction mode, and the derived intra prediction mode is more accurate. In addition, in the embodiments of the present application, the number of candidate samples used for deriving the intra prediction mode can be at least one, for example, 1, 2, 3 or more.

[0262] In some embodiments, determining the at least one candidate intra prediction mode and the gradient strength accumulation value corresponding to the at least one candidate intra prediction mode according to the reference intra prediction mode corresponding to each of the candidate samples and the gradient strength value corresponding to each of the candidate samples can include: determining the at least one candidate intra prediction mode with different characteristics according to the reference intra prediction mode corresponding to each of the candidate samples, and accumulating the gradient strength values belonging to the same candidate texture feature index to determine the gradient strength accumulation value corresponding to the at least one candidate intra prediction mode according to the gradient strength value corresponding to each of the candidate samples.

[0263] Thus, after the gradient strength accumulation value corresponding to the at least one candidate intra prediction mode is determined, the maximum gradient strength accumulation value is determined from the gradient strength accumulation value corresponding to the at least one candidate intra prediction mode, and the candidate intra prediction mode corresponding to the maximum gradient strength accumulation value is determined as the derived intra prediction mode of the current block.

[0264] In another possible implementation, the TIMD method can be used here to determine the derived intra prediction mode of the current block by using the first prediction block or the second prediction block.

[0265] In some embodiments, determining the derived intra prediction mode of the current block by using the first prediction block can include: determining a template region corresponding to the first prediction block; predicting the template region by using the at least one candidate intra prediction mode to determine the predicted sample of the template region corresponding to the at least one candidate intra prediction mode; performing cost calculation on the predicted sample of the template region and the reconstructed sample of the template region respectively according to the at least one candidate intra prediction mode to determine the cost result corresponding to the at least one candidate intra prediction mode; determining the minimum cost result in the cost result corresponding to the at least one candidate intra prediction mode, and determining the candidate intra prediction mode corresponding to the minimum cost result as the derived intra prediction mode of the current block.

[0266] In some embodiments, determining the derived intra prediction mode of the current block by using the second prediction block can include: determining a template region corresponding to the second prediction block; predicting the template region by using the at least one candidate intra prediction mode to determine the predicted sample of the template region corresponding to the at least one candidate intra prediction mode; performing cost calculation on the predicted sample of the template region and the reconstructed sample of the template region respectively according to the at least one candidate intra prediction mode to determine the cost result corresponding to the at least one candidate intra prediction mode; determining the minimum cost result in the cost result corresponding to the at least one candidate intra prediction mode, and determining the candidate intra prediction mode corresponding to the minimum cost result as the derived intra prediction mode of the current block.

[0267] It should be noted that in the embodiments of the present application, the cost calculation here can be determined according to the cost result of rate distortion optimization (RDO), or can be determined according to the cost result of sum of absolute difference (SAD), or can be determined according to the cost result of sum of absolute transformed difference (SATD), but here is not limited in any way.

[0268] That is, the process of using DIMD to derive the intra prediction mode in the embodiments of the present application can be replaced by the method of using TIMD to derive the intra prediction mode. The TIMD method is a template matching technology, and the template region can be set as the neighboring region of the current block or the size of the current block. By calculating the SAD or SATD of the prediction sample and the reconstructed sample, the cost result is obtained. Different intra prediction modes correspond to different prediction samples, and therefore the optimal intra prediction mode can be obtained according to the principle of minimum cost, which is used as the derived intra prediction mode of the current block.

[0269] In some embodiments, determining the transform set of the current block according to the derived intra prediction mode can include: determining the transform set index of the current block according to the first lookup table and the derived intra prediction mode; and determining the transform set of the current block according to the transform set index.

[0270] In the embodiments of the present application, the first lookup table is used to record the mapping relationship between the intra prediction mode and the transform set index. For example, in combination with Table 3 described above, after the derived intra prediction mode is obtained, the corresponding transform set index can be determined by looking up the table, and then the transform set of the current block is determined. The transform set of the current block can include at least one transform kernel.

[0271] It can be understood that in the embodiments of the present application, after the transform set of the current block is determined, the transform coefficient of the current block can be further determined. Referring to FIG. 13, the method can further include:

[0272] S1301, determining the transform kernel of the current block according to the transform set.

[0273] S1302, determining the residual sample of the current block according to the prediction sample in the second prediction block.

[0274] S1303, transforming the residual sample of the current block according to the transform kernel to determine the transform coefficient of the current block.

[0275] S1304, quantizing the transform coefficient of the current block to determine the quantized coefficient of the current block.

[0276] S1305, encode the quantization coefficient of the current block, and write the obtained encoding bits into the code stream.

[0277] In the embodiments of the present application, the execution order of steps S1301 and S1302 has no precedence. For example, step S1301 can be executed first, and then step S1302 can be executed; or step S1302 can be executed first, and then step S1301 can be executed; or steps S1301 and S1302 can be executed in parallel, which is not limited here.

[0278] In the embodiments of the present application, the residual sample of the current block is determined according to the prediction sample in the second prediction block of the current block. Specifically, the residual sample of the current block can be determined by difference calculation on the original sample of the current block and the prediction sample in the second prediction block.

[0279] In some embodiments, determining the transform kernel of the current block according to the transform set can include: determining at least two candidate transform kernels included in the transform set; calculating the encoding cost of the current block according to the at least two candidate transform kernels, determining the cost results corresponding to the at least two candidate transform kernels; determining the minimum cost result in the cost results corresponding to the at least two candidate transform kernels, and determining the candidate transform kernel corresponding to the minimum cost result as the transform kernel of the current block.

[0280] In some embodiments, the method can further include: determining the transform kernel index of the current block; wherein the transform kernel index is used to indicate the number of the transform kernel of the current block in the transform set; and performing encoding processing on the transform kernel index of the current block, and writing the obtained encoding bits into the code stream.

[0281] It should be noted that in the embodiments of the present application, when the current block uses the MIP mode, in addition to the prediction parameters such as the transposition identification information and the weight matrix index information, the transform kernel index information is also included in the code stream; so that the corresponding transform kernel can be obtained from the transform set, and the transform kernel is applied to the residual sample of the current block.

[0282] It should be further noted that in the embodiments of the present application, the transform set here can include the NSPT transform set and the LFNST transform set. The transform kernel for NSPT can be determined according to the NSPT transform set; and the transform kernel for LFNST can be determined according to the LFNST transform set.

[0283] In a possible implementation, determining the transform coefficient of the current block according to the transform of the residual sample of the current block can include: performing non-separable primary transform on the residual sample of the current block according to the transform kernel, and determining the transform coefficient of the current block.

[0284] In the embodiments of the present application, if the size parameter of the current block satisfies a first condition, for example, the size parameter of the current block is less than a certain threshold, that is, the size parameter of the current block is small, the transform kernel of NSPT can be used at this time. That is, for a block with a small size, the transform kernel of NSPT is used here, that is, the residual samples of the current block are subjected to NSPT transformation according to the transform kernel to determine the transform coefficients of the current block.

[0285] In another possible implementation, determining the transform coefficients of the current block according to the transform of the residual samples of the current block can include: performing discrete cosine transformation on the residual samples of the current block to determine transform samples of the current block; and performing low-frequency non-separable transformation on the transform samples of the current block according to the transform kernel to determine the transform coefficients of the current block.

[0286] In the embodiments of the present application, if the size parameter of the current block satisfies a second condition, for example, the size parameter of the current block is greater than a certain threshold, that is, the size parameter of the current block is large, the transform kernel of LFNST can be used at this time. That is, for a block with a large size, the transform kernel of LFNST is used here, that is, the residual samples of the current block are subjected to DCT2 transformation to determine the transform samples of the current block; and the transform samples of the current block are subjected to LFNST transformation according to the transform kernel to determine the transform coefficients of the current block.

[0287] It can be understood that in the embodiments of the present application, after the transform coefficients of the current block are determined, the transform coefficients of the current block are subjected to encoding processing, and the obtained encoding bits are written into the bitstream, which can include: quantizing the transform coefficients of the current block to determine the quantized coefficients of the current block; and encoding the quantized coefficients of the current block to write the obtained encoding bits into the bitstream.

[0288] In some embodiments, the embodiments of the present application also provide a bitstream, wherein the bitstream is generated by bit encoding according to to-be-encoded information; and the to-be-encoded information includes at least one of the following: the quantized coefficients of the current block, the transform kernel index of the current block, the prediction parameter of the current block, the value of the first syntax element, the value of the second syntax element, and the value of the third syntax element.

[0289] In the embodiments of the present application, the prediction parameter includes transposition identification information and weight matrix index information, the second syntax element is used to indicate whether the current sequence allows using the matrix-based intra prediction mode, the third syntax element is used to indicate the prediction type of the current block, the first syntax element is used to indicate whether the current block uses the matrix-based intra prediction mode, and the current sequence includes the current block.

[0290] It can also be understood that, in the embodiments of the present application, when the current block does not use the MIP mode, other prediction modes used by the current block and the like can also be written into the code stream. In this way, the subsequent decoding end can determine the other prediction modes used by the current block by decoding the code stream, and then determine the prediction samples of the current block by using the other prediction modes; and finally, the reconstructed samples of the current block can be determined by performing addition operation on the residual samples obtained by decoding and the prediction samples of the current block.

[0291] It can also be understood that, in the embodiments of the present application, after all the coding blocks in the current image are completely traversed, the code stream is transmitted to the decoding end after loop filtering, entropy coding and the like.

[0292] The embodiments of the present application provide an encoding method. When the current block uses the matrix-based intra prediction mode, a first prediction block of the current block is determined, and upsampling processing is performed on the first prediction block to determine a second prediction block of the current block; the derived intra prediction mode of the current block is determined by using the first prediction block or the second prediction block based on the size information of the current block, and the transform set of the current block is determined according to the derived intra prediction mode. In this way, when the current block uses the MIP mode, the first prediction block and the second prediction block of the current block are determined; then, according to the size information of the current block, the first prediction block or the second prediction block can be selected to determine the derived intra prediction mode of the current block, and then the transform set suitable for the current block can be determined, and the residual samples of the current block are transformed according to the transform kernel in the transform set, so that the accuracy of the transform prediction can be improved, the compression efficiency is improved, and the coding performance is improved.

[0293] In another embodiment of the present application, based on the coding method of the foregoing embodiment, the main idea of the technical solution is: for a non-square coding unit, if the MIP mode is used, DIMD is applied to the second prediction block after upsampling, but not applied to the first prediction block before upsampling, that is, the output vector of the MIP. For a square coding unit with the same width and height upsampling coefficients, if the MIP mode is used, DIMD is still applied to the output vector of the MIP, that is, the first prediction block before upsampling.

[0294] In addition, for coding units of different classIdx, such a distinction can also be made. For example, for a coding unit with classIdx being 0 or 1, DIMD is applied to the second prediction block after upsampling to derive the intra prediction mode for selecting the LFNST transform set; otherwise, DIMD is applied to the first prediction block before upsampling to derive the intra prediction mode for selecting the LFNST transform set.

[0295] In a specific embodiment, the current block takes the current coding unit as an example, and the specific steps at the encoding end are as follows:

[0296] The encoder traverses the prediction mode, if the current prediction mode type is intra prediction, the allowed use flag of the present technical solution is obtained, the flag is a sequence level flag, indicating that the current encoder allows using the matrix-based intra prediction technology, which can be in the form of sps mip enable flag.

[0297] Step 1, if the allowed use flag of MIP is true, step 2 is executed; if the allowed use flag of MIP or other conditions do not meet, the encoding end does not attempt the prediction method of MIP. That is, assuming that costMip is a very large value, for example, Max value for what double type allows, step 2 is skipped and step 3 is directly executed;

[0298] Step 2, according to the size of the current coding unit, classIdx (also called sizeId) is obtained, the input vector size of the current coding unit MIP is determined according to classIdx, and the adjacent boundary samples obtained are down-sampled to obtain the down-sampled input vector bdryred. The input vectors on both sides are spliced according to the MIP transpose identification information, and the input samples are processed.

[0299] According to the MIP mode index, the weight matrix is obtained, and the input vector bdry red The input is operated with the obtained weight matrix to obtain the output vector.

[0300] According to classIdx, the output vector size of the current coding unit MIP is determined, and each element in the output vector is placed in the corresponding position according to the arrangement order described in the foregoing. If the transpose identification information indicates that transposition is needed, each element in the output vector is placed column by column from top to bottom in order; otherwise, the transpose identification information indicates that transposition is not needed, and each element in the output vector is placed row by row from left to right in order. The interpolation order follows the horizontal linear interpolation first, and then the vertical linear interpolation. The method is similar to the horizontal interpolation. The missing sample points are linearly interpolated by using the left and right adjacent samples to obtain new sample points. After interpolation and up-sampling, a prediction block with the same size as the current coding unit is obtained. Here, the interpolation order is not limited.

[0301] If the current coding unit is a non-square block, DIMD is applied to the second prediction block after interpolation upsampling to derive the intra prediction mode; if the current coding unit is a square coding unit, DIMD is applied to the first prediction block before upsampling to derive the intra prediction mode. According to the derived intra prediction mode, the index of the LFNST transform set is obtained by searching the aforementioned table 3.

[0302] The residual sample is obtained by subtracting the predicted sample from the original sample of the current coding unit, and the frequency domain coefficient block is obtained after the primary transform. The secondary transform technique is applied to the frequency domain coefficient block. According to the LFNST transform set index obtained above, multiple transform kernels are traversed, and the obtained transform coefficients are quantized, dequantized and inverse transformed to obtain the reconstructed block of the current coding unit. The rate-distortion cost is calculated with the original block, and the minimum cost costMip is recorded. The MIP mode index, transposition identification information, LFNST transform kernel index and other information are saved.

[0303] Step 3: The encoding end continues to traverse other prediction modes and calculates the rate-distortion cost of each prediction mode. The prediction mode corresponding to the minimum rate-distortion cost is selected as the optimal prediction mode of the current coding unit.

[0304] If costMip is the minimum, the current coding unit uses MIP technology, and the MIP use identification position true is needed to indicate that the current coding unit uses MIP technology. The transposition identification information and the LFNST transform kernel index information are written into the code stream.

[0305] If the current coding unit costMip is not the minimum and the current sequence allows the use of MIP technology, the current coding unit does not use MIP technology, and the MIP coding unit level use identification position false is needed to indicate that the current coding unit does not use MIP technology, and is written into the code stream.

[0306] Step 4: After traversing all coding units, the code stream is output after loop filtering, entropy coding and other techniques.

[0307] The specific steps at the decoding end are as follows:

[0308] The decoding end parses or obtains the MIP allowed use flag, which is a sequence level flag (sps_mip_enable_flag) indicating that the current decoder allows the use of MIP technology.

[0309] Step 1, parse the bitstream to obtain the prediction type of the current coding unit, if it is intra prediction, parse the MIP using identification bit of the current coding unit. If the MIP using identification bit of the current coding unit is true, it means that the current coding unit uses MIP technology, then continue to parse the MIP transpose identification information, MIP mode index and LFNST transform core index of the current coding unit.

[0310] If the MIP using identification bit of the current coding unit is false, it means that the current coding unit does not use MIP technology, then execute step 3.

[0311] Step 2, obtain classIdx (also called sizeId) according to the size of the current coding unit, determine the input vector size of the MIP of the current coding unit according to the classIdx, and downsample the obtained adjacent boundary samples to obtain the downsampled input vector bdry red According to the MIP transpose identification information, splice the input vectors on both sides, and process the input samples.

[0312] According to the MIP mode index parsed, obtain the weight matrix, and take the input vector bdry red as input and the obtained weight matrix to obtain the output vector.

[0313] Determine the output vector size of the MIP of the current coding unit according to the classIdx, and place each element in the output vector in the corresponding position according to the arrangement order described in the foregoing according to the MIP transpose identification information. Among them, the interpolation order follows the horizontal linear interpolation first, and the missing sample points are linearly interpolated by means of the left and right adjacent samples to obtain new sample points, and then the vertical linear interpolation is performed, and the method is similar to the horizontal interpolation. Linearly interpolate the missing sample points by means of the upper and lower adjacent samples to obtain new sample points. After interpolation upsampling, a prediction block with the same size as the current coding unit is obtained. Here, the interpolation order is not limited.

[0314] If the current coding unit is a non-square block, DIMD is applied to the second prediction block after interpolation upsampling to obtain the derived intra prediction mode; if the current coding unit is a square coding unit, DIMD is applied to the first prediction block before upsampling to obtain the derived intra prediction mode. According to the derived intra prediction mode, the LFNST transform set index of the current coding unit under the MIP mode prediction is obtained by looking up the aforementioned table 3.

[0315] According to the LFNST transform core index obtained by analysis, a LFNST transform core is obtained from the LFNST transform set obtained by DIMD derivation, the transform core is applied to the transform coefficient after analysis and dequantization, inverse transform is performed to obtain a frequency domain coefficient block, and inverse transform is performed on the frequency domain coefficient block according to the main transform information obtained by analysis to obtain a residual block of the current coding unit.

[0316] The prediction block and the residual block are added to obtain a reconstructed sample block of the current coding unit.

[0317] Steps 3 and 4 are skipped.

[0318] Step 3, continue to analyze the usage identification bit or index of other prediction modes, and obtain a final prediction block of the current coding unit according to the other prediction modes;

[0319] Step 4, analyze the code stream and obtain residual information, obtain time domain residual information according to dequantization and inverse transform, and superimpose the final prediction block and the time domain residual information to obtain a reconstructed sample block;

[0320] Step 5, after all the reconstructed sample blocks pass through loop filtering and other technologies, a final reconstructed image is obtained, which can be used as video output and can also be used as a decoding reference.

[0321] In another embodiment of the present application, the current block is still taken as an example of the current coding unit, and the specific steps at the encoding end are as follows:

[0322] The encoder traverses the prediction mode, and if the current prediction mode type is intra prediction, an allowed usage flag of the present technical solution is obtained, the flag is a sequence level flag, and indicates that the current encoder allows to use the matrix-based intra prediction technology, which can be in the form of sps mip enable flag.

[0323] Step 1, if the allowed usage flag of MIP is true, step 2 is executed; if the allowed usage flag of MIP or other conditions does not meet the requirements, the encoding end does not attempt the prediction method of MIP. That is, assuming that costMip is a very large value, for example, Max value for what double type allows, step 2 is skipped and step 3 is directly executed;

[0324] Step 2, according to the size of the current coding unit, a classIdx (also called sizeId) is obtained, the input vector size of the MIP of the current coding unit is determined according to the classIdx, the adjacent boundary samples obtained are down-sampled to obtain a down-sampled input vector bdryred, the input vectors on both sides are spliced according to the MIP transpose identification information, and the input samples are processed.

[0325] According to the MIP mode index, a weight matrix is obtained, and the input vector bdry red The input is operated with the obtained weight matrix to obtain an output vector.

[0326] The output vector size of the current coding unit MIP is determined according to the classIdx, and each element in the output vector is placed in the corresponding position in the order described above. If the transpose identification information indicates that transposition is needed, each element in the output vector is placed in order from top to bottom and column by column; otherwise, the transpose identification information indicates that transposition is not needed, and each element in the output vector is placed in order from left to right and row by row. The interpolation order follows the horizontal linear interpolation first, and the missing sample points are linearly interpolated to obtain new sample points by using the left and right adjacent sample points, and then the vertical linear interpolation is performed, and the method is similar to the horizontal interpolation. The missing sample points are linearly interpolated to obtain new sample points by using the upper and lower adjacent sample points. After interpolation upsampling, a prediction block of the same size as the current coding unit is obtained. Here, the interpolation order is not limited.

[0327] If the current coding unit is a non-square block and classIdx is not 2, DIMD is applied to the second prediction block after interpolation upsampling to obtain the derived intra prediction mode; otherwise, DIMD is applied to the first prediction block before upsampling to obtain the derived intra prediction mode. According to the derived intra prediction mode, the LFNST transform set index of the current coding unit under the MIP mode prediction is obtained by searching the aforementioned table 3.

[0328] The residual samples are obtained by subtracting the predicted samples from the original samples of the current coding unit, and the frequency domain coefficient block is obtained after the primary transform. The secondary transform technique is applied to the frequency domain coefficient block. According to the LFNST transform set index obtained above, multiple transform kernels are traversed from the LFNST transform set index, and the obtained transform coefficients are quantized, dequantized and inverse transformed to obtain the reconstructed block of the current coding unit. The rate-distortion cost is calculated with the original block, and the minimum cost costMip is recorded. The MIP mode index, the transpose identification information, the LFNST transform kernel index and other information are saved.

[0329] Step 3, the encoding end continues to traverse other prediction modes, and the rate-distortion cost corresponding to each prediction mode is calculated. The prediction mode corresponding to the smallest cost value is selected as the optimal prediction mode of the current coding unit.

[0330] If costMip is the minimum, the current coding unit uses the MIP technology, and the MIP use identification position true is needed to indicate that the current coding unit uses the MIP technology. The transpose identification information and the LFNST transform kernel index information are written into the code stream.

[0331] If the current coding unit costMip is not the minimum and the current sequence allows the use of MIP technology, the current coding unit does not use MIP technology, the coding unit level use of MIP is identified as no, indicating that the current coding unit does not use MIP technology, and the flag bit is written into the code stream.

[0332] Step 4: After traversing all coding units, the code stream is output after loop filtering, entropy coding and other technologies.

[0333] The specific steps at the decoding end are as follows:

[0334] The decoding end parses or acquires the MIP use permission flag bit, which is a sequence level flag bit (sps_mip_enable_flag), indicating that the current decoder allows the use of MIP technology.

[0335] Step 1: Parse the code stream to obtain the prediction type of the current coding unit, and if it is intra prediction, parse the MIP use identification bit of the current coding unit. If the MIP use identification bit of the current coding unit is true, indicating that the current coding unit uses MIP technology, continue to parse the MIP transpose identification information, MIP mode index and LFNST transform core index of the current coding unit.

[0336] If the MIP use identification bit of the current coding unit is no, indicating that the current coding unit does not use MIP technology, step 3 is executed.

[0337] Step 2: According to the size of the current coding unit, obtain classIdx (also called sizeId), determine the input vector size of the current coding unit MIP according to classIdx, and downsample the obtained adjacent boundary samples to obtain the downsampled input vector bdry red According to the MIP transpose identification information, splice the input vectors on both sides, and process the input samples.

[0338] According to the parsed MIP mode index, obtain the weight matrix, and take the input vector bdry red as input and the obtained weight matrix to obtain the output vector.

[0339] According to classIdx, determine the output vector size of the current coding unit MIP, and according to the MIP transpose identification information, place each element in the output vector in the corresponding position according to the arrangement order described in the foregoing. The interpolation order follows the horizontal linear interpolation first, and the missing sample points are linearly interpolated by means of the left and right adjacent samples to obtain new sample points, and then the vertical linear interpolation is performed, and the method is similar to the horizontal interpolation. The missing sample points are linearly interpolated by means of the upper and lower adjacent samples to obtain new sample points. After interpolation and upsampling, a prediction block with the same size as the current coding unit is obtained. Here, the interpolation order is not limited.

[0340] If the current coding unit is a non-square block and classIdx is not equal to 2, DIMD is applied to the second prediction block after interpolation upsampling to derive the intra prediction mode; otherwise, DIMD is applied to the first prediction block before upsampling to derive the intra prediction mode. According to the derived intra prediction mode, the index of the LFNST transform set of the current coding unit under the MIP mode prediction is obtained by looking up Table 3.

[0341] According to the index of the LFNST transform core obtained from the LFNST transform set derived by DIMD from Table 3, the LFNST transform core is applied to the transform coefficients after parsing and dequantization to perform inverse transformation to obtain the frequency domain coefficient block, and the frequency domain coefficient block is dequantized according to the main transform information obtained from the parsing to obtain the residual block of the current coding unit.

[0342] The prediction block and the residual block are added to obtain the reconstructed sample block of the current coding unit.

[0343] Steps 3 and 4 are skipped.

[0344] Step 3, continue to parse the usage identification bit or index information of other prediction modes, and obtain the final prediction block of the current coding unit according to the parsed other prediction modes;

[0345] Step 4, parse the code stream and obtain the residual information, and obtain the time domain residual information according to the dequantization and inverse transformation. The final prediction block and the time domain residual information are superimposed to obtain the reconstructed sample block;

[0346] Step 5, after all the reconstructed sample blocks are filtered by loop filtering and other technologies, the final reconstructed image is obtained, which can be used as video output and can also be used as decoding reference.

[0347] That is, in the embodiments of the present application, the derivation method of the LFNST transform set index under the MIP mode is changed, and the first prediction block before upsampling and the second prediction block after upsampling are used to derive the intra prediction mode by DIMD to select the LFNST transform set index for square and non-square blocks, respectively. In addition, the derivation method of the LFNST transform set index under the MIP mode is changed, and the first prediction block before upsampling and the second prediction block after upsampling are used to derive the intra prediction mode by DIMD to select the LFNST transform set index for different classIdx.

[0348] Exemplarily, the method in the technical solution of the embodiments of the present application is integrated into the reference software ECM, and simulation experiments are carried out under the test conditions, and the results are shown in Table 4.

[0349] Table 4

[0350] According to Table 4, the numbers in the table are negative, indicating gain. It can be seen that the technical solution has certain performance. It should be noted that the encoding and decoding times (EncT and DecT) in Table 4 can be inaccurate because the anchor and test are simulated at different clusters and different times. As can be seen from Table 4, the performance of the 4K sequence A1 is better, which is the main development direction of the next generation of video coding.

[0351] In the embodiment of the present application, the intra prediction mode is derived using the pre-upsampled or post-upsampled prediction block for MIP prediction blocks of different coding sizes; in addition, for MIP prediction blocks that require post-upsampled prediction blocks to derive the intra prediction mode, PLANAR or other modes can be used instead of DIMD mode to derive the intra prediction mode; for MIP prediction blocks that require pre-upsampled prediction blocks to derive the intra prediction mode, PLANAR or other modes can be used instead of DIMD to derive the intra prediction mode.

[0352] In addition, in the embodiment of the present application, all the above processes using DIMD to derive the intra prediction mode can be derived using TIMD instead of the DIMD method. The TIMD method is a template matching technique, and the template region can be set to the neighboring region of the current coding unit or the size of the current coding unit. The cost is calculated by calculating the SAD or SATD of the prediction sample and the reconstructed sample. The prediction samples of different prediction modes are different, so the optimal intra prediction mode can be obtained according to the minimum cost principle.

[0353] In the embodiment of the present application, the specific implementation of the foregoing embodiment is described in detail. According to the technical solution of the foregoing embodiment, it can be seen that when the current block uses the MIP mode, the first prediction block and the second prediction block of the current block are determined; then according to the size information of the current block, the first prediction block or the second prediction block is selected to determine the derived intra prediction mode of the current block, and then the transform set suitable for the current block is determined, and the transform coefficients of the current block are transformed according to the transform kernel, so that the accuracy of the transform prediction is improved, the compression efficiency is improved, and the coding performance is improved.

[0354] In still another embodiment of the present application, based on the same inventive concept as the foregoing embodiments, FIG. 14 is a schematic diagram of the composition structure of an encoder provided in the embodiment of the present application. As shown in FIG. 14, the encoder 140 can include a first prediction unit 1401 and a first determination unit 1402, wherein:

[0355] The first prediction unit is configured to determine a first prediction block of the current block when the current block uses the matrix-based intra prediction mode, and to perform up-sampling processing on the first prediction block to determine a second prediction block of the current block.

[0356] The first determination unit is configured to determine a derived intra prediction mode of the current block based on the size information of the current block and using the first prediction block or the second prediction block, and to determine a transform set of the current block according to the derived intra prediction mode.

[0357] In some embodiments, referring to FIG. 14, the encoder 140 can further include a coding unit 1403; and the first determination unit 1402 is further configured to determine a transform kernel of the current block according to the transform set, to determine residual samples of the current block, and to perform transform on the residual samples of the current block according to the transform kernel to determine transform coefficients of the current block; and the coding unit 1403 is configured to perform coding processing on the transform coefficients of the current block and to write the obtained coding bits into the bitstream.

[0358] In some embodiments, the first determination unit 1402 is further configured to determine at least two candidate transform kernels included in the transform set, to perform coding cost calculation on the current block according to the at least two candidate transform kernels, to determine cost results corresponding to the at least two candidate transform kernels, and to determine a minimum cost result in the cost results corresponding to the at least two candidate transform kernels and determine a candidate transform kernel corresponding to the minimum cost result as the transform kernel of the current block.

[0359] In some embodiments, the first determination unit 1402 is further configured to determine a transform kernel index of the current block, where the transform kernel index is used to indicate a number of the transform kernel of the current block in the transform set; and the coding unit 1403 is further configured to perform coding processing on the transform kernel index of the current block and to write the obtained coding bits into the bitstream.

[0360] In some embodiments, the first determination unit 1402 is further configured to perform discrete cosine transform on the residual samples of the current block to determine transform samples of the current block, and to perform low-frequency non-separable transform on the transform samples of the current block according to the transform kernel to determine the transform coefficients of the current block.

[0361] In some embodiments, the first determination unit 1402 is further configured to perform non-separable primary transform on the residual samples of the current block according to the transform kernel to determine the transform coefficients of the current block.

[0362] In some embodiments, the first determination unit 1402 is further configured to perform quantization on the transform coefficients of the current block to determine quantized coefficients of the current block; and the coding unit 1403 is further configured to perform coding processing on the quantized coefficients of the current block and to write the obtained coding bits into the bitstream.

[0363] In some embodiments, the first determining unit 1402 is further configured to determine the derived intra prediction mode using the second prediction block when the size information of the current block satisfies a first condition, and determine the derived intra prediction mode using the first prediction block when the size information of the current block satisfies a second condition.

[0364] In some embodiments, the size information of the current block satisfying the first condition comprises that the current block is a non-square block, and the size information of the current block satisfying the second condition comprises that the current block is a square block.

[0365] In some embodiments, the size information of the current block satisfying the first condition comprises that the category index value of the current block is equal to 0 or 1, and the size information of the current block satisfying the second condition comprises that the category index value of the current block is equal to 2.

[0366] In some embodiments, the size information of the current block satisfying the first condition comprises that the current block is a non-square block and the category index value of the current block is not equal to 2, and the size information of the current block satisfying the second condition comprises that the current block is a square block or the category index value of the current block is equal to 2.

[0367] In some embodiments, the first determining unit 1402 is further configured to determine a preset intra prediction mode as the derived intra prediction mode of the current block when the size information of the current block satisfies the first condition, and determine a transform set of the current block according to the derived intra prediction mode.

[0368] In some embodiments, the first determining unit 1402 is further configured to determine a preset intra prediction mode as the derived intra prediction mode of the current block when the size information of the current block satisfies the second condition, and determine a transform set of the current block according to the derived intra prediction mode.

[0369] In some embodiments, the first determining unit 1402 is further configured to determine a transform set index of the current block according to the first lookup table and the derived intra prediction mode, and determine a transform set of the current block according to the transform set index, wherein the first lookup table is used to record a mapping relationship between the intra prediction mode and the transform set index.

[0370] In some embodiments, the first determining unit 1402 is further configured to determine a value of a second syntax element, wherein the second syntax element is used to indicate whether the matrix-based intra prediction mode is allowed to be used in a current sequence, and the encoding unit 1403 is further configured to perform encoding processing on the value of the second syntax element and write the obtained coded bits into a bitstream.

[0371] In some embodiments, the first determining unit 1402 is further configured to determine a value of a third syntax element when the current sequence allows the use of the matrix-based intra prediction mode, wherein the third syntax element is used to indicate a prediction type of the current block; and the encoding unit 1403 is further configured to encode the value of the third syntax element and write the obtained encoded bits into the bitstream.

[0372] In some embodiments, the first determining unit 1402 is further configured to determine a value of a first syntax element when the prediction type of the current block is intra prediction, wherein the first syntax element is used to indicate whether the current block uses the matrix-based intra prediction mode; and the encoding unit 1403 is further configured to encode the value of the first syntax element and write the obtained encoded bits into the bitstream.

[0373] In some embodiments, the first determining unit 1402 is further configured to determine a prediction parameter of the current block when the current block uses the matrix-based intra prediction mode, wherein the prediction parameter includes transposition identification information and weight matrix index information; and the encoding unit 1403 is further configured to encode the prediction parameter of the current block and write the obtained encoded bits into the bitstream.

[0374] It can be understood that, in the embodiments of the present application, the "unit" can be part of a circuit, part of a processor, part of a program or software, etc., and of course can also be a module, and can also be non-modular. Moreover, the various components in the embodiments can be integrated in one processing unit, or can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software function module.

[0375] In still another embodiment of the present application, FIG. 15 is a specific hardware structure diagram of an encoder provided by the embodiments of the present application. As shown in FIG. 15, the encoder 140 can include a first communication interface 1501, a first memory 1502 and a first processor 1503; the various components are coupled together through a first bus system 1504. It can be understood that the first bus system 1504 is used to realize the connection communication between the components. The first bus system 1504 includes not only a data bus, but also a power bus, a control bus and a status signal bus. However, for the purpose of clear illustration, all kinds of buses are marked as the first bus system 1504 in FIG. 15. Among them,

[0376] The first communication interface 1501 is used for receiving and sending signals in the process of transceiving information with other external network elements;

[0377] The first memory 1502 is used to store computer programs capable of running on the first processor 1503;

[0378] The first processor 1503 is configured to, when the computer program is running:

[0379] When the current block uses a matrix-based intra prediction mode, a first prediction block of the current block is determined, and the first prediction block is up-sampled to determine a second prediction block of the current block; based on size information of the current block, a derived intra prediction mode of the current block is determined using the first prediction block or the second prediction block, and a transform set of the current block is determined according to the derived intra prediction mode.

[0380] It can be understood that the first memory 1502 in the embodiments of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. The non-volatile memory can be a Read-Only Memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically EPROM (EEPROM), or a flash memory. The volatile memory can be a Random Access Memory (RAM) used as an external cache. By way of example, and not limitation, many forms of RAM can be used, such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous Dynamic RAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced SDRAM (ESDRAM), Synchlink DRAM (SLDRAM), and Direct Rambus RAM (DRRAM). The first memory 1502 of the system and method described herein is intended to include, without being limited to, these and any other suitable types of memory.

[0381] The first processor 1503 can be an integrated circuit chip that has the processing capability of signals. In the implementation process, each step of the above method can be completed by the integrated logic circuit of hardware or the instruction in the form of software in the first processor 1503. The first processor 1503 described above can be a general processor, a digital signal processor (Digital Signal Processor, DSP), an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component. Each method, step and logic block disclosed in the embodiment of the present application can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in combination with the embodiment of the present application can be directly embodied as a hardware code processor to execute, or be executed by a combination of hardware and software modules in the code processor. The software module can be located in a random memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register or other mature storage medium in the art. The storage medium is located in the first storage 1502, and the first processor 1503 reads the information in the first storage 1502 and combines the hardware to complete the steps of the above method.

[0382] It can be understood that the embodiments described in the present application can be realized by hardware, software, firmware, middleware, microcode or a combination thereof. For hardware implementation, the processing unit can be realized in one or more application specific integrated circuits (Application Specific Integrated Circuits, ASIC), digital signal processors (Digital Signal Processing, DSP), digital signal processing devices (DSP Device, DSPD), programmable logic devices (Programmable Logic Device, PLD), field programmable gate arrays (Field-Programmable Gate Array, FPGA), general processors, controllers, microcontrollers, microprocessors, other electronic units for executing functions described in the present application or a combination thereof. For software implementation, the technology described in the present application can be realized by modules (such as processes, functions, etc.) for executing functions described in the present application. The software code can be stored in the memory and executed by the processor. The memory can be implemented in the processor or outside the processor.

[0383] Optionally, as another embodiment, the first processor 1503 is further configured to execute the method of any one of the preceding embodiments when running the computer program.

[0384] The embodiment provides an encoder. When a MIP mode is used for a current block, first prediction blocks and second prediction blocks of the current block are determined; then, according to size information of the current block, the first prediction blocks or the second prediction blocks are selected to determine a derived intra prediction mode of the current block, and then a transform set suitable for the current block is determined, and a suitable transform kernel is selected from the transform set to perform transform, so that the accuracy of transform prediction is improved, the compression efficiency is improved, and the coding performance is improved.

[0385] In another embodiment of the present application, based on the same inventive concept as the preceding embodiments, FIG. 16 is a schematic diagram of the composition structure of a decoder provided by the embodiment of the present application. As shown in FIG. 16, the decoder 160 can include a decoding unit 1601, a second prediction unit 1602, and a second determination unit 1603, wherein:

[0386] The decoding unit 1601 is configured to decode a code stream and determine a value of a first syntax element.

[0387] The second prediction unit 1602 is configured to, when the first syntax element indicates that a matrix-based intra prediction mode is used for a current block, determine first prediction blocks of the current block, and perform up-sampling processing on the first prediction blocks to determine second prediction blocks of the current block.

[0388] The second determination unit 1603 is configured to determine a derived intra prediction mode of the current block by using the first prediction blocks or the second prediction blocks based on size information of the current block, and determine a transform set of the current block according to the derived intra prediction mode.

[0389] In some embodiments, the second determination unit 1603 is further configured to determine a transform kernel of the current block according to the transform set, the decoding unit 1601 is further configured to determine a transform coefficient of the current block, and the second determination unit 1603 is further configured to perform transform on the transform coefficient of the current block according to the transform kernel to determine residual samples of the current block.

[0390] In some embodiments, the decoding unit 1601 is further configured to decode the code stream and determine a transform kernel index of the current block, and the second determination unit 1603 is further configured to determine the transform kernel of the current block according to the transform set and the transform kernel index.

[0391] In some embodiments, the decoding unit 1601 is further configured to decode the code stream and determine a quantization coefficient of the current block, and the second determination unit 1603 is further configured to perform dequantization on the quantization coefficient of the current block to determine a transform coefficient of the current block.

[0392] In some embodiments, the second determining unit 1603 is further configured to determine transform samples of the current block according to a non-separable primary transform on transform coefficients of the current block; and determine the residual samples of the current block according to a discrete cosine transform on the transform samples of the current block.

[0393] In some embodiments, the second determining unit 1603 is further configured to determine residual samples of the current block according to a non-separable primary transform on transform coefficients of the current block.

[0394] In some embodiments, the second determining unit 1603 is further configured to determine the intra prediction mode by using the second prediction block when the size information of the current block satisfies a first condition; and determine the intra prediction mode by using the first prediction block when the size information of the current block satisfies a second condition.

[0395] In some embodiments, the size information of the current block satisfying the first condition comprises that the current block is a non-square block; and the size information of the current block satisfying the second condition comprises that the current block is a square block.

[0396] In some embodiments, the size information of the current block satisfying the first condition comprises that the category index value of the current block is equal to 0 or 1; and the size information of the current block satisfying the second condition comprises that the category index value of the current block is equal to 2.

[0397] In some embodiments, the size information of the current block satisfying the first condition comprises that the current block is a non-square block and the category index value of the current block is not equal to 2; and the size information of the current block satisfying the second condition comprises that the current block is a square block or the category index value of the current block is equal to 2.

[0398] In some embodiments, the second determining unit 1603 is further configured to determine a preset intra prediction mode as the derived intra prediction mode of the current block when the size information of the current block satisfies the first condition, and determine the transform set of the current block according to the derived intra prediction mode.

[0399] In some embodiments, the second determining unit 1603 is further configured to determine a preset intra prediction mode as the derived intra prediction mode of the current block when the size information of the current block satisfies the second condition, and determine the transform set of the current block according to the derived intra prediction mode.

[0400] In some embodiments, the second determining unit 1603 is further configured to determine a candidate sample for deriving the intra prediction mode based on the first prediction block or the second prediction block; determine at least one candidate intra prediction mode and a gradient strength accumulation value corresponding to the at least one candidate intra prediction mode according to a horizontal gradient value and a vertical gradient value of the candidate sample; and determine a maximum gradient strength accumulation value in the gradient strength accumulation values corresponding to the at least one candidate intra prediction mode, and determine a candidate intra prediction mode corresponding to the maximum gradient strength accumulation value as the derived intra prediction mode of the current block.

[0401] In some embodiments, the second determining unit 1603 is further configured to determine a template region corresponding to the first prediction block or the second prediction block; the second prediction unit 1602 is further configured to predict the template region by using the at least one candidate intra prediction mode to determine prediction samples of the template region corresponding to the at least one candidate intra prediction mode; the second determining unit 1603 is further configured to perform cost calculation on the prediction samples of the template region and the reconstructed samples of the template region corresponding to the at least one candidate intra prediction mode respectively to determine cost results corresponding to the at least one candidate intra prediction mode; and determine a minimum cost result in the cost results corresponding to the at least one candidate intra prediction mode, and determine a candidate intra prediction mode corresponding to the minimum cost result as the derived intra prediction mode of the current block.

[0402] In some embodiments, the second determining unit 1603 is further configured to determine a transform set index of the current block according to the first lookup table and the derived intra prediction mode; and determine a transform set of the current block according to the transform set index; wherein the first lookup table is used to record a mapping relationship between the intra prediction mode and the transform set index.

[0403] In some embodiments, the decoding unit 1601 is further configured to decode the bitstream to determine a value of the second syntax element; when the second syntax element indicates that the matrix-based intra prediction mode is allowed to be used in a current sequence, decode the bitstream to determine a value of the third syntax element; and when the third syntax element indicates that the prediction type of the current block is intra prediction, perform the step of decoding the bitstream to determine the value of the first syntax element.

[0404] In some embodiments, the decoding unit 1601 is further configured to, when the first syntax element indicates that the current block uses the matrix-based intra prediction mode, decode the code stream, determine the prediction parameter of the current block, the prediction parameter including the transposition identification information and the weight matrix index information; the second determining unit 1603 is further configured to perform down-sampling processing on the neighboring reference samples of the current block, determine an initial input vector of the current block, determine a target input vector of the current block according to the transposition identification information and the initial input vector, determine a weight matrix of the current block according to the weight matrix index information, and perform multiplication operation on the target input vector and the weight matrix to determine an output vector of the current block, and arrange and place the output vector of the current block according to the transposition identification information to obtain a first prediction block of the current block.

[0405] It can be understood that, in this embodiment, the "unit" can be a part of circuit, a part of processor, a part of program or software, etc., and of course can be a module, and can also be non-modular. Moreover, the components in this embodiment can be integrated in one processing unit, or can be physically present individually, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software function module.

[0406] In still another embodiment of the present application, Fig. 17 is a specific hardware structure schematic diagram of a decoder provided by the embodiment of the present application. As shown in Fig. 17, the decoder 160 can include a second communication interface 1701, a second memory 1702 and a second processor 1703; each component is coupled together through a second bus system 1704. It can be understood that the second bus system 1704 is used to realize the connection communication between the components. The second bus system 1704 includes not only a data bus, but also a power supply bus, a control bus and a state signal bus. However, for the purpose of clear illustration, all kinds of buses are marked as the second bus system 1704 in Fig. 17. Among them,

[0407] The second communication interface 1701 is used for receiving and sending signals in the process of transceiving information with other external network elements;

[0408] The second memory 1702 is used for storing computer programs capable of running on the second processor 1703;

[0409] The second processor 1703 is used for, when running the computer programs, performing:

[0410] The code stream is decoded to determine a value of the first syntax element; when the first syntax element indicates that the current block uses the matrix-based intra prediction mode, a first prediction block of the current block is determined, and the first prediction block is up-sampled to determine a second prediction block of the current block; the derived intra prediction mode of the current block is determined based on size information of the current block and the first prediction block or the second prediction block, and a transform set of the current block is determined according to the derived intra prediction mode.

[0411] Optionally, as another embodiment, the second processor 1703 is further configured to, when the computer program is executed, perform the method in any one of the preceding embodiments.

[0412] It can be understood that the second memory 1702 has similar hardware functions as the first memory 1502, and the second processor 1703 has similar hardware functions as the first processor 1503; and details are not described herein.

[0413] The embodiment provides a decoder, when the current block uses the MIP mode, first, a first prediction block and a second prediction block of the current block are determined; then, according to size information of the current block, the derived intra prediction mode of the current block can be determined by selecting the first prediction block or the second prediction block, and then a transform set suitable for the current block can be determined, and a suitable transform kernel is selected for transformation, so that the accuracy of the transform prediction can be improved, the compression efficiency is improved, and the coding and decoding performance is improved.

[0414] In still another embodiment of the present application, FIG. 18 is a schematic diagram of a composition structure of a coding system provided by the embodiment of the present application. As shown in FIG. 18, the coding system 180 can include an encoder 1801 and a decoder 1802.

[0415] In the embodiment of the present application, the encoder 1801 can be the encoder in any one of the preceding embodiments, and the decoder 1802 can be the decoder in any one of the preceding embodiments.

[0416] In some embodiments, the embodiment of the present application further provides a computer readable storage medium having a computer program stored thereon. The computer program is executed by a processor (for example, the first processor or the second processor) to implement the method in any one of the preceding embodiments.

[0417] In some embodiments, the embodiment of the present application further provides a computer program product including a computer program or instructions. The computer program or instructions are executed by a processor (for example, the first processor or the second processor) to implement the method in any one of the preceding embodiments.

[0418] In some embodiments, the embodiments of the present application further provide a computer program which, when executed by a processor (for example, the first processor or the second processor), implements the method according to any one of the preceding embodiments.

[0419] Those skilled in the art can clearly understand that the units and algorithm steps of each example described in combination with the embodiments disclosed in the present application can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical solution. A person skilled in the art can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0420] It can be clearly understood by those skilled in the art that, for the convenience and brevity of description, the specific working processes of the above-described devices and units can refer to the corresponding processes in the preceding method embodiments, which will not be described here.

[0421] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interface, device or unit, and can be electrical, mechanical or other forms.

[0422] The units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, that is, they can be located in one place, or can be distributed on a plurality of network units. According to actual needs, some or all of the units can be selected to achieve the purpose of the embodiments.

[0423] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit.

[0424] If the functions are implemented in the form of software function units and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application essentially or the parts that contribute to the prior art or parts of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (Read-Only Memory, ROM), a random access memory (Random Access Memory, RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0425] It should be noted that in the present application, the terms "include", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that the processes, methods, articles or devices including a series of elements not only include those elements, but also include other elements not explicitly listed or inherent to such processes, methods, articles or devices. Without more limitations, the element defined by the statement "including a" does not exclude the presence of another identical element in the process, method, article or device including the element.

[0426] The above-mentioned sequence numbers of the embodiments of the present application are only for description, and do not represent the advantages and disadvantages of the embodiments.

[0427] The methods disclosed in the several method embodiments provided by the present application can be combined arbitrarily without conflict to obtain new method embodiments.

[0428] The features disclosed in the several product embodiments provided by the present application can be combined arbitrarily without conflict to obtain new product embodiments.

[0429] The features disclosed in the several method or device embodiments provided by the present application can be combined arbitrarily without conflict to obtain new method or device embodiments.

[0430] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or replacements within the technical scope disclosed by the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims. Industrial applicability

[0431] In the embodiments of the present application, at the encoding end or the decoding end, when a matrix-based intra prediction mode is used for a current block, a first prediction block of the current block is determined, and upsampling processing is performed on the first prediction block to determine a second prediction block of the current block; based on size information of the current block, the derived intra prediction mode of the current block is determined by using the first prediction block or the second prediction block, and a transform set of the current block is determined according to the derived intra prediction mode. In this way, when the MIP mode is used for the current block, the first prediction block and the second prediction block of the current block are first determined; then, according to the size information of the current block, the derived intra prediction mode of the current block can be determined by selecting the first prediction block or the second prediction block, and then the transform set suitable for the current block can be determined, and a suitable transform kernel is selected for transformation, so that the accuracy of the transform prediction can be improved, the compression efficiency is improved, and the coding and decoding performance is improved.

Claims

1. A decoding method, applied to a decoder, comprising: Decoding the code stream and determining the value of the first syntax element; When the first syntax element indicates that the current block uses a matrix-based intra prediction mode, determining a first prediction block of the current block, and performing upsampling processing on the first prediction block to determine a second prediction block of the current block; Based on the size information of the current block, a derived intra prediction mode of the current block is determined using the first prediction block or the second prediction block, and a transform set of the current block is determined according to the derived intra prediction mode.

2. The method according to claim 1, wherein The method further comprises: determining a transform kernel of the current block according to the transform set; determining a transform coefficient of the current block; Transform coefficients of the current block are transformed according to the transform kernel to determine residual samples of the current block.

3. The method according to claim 2, wherein: The determining, according to the transform set, a transform kernel of the current block includes: Decoding a code stream to determine a transform kernel index of the current block; A transform core of the current block is determined according to the transform set and the transform core index.

4. The method according to claim 2, wherein: The determining of the transform coefficient of the current block includes: Decoding a code stream to determine a quantization coefficient of the current block; Dequantizing the quantized coefficients of the current block to determine the transform coefficients of the current block.

5. The method according to claim 2, wherein: The transforming the transform coefficients of the current block according to the transform kernel to determine the residual samples of the current block includes: performing a low-frequency inseparable transform on the transform coefficients of the current block according to the transform kernel to determine transform samples of the current block; Performing discrete cosine transform on the transformed samples of the current block to determine residual samples of the current block.

6. The method according to claim 2, wherein: The transforming the transform coefficients of the current block according to the transform kernel to determine the residual samples of the current block includes: Performing a non-separable main transform on the transform coefficients of the current block according to the transform kernel to determine residual samples of the current block.

7. The method according to claim 1, wherein The determining, based on the size information of the current block, a derived intra prediction mode of the current block by using the first prediction block or the second prediction block includes: When the size information of the current block meets a first condition, determining the derived intra prediction mode using the second prediction block; When the size information of the current block satisfies a second condition, the derived intra prediction mode is determined using the first prediction block.

8. The method according to claim 7, wherein: The size information of the current block satisfies a first condition, including: the current block is a non-square block; The size information of the current block satisfies a second condition, including: the current block is a square block.

9. The method according to claim 7, wherein: The size information of the current block satisfies a first condition, including: a category index value of the current block is equal to 0 or 1; The size information of the current block satisfies a second condition, including: a category index value of the current block is equal to 2.

10. The method according to claim 7, wherein: The size information of the current block satisfies a first condition, including: the current block is a non-square block and the category index value of the current block is not equal to 2; The size information of the current block satisfies a second condition, including: the current block is a square block, or the category index value of the current block is equal to 2.

11. The method according to claim 1, wherein The method further comprises: When the size information of the current block meets a first condition, a preset intra prediction mode is determined as a derived intra prediction mode of the current block, and a transform set of the current block is determined according to the derived intra prediction mode.

12. The method according to claim 1, wherein The method further comprises: When the size information of the current block meets the second condition, a preset intra prediction mode is determined as the derived intra prediction mode of the current block, and a transform set of the current block is determined according to the derived intra prediction mode.

13. The method according to claim 1, wherein The determining the derived intra prediction mode of the current block by using the first prediction block or the second prediction block includes: Determining, based on the first prediction block or the second prediction block, candidate samples for deriving an intra prediction mode; Determining at least one candidate intra prediction mode and a gradient strength accumulation value corresponding to the at least one candidate intra prediction mode according to the horizontal gradient value and the vertical gradient value of the candidate sample; A maximum gradient strength accumulated value is determined among the gradient strength accumulated values ​​corresponding to the at least one candidate intra prediction mode, and the candidate intra prediction mode corresponding to the maximum gradient strength accumulated value is determined as the derived intra prediction mode of the current block.

14. The method according to claim 1, wherein The determining the derived intra prediction mode of the current block by using the first prediction block or the second prediction block includes: determining a template region corresponding to the first prediction block or the second prediction block; Predicting the template region using at least one candidate intra prediction mode, and determining a prediction sample of the template region corresponding to the at least one candidate intra prediction mode; performing cost calculation on the prediction samples of the template region and the reconstructed samples of the template region according to the at least one candidate intra prediction mode, and determining a cost result corresponding to the at least one candidate intra prediction mode; A minimum cost result is determined among the cost results corresponding to the at least one candidate intra prediction mode, and the candidate intra prediction mode corresponding to the minimum cost result is determined as the derived intra prediction mode of the current block.

15. The method according to any one of claims 1 to 14, wherein The determining a transform set of the current block according to the derived intra prediction mode includes: Determining a transform set index of the current block according to a first lookup table and the derived intra prediction mode; According to the transform set index, the transform set of the current block is determined; wherein the first lookup table is used to record the mapping relationship between the intra prediction mode and the transform set index.

16. The method according to any one of claims 1 to 14, wherein The method further comprises: Decoding the code stream and determining a value of the second syntax element; When the second syntax element indicates that the current sequence allows the use of the matrix-based intra prediction mode, decoding the code stream and determining a value of the third syntax element; When the third syntax element indicates that the prediction type of the current block is intra prediction, the step of decoding the code stream and determining the value of the first syntax element is performed.

17. The method according to claim 1, wherein The determining the first prediction block of the current block includes: When the first syntax element indicates that the current block uses a matrix-based intra prediction mode, decoding the code stream to determine prediction parameters of the current block, where the prediction parameters include transposition identification information and weight matrix index information; Downsampling adjacent reference samples of the current block to determine an initial input vector of the current block; Determining a target input vector of the current block according to the transposition identification information and the initial input vector; Determining a weight matrix of the current block according to the weight matrix index information, and performing a multiplication operation on the target input vector and the weight matrix to determine an output vector of the current block; The output vectors of the current block are arranged and placed according to the transposition identification information to obtain a first prediction block of the current block.

18. A coding method, applied to an encoder, comprising: When a current block uses a matrix-based intra prediction mode, determining a first prediction block of the current block, and performing upsampling processing on the first prediction block to determine a second prediction block of the current block; Based on the size information of the current block, a derived intra prediction mode of the current block is determined using the first prediction block or the second prediction block, and a transform set of the current block is determined according to the derived intra prediction mode.

19. The method according to claim 18, wherein The method further comprises: determining a transform kernel of the current block according to the transform set; Determining residual samples of the current block, and transforming the residual samples of the current block according to the transform kernel to determine transform coefficients of the current block; The transform coefficients of the current block are coded, and the obtained coded bits are written into a bitstream.

20. The method according to claim 19, wherein The determining, according to the transform set, a transform kernel of the current block includes: determining at least two candidate transform kernels included in the transform set; performing encoding cost calculation on the current block according to the at least two candidate transform kernels, and determining cost results corresponding to the at least two candidate transform kernels; A minimum cost result is determined among the cost results corresponding to the at least two candidate transform cores, and the candidate transform core corresponding to the minimum cost result is determined as the transform core of the current block.

21. The method according to claim 19, wherein The method further comprises: Determine a transform core index of the current block; wherein the transform core index is used to indicate the number of the transform core of the current block in the transform set; The transform core index of the current block is coded, and the obtained coded bits are written into a bitstream.

22. The method according to claim 19, wherein The transforming the residual samples of the current block according to the transform kernel to determine the transform coefficients of the current block includes: Performing discrete cosine transform on the residual samples of the current block to determine transformed samples of the current block; Performing a low-frequency inseparable transform on the transform samples of the current block according to the transform kernel to determine transform coefficients of the current block.

23. The method according to claim 19, wherein The transforming the residual samples of the current block according to the transform kernel to determine the transform coefficients of the current block includes: Performing a non-separable main transform on the residual samples of the current block according to the transform kernel to determine transform coefficients of the current block.

24. The method according to claim 18, wherein The encoding process is performed on the transform coefficients of the current block, and the obtained encoding bits are written into a bitstream, including: quantizing the transform coefficients of the current block to determine the quantization coefficients of the current block; The quantized coefficients of the current block are coded, and the obtained coded bits are written into a bitstream.

25. The method according to claim 18, wherein The determining, based on the size information of the current block, a derived intra prediction mode of the current block by using the first prediction block or the second prediction block includes: When the size information of the current block meets a first condition, determining the derived intra prediction mode using the second prediction block; When the size information of the current block satisfies a second condition, the derived intra prediction mode is determined using the first prediction block.

26. The method according to claim 25, wherein The size information of the current block satisfies a first condition, including: the current block is a non-square block; The size information of the current block satisfies a second condition, including: the current block is a square block.

27. The method according to claim 25, wherein The size information of the current block satisfies a first condition, including: a category index value of the current block is equal to 0 or 1; The size information of the current block satisfies a second condition, including: a category index value of the current block is equal to 2.

28. The method according to claim 25, wherein The size information of the current block satisfies a first condition, including: the current block is a non-square block and the category index value of the current block is not equal to 2; The size information of the current block satisfies a second condition, including: the current block is a square block, or the category index value of the current block is equal to 2.

29. The method according to claim 18, wherein The method further comprises: When the size information of the current block meets a first condition, a preset intra prediction mode is determined as a derived intra prediction mode of the current block, and a transform set of the current block is determined according to the derived intra prediction mode.

30. The method of claim 18, wherein The method further comprises: When the size information of the current block meets the second condition, a preset intra prediction mode is determined as the derived intra prediction mode of the current block, and a transform set of the current block is determined according to the derived intra prediction mode.

31. The method according to claim 18, wherein The determining a transform set of the current block according to the derived intra prediction mode includes: Determining a transform set index of the current block according to a first lookup table and the derived intra prediction mode; According to the transform set index, the transform set of the current block is determined; wherein the first lookup table is used to record the mapping relationship between the intra prediction mode and the transform set index.

32. The method of claim 18, wherein: The method further comprises: Determining a value of a second syntax element; wherein the second syntax element is used to indicate whether the current sequence allows the use of a matrix-based intra prediction mode; Perform encoding processing on the value of the second syntax element, and write the obtained encoding bits into a bitstream.

33. The method according to claim 32, wherein The method further comprises: When the current sequence allows the use of a matrix-based intra prediction mode, determining a value of a third syntax element; wherein the third syntax element is used to indicate a prediction type of the current block; Perform encoding processing on the value of the third syntax element, and write the obtained encoding bits into a bitstream.

34. The method according to claim 33, wherein The method further comprises: When the prediction type of the current block is intra prediction, determining a value of a first syntax element; wherein the first syntax element is used to indicate whether the current block uses a matrix-based intra prediction mode; The value of the first syntax element is coded, and the obtained coded bits are written into a bitstream.

35. The method according to any one of claims 18 to 34, wherein The method further comprises: When the current block uses a matrix-based intra prediction mode, determining prediction parameters of the current block, the prediction parameters including transposition identification information and weight matrix index information; The prediction parameters of the current block are coded, and the obtained coded bits are written into a bitstream.

36. A code stream, wherein The bitstream is generated by bit encoding according to information to be encoded; wherein the information to be encoded includes at least one of the following: a quantization coefficient of a current block, a transform kernel index of the current block, a prediction parameter of the current block, a value of a first syntax element, a value of a second syntax element, and a value of a third syntax element; The prediction parameters include transposition identification information and weight matrix index information, the second syntax element is used to indicate whether the current sequence allows the use of a matrix-based intra-frame prediction mode, the third syntax element is used to indicate the prediction type of the current block, and the first syntax element is used to indicate whether the current block uses a matrix-based intra-frame prediction mode.

37. An encoder comprising a first prediction unit and a first determination unit, wherein: The first prediction unit is configured to, when the current block uses a matrix-based intra prediction mode, determine a first prediction block of the current block, and perform upsampling processing on the first prediction block to determine a second prediction block of the current block; The first determining unit is configured to determine a derived intra-frame prediction mode of the current block using the first prediction block or the second prediction block based on size information of the current block, and determine a transform set of the current block according to the derived intra-frame prediction mode.

38. An encoder comprising a first memory and a first processor, wherein: The first memory is used to store a computer program that can be run on the first processor; The first processor is configured to execute the method according to any one of claims 18 to 35 when running the computer program.

39. A decoder comprising a decoding unit, a second prediction unit, and a second determination unit, wherein: The decoding unit is configured to decode the code stream and determine a value of the first syntax element; The second prediction unit is configured to, when the first syntax element indicates that the current block uses a matrix-based intra prediction mode, determine a first prediction block of the current block, and perform upsampling processing on the first prediction block to determine a second prediction block of the current block; The second determining unit is configured to determine a derived intra-frame prediction mode of the current block using the first prediction block or the second prediction block based on size information of the current block, and determine a transform set of the current block according to the derived intra-frame prediction mode.

40. A decoder comprising a second memory and a second processor, wherein: The second memory is used to store a computer program that can be run on the second processor; The second processor is configured to execute the method according to any one of claims 1 to 17 when running the computer program.

41. A computer-readable storage medium having a computer program stored thereon, wherein: When the computer program is executed by a processor, the method according to any one of claims 1 to 17 or the method according to any one of claims 18 to 35 is implemented.

Images