Encoding and decoding method, encoder, decoder and storage medium

Abstract

The embodiment of the invention provides a decoding method, and the method comprises the steps: decoding a code stream at a decoding end, and determining a prediction mode parameter; when the prediction mode parameter indicates that the intra-frame prediction value is determined by using the MIP, decoding the code stream, and determining the MIP parameter of the current block; decoding the code stream, and determining a transformation coefficient and an LFNST index sequence number of the current block; when the LFNST index sequence number indicates that the current block uses the LFNST, determining a mapping mode of an LFNST transform set according to the MIP parameter; according to the mapping mode of the LFNST transform set, one LFNST transform kernel candidate set is selected from the multiple LFNST transform kernel candidate sets, and an LFNST transform kernel used by the current block is determined from the selected LFNST transform kernel candidate set; and performing conversion processing on the conversion coefficient by using the LFNST conversion kernel.

Description

Coding and decoding method, encoder, decoder and storage medium Technical Field

[0001] The embodiments of the present application relate to the field of image processing technology, and in particular to a coding and decoding method, an encoder, a decoder, and a storage medium. Background Art

[0002] As demand for video display quality increases, new video applications such as HD and UHD video have emerged. H.265 / High Efficiency Video Coding (HEVC) is no longer able to meet the demands of rapidly evolving video applications. The Joint Video Exploration Team (JVET) has proposed the next-generation video coding standard, H.266 / Versatile Video Coding (VVC), and its corresponding test model is the VVC Reference Software Test Platform (VVC Test Model, VTM). Based on VTM10.0, the Enhanced Compression Model (ECM) has begun to incorporate newer and more efficient compression algorithms.

[0003] Decoder-side Intra Mode Derivation (DIMD) is an intra-frame prediction technology for ECM. The core of this technology is that the decoder uses the same method as the encoder to derive the intra-frame prediction mode, thereby saving bit overhead.

[0004] However, the use of DIMD technology introduces greater complexity in both software and hardware, increasing the compression cost.

[0005] Summary of the Invention

[0006] The embodiments of the present application provide a coding and decoding method, an encoder, a decoder, and a storage medium, which can reduce computational complexity and thus improve coding efficiency.

[0007] The technical solution of the embodiment of the present application can be implemented as follows:

[0008] In a first aspect, an embodiment of the present application provides a decoding method, applied to a decoder, the method comprising:

[0009] Decode the code stream and determine the prediction mode parameters;

[0010] When the prediction mode parameter indicates that the intra-frame prediction value is determined using MIP, decoding the code stream to determine the MIP parameters of the current block;

[0011] Decoding the code stream to determine the transform coefficient and LFNST index number of the current block;

[0012] When the LFNST index number indicates that the current block uses LFNST, determining a mapping mode of an LFNST transform set according to the MIP parameter;

[0013] selecting, according to a mapping mode of the LFNST transform set, an LFNST transform core candidate set from a plurality of LFNST transform core candidate sets, and determining an LFNST transform core used for the current block from the selected LFNST transform core candidate set;

[0014] The transform coefficients are transformed using the LFNST transform kernel.

[0015] In a second aspect, an embodiment of the present application provides an encoding method, applied to an encoder, the method comprising:

[0016] determining prediction model parameters;

[0017] When the prediction mode parameter indicates that the current block uses MIP to determine an intra prediction value, determining an MIP parameter of the current block;

[0018] Determine an intra prediction block of the current block according to the MIP parameters, and calculate a residual block between the current block and the intra prediction value;

[0019] When the current block uses LFNST, determining a mapping mode of the LFNST transform set according to the MIP parameters;

[0020] Selecting an LFNST transform core candidate set from a plurality of LFNST transform core candidate sets according to a mapping mode of the LFNST transform set, determining an LFNST transform core used by the current block from the selected LFNST transform core candidate set, setting an LFNST index number, and writing the index number into a video stream;

[0021] The residual block is transformed using the LFNST transformation kernel.

[0022] In a third aspect, an embodiment of the present application provides an encoder, comprising a first determining unit, an encoding unit, and a first transforming unit; wherein,

[0023] The first determining unit is configured to determine a prediction mode parameter; when the prediction mode parameter indicates that the current block uses MIP to determine an intra prediction value, determine the MIP parameter of the current block; determine the intra prediction block of the current block based on the MIP parameter, and calculate a residual block between the current block and the intra prediction value; when the current block uses LFNST, determine a mapping mode of an LFNST transform set based on the MIP parameter; select an LFNST transform core candidate set from multiple LFNST transform core candidate sets based on the mapping mode of the LFNST transform set, determine the LFNST transform core used by the current block from the selected LFNST transform core candidate set, and set an LFNST index number;

[0024] The encoding unit is configured to write a video stream;

[0025] The first transformation unit is further configured to use the LFNST transformation kernel to perform transformation processing on the residual block.

[0026] In a fourth aspect, an embodiment of the present application provides an encoder, comprising a first memory and a first processor; wherein,

[0027] The first memory is used to store a computer program that can be run on the first processor;

[0028] The first processor is configured to execute the method according to the second aspect when running the computer program.

[0029] In a fifth aspect, an embodiment of the present application provides a decoder, comprising a second determining unit and a second transforming unit; wherein,

[0030] The second determining unit is configured to decode a bitstream and determine a prediction mode parameter; when the prediction mode parameter indicates that a MIP is used to determine an intra-frame prediction value, decode the bitstream and determine a MIP parameter of a current block; decode the bitstream and determine a transform coefficient and an LFNST index number of the current block; when the LFNST index number indicates that LFNST is used for the current block, determine a mapping mode of an LFNST transform set according to the MIP parameter; select an LFNST transform core candidate set from a plurality of LFNST transform core candidate sets according to the mapping mode of the LFNST transform set, and determine an LFNST transform core used for the current block from the selected LFNST transform core candidate set;

[0031] The second transform unit is configured to perform transform processing on the transform coefficients using the LFNST transform kernel.

[0032] In a sixth aspect, an embodiment of the present application provides a decoder, comprising a second memory and a second processor; wherein:

[0033] The second memory is used to store a computer program that can be run on the second processor;

[0034] The second processor is configured to execute the method according to the first aspect when running the computer program.

[0035] In a seventh aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed, the method as described in the first aspect or the method as described in the second aspect is implemented.

[0036] Embodiments of the present application provide a coding and decoding method, an encoder, a decoder, and a storage medium. At a decoding end, a bitstream is decoded to determine a prediction mode parameter; when the prediction mode parameter indicates the use of MIP to determine an intra-frame prediction value, the bitstream is decoded to determine the MIP parameter of the current block; the bitstream is decoded to determine the transform coefficient and LFNST index number of the current block; when the LFNST index number indicates that the current block uses LFNST, a mapping mode of an LFNST transform set is determined according to the MIP parameter; according to the mapping mode of the LFNST transform set, an LFNST transform core candidate set is selected from multiple LFNST transform core candidate sets, and an LFNST transform core used for the current block is determined from the selected LFNST transform core candidate set; and the transform coefficient is transformed using the LFNST transform core. At the encoding end, a prediction mode parameter is determined; when the prediction mode parameter indicates that the current block uses MIP to determine the intra-frame prediction value, the MIP parameter of the current block is determined; based on the MIP parameter, the intra-frame prediction block of the current block is determined, and the residual block between the current block and the intra-frame prediction value is calculated; when the current block uses LFNST, the mapping mode of the LFNST transform set is determined based on the MIP parameter; based on the mapping mode of the LFNST transform set, an LFNST transform core candidate set is selected from multiple LFNST transform core candidate sets, and the LFNST transform core used for the current block is determined from the selected LFNST transform core candidate set, the LFNST index sequence number is set and written into the video code stream; the residual block is transformed using the LFNST transform core. It can be seen that in an embodiment of the present application, when deriving the prediction block, the mapping mode of the LFNST transform set is determined based on the size parameter in the MIP parameter of the current block, wherein, for image blocks of larger sizes, the DIMD derivation mapping mode can be selected not to reduce the computational complexity, thereby improving the coding efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] FIG1 is a schematic diagram of a matrix-based intra-frame prediction technique;

[0038] FIG2 is a table showing the correspondence between intra prediction modes and transform sets;

[0039] FIG3 is a schematic diagram of a decoding-side intra-frame mode derivation technique;

[0040] FIG4 is a block diagram of a video encoding system provided in an embodiment of the present application;

[0041] FIG5 is a block diagram of a video decoding system provided in an embodiment of the present application;

[0042] FIG6 is a schematic diagram of an implementation flow of a decoding method proposed in an embodiment of the present application;

[0043] FIG7 is a schematic diagram of an implementation flow of the encoding method proposed in an embodiment of the present application;

[0044] FIG8 is a schematic diagram of the structure of the encoder;

[0045] FIG9 is a second schematic diagram of the structure of the encoder;

[0046] FIG10 is a schematic diagram of the structure of the decoder;

[0047] FIG11 is a second schematic diagram of the structure of the decoder. DETAILED DESCRIPTION

[0048] In order to enable a more detailed understanding of the features and technical contents of the embodiments of the present application, the implementation of the embodiments of the present application is described in detail below with reference to the accompanying drawings. The attached drawings are for reference only and are not used to limit the embodiments of the present application.

[0049] In the following description, reference is made to "some embodiments," which describe a subset of all possible embodiments. However, it is understood that "some embodiments" may be the same subset or different subsets of all possible embodiments, and may be combined with each other without conflict. It should also be noted that the terms "first, second, and third" in the embodiments of the present application are only used to distinguish similar objects and do not represent a specific ordering of the objects. It is understood that "first, second, and third" may be interchanged in a specific order or sequential order where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0050] In a video image, a first image component, a second image component, and a third image component are generally used to represent a coding block (CB); wherein the three image components are a luminance component, a blue chrominance component, and a red chrominance component, respectively. Specifically, the luminance component is usually represented by the symbol Y, the blue chrominance component is usually represented by the symbol Cb or U, and the red chrominance component is usually represented by the symbol Cr or V; thus, the video image can be represented in either the YCbCr format or the YUV format.

[0051] In the embodiment of the present application, the first image component may be a luminance component, the second image component may be a blue chrominance component, and the third image component may be a red chrominance component, but the embodiment of the present application does not make any specific limitation.

[0052] Common video codec standards all use a block-based hybrid coding framework. Each frame is divided into square Largest Coding Units (LCUs) or Coding Tree Units (CTUs) of the same size (e.g., 128×128, 64×64, etc.). Each LCU or CTU can be further divided into rectangular Coding Units (CUs) based on rules. Coding Units may also be further divided into smaller Prediction Units (PUs) and Transform Units (TUs), etc.

[0053] The hybrid coding framework can include modules such as prediction, transform, quantization, entropy coding, and in-loop filtering. The prediction module can include intra-frame prediction and inter-frame prediction, and inter-frame prediction can include motion estimation and motion compensation. Since there is a strong correlation between adjacent pixels within a frame of a video image, the use of intra-frame prediction in video coding and decoding technology can eliminate spatial redundancy between adjacent pixels. However, since there is also a strong similarity between adjacent frames in a video image, the use of inter-frame prediction in video coding and decoding technology can eliminate temporal redundancy between adjacent frames, thereby improving coding and decoding efficiency.

[0054] The basic process of a video codec is as follows: On the encoder side, a frame is divided into blocks. Intra-frame prediction or inter-frame prediction is used on the current block to generate a predicted block for the current block. The predicted block is subtracted from the original block to obtain a residual block. The residual block is transformed and quantized to obtain a quantization coefficient matrix. This quantization coefficient matrix is ​​entropy encoded and output to the bitstream. On the decoder side, intra-frame prediction or inter-frame prediction is used on the current block to generate a predicted block for the current block. The decoded bitstream is then decoded to obtain a quantization coefficient matrix. This quantization coefficient matrix is ​​inversely quantized and inversely transformed to obtain a residual block. The predicted block and residual block are added together to obtain a reconstructed block. The reconstructed blocks form a reconstructed image, which is then subjected to image-based or block-based loop filtering to obtain the decoded image. The encoder side also performs similar operations to the decoder side to obtain the decoded image. The decoded image can serve as a reference frame for inter-frame prediction in subsequent frames. Block division information, prediction, transform, quantization, entropy coding, loop filtering, and other mode or parameter information determined by the encoder are output to the bitstream if necessary. The decoding end determines the same block division information as the encoding end by decoding and analyzing the existing information, as well as the mode information or parameter information such as prediction, transformation, quantization, entropy coding, and loop filtering, thereby ensuring that the decoded image obtained by the encoding end is the same as the decoded image obtained by the decoding end. The decoded image obtained by the encoding end is also usually called a reconstructed image. The current block can be divided into prediction units during prediction, and the current block can be divided into transformation units during transformation. The division of prediction units and transformation units can be different. The above is the basic process of the video codec under the block-based hybrid coding framework. With the development of technology, some modules or steps of the framework or process may be optimized. The embodiment of the present application is applicable to the basic process of the video codec under the block-based hybrid coding framework, but is not limited to the framework and process.

[0055] The current block may be a current coding unit (CU) or a current prediction unit (PU), etc.

[0056] JVET, the international video coding standards organization, has established a group to research coding models that surpass H.266 / VVC. This model, also known as platform testing software, has been named the Enhanced Compression Model (ECM). Based on VTM10.0, ECM incorporates newer and more efficient compression algorithms, currently surpassing VVC's coding performance by approximately 13%. ECM not only increases the coding unit size for specific resolutions but also integrates numerous intra- and inter-frame prediction techniques.

[0057] The following describes relevant technical solutions of the matrix-based intra prediction (MIP) technology.

[0058] The matrix-based intra-frame prediction technology, namely MIP technology, can be divided into three main steps, namely downsampling, matrix multiplication and upsampling. The first step is to downsample the spatially adjacent reconstructed samples to obtain the downsampled sample sequence as the input vector of the second step; the second step is to use the output vector of the first step as input, multiply it with the pre-set matrix and add the bias vector, and output the calculated sample vector; the third step is to use the output vector of the second step as input to upsample it into the final prediction block. Figure 1 is a schematic diagram of the matrix-based intra-frame prediction technology. The above process is shown in Figure 1. In the first step, the MIP technology obtains the upper adjacent downsampled reconstructed sample vector by averaging the upper adjacent reconstructed samples of the current coding unit, and obtains the left adjacent downsampled reconstructed sample vector by averaging the left adjacent reconstructed samples. The upper vector and the left vector are used as the input of the matrix-vector multiplication in the second step. A k is a pre-set matrix, b k is a pre-set bias vector, where k is the MIP mode index. In the third step, the result obtained in the second step is linearly interpolated and up-sampled to obtain a prediction sample block that matches the number of samples in the actual coding unit.

[0059] The number of MIP modes varies for coding units of different block sizes. Taking H.266 / VVC as an example, for a 4×4 coding unit, MIP has 16 prediction modes; for an 8×8 coding unit, or a coding unit with a width and height equal to 4, MIP has 8 prediction modes; for coding units of other sizes, MIP has 6 prediction modes. Furthermore, MIP technology includes a transposition function. For prediction modes that match the current size, MIP will attempt a transposition calculation on the encoding side. If transposition is required, the order of the upper and left input vectors is swapped, and the output is then swapped after the matrix calculation.

[0060] Therefore, MIP not only requires a flag to indicate whether the current coding unit uses the MIP technology, but also, if the current coding unit uses the MIP technology, it is necessary to additionally transmit a transposition flag and a MIP mode index to the decoding end.

[0061] In the VVC standard text, the transposition flag of the MIP is binarized by the fixed length coding method (FL) with a length of 1. The mode index of the MIP is binarized by the truncated binary coding method (TB).

[0062] The Low-Frequency Non-Separable Transform (LFNST) technology is also a technology adopted in the VVC text. The following describes the relevant technical solutions of LFNST.

[0063] LFNST is applied between the forward main transform and quantization on the encoder side, and between inverse quantization and the inverse main transform on the decoder side. After the main transform, the residual of the current coding block is converted to frequency domain coefficients. LFNST then performs a frequency domain conversion on some of these coefficients, transforming some of the frequency domain coefficients to obtain coefficients in another domain. Quantization and entropy coding are then performed. LFNST further removes statistical redundancy and demonstrates excellent performance on VTM, the VVC reference software.

[0064] LFNST mainly performs a secondary transform on the 4×4 or 8×8 area in the upper left corner of the transform block. In addition, the transform kernels of LFNST are mainly classified into four transform sets in VVC, each with two candidate transform kernels. In ECM, the transform kernels of LFNST have been expanded from the original four transform sets to 35 transform sets, and from the original two candidate transform kernels per transform set to three candidate transform kernels per transform set.

[0065] LFNST allows for both intra-frame prediction and inter-frame prediction. LFNST saves bit overhead by selecting transform sets corresponding to intra-frame prediction modes. Since intra-frame prediction typically has corresponding intra-frame prediction modes, namely DC mode, PLANAR mode, or angular prediction mode, these intra-frame prediction modes are bound to the LFNST transform set. For example, in VVC, DC mode and PLANAR mode correspond to the first transform set, as shown in Table 1 below.

[0066] Table 1

[0067] predModeIntraSetIdxpredModeIntra<010<=predModeIntra<=102<=predModeIntra<=12113<=predModeIntra <=23224<=predModeIntra<=44345<=predModeIntra<=55256<=predModeIntra<=80181<=IntraPredMode<=830

[0068] Where predModeIntra can be an intra prediction mode indicator, and SetIdx can be an LFNST index number. Here, the value of the LFNST index number is set to indicate that LFNST is used for the current block and the index number of the LFNST transform kernel in the LFNST transform kernel candidate set. For example, if the LFNST transform set includes four transform kernel candidate sets (set0, set1, set2, set3), the corresponding SetIdx values ​​are 0, 1, 2, and 3, respectively.

[0069] Correspondingly, after the LFNST transform kernel is expanded in ECM, the LFNST transform sets corresponding to different intra-frame prediction modes will be more fine-grained. For example, Figure 2 is a correspondence table between intra-frame prediction modes and transform sets. As shown in Figure 2, there are 35 transform sets after expansion.

[0070] The following describes relevant technical solutions for the decoder side intra mode derivation (DIMD) technology.

[0071] DIMD is an intra-frame prediction technology for ECM, which is not included in VVC. The core point of this technology is that the intra-frame prediction mode is derived at the decoding end using the same method as the encoding end, thereby avoiding the transmission of the intra-frame prediction mode index of the current coding unit in the bitstream, achieving the purpose of saving bit overhead. The specific method is divided into two main steps. The first step is to derive the prediction mode, using the same prediction mode strength calculation method at the codec end. The encoder uses the Sobel operator to calculate the gradient histogram (histogram of gradients) under each prediction mode. The operating area is the three rows of adjacent reconstructed samples above the current block, the three columns of adjacent reconstructed samples to the left, and the corresponding adjacent reconstructed samples to the upper left. By calculating the gradient histogram of the aforementioned L-shaped area, the first prediction mode corresponding to the largest amplitude in the histogram and the second prediction mode corresponding to the second largest amplitude can be obtained. The decoding end derives the first and second prediction modes using the same steps; the second step is to derive the prediction block, using the same prediction block derivation method at the codec end to obtain the current prediction block. The encoder checks the following two conditions: 1. The gradient of the second prediction mode is not zero; 2. Neither the first prediction mode nor the second prediction mode is a PLANAR or DC prediction mode. If neither of these two conditions is met, the current prediction block uses only the first prediction mode to calculate the predicted sample values ​​for the current block, applying the normal prediction process to the first prediction mode. Otherwise, if both conditions are met, the current prediction block is derived using a weighted averaging method. Specifically, the PLANAR mode accounts for one-third of the weighting, while the remaining two-thirds are weighted by the first prediction mode based on the ratio of the first prediction mode's gradient strength to the sum of the gradient strengths of the first and second prediction modes, and by the second prediction mode based on the ratio of the second prediction mode's gradient strength to the sum of the gradient strengths of the first and second prediction modes. The three prediction modes—PLANAR, the first prediction mode, and the second prediction mode—are weighted and averaged to obtain the prediction block for the current coding unit. The decoder follows the same steps to obtain the prediction block. Figure 3 is a schematic diagram of the intra mode derivation technique at the decoder end, illustrating the specific operation process described above.

[0072] Among them, in the above second step, the specific weight calculation method is shown in the following formula:

[0073] Weight(PLANAR)=1 / 3 (1)

[0074] Weight(mode1)=2 / 3×(amp1 / (amp1+amp2)) (2)

[0075] Weight(mode2)=1–Weight(PLANAR)–Weight(mode1) (3)

[0076] Wherein, mode1 and mode2 represent the first prediction mode and the second prediction mode, respectively, and amp1 and amp2 represent the gradient amplitude value of the first prediction mode and the gradient amplitude value of the second prediction mode, respectively. For DIMD technology, a flag bit needs to be transmitted to the decoding end to indicate whether the current coding unit uses DIMD technology.

[0077] To improve MIP and LFNST, the relationship between MIP and LFNST has been simplified in VVC and ECM, and all MIP prediction modes are defaulted to PLANAR mode before being mapped to the LFNST transform set. This is because LFNST used intra-frame prediction mode as training input in its early design stages, and the LFNST transform kernel coefficients were obtained through deep learning training. The MIP prediction mode is expressed differently from the traditional intra-frame prediction mode. The MIP prediction mode represents a certain prediction matrix coefficient, while the traditional prediction mode represents directionality. At the same time, the prediction results of MIP are similar to those of the traditional PLANAR mode, so all MIP prediction modes are mapped to the LFNST transform set using PLANAR.

[0078] You can choose to use DIMD to sort the gradient magnitude values ​​of each traditional intra prediction mode for the MIP's prediction block, and demap the best possible traditional prediction mode to the LFNST transform set. At the same time, you can also choose to expand the range of coding unit sizes that allow the use of LFNST in the original MIP. In VVC and ECM, LFNST is only allowed when the width and height of the current coding unit are greater than or equal to 16. After the expansion, LFNST is allowed when the width and height of the current coding unit are greater than or equal to 4.

[0079] Although the above method effectively solves the problem of mapping the MIP prediction mode to LFNST and improves coding efficiency, it also introduces corresponding complexity, whether it is the encoding and decoding time in software or the cache and timing issues in hardware implementation.

[0080] It is also proposed to expand the scope of using LFNST for MIP, allowing the condition to be changed to that both width and height are greater than or equal to 4 to use LFNST. In this way, the scope of using DIMD to derive traditional prediction modes for MIP prediction blocks is expanded, and the encoding and decoding complexity will continue to increase.

[0081] Compared to the encoding side, the added complexity on the decoding side is more costly. The 4×4 coding unit range of LFNST is already supported by both VVC and ECM, so it's not expected to introduce significant additional concerns. However, the DIMD derivation process for prediction blocks is a technology and operation currently unavailable in both VVC and ECM. Reducing the requirements for DIMD usage or simplifying the steps while maintaining encoding performance can significantly improve decoding efficiency on the decoding side.

[0082] In other words, common encoding and decoding solutions based on DIMD technology will introduce greater complexity in both software and hardware, increase the compression cost, and reduce encoding efficiency.

[0083] To solve the above problems, in an embodiment of the present application, at the decoding end, a bitstream is decoded to determine a prediction mode parameter; when the prediction mode parameter indicates the use of MIP to determine an intra-frame prediction value, the bitstream is decoded to determine the MIP parameter of the current block; the bitstream is decoded to determine the transform coefficient and LFNST index number of the current block; when the LFNST index number indicates that the current block uses LFNST, a mapping mode of the LFNST transform set is determined based on the MIP parameter; based on the mapping mode of the LFNST transform set, an LFNST transform core candidate set is selected from multiple LFNST transform core candidate sets, and an LFNST transform core used for the current block is determined from the selected LFNST transform core candidate set; and the transform coefficient is transformed using the LFNST transform core. At the encoding end, a prediction mode parameter is determined; when the prediction mode parameter indicates that the current block uses MIP to determine the intra-frame prediction value, the MIP parameter of the current block is determined; based on the MIP parameter, the intra-frame prediction block of the current block is determined, and the residual block between the current block and the intra-frame prediction value is calculated; when the current block uses LFNST, the mapping mode of the LFNST transform set is determined based on the MIP parameter; based on the mapping mode of the LFNST transform set, an LFNST transform core candidate set is selected from multiple LFNST transform core candidate sets, and the LFNST transform core used for the current block is determined from the selected LFNST transform core candidate set, the LFNST index sequence number is set and written into the video code stream; the residual block is transformed using the LFNST transform core. It can be seen that in an embodiment of the present application, when deriving the prediction block, the mapping mode of the LFNST transform set is determined based on the size parameter in the MIP parameter of the current block, wherein, for image blocks of larger sizes, the DIMD derivation mapping mode can be selected not to reduce the computational complexity, thereby improving the coding efficiency.

[0084] Refer to Figure 4, which shows an example of a block diagram of a video coding system provided by an embodiment of the present application; as shown in Figure 4, the video coding system 10 includes a transform and quantization unit 101, an intra-frame estimation unit 102, an intra-frame prediction unit 103, a motion compensation unit 104, a motion estimation unit 105, an inverse transform and inverse quantization unit 106, a filter control analysis unit 107, a filtering unit 108, an encoding unit 109 and a decoded image cache unit 110, etc., wherein the filtering unit 108 can implement deblocking filtering and sample adaptive offset (SAO) filtering, and the encoding unit 109 can implement header information encoding and context-based adaptive binary arithmetic coding (CABAC).For the input original video signal, a video coding block can be obtained by dividing it into coding tree units (CTUs). Then, the residual pixel information obtained after intra-frame or inter-frame prediction is transformed by the transform and quantization unit 101, including transforming the residual information from the pixel domain to the transform domain and quantizing the obtained transform coefficients to further reduce the bit rate; the intra-frame estimation unit 102 and the intra-frame prediction unit 103 are used to perform intra-frame prediction on the video coding block; specifically, the intra-frame estimation unit 102 and the intra-frame prediction unit 103 are used to determine the intra-frame prediction mode to be used to encode the video coding block; the motion compensation unit 104 and the motion estimation unit 105 are used to perform inter-frame prediction coding on the received video coding block relative to one or more blocks in one or more reference frames to provide temporal prediction information; the motion estimation performed by the motion estimation unit 105 is the process of generating a motion vector, which can estimate the motion of the video coding block, and the motion compensation unit 104 then calculates the motion vector based on the motion vector determined by the motion estimation unit 105. After determining the intra-frame prediction mode, the intra-frame prediction unit 103 is further configured to provide the selected intra-frame prediction data to the encoding unit 109, and the motion estimation unit 105 also sends the calculated motion vector data to the encoding unit 109. In addition, the inverse transform and inverse quantization unit 106 is configured to reconstruct the video coding block and reconstruct a residual block in the pixel domain. The reconstructed residual block is subjected to the filter control analysis unit 107 and the filtering unit 108 to remove the block effect artifacts. The reconstructed residual block is then added to a predictive block in the frame of the decoded image buffer unit 110 to generate a reconstructed video coding block. The encoding unit 109 is configured to encode various coding parameters and quantized transform coefficients. In the CABAC-based coding algorithm, the context content can be based on adjacent coding blocks and can be used to encode information indicating the determined intra-frame prediction mode, and output the code stream of the video signal. The decoded image buffer unit 110 is configured to store the reconstructed video coding block for prediction reference. As the video image encoding proceeds, new reconstructed video encoding blocks are continuously generated, and these reconstructed video encoding blocks are stored in the decoded image buffer unit 110 .

[0085] Referring to FIG5 , there is shown an example block diagram of a video decoding system provided by an embodiment of the present application. As shown in FIG5 , the video decoding system 20 includes a decoding unit 201, an inverse transform and inverse quantization unit 202, an intra-frame prediction unit 203, a motion compensation unit 204, a filtering unit 205, and a decoded image buffer unit 206. The decoding unit 201 can implement header information decoding and CABAC decoding, and the filtering unit 205 can implement deblocking filtering and SAO filtering. After the input video signal undergoes the encoding process shown in FIG4 , a bitstream of the video signal is output. The bitstream is input to the video decoding system 20 and first passes through the decoding unit 201 to obtain decoded transform coefficients. The transform coefficients are processed by the inverse transform and inverse quantization unit 202 to generate residual blocks in the pixel domain. The intra-frame prediction unit 203 can be used to generate prediction data for the current video decoding block based on the determined intra-frame prediction mode and data from previously decoded blocks of the current frame or picture. The motion compensation unit 204 determines prediction information for the video decoding block by analyzing motion vectors and other associated syntax elements, and uses The prediction information is used to generate a predictive block for the video decoding block being decoded; a decoded video block is formed by summing the residual block from the inverse transform and inverse quantization unit 202 with the corresponding predictive block generated by the intra-frame prediction unit 203 or the motion compensation unit 204; the decoded video signal passes through the filtering unit 205 to remove blocking artifacts, thereby improving video quality; the decoded video block is then stored in the decoded image buffer unit 206, which stores reference images used for subsequent intra-frame prediction or motion compensation, and is also used for outputting the video signal, thereby obtaining the restored original video signal.

[0086] The encoding method in the embodiment of the present application can be applied to the intra-frame estimation unit 102 and the intra-frame prediction unit 103 as shown in Figure 4. In addition, the decoding method in the embodiment of the present application can also be applied to the intra-frame prediction unit 203 as shown in Figure 5. That is to say, the encoding and decoding method in the embodiment of the present application can be applied to both video encoding systems and video decoding systems, and can even be applied to both video encoding systems and video decoding systems at the same time, but the embodiment of the present application does not make specific limitations. It should also be noted that when the encoding and decoding method is applied to a video encoding system, the "current block" specifically refers to the current encoding block in the intra-frame prediction; when the encoding and decoding method is applied to a video decoding system, the "current block" specifically refers to the current decoding block in the intra-frame prediction.

[0087] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application.

[0088] An embodiment of the present application proposes a decoding method. FIG6 is a schematic diagram of an implementation flow of the decoding method proposed in the embodiment of the present application. As shown in FIG6 , the method for performing decoding processing by the decoder may include the following steps:

[0089] Step 101: Decode the code stream and determine prediction mode parameters.

[0090] In an embodiment of the present application, the decoder may first determine prediction mode parameters when decoding a bitstream.

[0091] It should be noted that, in the embodiments of the present application, the prediction mode parameter indicates the coding mode of the current block and parameters related to the coding mode. Among them, the prediction mode generally includes a traditional intra-frame prediction mode and a non-traditional intra-frame prediction mode, and the traditional intra-frame prediction mode can include a direct current (DC) mode, a planar (PLANAR) mode, and an angle mode, etc., and the non-traditional intra-frame prediction mode can include a MIP mode, a cross-component linear model prediction (CCLM) mode, an intra-block copy (IBC) mode, and a PLT (Palette) mode, etc.

[0092] It can be understood that in the embodiments of the present application, on the encoding side, predictive encoding can be performed on the current block. During the predictive encoding process, the prediction mode of the current block can be determined, and the corresponding prediction mode parameters can be written into the bitstream, so that the prediction mode parameters can be transmitted from the encoder to the decoder.

[0093] Correspondingly, on the decoding side, the intra-frame prediction mode of the luminance or chrominance component of the current block or the coding block where the current block is located can be obtained by decoding the code stream. At this time, the value of predModeIntra (intra-frame prediction mode indicator) can be determined. The calculation formula is as follows:

[0094]

[0095] The image component indicator (which can be represented by cIdx) is used to indicate the luma component or chroma component of the current block. Here, if the current block is predicted as a luma component, cIdx is equal to 0; if the current block is predicted as a chroma component, cIdx is equal to 1. In addition, (xTbY, yTbY) are the coordinates of the upper left corner sampling point of the current block, IntraPredModeY[xTbY][yTbY] is the intra prediction mode of the luma component, and IntraPredModeC[xTbY][yTbY] is the intra prediction mode of the chroma component.

[0096] Furthermore, in the embodiment of the present application, by acquiring the prediction mode parameters, it can be determined based on the prediction mode parameters whether to use MIP to determine the intra-frame prediction value when performing intra-frame prediction.

[0097] Step 102: When the prediction mode parameter indicates that the intra-frame prediction value is determined using MIP, the code stream is decoded to determine the MIP parameters of the current block.

[0098] In an embodiment of the present application, after the prediction mode parameters are determined, if the prediction mode parameters indicate that the intra-frame prediction value is determined using MIP, the code stream may be further decoded to determine the MIP parameters of the current block.

[0099] It should be noted that, in the embodiment of the present application, the MIP parameters may include parameters such as the MIP transposition indication parameter (which can be represented by isTransposed), the MIP mode index number (which can be represented by modeId), the size of the current block, the category of the current block (which can be represented by mipSizeId), etc.; the values ​​of these parameters can be obtained by decoding the code stream.

[0100] That is, in the embodiment of the present application, the MIP parameters determined by decoding the code stream can indicate at least one of the MIP transposition indication parameter, the MIP mode index number, the size of the current block, the category of the current block, and the like.

[0101] Furthermore, in an embodiment of the present application, the value of isTransposed can be determined by decoding the code stream; when the value of isTransposed is equal to 1, it can be determined that the sampling point input vector used by the MIP mode needs to be transposed; when the value of isTransposed is equal to 0, it can be determined that the sampling point input vector used by the MIP mode does not need to be transposed; that is, the MIP transposition indication parameter isTransposed can be used to indicate whether the sampling point input vector used by the MIP mode is transposed.

[0102] Furthermore, in an embodiment of the present application, a MIP mode index number modeId can be determined by decoding the bitstream. The MIP mode index number can be used to indicate the MIP mode used by the current block, and the MIP mode can be used to indicate the calculation and derivation method for determining the intra-frame prediction block of the current block using the MIP. In other words, different MIP modes have different corresponding MIP mode index numbers. Here, the MIP mode index number can have a value of 0, 1, 2, 3, 4, or 5.

[0103] Furthermore, in the embodiments of the present application, by decoding the bitstream, parameter information such as the size, aspect ratio, and mipSizeId of the current block can also be determined. Thus, after the MIP parameters are determined, it is convenient to subsequently select the LFNST transform kernel (which can be represented by kernel) for the current block based on the determined MIP parameters.

[0104] That is, in the embodiment of the present application, the MIP parameter may determine the size parameter of the current block, wherein the size parameter may represent the size of the current block, and may be either the height and width of the current block or the aspect ratio of the current block.

[0105] Step 103: Decode the code stream to determine the transform coefficient and LFNST index number of the current block.

[0106] In an embodiment of the present application, if the prediction mode parameter indicates that the intra prediction value is determined using MIP, then after determining the MIP parameter of the current block, the code stream can be decoded to determine the transform coefficient and LFNST index number of the current block.

[0107] It should be noted that, in the embodiment of the present application, the value of the LFNST index number can be used to indicate whether the current block uses LFNST, and can also be used to indicate the index number of the LFNST transform core in the LFNST transform core candidate set.

[0108] That is, in an embodiment of the present application, after the LFNST index number is decoded, when the value of the LFNST index number is equal to 0, it indicates that the current block does not use LFNST; and when the value of the LFNST index number is greater than 0, it indicates that the current block uses LFNST. In this case, the index number of the transform core can be equal to the value of the LFNST index number, or the index number of the transform core can also be equal to the value of the LFNST index number minus 1.

[0109] Step 104: When the LFNST index number indicates that the current block uses LFNST, a mapping mode of the LFNST transform set is determined according to the MIP parameters.

[0110] In an embodiment of the present application, after determining the transform coefficient and LFNST index number of the current block, if the LFNST index number indicates that the current block uses LFNST, then the mapping mode of the LFNST transform set may be further determined according to the MIP parameter.

[0111] It should be noted that, in the embodiment of the present application, the MIP parameter may be a size parameter of the current block, wherein the size parameter may represent the size of the current block, and may be either the height and width of the current block or the aspect ratio of the current block.

[0112] That is, in embodiments of the present application, when determining the mapping mode of the LFNST transform set, the size parameter of the current block may be referenced. For example, the mapping mode of the LFNST transform set may be determined based on the height and width of the current block, or based on the aspect ratio of the current block.

[0113] Furthermore, in an embodiment of the present application, when determining the mapping mode of the LFNST transform set based on the size parameter of the current block, it can be first determined whether the size parameter satisfies a first preset condition. If the size parameter satisfies the first preset condition, the first preset prediction mode can be determined as the mapping mode of the LFNST transform set; if the size parameter does not satisfy the first preset condition, DIMD is used to determine the mapping mode of the LFNST transform set.

[0114] It should be noted that in the embodiments of the present application, the first preset condition can be used to limit the size of the current block. The first preset condition corresponds to a size parameter of the current block. If the size parameter of the current block is the height and width of the current block, the first preset condition can limit the height and width separately; if the size parameter of the current block is the aspect ratio of the current block, the first preset condition can limit the aspect ratio.

[0115] For example, in an embodiment of the present application, assuming that the size parameters of the current block are the height and width of the current block, the first preset condition can be set as the width being greater than or equal to a preset width threshold, and / or the height being greater than or equal to a preset height threshold. For example, if the height of the current block is greater than or equal to the preset height threshold, or the width of the current block is greater than or equal to the preset width threshold, then it can be determined that the size parameter meets the first preset condition; if the height of the current block is less than the preset height threshold, and the width of the current block is less than the preset width threshold, then it can be determined that the size parameter does not meet the first preset condition.

[0116] It is understandable that, in the embodiment of the present application, the preset width threshold and the preset height threshold can be any value greater than or equal to 0. For example, the preset width threshold is 32, and the preset height threshold is also 32. That is, if the height or width of the current block is greater than or equal to 32, then it can be determined that the current block meets the first preset condition. The preset width threshold is 32, and the preset height threshold is also 16. That is, if the height of the current block is greater than or equal to 32, or the width of the current block is greater than or equal to 16, then it can be determined that the current block meets the first preset condition.

[0117] It can be understood that in the embodiment of the present application, the use of DIMD can be restricted by the first preset condition, that is, the use of DIMD to determine the mapping mode of the LFNST transform set is allowed only when the size parameter of the current block does not meet the first preset condition.

[0118] As can be seen, the decoding method proposed in the embodiment of the present application can use the first preset condition to limit the size of the image blocks using DIMD when determining the mapping mode of the LFNST transform set, allowing DIMD to be used only for some image blocks, thereby effectively reducing the computational complexity. For example, DIMD can be used only for smaller image blocks to determine the mapping mode of the LFNST transform set.

[0119] Furthermore, in an embodiment of the present application, if the size parameter of the current block satisfies a first preset condition, the first preset prediction mode may be directly determined as the mapping mode of the LFNST transform set, wherein the first preset prediction mode may be a PLANAR mode or a DC mode.

[0120] That is, in embodiments of the present application, when determining the mapping mode of the LFNST transform set, in combination with the first preset condition, the mapping mode may be directly set for some image blocks. For example, for larger image blocks, the PLANAR mode or the DC mode may be directly determined as the mapping mode of the LFNST transform set.

[0121] Furthermore, in an embodiment of the present application, when using DIMD to determine the mapping mode of the LFNST transform set, at least one intra-frame prediction mode may be first traversed to determine at least one gradient information corresponding to the current block; then, the mapping mode of the LFNST transform set may be determined based on the at least one gradient information. Each intra-frame prediction mode corresponds to one gradient information, and the gradient information may be a gradient histogram.

[0122] It is understood that in the embodiments of the present application, when determining the mapping mode of the LFNST transform set based on at least one gradient information, the gradient magnitude value corresponding to each intra-frame prediction mode can be determined based on the at least one gradient information; then, the intra-frame prediction mode with the largest gradient magnitude value among the at least one intra-frame prediction mode can be determined as the mapping mode of the LFNST transform set. One intra-frame prediction mode corresponds to one gradient magnitude value.

[0123] That is to say, in an embodiment of the present application, based on the DIMD technology, it is possible to choose to use the same method as the encoding end to derive the intra-frame prediction mode at the decoding end to save bit overhead. It mainly includes two steps: the first step is to derive the prediction mode, and the same prediction mode strength calculation method is used at the codec end. For example, the Sobel operator is used to count the gradient histograms under each prediction mode, and the effective area is an L-shaped area composed of the three rows of adjacent reconstructed samples above the current block, the three columns of adjacent reconstructed samples on the left, and the corresponding adjacent reconstructed samples on the upper left. By calculating the gradient histogram of the L-shaped area, the first prediction mode corresponding to the largest amplitude in the histogram and the second prediction mode corresponding to the second largest amplitude can be obtained; the second step is to derive the prediction block, and the same prediction block derivation method is used at the codec end to obtain the current prediction block. For example, the following two conditions are judged: 1. The gradient of the second prediction mode is not 0; 2. Both the first prediction mode and the second prediction mode are not PLANAR or DC prediction modes. If neither of the above two conditions holds, the current prediction block uses only the first prediction mode to calculate the predicted sample values ​​for the current block, i.e., applying the normal prediction process to the first prediction mode. Otherwise, if both of the above conditions hold, the current prediction block is derived using a weighted averaging method. Specifically, the PLANAR mode accounts for 1 / 3 of the weighting, while the remaining 2 / 3 is weighted by the first prediction mode based on the ratio of the first prediction mode's gradient strength to the sum of the gradient strengths of the first and second prediction modes, and by the second prediction mode based on the ratio of the second prediction mode's gradient strength to the sum of the gradient strengths of the first and second prediction modes. The three prediction modes—PLANAR, the first prediction mode, and the second prediction mode—are weighted and averaged to obtain the prediction block for the current coding unit. The decoding end uses the same steps to obtain the prediction block.

[0124] Furthermore, in an embodiment of the present application, a downsampling vector of the current block can be first determined based on the MIP parameters; then, a matrix multiplication calculation is performed based on the downsampling vector to obtain a MIP output vector; then, a MIP prediction block of the current block is determined based on the MIP output vector; finally, at least one intra-frame prediction mode is traversed on the MIP prediction block to obtain at least one gradient information.

[0125] It can be understood that in the embodiment of the present application, the obtained peripheral reference reconstructed samples can be Haar down-sampled according to the size parameters of the current block, and the sampling step size is determined by the size parameters of the current block. Among them, the splicing order of the reference reconstructed samples after the upper downsampling and the reference reconstructed samples after the left downsampling is adjusted according to the MIP transposition indication parameter obtained by decoding. If transposition is not required, the reference reconstructed samples after the left downsampling are spliced ​​after the reference reconstructed samples after the upper downsampling, and the obtained vector is used as input (downsampling vector); if transposition is required, the reference reconstructed samples after the upper downsampling are spliced ​​after the reference reconstructed samples after the left downsampling, and the obtained vector is used as input (downsampling vector).

[0126] Next, the MIP matrix coefficients can be obtained based on the decoded MIP mode index number, and the output vector (MIP output vector) can be calculated with the input (downsampled vector). The output vector is then upsampled based on the number of output vectors and the size parameter of the current block. If upsampling is not required, the vectors are filled in the horizontal direction in sequence as the MIP prediction block output of the current block. If upsampling is required, the horizontal direction is first upsampled and then the vertical direction is downsampled. After upsampling to the same size as the template, it is output as the MIP prediction block of the current block.

[0127] Next, if DIMD is determined to be used for the current block, the optimal traditional intra prediction mode can be directly derived using the DIMD method for the MIP prediction block of the current block as the mapping mode of the LFNST transform set. That is, the MIP prediction block of the current block is traversed through at least one intra prediction mode, and the gradient information of at least one intra prediction mode on the MIP prediction block of the current block is calculated.

[0128] That is, in the embodiment of the present application, the DIMD calculation process can be performed after the MIP output vector is up-sampled.

[0129] Furthermore, in an embodiment of the present application, a downsampling vector of the current block can be determined based on the MIP parameters; then, matrix multiplication calculation is performed based on the downsampling vector to obtain a MIP output vector; finally, the MIP output vector is traversed through at least one intra-frame prediction mode to obtain at least one gradient information.

[0130] It can be understood that in the embodiment of the present application, the obtained peripheral reference reconstructed samples can be Haar down-sampled according to the size parameters of the current block, and the sampling step size is determined by the size parameters of the current block. Among them, the splicing order of the reference reconstructed samples after the upper downsampling and the reference reconstructed samples after the left downsampling is adjusted according to the MIP transposition indication parameter obtained by decoding. If transposition is not required, the reference reconstructed samples after the left downsampling are spliced ​​after the reference reconstructed samples after the upper downsampling, and the obtained vector is used as input (downsampling vector); if transposition is required, the reference reconstructed samples after the upper downsampling are spliced ​​after the reference reconstructed samples after the left downsampling, and the obtained vector is used as input (downsampling vector).

[0131] Next, the MIP matrix coefficients can be obtained according to the MIP mode index number obtained by decoding, and the output vector (MIP output vector) can be calculated with the input (down-sampling vector).

[0132] Next, if it is determined that DIMD is used for the current block, then the DIMD method can be used directly on the MIP output vector of the current block to derive the optimal traditional intra-frame prediction mode as the mapping mode of the LFNST transform set. That is, the MIP output vector is traversed through at least one intra-frame prediction mode, and the gradient information of at least one intra-frame prediction mode on the MIP prediction block of the current block is calculated. Then, based on the number of output vectors (MIP output vectors) and the size parameters of the current block, the output vector is upsampled. If upsampling is not required, the vector is filled in the horizontal direction in sequence as the MIP prediction block output of the current block. If upsampling is required, the horizontal direction is upsampled first and then the vertical direction is downsampled. After upsampling to the same size as the template, it is output as the MIP prediction block of the current block.

[0133] That is, in the embodiment of the present application, the DIMD calculation process may also be performed before the MIP output vector is up-sampled.

[0134] It should be noted that, in an embodiment of the present application, when using DIMD to determine the mapping mode of the LFNST transform set, you can choose to traverse all 67 intra-frame prediction modes to obtain the corresponding 67 gradient information; you can also choose to traverse a part of the intra-frame prediction modes among all 67 intra-frame prediction modes to obtain the corresponding gradient information.

[0135] That is to say, in an embodiment of the present application, in order to further reduce the complexity of the codec end, for the use of DIMD, 67 intra-frame prediction modes can be selectively skipped to reduce the number of traversed intra-frame prediction modes, for example, selective screening can be performed with a step size of 1.

[0136] Step 105 : Select an LFNST transform core candidate set from multiple LFNST transform core candidate sets according to the mapping mode of the LFNST transform set, and determine an LFNST transform core used for the current block from the selected LFNST transform core candidate set.

[0137] In an embodiment of the present application, if the LFNST index number indicates that the current block uses LFNST, then after determining the mapping mode of the LFNST transform set according to the MIP parameter, an LFNST transform core candidate set can be selected from multiple LFNST transform core candidate sets according to the mapping mode of the LFNST transform set, and then the LFNST transform core used by the current block is determined from the selected LFNST transform core candidate set.

[0138] Furthermore, in an embodiment of the present application, when determining the LFNST transform kernel used for the current block, the index number of the mapping mode of the LFNST transform set can be determined first; then, based on the value of the index number, the value of the LFNST intra-frame prediction mode index number can be determined; then, based on the value of the LFNST intra-frame prediction mode index number, an LFNST transform kernel candidate set can be selected from multiple LFNST transform kernel candidate sets; finally, the transform kernel indicated by the LFNST index number can be selected from the selected LFNST transform kernel candidate set and set as the LFNST transform kernel used for the current block.

[0139] That is, in an embodiment of the present application, after determining the mapping mode of the LFNST transform set, the index number of the mapping mode of the LFNST transform set can be further determined, and then the value of the index number of the mapping mode of the LFNST transform set can be converted into the value of the LFNST intra-frame prediction mode index number (which can be represented by predModeIntra); then, based on the value of predModeIntra, an LFNST transform core candidate set is selected from multiple LFNST transform core candidate sets to determine the transform core candidate set; and in the selected LFNST transform core candidate set, the transform core indicated by the LFNST index number is selected and set as the LFNST transform core used for the current block.

[0140] It should be noted that, in an embodiment of the present application, when determining the LFNST transform core candidate set and the corresponding LFNST transform core, the multiple LFNST transform core candidate sets may include 4 LFNST transform core candidate sets, wherein each LFNST transform core candidate set includes 2 LFNST transform cores; accordingly, the first lookup table can be used to determine the value of the LFNST intra-frame prediction mode index number corresponding to the value of the index number.

[0141] It can be understood that in the embodiment of the present application, based on the first lookup table, the DC mode, the PLANAR mode or the angle prediction mode and the transformation set of the LFNST can be bound, such as the first lookup table shown in Table 1.

[0142] It should be noted that, in an embodiment of the present application, when determining the LFNST transform core candidate set and the corresponding LFNST transform core, the multiple LFNST transform core candidate sets may also include 35 LFNST transform core candidate sets, wherein each LFNST transform core candidate set includes 3 LFNST transform cores; accordingly, the second lookup table can be used to determine the value of the LFNST intra-frame prediction mode index number corresponding to the value of the index number.

[0143] It can be understood that in the embodiment of the present application, based on the second lookup table, the LFNST transform sets corresponding to different intra prediction modes will be finer-grained, such as the second lookup table shown in FIG. 2 .

[0144] Furthermore, in an embodiment of the present application, the MIP parameters may further include a MIP transposition indication parameter, wherein the value of the MIP transposition indication parameter is used to indicate whether to perform transposition processing on the sampling point input vector used by the MIP mode. Accordingly, when the value of the MIP transposition indication parameter indicates that the sampling point input vector used by the MIP mode is to be transposed, a matrix transposition processing may be performed on the transform kernel indicated by the LFNST index number to obtain the LFNST transform kernel used by the current block.

[0145] It can be understood that in an embodiment of the present application, when the value of the MIP transposition indication parameter is equal to 1, it can be considered that the value of the MIP transposition indication parameter indicates that the sampling point input vector used by the MIP mode is transposed. At this time, it is necessary to perform corresponding matrix transposition processing on the selected transform kernel, so as to obtain the LFNST transform kernel used by the current block.

[0146] Step 106: Use the LFNST transformation kernel to transform the transformation coefficients.

[0147] In an embodiment of the present application, after selecting an LFNST transform kernel candidate set from multiple LFNST transform kernel candidate sets according to the mapping mode of the LFNST transform set, and determining the LFNST transform kernel used for the current block from the selected LFNST transform kernel candidate set, the LFNST transform kernel can be used to perform transform processing on the transform coefficients.

[0148] Furthermore, in an embodiment of the present application, the LFNST transform kernel determined from the selected LFNST transform kernel candidate set is the LFNST transform kernel used for the current block. The LFNST transform kernel can be a transform matrix for transforming transform coefficients. Furthermore, a secondary transform coefficient vector can be used as input and multiplied with the transform matrix (transform kernel) to obtain a primary transform coefficient vector. Thus, after matrix calculation, the transform coefficients can be transformed.

[0149] For example, in one possible embodiment, at the decoding end, the decoder can decode the coding unit level type flag. If it indicates intra mode, the decoder obtains the MIP enable flag (prediction mode parameter). This flag can be a sequence level flag that indicates whether the current decoder allows the use of MIP technology. The sequence level flag can be expressed in the form of sps_mip_enable_flag.

[0150] Next, if the MIP usage flag is true, the MIP usage flag of the current coding unit (current block) is decoded. Otherwise, the current decoding process does not need to decode the MIP usage flag at the coding unit level, and the MIP usage flag at the coding unit level defaults to false.

[0151] If the MIP usage flag of the current coding unit is true, decoding is performed to obtain the MIP parameters of the current coding unit, where the MIP parameters may include at least one of the following information: a MIP transposition indication parameter, a MIP mode index number, the size of the current block, the category of the current block, etc. Otherwise, decoding is continued to determine the usage flags or indices of other intra-frame prediction techniques, and the final prediction block of the current coding unit is obtained based on the decoded information.

[0152] After decoding and obtaining the MIP parameters, the obtained peripheral reference reconstructed samples can be Haar down-sampled according to the current coding unit size (the size parameter of the current block). The sampling step size is determined according to the coding unit size. At the same time, the splicing order of the reference reconstructed samples after the upper down-sampling and the reference reconstructed samples after the left down-sampling can be adjusted in combination with the MIP transposition indication parameter obtained by decoding. If transposition is not required, the reference reconstructed samples after the left down-sampling are spliced ​​after the reference reconstructed samples after the upper down-sampling, and the resulting vector is used as input (down-sampling vector); if transposition is required, the reference reconstructed samples after the upper down-sampling are spliced ​​after the reference reconstructed samples after the left down-sampling, and the resulting vector is used as input (down-sampling vector).

[0153] Next, the MIP matrix coefficients are obtained based on the decoded MIP prediction mode and calculated with the input (downsampled vector) to obtain the output vector (MIP output vector). The output vector is then upsampled based on the number of output vectors and the current coding unit size. If upsampling is not required, the vectors are padded horizontally and output as the current coding unit prediction block (the MIP prediction block of the current block). If upsampling is required, the vectors are first upsampled horizontally and then downsampled vertically. After upsampling to the same size as the template, the vectors are output as the current coding unit prediction block (the MIP prediction block of the current block).

[0154] It should be noted that if the size parameter of the current block meets the first preset condition, the first preset prediction mode is directly determined as the mapping mode of the LFNST transform set. For example, if the width and height of the current coding unit are both greater than or equal to 32, the PLANAR mode (the first preset prediction mode) can be used as the mapping mode of the LFNST transform set. If the size parameter of the current block meets the first preset condition, the optimal traditional intra-frame prediction mode derived from the current MIP prediction block using the DIMD method can be selected as the mapping mode of the LFNST transform set.

[0155] Furthermore, when using the DIMD method to derive the mapping mode of the traditional intra-frame prediction mode to the LFNST transform set, the current MIP prediction block can be traversed through 67 intra-frame prediction modes in the current VVC and ECM (or some intra-frame prediction modes), and the gradient information of each intra-frame prediction mode on the current MIP prediction block is calculated. Then, the corresponding gradient amplitude value is determined based on the gradient information, and the traversed intra-frame prediction modes are sorted according to the gradient amplitude values. The intra-frame prediction mode with the largest amplitude is the optimal mode, that is, the mapping mode of the LFNST transform set used for the inverse transformation process in the subsequent step.

[0156] After determining the mapping mode of the LFNST transform set, the user identification bits or indexes of other intra-frame prediction technologies can be decoded, and the final prediction block of the current coding unit can be obtained based on the decoded information. Furthermore, the bitstream can be decoded to obtain residual information, and the time domain residual information can be obtained through inverse quantization and inverse transformation. The final prediction block and the time domain residual information are superimposed to obtain a reconstructed sample block. After all reconstructed sample blocks are subjected to loop filtering and other technologies, the final reconstructed image is obtained, which can be used as both video output and as a reference for subsequent decoding.

[0157] For example, in another possible embodiment, at the decoding end, the decoder can decode the coding unit level type flag. If it indicates intra mode, the decoder obtains the MIP enable flag (prediction mode parameter). This flag can be a sequence level flag, which is used to indicate whether the current decoder allows the use of MIP technology. The sequence level flag can be expressed in the form of sps_mip_enable_flag.

[0158] Next, if the MIP usage flag is true, the MIP usage flag of the current coding unit (current block) is decoded. Otherwise, the current decoding process does not need to decode the MIP usage flag at the coding unit level, and the MIP usage flag at the coding unit level defaults to false.

[0159] If the MIP usage flag of the current coding unit is true, decoding is performed to obtain the MIP parameters of the current coding unit, where the MIP parameters may include at least one of the following information: a MIP transposition indication parameter, a MIP mode index number, the size of the current block, the category of the current block, etc. Otherwise, decoding is continued to determine the usage flags or indices of other intra-frame prediction techniques, and the final prediction block of the current coding unit is obtained based on the decoded information.

[0160] After decoding and obtaining the MIP parameters, the obtained peripheral reference reconstructed samples can be Haar down-sampled according to the current coding unit size (the size parameter of the current block). The sampling step size is determined according to the coding unit size. At the same time, the splicing order of the reference reconstructed samples after the upper down-sampling and the reference reconstructed samples after the left down-sampling can be adjusted in combination with the MIP transposition indication parameter obtained by decoding. If transposition is not required, the reference reconstructed samples after the left down-sampling are spliced ​​after the reference reconstructed samples after the upper down-sampling, and the resulting vector is used as input (down-sampling vector); if transposition is required, the reference reconstructed samples after the upper down-sampling are spliced ​​after the reference reconstructed samples after the left down-sampling, and the resulting vector is used as input (down-sampling vector).

[0161] Next, the MIP matrix coefficients are obtained according to the decoded MIP prediction mode, and the output vector (MIP output vector) is calculated with the input (downsampled vector).

[0162] It should be noted that if the size parameter of the current block meets the first preset condition, the first preset prediction mode is directly determined as the mapping mode of the LFNST transform set. For example, if the width and height of the current coding unit are both greater than or equal to 32, then the PLANAR mode (the first preset prediction mode) can be used as the mapping mode of the LFNST transform set. If the size parameter of the current block meets the first preset condition, the optimal traditional intra-frame prediction mode derived from the current MIP output vector using the DIMD method can be selected as the mapping mode of the LFNST transform set.

[0163] Furthermore, when using the DIMD method to derive the mapping mode of the traditional intra-frame prediction mode to the LFNST transform set, the gradient information of the current MIP output vector (MIP output vector) in the 67 intra-frame prediction modes in the current VVC and ECM (or some intra-frame prediction modes) can be calculated for each intra-frame prediction mode on the current MIP output vector, and then the corresponding gradient amplitude value is determined based on the gradient information. Then, the traversed intra-frame prediction modes are sorted according to the gradient amplitude values. The intra-frame prediction mode with the largest amplitude is the optimal mode, that is, the mapping mode of the LFNST transform set used as the inverse transformation process in the subsequent steps.

[0164] Furthermore, the output vectors can be upsampled based on the number of output vectors and the current coding unit size. If upsampling is not required, the vectors are padded horizontally and output as the current coding unit prediction block (the MIP prediction block of the current block). If upsampling is required, the vectors are first upsampled horizontally and then downsampled vertically. After upsampling to the same size as the template, they are output as the current coding unit prediction block (the MIP prediction block of the current block).

[0165] After determining the mapping mode of the LFNST transform set, the user identification bits or indexes of other intra-frame prediction technologies can be decoded, and the final prediction block of the current coding unit can be obtained based on the decoded information. Furthermore, the bitstream can be decoded to obtain residual information, and the time domain residual information can be obtained through inverse quantization and inverse transformation. The final prediction block and the time domain residual information are superimposed to obtain a reconstructed sample block. After all reconstructed sample blocks are subjected to loop filtering and other technologies, the final reconstructed image is obtained, which can be used as both video output and as a reference for subsequent decoding.

[0166] It should be noted that the encoding and decoding method proposed in the embodiment of the present application is applicable to the intra-frame prediction part of the encoding and decoding end. After the solution proposed in the embodiment of the present application is integrated into the JVET-Z008, the test results under the general test condition AI are as follows Table 2 and Table 3:

[0167] Table 2

[0168]

[0169]

[0170] Table 3

[0171]

[0172] Using the solution proposed in the embodiment of this application, anchor and test were run simultaneously on class D, providing relatively accurate results, as shown in Tables 4 and 5 below:

[0173] Table 4

[0174]

[0175] Table 5

[0176]

[0177] The test results of the solution proposed in the embodiment of this application (JVET-Z0048 + proposed method) with ECM4.0 as the anchor are shown in Table 6 below:

[0178] Table 6

[0179]

[0180] The above test results show that the decoding method proposed in this embodiment reduces the software and hardware complexity of the JVET-Z0048 solution while maintaining similar performance, with no performance change in the luminance component. Compared with ECM4.0, it maintains the same performance as JVET-Z0048.

[0181] Furthermore, in an embodiment of the present application, taking into account the different requirements of the hardware decoder for I frames and B frames, the decoding method proposed in the embodiment of the present application can be used only on B frames, or can be used on both I frames and B frames. Alternatively, the decoding method proposed in the embodiment of the present application can also be used only on I frames. Alternatively, the conditions under which the decoding method proposed in the embodiment of the present application allows use are different on B frames or I frames. For example, I frames allow coding units of all sizes to use the decoding method proposed in the embodiment of the present application, while B frames only allow coding units of small sizes to use the decoding method proposed in the embodiment of the present application.

[0182] Furthermore, in an embodiment of the present application, if the luminance component of the current coding unit (current block) uses the MIP prediction mode, and the chrominance component does not use the MIP prediction mode and does not use the traditional intra-frame prediction mode, the LFNST transform set of the chrominance component can inherit the LFNST transform set of the luminance component.

[0183] Furthermore, in an embodiment of the present application, if the current coding unit (current block) does not use the traditional intra-frame prediction mode, then the LFNST transform set of the luminance and chrominance components can be solved according to the decoding method proposed in the embodiment of the present application.

[0184] In summary, the decoding method proposed in the embodiment of the present application involves a method for deriving a MIP prediction block mapping LFNST transform set using DIMD. On the one hand, it is proposed to limit the size of the coding unit used by DIMD. For larger image blocks, the MIP output vector is more upsampled and the directional information is not obvious. Therefore, the process of deriving the traditional prediction mode by DIMD is skipped to reduce computational complexity. On the other hand, it is proposed to further reduce computational complexity based on limiting the size of the coding unit used by DIMD, using the MIP output vector before upsampling as the input of DIMD and deriving the optimal traditional intra-frame prediction mode.

[0185] The embodiment of the present application provides a decoding method, at the decoding end, decoding a bitstream and determining a prediction mode parameter; when the prediction mode parameter indicates that a MIP is used to determine an intra-frame prediction value, decoding the bitstream and determining the MIP parameter of the current block; decoding the bitstream and determining the transform coefficient and LFNST index number of the current block; when the LFNST index number indicates that the current block uses LFNST, determining the mapping mode of the LFNST transform set according to the MIP parameter; selecting an LFNST transform core candidate set from multiple LFNST transform core candidate sets according to the mapping mode of the LFNST transform set, and determining the LFNST transform core used for the current block from the selected LFNST transform core candidate set; using the LFNST transform core, transforming the transform coefficient. It can be seen that in the embodiment of the present application, when deriving a prediction block, the mapping mode of the LFNST transform set is determined according to the size parameter in the MIP parameter of the current block, wherein, for image blocks of larger sizes, it is possible to choose not to use the DIMD derivation mapping mode, which can reduce computational complexity and thus improve coding efficiency.

[0186] Based on the above embodiments, an encoding method is proposed in an embodiment of the present application. FIG7 is a schematic diagram of an implementation flow of the encoding method proposed in an embodiment of the present application. As shown in FIG7 , the encoding method of the encoder may include the following steps:

[0187] Step 201: Determine prediction model parameters.

[0188] In an embodiment of the present application, the encoder may first determine prediction mode parameters.

[0189] It should be noted that, in the embodiments of the present application, a video image may be divided into multiple image blocks, and each image block currently to be encoded may be referred to as a coding block (CB). Here, each coding block may include a first image component, a second image component, and a third image component; and the current block is a coding block in the video image for which prediction of the first image component, the second image component, or the third image component is currently to be performed.

[0190] Here, assuming that the current block performs a first image component prediction, and the first image component is a luminance component, that is, the image component to be predicted is a luminance component, then the current block can also be called a luminance block; or, assuming that the current block performs a second image component prediction, and the second image component is a chrominance component, that is, the image component to be predicted is a chrominance component, then the current block can also be called a chrominance block.

[0191] It should be noted that the prediction mode parameters indicate the coding mode of the current block and parameters related to the mode. Usually, the prediction mode parameters of the current block can be determined by using Rate Distortion Optimization (RDO).

[0192] Exemplarily, in an embodiment of the present application, when determining the prediction mode parameters of the current block, the image components to be predicted of the current block can be determined first; then, based on the parameters of the current block, multiple prediction modes are used to perform prediction encoding on the image components to be predicted respectively, and the rate-distortion cost results corresponding to each prediction mode under the multiple prediction modes are calculated; finally, the minimum rate-distortion cost result can be selected from the multiple rate-distortion cost results calculated, and the prediction mode corresponding to the minimum rate-distortion cost result is determined as the prediction mode parameter of the current block.

[0193] That is to say, in an embodiment of the present application, on the encoding side, multiple prediction modes can be used for the current block to encode the image components to be predicted respectively. Here, the multiple prediction modes generally include traditional intra-frame prediction modes and non-traditional intra-frame prediction modes, and the traditional intra-frame prediction modes can include direct current (DC) mode, planar (PLANAR) mode and angle mode, etc., and the non-traditional intra-frame prediction modes can include MIP mode, cross-component linear model prediction (CCLM) mode, intra-block copy (IBC) mode and PLT (Palette) mode, etc.

[0194] In this way, after encoding the current block using multiple prediction modes respectively, the rate-distortion cost results corresponding to each prediction mode can be obtained; then the minimum rate-distortion cost result is selected from the multiple rate-distortion cost results obtained, and the prediction mode corresponding to the minimum rate-distortion cost result is determined as the prediction mode parameter of the current block; in this way, the current block can finally be encoded using the determined prediction mode, and under this prediction mode, the prediction residual can be made small, which can improve the coding efficiency.

[0195] It can be understood that in the embodiments of the present application, on the encoding side, predictive encoding can be performed on the current block. During the predictive encoding process, the prediction mode of the current block can be determined, and the corresponding prediction mode parameters can be written into the bitstream, so that the prediction mode parameters can be transmitted from the encoder to the decoder.

[0196] Correspondingly, on the decoding side, the intra-frame prediction mode of the luminance or chrominance component of the current block or the coding block where the current block is located can be obtained by decoding the code stream, and the value of predModeIntra (intra-frame prediction mode indicator) can be determined at this time.

[0197] Furthermore, in the embodiment of the present application, by acquiring the prediction mode parameters, it can be determined based on the prediction mode parameters whether to use MIP to determine the intra-frame prediction value when performing intra-frame prediction.

[0198] Step 202: When the prediction mode parameter indicates that the intra prediction value is determined using MIP, determine the MIP parameters of the current block.

[0199] In an embodiment of the present application, after the prediction mode parameters are determined, if the prediction mode parameters indicate that an intra-frame prediction value is determined using MIP, then the MIP parameters of the current block may be determined.

[0200] It should be noted that, in an embodiment of the present application, the MIP parameters may include a MIP transposition indication parameter (which can be represented by isTransposed), a MIP mode index number (which can be represented by modeId), the size of the current block, the category of the current block (which can be represented by mipSizeId), and other parameters.

[0201] That is, in the embodiment of the present application, at least one of the MIP transposition indication parameter, the MIP mode index number, the size of the current block, the category of the current block, etc. can be indicated through the determined MIP parameters.

[0202] Furthermore, in an embodiment of the present application, the MIP parameters may include a MIP transposition indication parameter (which may be represented by isTransposed); here, the value of the MIP transposition indication parameter is used to indicate whether to perform transposition processing on the sampling point input vector used by the MIP mode.

[0203] Specifically, in MIP mode, based on the reference sample values ​​corresponding to the adjacent reference pixels on the left side of the current block and the reference sample values ​​corresponding to the adjacent reference pixels on the top side, a set of adjacent reference samples can be obtained. Thus, after obtaining the adjacent reference sample sets, an input reference sample value set can be constructed, i.e., the sampling point input vector used by the MIP mode. However, the construction of the input reference sample value set differs between the encoding and decoding sides, primarily due to the value of the MIP transposition indicator parameter.

[0204] When applied to the encoding side, the value of the MIP transposition indication parameter can still be determined by utilizing rate-distortion optimization. Specifically, it can include:

[0205] Calculate the first generation value with transposition processing and the second generation value without transposition processing respectively;

[0206] If the first generation value is less than the second generation value, then the value of the MIP transposition indicator parameter can be determined to be 1;

[0207] If the first generation value is not less than the second generation value, then the value of the MIP transposition indicator parameter can be determined to be 0.

[0208] Furthermore, when the value of the MIP transposition indication parameter is 0, the reference sample value corresponding to the upper side of the adjacent reference sample set can be stored before the reference sample value corresponding to the left side in the buffer. In this case, no transposition processing is required, that is, the sampling point input vector used by the MIP mode does not need to be transposed, and the buffer can be directly determined as the input reference sample value set; when the value of the MIP transposition indication parameter is 1, the reference sample value corresponding to the upper side of the adjacent reference sample set can be stored after the reference sample value corresponding to the left side in the buffer. In this case, the buffer is transposed, that is, the sampling point input vector used by the MIP mode needs to be transposed, and then the transposed buffer is determined as the input reference sample value set. In this way, after obtaining the input reference sample value set, it can be used in the process of determining the intra-frame prediction value corresponding to the current block in the MIP mode.

[0209] It should also be noted that, on the encoding side, after determining the value of the MIP transposition indication parameter, the determined value of the MIP transposition indication parameter needs to be written into the bitstream to facilitate subsequent decoding processing on the decoder side.

[0210] Furthermore, in an embodiment of the present application, the MIP parameter may also include a MIP mode index number (which may be represented by modeId), wherein the MIP mode index number is used to indicate the MIP mode used by the current block, and the MIP mode is used to indicate the calculation and derivation method for determining the intra-frame prediction block of the current block using the MIP. That is, in the MIP mode, since there are many types of MIP modes, these multiple MIP modes can be distinguished by the MIP mode index number, that is, different MIP modes have different MIP mode index numbers; in this way, according to the calculation and derivation method for determining the intra-frame prediction block of the current block using the MIP, the specific MIP mode can be determined, and thus the corresponding MIP mode index number can be obtained; in the embodiment of the present application, the value of the MIP mode index number can be 0, 1, 2, 3, 4 or 5.

[0211] Furthermore, in an embodiment of the present application, the MIP parameters may also include parameters such as the size and aspect ratio of the current block; wherein, based on the size of the current block (i.e., the width and height of the current block), the category of the current block (which can be represented by mipSizeId) may also be determined.

[0212] For example, if the width and height of the current block are both equal to 4, then the value of mipSizeId can be set to 0; conversely, if one of the width and height of the current block is equal to 4, or the width and height of the current block are both equal to 8, then the value of mipSizeId can be set to 1; conversely, if the current block is a block of other sizes, then the value of mipSizeId can be set to 2.

[0213] For example, if the width and height of the current block are both equal to 4, then the value of mipSizeId can be set to 0; conversely, if one of the width and height of the current block is equal to 4, then the value of mipSizeId can be set to 1; conversely, if the current block is a block of another size, then the value of mipSizeId can be set to 2.

[0214] In this way, in the process of using MIP to determine the intra-frame prediction value, MIP parameters can also be determined, so as to facilitate determining the LFNST transformation kernel (which can be represented by kernel) used for the current block according to the determined MIP parameters.

[0215] That is, in the embodiment of the present application, the MIP parameter may determine the size parameter of the current block, wherein the size parameter may represent the size of the current block, and may be either the height and width of the current block or the aspect ratio of the current block.

[0216] Step 203: Determine the intra prediction block of the current block according to the MIP parameters, and calculate the residual block between the current block and the intra prediction value.

[0217] In an embodiment of the present application, if the prediction mode parameters indicate the use of MIP to determine the intra-frame prediction value, then after determining the MIP parameters of the current block, the intra-frame prediction block of the current block can be further determined based on the MIP parameters, and the residual block between the current block and the intra-frame prediction value can be calculated.

[0218] It should be noted that, in the embodiments of the present application, for the MIP mode, the input data of the MIP prediction include: the position of the current block (xTbCmp, yTbCmp), the MIP prediction mode applied to the current block (which can be represented by modeId), the height of the current block (represented by nTbH), the width of the current block (represented by nTbW), and a transposition processing indication flag indicating whether transposition is required (which can be represented by isTransposed), etc.; the output data of the MIP prediction include: the prediction block of the current block, the intra-frame prediction value corresponding to the pixel coordinates [x][y] in the prediction block is predSamples[x][y]; where x=0, 1,…, nTbW-1; y=0, 1,…, nTbH-1.

[0219] It can be understood that in the embodiment of the present application, the MIP prediction process can be divided into four steps: configuring core parameters, obtaining reference pixels, constructing input samples, and generating predicted values. Among them, for configuring core parameters, the current block can be divided into three categories according to the size of the current block in the frame, and mipSizeId is used to record the type of the current block; and the number of reference sampling points and the number of matrix multiplication output sampling points are different for different types of current blocks. For obtaining reference pixels, when predicting the current block, the upper block and the left block of the current block are both encoded blocks. The reference pixels of the MIP technology are the reconstructed values ​​of the upper row of pixels and the left column of pixels of the current block. The process of obtaining the reference pixels adjacent to the upper side of the current block (represented by refT) and the reference pixels adjacent to the left side (represented by refL) is the reference pixel acquisition process. For constructing input samples, this step is used for the input of matrix multiplication, and can mainly include: obtaining reference samples, constructing a reference sample buffer, and deriving matrix multiplication input samples; wherein, the process of obtaining reference samples is a downsampling process, and constructing a reference sample buffer can include a buffer filling method when transposition is not required and a buffer filling method when transposition is required. For generating prediction values, this step is used to obtain the MIP prediction value of the current block, and can mainly include: constructing a matrix multiplication output sample block, matrix multiplication output sample embedding, matrix multiplication output sample transposition, and generating the MIP final prediction value; wherein, constructing a matrix multiplication output sample block can include obtaining a weight matrix, obtaining a shift factor and an offset factor, and a matrix multiplication operation, and generating the MIP final prediction value can include generating a prediction value that does not require upsampling and generating a prediction value that requires upsampling. In this way, after these four steps, the intra-frame prediction block of the current block can be obtained.

[0220] It can be understood that in an embodiment of the present application, after determining the intra-frame prediction block of the current block, a difference calculation can be performed based on the actual pixel value of the current block and the intra-frame prediction value, and the calculated difference is used as a residual block to facilitate subsequent transformation processing of the residual block.

[0221] Step 204: When the current block uses LFNST, determine the mapping mode of the LFNST transform set according to the MIP parameters.

[0222] In the embodiment of the present application, after determining that the current block uses LFNST, the mapping mode of the LFNST transform set may be further determined according to the MIP parameters.

[0223] It should be noted that in the embodiments of the present application, not all current blocks can be subjected to LFNST. In one possible embodiment, LFNST can be performed on the current block only when the current block satisfies the following conditions at the same time. These conditions include: (a) the width and height of the current block are both greater than or equal to 4; (b) the width and height of the current block are both less than or equal to the maximum size of the transform block; (c) the prediction mode of the current block or the current coding block is an intra-frame prediction mode; (d) the single transform of the current block is a two-dimensional forward single transform (DCT2) in both the horizontal and vertical directions; (e) the intra-frame prediction mode of the current block or the coding block where the current block is located is a non-MIP mode or the prediction mode of the transform block is a MIP mode and the width and height of the transform block are both greater than or equal to 16.

[0224] Furthermore, when it is determined that the current block can perform LFNST, it is also necessary to determine the LFNST transformation kernel (which can be represented by kernel) used by the current block.

[0225] It should be noted that, in the embodiment of the present application, the MIP parameter may be a size parameter of the current block, wherein the size parameter may represent the size of the current block, and may be either the height and width of the current block or the aspect ratio of the current block.

[0226] That is, in embodiments of the present application, when determining the mapping mode of the LFNST transform set, the size parameter of the current block may be referenced. For example, the mapping mode of the LFNST transform set may be determined based on the height and width of the current block, or based on the aspect ratio of the current block.

[0227] Furthermore, in an embodiment of the present application, when determining the mapping mode of the LFNST transform set based on the size parameter of the current block, it can be first determined whether the size parameter satisfies a first preset condition. If the size parameter satisfies the first preset condition, the first preset prediction mode can be determined as the mapping mode of the LFNST transform set; if the size parameter does not satisfy the first preset condition, DIMD is used to determine the mapping mode of the LFNST transform set.

[0228] It should be noted that in the embodiments of the present application, the first preset condition can be used to limit the size of the current block. The first preset condition corresponds to a size parameter of the current block. If the size parameter of the current block is the height and width of the current block, the first preset condition can limit the height and width separately; if the size parameter of the current block is the aspect ratio of the current block, the first preset condition can limit the aspect ratio.

[0229] For example, in an embodiment of the present application, assuming that the size parameters of the current block are the height and width of the current block, the first preset condition can be set as the width being greater than or equal to a preset width threshold, and / or the height being greater than or equal to a preset height threshold. For example, if the height of the current block is greater than or equal to the preset height threshold, or the width of the current block is greater than or equal to the preset width threshold, then it can be determined that the size parameter meets the first preset condition; if the height of the current block is less than the preset height threshold, and the width of the current block is less than the preset width threshold, then it can be determined that the size parameter does not meet the first preset condition.

[0230] It is understandable that, in the embodiment of the present application, the preset width threshold and the preset height threshold can be any value greater than or equal to 0. For example, the preset width threshold is 32, and the preset height threshold is also 32. That is, if the height or width of the current block is greater than or equal to 32, then it can be determined that the current block meets the first preset condition. The preset width threshold is 32, and the preset height threshold is also 16. That is, if the height of the current block is greater than or equal to 32, or the width of the current block is greater than or equal to 16, then it can be determined that the current block meets the first preset condition.

[0231] It can be understood that in the embodiment of the present application, the use of DIMD can be restricted by the first preset condition, that is, the use of DIMD to determine the mapping mode of the LFNST transform set is allowed only when the size parameter of the current block does not meet the first preset condition.

[0232] As can be seen, the encoding method proposed in the embodiment of the present application can use the first preset condition to limit the size of the image blocks using DIMD when determining the mapping mode of the LFNST transform set, allowing DIMD to be used only for some image blocks, thereby effectively reducing the computational complexity. For example, DIMD can be used only for smaller image blocks to determine the mapping mode of the LFNST transform set.

[0233] Furthermore, in an embodiment of the present application, if the size parameter of the current block satisfies a first preset condition, the first preset prediction mode may be directly determined as the mapping mode of the LFNST transform set, wherein the first preset prediction mode may be a PLANAR mode or a DC mode.

[0234] That is, in embodiments of the present application, when determining the mapping mode of the LFNST transform set, in combination with the first preset condition, the mapping mode may be directly set for some image blocks. For example, for larger image blocks, the PLANAR mode or the DC mode may be directly determined as the mapping mode of the LFNST transform set.

[0235] Furthermore, in an embodiment of the present application, when using DIMD to determine the mapping mode of the LFNST transform set, at least one intra-frame prediction mode may be first traversed to determine at least one gradient information corresponding to the current block; then, the mapping mode of the LFNST transform set may be determined based on the at least one gradient information. Each intra-frame prediction mode corresponds to one gradient information, and the gradient information may be a gradient histogram.

[0236] It is understood that in the embodiments of the present application, when determining the mapping mode of the LFNST transform set based on at least one gradient information, the gradient magnitude value corresponding to each intra-frame prediction mode can be determined based on the at least one gradient information; then, the intra-frame prediction mode with the largest gradient magnitude value among the at least one intra-frame prediction mode can be determined as the mapping mode of the LFNST transform set. One intra-frame prediction mode corresponds to one gradient magnitude value.

[0237] That is to say, in an embodiment of the present application, based on the DIMD technology, it is possible to choose to use the same method as the encoding end to derive the intra-frame prediction mode at the decoding end to save bit overhead. It mainly includes two steps: the first step is to derive the prediction mode, and the same prediction mode strength calculation method is used at the codec end. For example, the Sobel operator is used to count the gradient histograms under each prediction mode, and the effective area is an L-shaped area composed of the three rows of adjacent reconstructed samples above the current block, the three columns of adjacent reconstructed samples on the left, and the corresponding adjacent reconstructed samples on the upper left. By calculating the gradient histogram of the L-shaped area, the first prediction mode corresponding to the largest amplitude in the histogram and the second prediction mode corresponding to the second largest amplitude can be obtained; the second step is to derive the prediction block, and the same prediction block derivation method is used at the codec end to obtain the current prediction block. For example, the following two conditions are judged: 1. The gradient of the second prediction mode is not 0; 2. Both the first prediction mode and the second prediction mode are not PLANAR or DC prediction modes. If neither of the above two conditions holds, the current prediction block uses only the first prediction mode to calculate the predicted sample values ​​for the current block, i.e., applying the normal prediction process to the first prediction mode. Otherwise, if both of the above conditions hold, the current prediction block is derived using a weighted averaging method. Specifically, the PLANAR mode accounts for 1 / 3 of the weighting, while the remaining 2 / 3 is weighted by the first prediction mode based on the ratio of the first prediction mode's gradient strength to the sum of the gradient strengths of the first and second prediction modes, and by the second prediction mode based on the ratio of the second prediction mode's gradient strength to the sum of the gradient strengths of the first and second prediction modes. The three prediction modes—PLANAR, the first prediction mode, and the second prediction mode—are weighted and averaged to obtain the prediction block for the current coding unit. The decoding end uses the same steps to obtain the prediction block.

[0238] Furthermore, in an embodiment of the present application, a downsampling vector of the current block can be first determined based on the MIP parameters; then, a matrix multiplication calculation is performed based on the downsampling vector to obtain a MIP output vector; then, a MIP prediction block of the current block is determined based on the MIP output vector; finally, at least one intra-frame prediction mode is traversed on the MIP prediction block to obtain at least one gradient information.

[0239] It can be understood that in the embodiment of the present application, the obtained peripheral reference reconstructed samples can be Haar down-sampled according to the size parameters of the current block, and the sampling step size is determined by the size parameters of the current block. Among them, the splicing order of the reference reconstructed samples after the upper downsampling and the reference reconstructed samples after the left downsampling is adjusted according to the MIP transposition indication parameter. If transposition is not required, the reference reconstructed samples after the left downsampling are spliced ​​after the reference reconstructed samples after the upper downsampling, and the obtained vector is used as input (downsampling vector); if transposition is required, the reference reconstructed samples after the upper downsampling are spliced ​​after the reference reconstructed samples after the left downsampling, and the obtained vector is used as input (downsampling vector).

[0240] Next, the MIP matrix coefficients can be obtained using the traversed prediction mode as an index, and the output vector (MIP output vector) can be calculated with the input (downsampled vector). The output vector is then upsampled based on the number of output vectors and the size parameter of the current block. If upsampling is not required, the vectors are filled in the horizontal direction in sequence as the MIP prediction block output of the current block. If upsampling is required, the horizontal direction is first upsampled and then the vertical direction is downsampled. After upsampling to the same size as the template, it is output as the MIP prediction block of the current block.

[0241] Next, if DIMD is determined to be used for the current block, the optimal traditional intra prediction mode can be directly derived using the DIMD method for the MIP prediction block of the current block as the mapping mode of the LFNST transform set. That is, the MIP prediction block of the current block is traversed through at least one intra prediction mode, and the gradient information of at least one intra prediction mode on the MIP prediction block of the current block is calculated.

[0242] That is, in the embodiment of the present application, the DIMD calculation process can be performed after the MIP output vector is up-sampled.

[0243] Furthermore, in an embodiment of the present application, a downsampling vector of the current block can be determined based on the MIP parameters; then, matrix multiplication calculation is performed based on the downsampling vector to obtain a MIP output vector; finally, the MIP output vector is traversed through at least one intra-frame prediction mode to obtain at least one gradient information.

[0244] It can be understood that in the embodiment of the present application, the obtained peripheral reference reconstructed samples can be Haar down-sampled according to the size parameters of the current block, and the sampling step size is determined by the size parameters of the current block. Among them, the splicing order of the reference reconstructed samples after the upper downsampling and the reference reconstructed samples after the left downsampling is adjusted according to the MIP transposition indication parameter. If transposition is not required, the reference reconstructed samples after the left downsampling are spliced ​​after the reference reconstructed samples after the upper downsampling, and the obtained vector is used as input (downsampling vector); if transposition is required, the reference reconstructed samples after the upper downsampling are spliced ​​after the reference reconstructed samples after the left downsampling, and the obtained vector is used as input (downsampling vector).

[0245] Next, the MIP matrix coefficients can be obtained according to the traversed prediction mode as an index, and the output vector (MIP output vector) can be calculated with the input (downsampled vector).

[0246] Next, if it is determined that DIMD is used for the current block, then the DIMD method can be used directly on the MIP output vector of the current block to derive the optimal traditional intra-frame prediction mode as the mapping mode of the LFNST transform set. That is, the MIP output vector is traversed through at least one intra-frame prediction mode, and the gradient information of at least one intra-frame prediction mode on the MIP prediction block of the current block is calculated. Then, based on the number of output vectors (MIP output vectors) and the size parameters of the current block, the output vector is upsampled. If upsampling is not required, the vector is filled in the horizontal direction in sequence as the MIP prediction block output of the current block. If upsampling is required, the horizontal direction is upsampled first and then the vertical direction is downsampled. After upsampling to the same size as the template, it is output as the MIP prediction block of the current block.

[0247] That is, in the embodiment of the present application, the DIMD calculation process may also be performed before the MIP output vector is up-sampled.

[0248] It should be noted that, in an embodiment of the present application, when using DIMD to determine the mapping mode of the LFNST transform set, you can choose to traverse all 67 intra-frame prediction modes to obtain the corresponding 67 gradient information; you can also choose to traverse a part of the intra-frame prediction modes among all 67 intra-frame prediction modes to obtain the corresponding gradient information.

[0249] That is to say, in an embodiment of the present application, in order to further reduce the complexity of the codec end, for the use of DIMD, 67 intra-frame prediction modes can be selectively skipped to reduce the number of traversed intra-frame prediction modes, for example, selective screening can be performed with a step size of 1.

[0250] Step 205: Select an LFNST transform core candidate set from multiple LFNST transform core candidate sets according to the mapping mode of the LFNST transform set, determine the LFNST transform core used for the current block from the selected LFNST transform core candidate set, set the LFNST index sequence number and write it into the video stream.

[0251] In an embodiment of the present application, if the current block uses LFNST, then after determining the mapping mode of the LFNST transform set according to the MIP parameters, an LFNST transform core candidate set can be selected from multiple LFNST transform core candidate sets according to the mapping mode of the LFNST transform set, and then the LFNST transform core used for the current block is determined from the selected LFNST transform core candidate set, and then the LFNST index number can be set and the LFNST index number can be written into the video code stream.

[0252] Furthermore, in an embodiment of the present application, when determining the LFNST transform kernel used for the current block, the index number of the mapping mode of the LFNST transform set can be determined first; then, based on the value of the index number, the value of the LFNST intra-frame prediction mode index number can be determined; then, based on the value of the LFNST intra-frame prediction mode index number, an LFNST transform kernel candidate set can be selected from multiple LFNST transform kernel candidate sets; finally, the LFNST transform kernel used for the current block can be selected from the selected LFNST transform kernel candidate set, and then the LFNST index number can be set and written into the video code stream.

[0253] That is, in an embodiment of the present application, after determining the mapping mode of the LFNST transform set, the index number of the mapping mode of the LFNST transform set can be further determined, and then the value of the index number of the mapping mode of the LFNST transform set is converted into the value of the LFNST intra-frame prediction mode index number (which can be represented by predModeIntra); then, based on the value of predModeIntra, an LFNST transform core candidate set is selected from multiple LFNST transform core candidate sets to determine the transform core candidate set; and from the selected LFNST transform core candidate set, the LFNST transform core used by the current block is selected.

[0254] It should be noted that, in an embodiment of the present application, when determining the LFNST transform core candidate set and the corresponding LFNST transform core, the multiple LFNST transform core candidate sets may include 4 LFNST transform core candidate sets, wherein each LFNST transform core candidate set includes 2 LFNST transform cores; accordingly, the first lookup table can be used to determine the value of the LFNST intra-frame prediction mode index number corresponding to the value of the index number.

[0255] It can be understood that in the embodiment of the present application, based on the first lookup table, the DC mode, the PLANAR mode or the angle prediction mode and the transformation set of the LFNST can be bound, such as the first lookup table shown in Table 1.

[0256] It should be noted that, in an embodiment of the present application, when determining the LFNST transform core candidate set and the corresponding LFNST transform core, the multiple LFNST transform core candidate sets may also include 35 LFNST transform core candidate sets, wherein each LFNST transform core candidate set includes 3 LFNST transform cores; accordingly, the second lookup table can be used to determine the value of the LFNST intra-frame prediction mode index number corresponding to the value of the index number.

[0257] It can be understood that in the embodiment of the present application, based on the second lookup table, the LFNST transform sets corresponding to different intra prediction modes will be finer-grained, such as the second lookup table shown in FIG. 2 .

[0258] It should be noted that, in the embodiments of the present application, the LFNST transformation kernel can be understood as the transformation matrix of LFNST, which is a plurality of fixed coefficient matrices obtained through training.

[0259] It should be noted that in the embodiment of the present application, since the LFNST transform core candidate set includes two or more transform cores preset for MIP, a rate-distortion optimization method can be used to select the transform core used for the current block. Specifically, a rate-distortion cost (RDCost) can be calculated for each transform core using a rate-distortion optimization method, and then the transform core with the smallest rate-distortion cost is selected as the transform core used for the current block.

[0260] That is, on the encoding side, a set of LFNST transform kernels can be selected through RDCost, and the index numbers corresponding to the LFNST transform kernels (which can be represented by lfnst_idx) are written into the video stream and transmitted to the decoding side. When the first set of LFNST transform kernels (i.e., the first set of transform matrices) in the LFNST transform kernel candidate set is selected, lfnst_idx is set to 1; when the second set of LFNST transform kernels (i.e., the second set of transform matrices) in the LFNST transform kernel candidate set is selected, lfnst_idx is set to 2.

[0261] It should be noted that, in the embodiment of the present application, the value of the LFNST index number can be used to indicate whether the current block uses LFNST, and can also be used to indicate the index number of the LFNST transform core in the LFNST transform core candidate set.

[0262] That is, in the embodiment of the present application, when the value of the LFNST index number (i.e., lfnst_idx) is 0, LFNST is not used; however, when the value of the LFNST index number is greater than 0, LFNST is used, and the index number of the transform core is equal to the value of the LFNST index number, or the value of the LFNST index number minus 1. In this way, the LFNST transform core used for the current block can be determined based on the LFNST index number.

[0263] Furthermore, in an embodiment of the present application, the MIP parameters may further include a MIP transposition indication parameter, wherein the value of the MIP transposition indication parameter is used to indicate whether to perform transposition processing on the sampling point input vector used by the MIP mode. Accordingly, when the value of the MIP transposition indication parameter indicates that the sampling point input vector used by the MIP mode is to be transposed, a matrix transposition processing may be performed on the selected transform kernel to obtain the LFNST transform kernel used by the current block.

[0264] It can be understood that in an embodiment of the present application, when the value of the MIP transposition indication parameter is equal to 1, it can be considered that the value of the MIP transposition indication parameter indicates that the sampling point input vector used by the MIP mode is transposed. At this time, it is necessary to perform corresponding matrix transposition processing on the selected transform kernel, so as to obtain the LFNST transform kernel used by the current block.

[0265] Step 206: Use the LFNST transformation kernel to transform the residual block.

[0266] In an embodiment of the present application, after selecting an LFNST transform kernel candidate set from multiple LFNST transform kernel candidate sets according to the mapping mode of the LFNST transform set, and determining the LFNST transform kernel used for the current block from the selected LFNST transform kernel candidate set, the LFNST transform kernel, that is, the transform matrix selected for the current block, can be used to perform transform processing on the residual block.

[0267] For example, in one possible embodiment, at the encoding end, the encoder traverses the prediction mode. If the current coding unit (current block) is in intra mode, the encoder obtains the flag bit that allows the use of the encoding and decoding method proposed in the embodiment of the present application, that is, obtains the MIP use permission flag bit (prediction mode parameter). This flag bit can be a sequence-level flag bit used to indicate whether the current decoder allows the use of MIP technology. The sequence-level flag bit can be expressed in the form of sps_mip_enable_flag.

[0268] Next, if the MIP's allowed flag is true, the encoder tries the MIP's prediction method and calculates the corresponding rate-distortion cost, which is recorded as cost1. If the MIP's allowed flag is false, the encoder does not try the MIP's prediction method, but continues to traverse other intra-frame prediction techniques and calculates the corresponding rate-distortion costs, which are recorded as cost2…costN.

[0269] Among them, if the MIP permission flag is true, the obtained peripheral reference reconstructed samples can be Haar down-sampled according to the current coding unit size (the size parameter of the current block), and the sampling step size is determined according to the coding unit size. At the same time, the splicing order of the reference reconstructed samples after the upper downsampling and the reference reconstructed samples after the left downsampling can be adjusted in combination with the MIP transposition indication parameter. Among them, if transposition is not required, the reference reconstructed samples after the left downsampling are spliced ​​after the reference reconstructed samples after the upper downsampling, and the resulting vector is used as input (downsampling vector); if transposition is required, the reference reconstructed samples after the upper downsampling are spliced ​​after the reference reconstructed samples after the left downsampling, and the resulting vector is used as input (downsampling vector).

[0270] Next, the MIP matrix coefficients are obtained using the traversed prediction mode as an index, and the output vector (MIP output vector) is calculated with the input (downsampled vector). The output vector is then upsampled based on the number of output vectors and the current coding unit size. If upsampling is not required, the vectors are padded horizontally and output as the current coding unit prediction block (the MIP prediction block of the current block). If upsampling is required, the vectors are first upsampled horizontally and then downsampled vertically. After upsampling to the same size as the template, the vectors are output as the current coding unit prediction block (the MIP prediction block of the current block).

[0271] It should be noted that if the size parameter of the current block meets the first preset condition, the first preset prediction mode is directly determined as the mapping mode of the LFNST transform set. For example, if the width and height of the current coding unit are both greater than or equal to 32, the PLANAR mode (the first preset prediction mode) can be used as the mapping mode of the LFNST transform set. If the size parameter of the current block meets the first preset condition, the optimal traditional intra-frame prediction mode derived from the current MIP prediction block using the DIMD method can be selected as the mapping mode of the LFNST transform set.

[0272] Furthermore, when using the DIMD method to derive the mapping mode of the traditional intra-frame prediction mode to the LFNST transform set, the current MIP prediction block can be traversed through 67 intra-frame prediction modes in the current VVC and ECM (or some intra-frame prediction modes), and the gradient information of each intra-frame prediction mode on the current MIP prediction block is calculated. Then, the corresponding gradient amplitude value is determined based on the gradient information, and the traversed intra-frame prediction modes are sorted according to the gradient amplitude value. The intra-frame prediction mode with the largest amplitude is the optimal mode, that is, the optimal mode is used to map the LFNST transform set of the current coding unit.

[0273] After determining the mapping mode of the LFNST transform set, the original image block of the current coding unit is subtracted from the prediction block (the MIP prediction block of the current block) to obtain the residual block of the current coding unit (current block). After the residual block undergoes the main transform, it obtains the frequency domain coefficient block. The region of interest of the frequency domain coefficient block is then subjected to a secondary transform using LFNST, where the mapping prediction mode of the LFNST transform set has been determined using the above method. After quantization, inverse quantization, and inverse transform, the rate-distortion cost of the current coding unit is calculated, denoted as cost1.

[0274] Furthermore, other intra-frame prediction technologies can be traversed and the corresponding rate-distortion costs can be calculated and recorded as cost2…costN; if cost1 is the smallest among all rate-distortion costs, the current coding unit adopts the MIP technology, and the MIP usage flag of the current coding unit and the corresponding MIP transposition flag (MIP transposition indication parameter) are set to true and written into the bitstream; if cost1 is not the minimum rate-distortion cost, the current coding unit adopts other intra-frame prediction technologies, and the MIP usage flag of the current coding unit is set to false and written into the bitstream; information such as the flags or indexes of other intra-frame prediction technologies is transmitted according to the definition.

[0275] For example, in another possible embodiment, the encoder traverses the prediction mode. If the current coding unit (current block) is in intra mode, the encoder obtains the flag that allows the use of the encoding and decoding method proposed in the embodiment of the present application, that is, obtains the MIP use permission flag (prediction mode parameter). This flag can be a sequence-level flag that indicates whether the current decoder allows the use of MIP technology. The sequence-level flag can be expressed in the form of sps_mip_enable_flag.

[0276] Next, if the MIP's allowed flag is true, the encoder tries the MIP's prediction method and calculates the corresponding rate-distortion cost, which is recorded as cost1. If the MIP's allowed flag is false, the encoder does not try the MIP's prediction method, but continues to traverse other intra-frame prediction techniques and calculates the corresponding rate-distortion costs, which are recorded as cost2…costN.

[0277] Among them, if the MIP permission flag is true, the obtained peripheral reference reconstructed samples can be Haar down-sampled according to the current coding unit size (the size parameter of the current block), and the sampling step size is determined according to the coding unit size. At the same time, the splicing order of the reference reconstructed samples after the upper downsampling and the reference reconstructed samples after the left downsampling can be adjusted in combination with the MIP transposition indication parameter. Among them, if transposition is not required, the reference reconstructed samples after the left downsampling are spliced ​​after the reference reconstructed samples after the upper downsampling, and the resulting vector is used as input (downsampling vector); if transposition is required, the reference reconstructed samples after the upper downsampling are spliced ​​after the reference reconstructed samples after the left downsampling, and the resulting vector is used as input (downsampling vector).

[0278] Next, the MIP matrix coefficients are obtained according to the traversed prediction mode as an index, and the output vector (MIP output vector) is calculated with the input (downsampled vector).

[0279] It should be noted that if the size parameter of the current block meets the first preset condition, the first preset prediction mode is directly determined as the mapping mode of the LFNST transform set. For example, if the width and height of the current coding unit are both greater than or equal to 32, then the PLANAR mode (the first preset prediction mode) can be used as the mapping mode of the LFNST transform set. If the size parameter of the current block meets the first preset condition, the optimal traditional intra-frame prediction mode derived from the current MIP output vector using the DIMD method can be selected as the mapping mode of the LFNST transform set.

[0280] Furthermore, when using the DIMD method to derive the mapping mode of the traditional intra-frame prediction mode to the LFNST transform set, the gradient information of the current MIP output vector (MIP output vector) in the 67 intra-frame prediction modes in the current VVC and ECM (or some intra-frame prediction modes) can be calculated for each intra-frame prediction mode on the current MIP output vector, and then the corresponding gradient amplitude value is determined based on the gradient information. The traversed intra-frame prediction modes are then sorted according to the gradient amplitude values. The intra-frame prediction mode with the largest amplitude is the optimal mode, that is, the optimal mode is used to map the LFNST transform set of the current coding unit.

[0281] Furthermore, the output vectors can be upsampled based on the number of output vectors and the current coding unit size. If upsampling is not required, the vectors are padded horizontally and output as the current coding unit prediction block (the MIP prediction block of the current block). If upsampling is required, the vectors are first upsampled horizontally and then downsampled vertically. After upsampling to the same size as the template, they are output as the current coding unit prediction block (the MIP prediction block of the current block).

[0282] After determining the mapping mode of the LFNST transform set, the original image block of the current coding unit is subtracted from the prediction block (the MIP prediction block of the current block) to obtain the residual block of the current coding unit (current block). After the residual block undergoes the main transform, it obtains the frequency domain coefficient block. The region of interest of the frequency domain coefficient block is then subjected to a secondary transform using LFNST, where the mapping prediction mode of the LFNST transform set has been determined using the above method. After quantization, inverse quantization, and inverse transform, the rate-distortion cost of the current coding unit is calculated, denoted as cost1.

[0283] Furthermore, other intra-frame prediction technologies can be traversed and the corresponding rate-distortion costs can be calculated and recorded as cost2…costN; if cost1 is the smallest among all rate-distortion costs, the current coding unit adopts the MIP technology, and the MIP usage flag of the current coding unit and the corresponding MIP transposition flag (MIP transposition indication parameter) are set to true and written into the bitstream; if cost1 is not the minimum rate-distortion cost, the current coding unit adopts other intra-frame prediction technologies, and the MIP usage flag of the current coding unit is set to false and written into the bitstream; information such as the flags or indexes of other intra-frame prediction technologies is transmitted according to the definition.

[0284] The encoding method proposed in the embodiment of the present application reduces the software and hardware complexity of the JVET-Z0048 solution while maintaining similar performance, with no performance change in the luminance component. Compared with ECM4.0, it maintains the same performance as JVET-Z0048.

[0285] Furthermore, in an embodiment of the present application, taking into account the different requirements of the hardware decoder for I frames and B frames, the encoding method proposed in the embodiment of the present application can be used only in B frames, or can be used on both I frames and B frames. Alternatively, the encoding method proposed in the embodiment of the present application can also be used only on I frames. Alternatively, the conditions under which the encoding method proposed in the embodiment of the present application allows use are different on B frames or I frames. For example, I frames allow encoding units of all sizes to use the encoding method proposed in the embodiment of the present application, while B frames only allow encoding units of small sizes to use the encoding method proposed in the embodiment of the present application.

[0286] Furthermore, in an embodiment of the present application, if the luminance component of the current coding unit (current block) uses the MIP prediction mode, and the chrominance component does not use the MIP prediction mode and does not use the traditional intra-frame prediction mode, the LFNST transform set of the chrominance component can inherit the LFNST transform set of the luminance component.

[0287] Furthermore, in an embodiment of the present application, if the current coding unit (current block) does not use the traditional intra-frame prediction mode, then the LFNST transform set of the luminance and chrominance components can be solved according to the encoding method proposed in the embodiment of the present application.

[0288] In summary, the encoding method proposed in the embodiment of the present application involves a method for deriving a MIP prediction block mapping LFNST transform set using DIMD. On the one hand, it is proposed to limit the size of the coding unit using DIMD. For larger image blocks, the MIP output vector is upsampled more and the directional information is not obvious. Therefore, the process of deriving the traditional prediction mode by DIMD is skipped to reduce computational complexity. On the other hand, it is proposed to further reduce computational complexity based on limiting the size of the coding unit used by DIMD, using the MIP output vector before upsampling as the input of DIMD and deriving the optimal traditional intra-frame prediction mode.

[0289] The embodiment of the present application provides an encoding method, which determines a prediction mode parameter at the encoding end; when the prediction mode parameter indicates that the current block uses MIP to determine an intra-frame prediction value, determines the MIP parameter of the current block; determines the intra-frame prediction block of the current block based on the MIP parameter, and calculates the residual block between the current block and the intra-frame prediction value; when the current block uses LFNST, determines the mapping mode of the LFNST transform set based on the MIP parameter; selects an LFNST transform core candidate set from multiple LFNST transform core candidate sets based on the mapping mode of the LFNST transform set, determines the LFNST transform core used by the current block from the selected LFNST transform core candidate set, sets the LFNST index sequence number and writes it into the video code stream; uses the LFNST transform core to transform the residual block. It can be seen that in the embodiment of the present application, when deriving the prediction block, the mapping mode of the LFNST transform set is determined based on the size parameter in the MIP parameter of the current block, wherein, for image blocks of larger sizes, it is possible to choose not to use the DIMD derivation mapping mode, which can reduce computational complexity and thus improve encoding efficiency.

[0290] Based on the above embodiment, in another embodiment of the present application, based on the same inventive concept as the above embodiment, FIG8 is a schematic diagram of the composition structure of the encoder. As shown in FIG8 , the encoder 110 may include: a first determining unit 111, an encoding unit 112, and a first transforming unit 113; wherein,

[0291] The first determining unit 111 is configured as the first determining unit and is configured to determine a prediction mode parameter; when the prediction mode parameter indicates that the current block uses MIP to determine an intra prediction value, determine the MIP parameter of the current block; determine the intra prediction block of the current block according to the MIP parameter, and calculate the residual block between the current block and the intra prediction value; when the current block uses LFNST, determine a mapping mode of an LFNST transform set according to the MIP parameter; select an LFNST transform core candidate set from multiple LFNST transform core candidate sets according to the mapping mode of the LFNST transform set, determine the LFNST transform core used by the current block from the selected LFNST transform core candidate set, and set an LFNST index number;

[0292] The encoding unit 112 is configured to write a video stream;

[0293] The first transform unit 113 is configured to use the LFNST transform kernel to perform transform processing on the residual block.

[0294] It is understood that in this embodiment, a "unit" can be a portion of a circuit, a portion of a processor, a portion of a program or software, etc., and can also be a module or a non-modular system. Furthermore, the various components in this embodiment can be integrated into a single processing unit, or each unit can exist physically separately, or two or more units can be integrated into a single unit. The aforementioned integrated units can be implemented in the form of hardware or software functional modules.

[0295] If the integrated unit is implemented as a software functional module and not sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this embodiment, or the portion that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) or a processor to perform all or part of the steps of the method described in this embodiment. The aforementioned storage medium includes various media that can store program code, such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0296] Therefore, an embodiment of the present application provides a computer-readable storage medium, which is applied to the encoder 110. The computer-readable storage medium stores a computer program, and when the computer program is executed by the first processor, it implements the method described in any one of the aforementioned embodiments.

[0297] Based on the composition of the above-mentioned encoder 110 and the computer-readable storage medium, Figure 9 is a second schematic diagram of the composition structure of the encoder. As shown in Figure 9, the encoder 110 may include: a first memory 114 and a first processor 115, a first communication interface 116 and a first bus system 117. The first memory 114, the first processor 115, and the first communication interface 116 are coupled together through the first bus system 117. It can be understood that the first bus system 117 is used to achieve connection and communication between these components. In addition to the data bus, the first bus system 117 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, various buses are labeled as the first bus system 117 in Figure 9. Among them,

[0298] The first communication interface 116 is used to receive and send signals during the process of sending and receiving information between other external network elements;

[0299] The first memory 114 is configured to store a computer program that can be run on the first processor;

[0300] The first processor 115 is configured to, when running the computer program, determine a prediction mode parameter; when the prediction mode parameter indicates that the current block uses MIP to determine an intra-frame prediction value, determine the MIP parameter of the current block; determine the intra-frame prediction block of the current block based on the MIP parameter, and calculate a residual block between the current block and the intra-frame prediction value; when the current block uses LFNST, determine a mapping mode of an LFNST transform set based on the MIP parameter; select an LFNST transform core candidate set from multiple LFNST transform core candidate sets based on the mapping mode of the LFNST transform set, determine the LFNST transform core used by the current block from the selected LFNST transform core candidate set, set an LFNST index number and write it into a video bitstream; and use the LFNST transform core to perform transform processing on the residual block.

[0301] It is understood that the first memory 114 in the embodiment of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct RAM bus random access memory (DRRAM). The first memory 114 of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0302] The first processor 115 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by hardware integrated logic circuits or software instructions in the first processor 115. The above-mentioned first processor 115 can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The various methods, steps, and logic block diagrams disclosed in the embodiments of this application can be implemented or executed. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of this application can be directly implemented and executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium mature in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in the first memory 114 , and the first processor 115 reads the information in the first memory 114 and completes the steps of the above method in combination with its hardware.

[0303] It is to be understood that these embodiments described in the present application can be implemented with hardware, software, firmware, middleware, microcode or its combination.For hardware implementation, the processing unit can be implemented in one or more application specific integrated circuits (Application Specific Integrated Circuits, ASIC), digital signal processor (Digital Signal Processing, DSP), digital signal processing equipment (DSP Device, DSPD), programmable logic device (Programmable Logic Device, PLD), field programmable gate array (Field-Programmable Gate Array, FPGA), general-purpose processor, controller, microcontroller, microprocessor, other electronic units for performing functions described in the present application or its combination.For software implementation, the technology described in the present application can be realized by the module (such as process, function etc.) that performs functions described in the present application. The software code can be stored in a memory and executed by a processor. The memory can be implemented in the processor or outside the processor.

[0304] Optionally, as another embodiment, the first processor 115 is further configured to execute the method described in any one of the aforementioned embodiments when running the computer program.

[0305] FIG10 is a schematic diagram of the first structure of the decoder. As shown in FIG10 , the decoder 120 may include: a second determining unit 121 and a second transforming unit 122; wherein,

[0306] The second determining unit 121 is configured to decode a bitstream and determine a prediction mode parameter; when the prediction mode parameter indicates that a MIP is used to determine an intra-frame prediction value, decode the bitstream and determine the MIP parameter of the current block; decode the bitstream and determine the transform coefficient and LFNST index number of the current block; when the LFNST index number indicates that LFNST is used for the current block, determine a mapping mode of an LFNST transform set according to the MIP parameter; select an LFNST transform kernel candidate set from a plurality of LFNST transform kernel candidate sets according to the mapping mode of the LFNST transform set, and determine an LFNST transform kernel used for the current block from the selected LFNST transform kernel candidate set;

[0307] The second transform unit 122 is configured to use the LFNST transform kernel to perform transform processing on the transform coefficients.

[0308] It is understood that in this embodiment, a "unit" can be a portion of a circuit, a portion of a processor, a portion of a program or software, etc., and can also be a module or a non-modular system. Furthermore, the various components in this embodiment can be integrated into a single processing unit, or each unit can exist physically separately, or two or more units can be integrated into a single unit. The aforementioned integrated units can be implemented in the form of hardware or software functional modules.

[0309] If the integrated unit is implemented as a software functional module and is not sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this embodiment, or the portion that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute all or part of the steps of the method described in this embodiment. The aforementioned storage medium includes various media that can store program code, such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0310] Therefore, an embodiment of the present application provides a computer-readable storage medium, which is applied to the decoder 120. The computer-readable storage medium stores a computer program, and when the computer program is executed by the first processor, it implements the method described in any one of the aforementioned embodiments.

[0311] Based on the composition of the above-mentioned decoder 120 and the computer-readable storage medium, Figure 11 is a second schematic diagram of the composition structure of the decoder. As shown in Figure 11, the decoder 120 may include: a second memory 123 and a second processor 124, a second communication interface 125 and a second bus system 126. The second memory 123 and the second processor 124, and the second communication interface 125 are coupled together through the second bus system 126. It can be understood that the second bus system 126 is used to realize the connection and communication between these components. In addition to the data bus, the second bus system 126 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, various buses are labeled as the second bus system 126 in Figure 11. Among them,

[0312] The second communication interface 125 is used for receiving and sending signals during the process of sending and receiving information with other external network elements;

[0313] The second memory 123 is used to store computer programs that can be run on the second processor;

[0314] The second processor 124 is configured to decode the bit stream and determine prediction mode parameters when running the computer program;

[0315] When the prediction mode parameter indicates that MIP is used to determine the intra-frame prediction value, the code stream is decoded to determine the MIP parameters of the current block; the code stream is decoded to determine the transform coefficient and LFNST index number of the current block; when the LFNST index number indicates that LFNST is used for the current block, a mapping mode of an LFNST transform set is determined according to the MIP parameters; according to the mapping mode of the LFNST transform set, an LFNST transform core candidate set is selected from multiple LFNST transform core candidate sets, and an LFNST transform core used for the current block is determined from the selected LFNST transform core candidate set; and the transform coefficient is transformed using the LFNST transform core.

[0316] It is understood that the second memory 123 in the embodiment of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct RAM bus random access memory (DRRAM). The second memory 123 of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0317] The second processor 124 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by hardware integrated logic circuits or software instructions in the second processor 124. The above-mentioned second processor 124 can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of this application can be directly implemented and executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium mature in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in the second memory 123 , and the second processor 124 reads the information in the second memory 123 and completes the steps of the above method in combination with its hardware.

[0318] It is to be understood that these embodiments described in the present application can be implemented with hardware, software, firmware, middleware, microcode or its combination.For hardware implementation, the processing unit can be implemented in one or more application specific integrated circuits (Application Specific Integrated Circuits, ASIC), digital signal processor (Digital Signal Processing, DSP), digital signal processing equipment (DSP Device, DSPD), programmable logic device (Programmable Logic Device, PLD), field programmable gate array (Field-Programmable Gate Array, FPGA), general-purpose processor, controller, microcontroller, microprocessor, other electronic units for performing functions described in the present application or its combination.For software implementation, the technology described in the present application can be realized by the module (such as process, function etc.) that performs functions described in the present application. The software code can be stored in a memory and executed by a processor. The memory can be implemented in the processor or outside the processor.

[0319] An embodiment of the present application provides an encoder and a decoder, which, when deriving a prediction block, determines the mapping mode of the LFNST transform set based on the size parameter in the MIP parameter of the current block. For image blocks of larger sizes, the DIMD derivation mapping mode can be selected not to be used, which can reduce computational complexity and thus improve coding efficiency.

[0320] It should be noted that, in the embodiments of the present application, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or apparatus comprising the element.

[0321] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0322] The methods disclosed in the several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0323] The features disclosed in the several product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0324] The features disclosed in the several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments or device embodiments.

[0325] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims. Industrial Applicability

[0326] Embodiments of the present application provide a coding and decoding method, an encoder, a decoder, and a storage medium. At a decoding end, a bitstream is decoded to determine a prediction mode parameter; when the prediction mode parameter indicates the use of MIP to determine an intra-frame prediction value, the bitstream is decoded to determine the MIP parameter of the current block; the bitstream is decoded to determine the transform coefficient and LFNST index number of the current block; when the LFNST index number indicates that the current block uses LFNST, a mapping mode of an LFNST transform set is determined according to the MIP parameter; according to the mapping mode of the LFNST transform set, an LFNST transform core candidate set is selected from multiple LFNST transform core candidate sets, and an LFNST transform core used for the current block is determined from the selected LFNST transform core candidate set; and the transform coefficient is transformed using the LFNST transform core. At the encoding end, a prediction mode parameter is determined; when the prediction mode parameter indicates that the current block uses MIP to determine the intra-frame prediction value, the MIP parameter of the current block is determined; based on the MIP parameter, the intra-frame prediction block of the current block is determined, and the residual block between the current block and the intra-frame prediction value is calculated; when the current block uses LFNST, the mapping mode of the LFNST transform set is determined based on the MIP parameter; based on the mapping mode of the LFNST transform set, an LFNST transform core candidate set is selected from multiple LFNST transform core candidate sets, and the LFNST transform core used for the current block is determined from the selected LFNST transform core candidate set, the LFNST index sequence number is set and written into the video code stream; the residual block is transformed using the LFNST transform core. It can be seen that in an embodiment of the present application, when deriving the prediction block, the mapping mode of the LFNST transform set is determined based on the size parameter in the MIP parameter of the current block, wherein, for image blocks of larger sizes, the DIMD derivation mapping mode can be selected not to reduce the computational complexity, thereby improving the coding efficiency.

Claims

1. A decoding method, applied to a decoder, comprising: Decode the code stream and determine the prediction mode parameters; When the prediction mode parameter indicates that a matrix-based intra-frame prediction MIP is used to determine an intra-frame prediction value, decoding the code stream to determine an MIP parameter of the current block; Decoding the bitstream to determine the transform coefficient and the low-frequency non-separable secondary transform LFNST index number of the current block; When the LFNST index number indicates that the current block uses LFNST, determining a mapping mode of an LFNST transform set according to the MIP parameter; selecting, according to a mapping mode of the LFNST transform set, an LFNST transform core candidate set from a plurality of LFNST transform core candidate sets, and determining an LFNST transform core used for the current block from the selected LFNST transform core candidate set; The transform coefficients are transformed using the LFNST transform kernel.

2. The method according to claim 1, wherein The MIP parameter includes a size parameter of the current block, and the method further includes: If the size parameter satisfies a first preset condition, determining the first preset prediction mode as the mapping mode of the LFNST transform set; If the size parameter does not meet the first preset condition, the decoding-side intra mode is used to derive the DIMD to determine the mapping mode of the LFNST transform set.

3. The method according to claim 2, wherein: The method further comprises: Traversing at least one intra prediction mode to determine at least one gradient information corresponding to the current block; A mapping mode of the LFNST transformation set is determined according to the at least one gradient information.

4. The method according to claim 3, wherein: The method further comprises: Determining a gradient magnitude value corresponding to each intra prediction mode based on the at least one gradient information; The intra-frame prediction mode with the largest gradient magnitude value among the at least one intra-frame prediction mode is determined as the mapping mode of the LFNST transform set.

5. The method according to claim 3 or 4, wherein: The method further comprises: Determining a downsampling vector of the current block according to the MIP parameters; Performing matrix multiplication calculation according to the down-sampling vector to obtain a MIP output vector; Determine a MIP prediction block for the current block according to the MIP output vector; The at least one intra prediction mode is traversed for the MIP prediction block to obtain the at least one gradient information.

6. The method according to claim 3 or 4, wherein: The method further comprises: Determining a downsampling vector of the current block according to the MIP parameters; Performing matrix multiplication calculation according to the down-sampling vector to obtain a MIP output vector; The MIP output vector is traversed through the at least one intra prediction mode to obtain the at least one gradient information.

7. The method according to claim 2, wherein: The size parameters include the height and width of the current block, and the method further includes: If the height of the current block is greater than or equal to a preset height threshold, and / or the width of the current block is greater than or equal to a preset width threshold, determining that the size parameter meets the first preset condition; If the height of the current block is smaller than a preset height threshold, and / or the width of the current block is smaller than a preset width threshold, it is determined whether the size parameter satisfies the first preset condition.

8. The method according to claim 2 or 7, wherein: The first preset prediction mode is the PLANAR mode or the DC mode.

9. The method according to claim 2, wherein: The method further comprises: Determine the index number of the mapping mode of the LFNST transformation set; Determine the value of the LFNST intra prediction mode index number according to the value of the index number; Selecting an LFNST transform kernel candidate set from a plurality of LFNST transform kernel candidate sets according to the value of the LFNST intra prediction mode index sequence number; A transform core indicated by the LFNST index number is selected from the selected LFNST transform core candidate set, and is set as the LFNST transform core used by the current block.

10. The method according to claim 9, wherein: The multiple LFNST transform core candidate sets include 4 LFNST transform core candidate sets, wherein each LFNST transform core candidate set includes 2 LFNST transform cores; accordingly, A first lookup table is used to determine the value of the LFNST intra prediction mode index number corresponding to the value of the index number.

11. The method according to claim 9, wherein: The method further comprises: The multiple LFNST transform core candidate sets include 35 LFNST transform core candidate sets, wherein each LFNST transform core candidate set includes 3 LFNST transform cores; accordingly, A second lookup table is used to determine the value of the LFNST intra prediction mode index number corresponding to the value of the index number.

12. The method according to claim 10 or 11, wherein: The MIP parameters include a MIP transposition indication parameter, and the value of the MIP transposition indication parameter is used to indicate whether to perform transposition processing on the sampling point input vector used by the MIP mode.

13. The method according to claim 12, wherein: The method further comprises: When the value of the MIP transposition indication parameter indicates that a transposition process is to be performed on the sampling point input vector used in the MIP mode, a matrix transposition process is performed on the transformation kernel indicated by the LFNST index number to obtain the LFNST transformation kernel used by the current block.

14. A coding method, applied to an encoder, comprising: determining prediction model parameters; When the prediction mode parameter indicates that the current block uses MIP to determine an intra prediction value, determining an MIP parameter of the current block; Determine an intra prediction block of the current block according to the MIP parameters, and calculate a residual block between the current block and the intra prediction value; When the current block uses LFNST, determining a mapping mode of the LFNST transform set according to the MIP parameters; Selecting an LFNST transform core candidate set from a plurality of LFNST transform core candidate sets according to a mapping mode of the LFNST transform set, determining an LFNST transform core used by the current block from the selected LFNST transform core candidate set, setting an LFNST index number, and writing the index number into a video stream; The residual block is transformed using the LFNST transformation kernel.

15. The method according to claim 14, wherein The MIP parameter includes a size parameter of the current block, and the method further includes: If the size parameter satisfies a first preset condition, determining the first preset prediction mode as the mapping mode of the LFNST transform set; If the size parameter does not meet the first preset condition, a mapping mode of the LFNST transformation set is determined using DIMD.

16. The method according to claim 15, wherein The method further comprises: Traversing at least one intra prediction mode to determine at least one gradient information corresponding to the current block; A mapping mode of the LFNST transformation set is determined according to the at least one gradient information.

17. The method according to claim 16, wherein The method further comprises: Determining a gradient magnitude value corresponding to each intra prediction mode based on the at least one gradient information; The intra-frame prediction mode with the largest gradient magnitude value among the at least one intra-frame prediction mode is determined as the mapping mode of the LFNST transform set.

18. The method according to claim 16 or 17, wherein The method further comprises: Determining a downsampling vector of the current block according to the MIP parameters; Performing matrix multiplication calculation according to the down-sampling vector to obtain a MIP output vector; Determine a MIP prediction block for the current block according to the MIP output vector; The at least one intra prediction mode is traversed for the MIP prediction block to obtain the at least one gradient information.

19. The method according to claim 16 or 17, wherein: The method further comprises: Determining a downsampling vector of the current block according to the MIP parameters; Performing matrix multiplication calculation according to the down-sampling vector to obtain a MIP output vector; The MIP output vector is traversed through the at least one intra prediction mode to obtain the at least one gradient information.

20. The method according to claim 15, wherein The size parameters include the height and width of the current block, and the method further includes: If the height of the current block is greater than or equal to a preset height threshold, and / or the width of the current block is greater than or equal to a preset width threshold, determining that the size parameter meets the first preset condition; If the height of the current block is smaller than a preset height threshold, and / or the width of the current block is smaller than a preset width threshold, it is determined whether the size parameter satisfies the first preset condition.

21. The method according to claim 15 or 19, wherein The first preset prediction mode is the PLANAR mode or the DC mode.

22. The method according to claim 15, wherein The method further comprises: Determine the index number of the mapping mode of the LFNST transformation set; Determine the value of the LFNST intra prediction mode index number according to the value of the index number; selecting an LFNST transform kernel candidate set from a plurality of LFNST transform kernel candidate sets according to the value of the LFNST intra prediction mode index sequence number; A transform kernel used for the current block is selected from the selected LFNST transform kernel candidate set.

23. The method according to claim 22, wherein The multiple LFNST transform core candidate sets include 4 LFNST transform core candidate sets, wherein each LFNST transform core candidate set includes 2 LFNST transform cores; accordingly, A first lookup table is used to determine the value of the LFNST intra prediction mode index number corresponding to the value of the index number.

24. The method according to claim 22, wherein The method further comprises: The multiple LFNST transform core candidate sets include 35 LFNST transform core candidate sets, wherein each LFNST transform core candidate set includes 3 LFNST transform cores; accordingly, A second lookup table is used to determine the value of the LFNST intra prediction mode index number corresponding to the value of the index number.

25. The method according to claim 23 or 24, wherein The MIP parameters include a MIP transposition indication parameter, and the value of the MIP transposition indication parameter is used to indicate whether to perform transposition processing on the sampling point input vector used by the MIP mode.

26. The method according to claim 23, wherein The method further comprises: When the value of the MIP transposition indication parameter indicates that a transposition process is to be performed on the sampling point input vector used in the MIP mode, a matrix transposition process is performed on the selected transformation kernel to obtain the LFNST transformation kernel used by the current block.

27. An encoder comprising a first determining unit, an encoding unit, and a first transforming unit; wherein: The first determining unit is configured to determine a prediction mode parameter; when the prediction mode parameter indicates that the current block uses MIP to determine an intra-frame prediction value, determine the MIP parameter of the current block; determine the intra-frame prediction block of the current block based on the MIP parameter, and calculate a residual block between the current block and the intra-frame prediction value; When the current block uses LFNST, determining a mapping mode of an LFNST transform set according to the MIP parameter; selecting an LFNST transform core candidate set from a plurality of LFNST transform core candidate sets according to the mapping mode of the LFNST transform set, determining an LFNST transform core used for the current block from the selected LFNST transform core candidate set, and setting an LFNST index number; The encoding unit is configured to write a video stream; The first transformation unit is configured to use the LFNST transformation kernel to perform transformation processing on the residual block.

28. An encoder comprising a first memory and a first processor; wherein: The first memory is used to store a computer program that can be run on the first processor; The first processor is configured to execute the method according to any one of claims 14 to 26 when running the computer program.

29. A decoder comprising a second determining unit and a second transforming unit; wherein: The second determining unit is configured to decode a bitstream and determine a prediction mode parameter; when the prediction mode parameter indicates that a MIP is used to determine an intra-frame prediction value, decode the bitstream and determine a MIP parameter of a current block; decode the bitstream and determine a transform coefficient and an LFNST index number of the current block; when the LFNST index number indicates that LFNST is used for the current block, determine a mapping mode of an LFNST transform set according to the MIP parameter; select an LFNST transform core candidate set from a plurality of LFNST transform core candidate sets according to the mapping mode of the LFNST transform set, and determine an LFNST transform core used for the current block from the selected LFNST transform core candidate set; The second transform unit is configured to perform transform processing on the transform coefficients using the LFNST transform kernel.

30. A decoder comprising a second memory and a second processor; wherein: The second memory is used to store a computer program that can be run on the second processor; The second processor is configured to execute the method according to any one of claims 1 to 13 when running the computer program.

31. A computer-readable storage medium storing a computer program, wherein the computer program, when executed, implements the method according to any one of claims 1 to 13, or implements the method according to any one of claims 14 to 26.

Images