Transformation method, encoder, decoder, and storage medium

Abstract

The embodiment of the invention discloses a transformation method, an encoder, a decoder and a storage medium. The method comprises the following steps: determining a prediction mode parameter of a current block; when the prediction mode parameter indicates that the current block determines an intra prediction value using matrix-based intra prediction (MIP), determining an MIP parameter; determining an intra-frame prediction value of the current block according to the MIP parameter, and calculating a prediction difference value between the current block and the intra-frame prediction value; when the current block uses low-frequency inseparable secondary transform (LFNST), determining an LFNST transform kernel used by the current block according to the MIP parameter, setting an LFNST index sequence number, and writing the LFNST index sequence number into a video code stream; and carrying out transformation processing on the prediction difference value by using the LFNST transformation kernel.

Description

[0001] Description of the case

[0002] This application is a divisional application based on a Chinese patent with an application date of December 30, 2019, application number 201980103146.X, and invention name “Transformation method, encoder, decoder and storage medium”. Technical Field

[0003] The embodiments of the present application relate to the field of image processing technology, and in particular to a transformation method, an encoder, a decoder, and a storage medium. Background Art

[0004] As people's requirements for video display quality increase, new video applications such as high-definition and ultra-high-definition video have emerged. H.265 / High Efficiency Video Coding (HEVC) can no longer meet the needs of the rapid development of video applications. The Joint Video Exploration Team (JVET) has proposed the next-generation video coding standard H.266 / Versatile Video Coding (VVC), and its corresponding test model is the VVC reference software test platform (VVC Test Model, VTM).

[0005] In H.266 / VVC, the Reduced Second Transform (RST) technology has been accepted and renamed as Low-Frequency Non-Separable Transform (LFNST) technology. Since the selection of the transform set in the LFNST technology is based on the intra-frame prediction mode, but for non-traditional intra-frame prediction modes, there is a lack of variability when performing the LFNST transform, which reduces the coding efficiency. Summary of the invention

[0006] The embodiments of the present application provide a transformation method, an encoder, a decoder, and a storage medium, which can improve the applicability of LFNST technology to non-traditional intra-frame prediction modes, make the selection of transformation sets more flexible, and thus improve the encoding and decoding efficiency.

[0007] The technical solution of the embodiment of the present application can be implemented as follows:

[0008] In a first aspect, an embodiment of the present application provides a transformation method, which is applied to an encoder, and the method includes:

[0009] Determining prediction mode parameters for the current block;

[0010] When the prediction mode parameter indicates that the current block uses a matrix-based intra prediction MIP to determine an intra prediction value, determining a MIP parameter;

[0011] Determine an intra-frame prediction value of the current block according to the MIP parameter, and calculate a prediction difference between the current block and the intra-frame prediction value;

[0012] When the current block uses a low frequency non-separable secondary transform LFNST, determining the LFNST transform kernel used by the current block according to the MIP parameter, setting the LFNST index number and writing it into the video bitstream;

[0013] The prediction difference is transformed using the LFNST transformation kernel.

[0014] In a second aspect, an embodiment of the present application provides a transformation method, which is applied to a decoder, and the method includes:

[0015] Parse the bitstream and determine the prediction mode parameters of the current block;

[0016] When the prediction mode parameter indicates that the current block uses MIP to determine the intra-frame prediction value, parsing the bitstream to determine the MIP parameter;

[0017] Parsing the bitstream to determine the transform coefficient and LFNST index number of the current block;

[0018] When the LFNST index number indicates that the current block uses LFNST, determining, according to the MIP parameter, an LFNST transformation kernel used by the current block;

[0019] The transform coefficients are transformed using the LFNST transform kernel.

[0020] In a third aspect, an embodiment of the present application provides an encoder, the encoder comprising a first determination unit, a first calculation unit and a first transformation unit; wherein,

[0021] A first determining unit, configured to determine a prediction mode parameter of a current block;

[0022] The first determining unit is further configured to determine a MIP parameter when the prediction mode parameter indicates that the current block uses a matrix-based intra prediction MIP to determine an intra prediction value;

[0023] A first calculation unit is configured to determine an intra-frame prediction value of the current block according to the MIP parameter, and calculate a prediction difference between the current block and the intra-frame prediction value;

[0024] The first determining unit is further configured to determine, when the current block uses a low frequency non-separable secondary transform LFNST, a LFNST transform kernel used by the current block according to the MIP parameter, set a LFNST index number and write it into a video bitstream;

[0025] The first transform unit is configured to use the LFNST transform kernel to perform transform processing on the prediction difference.

[0026] In a fourth aspect, an embodiment of the present application provides an encoder, the encoder comprising a first memory and a first processor; wherein:

[0027] A first memory, for storing a computer program that can be run on the first processor;

[0028] The first processor is configured to execute the method according to the first aspect when running the computer program.

[0029] In a fifth aspect, an embodiment of the present application provides a decoder, the decoder comprising a parsing unit, a second determining unit, and a second transforming unit; wherein:

[0030] A parsing unit, configured to parse a bitstream to determine a prediction mode parameter of a current block; and further configured to parse the bitstream to determine the MIP parameter when the prediction mode parameter indicates that the current block uses MIP to determine an intra-frame prediction value; and further configured to parse the bitstream to determine a transform coefficient and a LFNST index number of the current block;

[0031] A second determining unit is configured to determine, when the LFNST index number indicates that the current block uses LFNST, according to the MIP parameter, an LFNST transform kernel used by the current block;

[0032] The second transform unit is configured to use the LFNST transform kernel to perform transform processing on the transform coefficients.

[0033] In a sixth aspect, an embodiment of the present application provides a decoder, the decoder comprising a second memory and a second processor; wherein:

[0034] A second memory for storing a computer program that can be run on a second processor;

[0035] The second processor is used to execute the method described in the second aspect when running the computer program.

[0036] In a seventh aspect, an embodiment of the present application provides a computer storage medium storing a computer program, wherein the computer program implements the method described in the first aspect when executed by a first processor, or implements the method described in the second aspect when executed by a second processor.

[0037] The embodiment of the present application provides a transformation method, an encoder, a decoder and a storage medium, by determining the prediction mode parameters of the current block; when the prediction mode parameters indicate that the current block uses MIP to determine the intra-frame prediction value, determining the MIP parameters; determining the intra-frame prediction value of the current block according to the MIP parameters, and calculating the prediction difference between the current block and the intra-frame prediction value; when the current block uses LFNST, determining the LFNST transformation core used by the current block according to the MIP parameters, setting the LFNST index number and writing it into the video bitstream; using the LFNST transformation core, transforming the prediction difference. In this way, for the current block using the MIP mode, since the MIP parameters are introduced when performing the LFNST transformation, the selection of the LFNST transformation core is more flexible, which not only improves the applicability of the LFNST technology to non-traditional intra-frame prediction modes, but also improves the encoding and decoding efficiency, and can also improve the video image quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 A schematic diagram of the application location of a LFNST technology provided for a related technical solution;

[0039] Figure 2A A block diagram of a video encoding system provided in an embodiment of the present application;

[0040] Figure 2B A block diagram of a video decoding system provided in an embodiment of the present application;

[0041] Figure 3 A flowchart of a transformation method provided in an embodiment of the present application;

[0042] Figure 4 A flowchart of a MIP prediction process provided in an embodiment of the present application;

[0043] Figure 5 A schematic diagram of the structure of a calculation process of matrix multiplication of LFNST technology provided in an embodiment of the present application;

[0044] Fig. 6A A structural block diagram of LFNST transformation provided for related technical solutions;

[0045] Figure 6B A structural block diagram of another LFNST transformation provided for the related technical solution;

[0046] Figure 6C A structural block diagram of another LFNST transformation provided for the related technical solution;

[0047] Fig.6DA structural block diagram of another LFNST transformation provided for the related technical solution;

[0048] Figure 7 A flowchart of another transformation method provided in an embodiment of the present application;

[0049] Figure 8 A flowchart of a specific LFNST process provided in an embodiment of the present application;

[0050] Fig. 9 A schematic diagram of the structure of an encoder provided in an embodiment of the present application;

[0051] Fig.10 A schematic diagram of a specific hardware structure of an encoder provided in an embodiment of the present application;

[0052] Fig.11 A schematic diagram of the structure of a decoder provided in an embodiment of the present application;

[0053] Fig.12 A schematic diagram of the specific hardware structure of a decoder provided in an embodiment of the present application. DETAILED DESCRIPTION

[0054] In order to enable a more detailed understanding of the features and technical contents of the embodiments of the present application, the implementation of the embodiments of the present application is described in detail below in conjunction with the accompanying drawings. The attached drawings are for reference only and are not used to limit the embodiments of the present application.

[0055] In a video image, a first image component, a second image component and a third image component are generally used to represent a coding block (Coding Block, CB); wherein the three image components are a brightness component, a blue chrominance component and a red chrominance component, respectively. Specifically, the brightness component is usually represented by the symbol Y, the blue chrominance component is usually represented by the symbol Cb or U, and the red chrominance component is usually represented by the symbol Cr or V; thus, the video image can be represented in the YCbCr format or in the YUV format.

[0056] In the embodiment of the present application, the first image component may be a brightness component, the second image component may be a blue chrominance component, and the third image component may be a red chrominance component, but the embodiment of the present application does not make any specific limitation.

[0057] The following is a description of the relevant technical solutions for the current LFNST technology.

[0058] See also Figure 1 , which shows a schematic diagram of the application location of a LFNST technology provided by a related technical solution. Figure 1As shown, in the intra prediction mode, for the encoder side, the LFNST technology is applied between the forward primary transform unit 11 and the quantization unit 12, and the LFNST technology is applied between the inverse quantization unit 13 and the inverse primary transform unit 14.

[0059] Specifically, on the encoder side, first, the data, such as the prediction residual (which can be represented by residual), is subjected to a first transformation (which can be called "Core Transform" or "primary transform" or "main transform") through the positive primary transform unit 11 to obtain a transform coefficient matrix after the first transformation; then, an LFNST transform (which can be called "Secondary Transform" or "secondary transform") is performed on the coefficients in the transform coefficient matrix to obtain a LFNST transform coefficient matrix, and finally, the LFNST transform coefficient matrix is ​​quantized through the quantization unit 12, and the final quantized value is written into the video bitstream.

[0060] On the decoder side, by parsing the bitstream, the quantized value of the LFNST transform coefficient matrix can be obtained, and the quantized value is dequantized by the dequantization unit 13 (which can be called "Scaling") to obtain the restored value of the LFNST transform coefficient matrix. The restored value is subjected to an inverse LFNST transform to obtain the coefficient matrix; and then the coefficient matrix is ​​subjected to an inverse transform corresponding to the "Core Transform" on the encoder side by the inverse primary transform unit 14, and finally the restored value of the residual is obtained. It should be noted that the standard only defines the "inverse transform" operation on the decoder side, so the "inverse LFNST transform" is also referred to as the "LFNST transform" in the standard; here, in order to distinguish it from the transform on the encoder side, the "LFNST transform" on the encoder side can be referred to as the "forward LFNST transform", and the "LFNST transform" on the decoder side can be referred to as the "inverse LFNST transform".

[0061] That is to say, on the encoder side, the prediction residual of the current transform block can obtain the primary transform coefficient through the positive primary transform, and then some of the primary transform coefficients are transformed secondary through matrix multiplication to obtain fewer and more concentrated secondary transform coefficients, which are then quantized; on the decoder side, after parsing the quantization value, it is inverse quantized, the inverse quantized coefficients are inversely transformed secondary through matrix multiplication, and then the coefficients after the inverse secondary transform are inversely transformed primary to restore the prediction residual.

[0062] In LFNST technology, the LFNST transformation process may include: configuring core parameters, intra-frame prediction mode mapping, selecting transformation matrix, calculating matrix multiplication, and constructing inverse primary transformation coefficient matrix; after these steps, the LFNST transformation is completed. However, in the step of selecting the transformation matrix, it is necessary to select the transformation set first. Since the transformation matrix is ​​related to the directional characteristics of the prediction mode, the transformation set is currently selected according to the intra-frame prediction mode. Among them, for the traditional intra-frame prediction mode, the value of the intra-frame prediction mode indicator (which can be represented by predModeIntra) can be determined according to the number of the traditional intra-frame prediction mode, and then the transform set index number can be determined according to the value of predModeIntra; however, for non-traditional intra-frame prediction modes, especially the matrix-based intra-frame prediction (Matrix-based IntraPrediction, MIP) mode, the value of predModeIntra is directly set to indicate the intra-frame prediction mode index number corresponding to the PLANAR mode (i.e., 0), so that the current block in the MIP mode can only select the transform set with a transform set index number of 0, which makes the current block in the MIP mode lack variability when performing LFNST transformation, thereby causing the LFNST technology to be not well applicable to the MIP mode and also reducing the coding efficiency.

[0063] The embodiment of the present application provides a transformation method, which is applied to an encoder. By determining the prediction mode parameters of the current block; when the prediction mode parameters indicate that the current block uses a matrix-based intra-frame prediction MIP to determine the intra-frame prediction value, determine the MIP parameters; according to the MIP parameters, determine the intra-frame prediction value of the current block, and calculate the prediction difference between the current block and the intra-frame prediction value; when the current block uses a low-frequency inseparable secondary transform LFNST, according to the MIP parameters, determine the LFNST transformation core used by the current block, set the LFNST index number and write it into the video bitstream; use the LFNST transformation core to transform the prediction difference. In this way, for the current block using the MIP mode, since the MIP parameters are introduced when performing the LFNST transformation, the selection of the LFNST transformation core is more flexible, which not only improves the applicability of the LFNST technology to non-traditional intra-frame prediction modes, but also improves the encoding and decoding efficiency, and can also improve the video image quality.

[0064] The embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0065] See also Figure 2A , which shows an example of a block diagram of a video encoding system provided by an embodiment of the present application; Figure 2AAs shown, the video coding system 10 includes a transform and quantization unit 101, an intra-frame estimation unit 102, an intra-frame prediction unit 103, a motion compensation unit 104, a motion estimation unit 105, an inverse transform and inverse quantization unit 106, a filter control analysis unit 107, a filtering unit 108, a coding unit 109 and a decoded image cache unit 110, etc., wherein the filtering unit 108 can implement deblocking filtering and sample adaptive offset (Sample Adaptive0ffset, SAO) filtering, and the coding unit 109 can implement header information encoding and context-based adaptive binary arithmetic coding (Context-based Adaptive Binary Arithmatic Coding, CABAC). For the input original video signal, through the coding tree block (Coding A video coding block can be obtained by dividing the video coding block into a plurality of blocks (CTUs) by a plurality of tree units (CTUs). Then, the residual pixel information obtained after intra-frame or inter-frame prediction is transformed by a transform and quantization unit 101, including transforming the residual information from a pixel domain to a transform domain, and quantizing the obtained transform coefficients to further reduce the bit rate. An intra-frame estimation unit 102 and an intra-frame prediction unit 103 are used to perform intra-frame prediction on the video coding block. Specifically, the intra-frame estimation unit 102 and the intra-frame prediction unit 103 are used to determine an intra-frame prediction mode to be used for encoding the video coding block. A motion compensation unit 104 and a motion estimation unit 105 are used to perform inter-frame prediction coding of the received video coding block relative to one or more blocks in one or more reference frames to provide temporal prediction information. The motion estimation performed by the motion estimation unit 105 is a process of generating a motion vector, wherein the motion vector can estimate the motion of the video coding block. Then, the motion compensation unit 104 calculates the motion vector based on the motion vector determined by the motion estimation unit 105. The motion vector performs motion compensation; after determining the intra-frame prediction mode, the intra-frame prediction unit 103 is also used to provide the selected intra-frame prediction data to the encoding unit 109, and the motion estimation unit 105 also sends the calculated motion vector data to the encoding unit 109; in addition, the inverse transform and inverse quantization unit 106 is used to reconstruct the video coding block, reconstruct the residual block in the pixel domain, and the reconstructed residual block is removed by the filter control analysis unit 107 and the filtering unit 108. The block effect artifacts are then removed, and the reconstructed residual block is then added to a predictive block in the frame of the decoded image cache unit 110 to generate a reconstructed video coding block; the encoding unit 109 is used to encode various coding parameters and quantized transform coefficients. In the CABAC-based coding algorithm, the context content can be based on adjacent coding blocks and can be used to encode information indicating the determined intra-frame prediction mode and output the code stream of the video signal; and the decoded image cache unit 110 is used to store the reconstructed video coding block for prediction reference.As the video image encoding proceeds, new reconstructed video encoding blocks are continuously generated, and these reconstructed video encoding blocks are stored in the decoded image buffer unit 110 .

[0066] See also Figure 2B , which shows an example of a block diagram of a video decoding system provided by an embodiment of the present application; Figure 2B As shown, the video decoding system 20 includes a decoding unit 201, an inverse transform and inverse quantization unit 202, an intra-frame prediction unit 203, a motion compensation unit 204, a filtering unit 205, and a decoded image buffer unit 206, etc., wherein the decoding unit 201 can implement header information decoding and CABAC decoding, and the filtering unit 205 can implement deblocking filtering and SAO filtering. The input video signal is Figure 2A After the encoding process, the code stream of the video signal is output; the code stream is input into the video decoding system 20, and first passes through the decoding unit 201 to obtain the decoded transform coefficient; the transform coefficient is processed by the inverse transform and inverse quantization unit 202 to generate a residual block in the pixel domain; the intra-frame prediction unit 203 can be used to generate prediction data of the current video decoding block based on the determined intra-frame prediction mode and the data of the previously decoded block from the current frame or picture; the motion compensation unit 204 determines the prediction information for the video decoding block by analyzing the motion vector and other associated syntax elements, and uses the prediction information The decoded video block is formed by summing the residual block from the inverse transform and inverse quantization unit 202 and the corresponding predictive block generated by the intra-frame prediction unit 203 or the motion compensation unit 204; the decoded video signal is passed through the filtering unit 205 to remove the block effect artifacts, which can improve the video quality; the decoded video block is then stored in the decoded image cache unit 206, which stores the reference image used for subsequent intra-frame prediction or motion compensation, and is also used for the output of the video signal, that is, the restored original video signal is obtained.

[0067] The transformation method in the embodiment of the present application can be applied to Figure 2A The transformation and quantization unit 101 shown in FIG. 1 includes Figure 1 The first transform unit 11 and the quantization unit 12 shown in the figure, at this time, the transform method is specifically applied to the part between the transform and the quantization. In addition, the transform method in the embodiment of the present application can also be applied to the following Figure 2A The inverse transform and inverse quantization unit 106 shown in FIG. Figure 2B The inverse transform and inverse quantization unit 202 shown in the figure, whether it is the inverse transform and inverse quantization unit 106 or the inverse transform and inverse quantization unit 202, can include Figure 1The inverse quantization unit 13 and the inverse primary transform unit 14 shown, at this time, the transformation method is specifically applied to the part between the inverse quantization and the inverse transform. That is to say, the transformation method in the embodiment of the present application can be applied to both the video encoding system and the video decoding system, and can even be applied to both the video encoding system and the video decoding system at the same time, but the embodiment of the present application does not make specific limitations. It should also be noted that when the transformation method is applied to the video encoding system, the "current block" specifically refers to the current encoding block in the intra-frame prediction; when the transformation method is applied to the video decoding system, the "current block" specifically refers to the current decoding block in the intra-frame prediction.

[0068] Based on the above Figure 2A For an example of an application scenario, see Figure 3 , which shows a flow chart of a transformation method provided in an embodiment of the present application. Figure 3 As shown, the method may include:

[0069] S301: Determine prediction mode parameters of the current block;

[0070] It should be noted that the video image can be divided into multiple image blocks, and each image block currently to be encoded can be called a coding block (Coding Block, CB). Here, each coding block may include a first image component, a second image component, and a third image component; and the current block is a coding block in the video image that is currently to be predicted for the first image component, the second image component, or the third image component.

[0071] Among them, assuming that the current block predicts the first image component, and the first image component is a luminance component, that is, the image component to be predicted is a luminance component, then the current block can also be called a luminance block; or, assuming that the current block predicts the second image component, and the second image component is a chrominance component, that is, the image component to be predicted is a chrominance component, then the current block can also be called a chrominance block.

[0072] It should also be noted that the prediction mode parameters indicate the coding mode of the current block and parameters related to the mode. The prediction mode parameters of the current block can usually be determined by using rate distortion optimization (RDO).

[0073] Specifically, in some embodiments, for S301, determining the prediction mode parameters of the current block may include:

[0074] Determining the image component to be predicted of the current block;

[0075] Based on the parameters of the current block, using multiple prediction modes to respectively perform predictive coding on the image components to be predicted, and calculating a rate-distortion cost result corresponding to each prediction mode under the multiple prediction modes;

[0076] A minimum rate-distortion cost result is selected from the multiple rate-distortion cost results obtained by calculation, and a prediction mode corresponding to the minimum rate-distortion cost result is determined as a prediction mode parameter of the current block.

[0077] That is to say, on the encoder side, multiple prediction modes can be used for the current block to encode the predicted image components respectively. Here, multiple prediction modes generally include traditional intra-frame prediction modes and non-traditional intra-frame prediction modes, and traditional intra-frame prediction modes can include direct current (DC) mode, plane (PLANAR) mode and angle mode, etc., and non-traditional intra-frame prediction modes can include MIP mode, cross-component linear model prediction (CCLM) mode, intra-frame block copy (IBC) mode and PLT (Palette) mode, etc.

[0078] In this way, after encoding the current block respectively using multiple prediction modes, the rate-distortion cost result corresponding to each prediction mode can be obtained; then the minimum rate-distortion cost result is selected from the multiple rate-distortion cost results obtained, and the prediction mode corresponding to the minimum rate-distortion cost result is determined as the prediction mode parameter of the current block; in this way, the current block can finally be encoded using the determined prediction mode, and in this prediction mode, the prediction residual can be made small, which can improve the coding efficiency.

[0079] S302: When the prediction mode parameter indicates that the current block uses MIP to determine an intra-frame prediction value, determine MIP parameters;

[0080] S303: determining an intra-frame prediction value of a current block according to the MIP parameter, and calculating a prediction difference between the current block and the intra-frame prediction value;

[0081] It should be noted that, for the MIP mode, the input data of MIP prediction include: the position of the current block (xTbCmp, yTbCmp), the MIP prediction mode applied to the current block (which can be represented by modeId), the height of the current block (represented by nTbH), the width of the current block (represented by nTbW), and a transposition processing indication flag indicating whether transposition is required (represented by isTransposed), etc.; the output data of MIP prediction include: the prediction block of the current block, the intra-frame prediction value corresponding to the pixel coordinate [x][y] in the prediction block is predSamples[x][y]; where x=0,1,…,nTbW-1; y=0,1,…,nTbH-1.

[0082] Specifically, Figure 4 As shown, the MIP prediction process can be divided into four steps: configuring core parameters 41, obtaining reference pixels 42, constructing input samples 43, and generating prediction values ​​44. Among them, for configuring core parameters 41, according to the size of the current block in the frame, the current block can be divided into three categories, and mipSizeId is used to record the type of the current block; and the number of reference sampling points and the number of matrix multiplication output sampling points are different for different types of current blocks. For obtaining reference pixels 42, when predicting the current block, the upper block and the left block of the current block are both coded blocks. The reference pixels of the MIP technology are the reconstructed values ​​of the upper row of pixels and the left column of pixels of the current block. The process of obtaining the reference pixels adjacent to the upper side of the current block (represented by refT) and the reference pixels adjacent to the left side (represented by refL) is the reference pixel acquisition process. For constructing input samples 43, this step is used for the input of matrix multiplication, and can mainly include: obtaining reference samples 431, constructing reference sample buffers 432, and deriving matrix multiplication input samples 433; wherein, the process of obtaining reference samples is a downsampling process, and constructing reference sample buffers 432 can include a buffer filling method 4321 when transposition is not required and a buffer filling method 4322 when transposition is required. For generating prediction values ​​44, this step is used to obtain the MIP prediction value of the current block, and can mainly include: constructing a matrix multiplication output sample block 441, matrix multiplication output sample embedding 442, matrix multiplication output sample transposition 443, and generating a MIP final prediction value 444; wherein, constructing a matrix multiplication output sample block 441 can include obtaining a weight matrix 4411, obtaining a shift factor and an offset factor 4412, and a matrix multiplication operation 4413, and generating a MIP final prediction value 444 can include generating a prediction value that does not require upsampling 4441 and generating a prediction value that requires upsampling 4442. In this way, after these four steps, the intra-frame prediction value of the current block can be obtained.

[0083] In this way, after determining the intra-frame prediction value of the current block, a difference calculation can be performed based on the actual pixel value of the current block and the intra-frame prediction value, and the calculated difference is used as the prediction difference to facilitate subsequent transformation processing on the prediction difference.

[0084] Furthermore, in the MIP prediction process, MIP parameters also need to be determined.

[0085] In some embodiments, the MIP parameters may include a MIP transposition indication parameter (which may be represented by isTransposed); here, the value of the MIP transposition indication parameter is used to indicate whether to perform transposition processing on the sampling point input vector used by the MIP mode.

[0086] Specifically, in the MIP mode, according to the reference sampling values ​​corresponding to the adjacent reference pixels on the left side of the current block and the reference sampling values ​​corresponding to the adjacent reference pixels on the upper side, an adjacent reference sampling set can be obtained; thus, after obtaining the adjacent reference sampling set, an input reference sample value set can be constructed, that is, the sampling point input vector used by the MIP mode. However, the construction of the input reference sample value set is different on the encoder side and the decoder side, which is mainly related to the value of the MIP transposition indication parameter.

[0087] When applied to the encoder side, the value of the MIP transposition indication parameter can still be determined by utilizing the rate-distortion optimization method, specifically, it can include:

[0088] Calculate the first generation value with transposition processing and the second generation value without transposition processing respectively;

[0089] If the first generation value is less than the second generation value, then it can be determined that the value of the MIP transposition indicator parameter is 1;

[0090] If the first generation value is not less than the second generation value, then it can be determined that the value of the MIP transposition indication parameter is 0.

[0091] Further, when the value of the MIP transposition indication parameter is 0, in the buffer, the reference sampling value corresponding to the upper side of the adjacent reference sampling set can be stored before the reference sampling value corresponding to the left side. At this time, no transposition processing is required, that is, the sampling point input vector used by the MIP mode does not need to be transposed, and the buffer can be directly determined as the input reference sample value set; when the value of the MIP transposition indication parameter is 1, in the buffer, the reference sampling value corresponding to the upper side of the adjacent reference sampling set can be stored after the reference sampling value corresponding to the left side. At this time, the buffer is transposed, that is, the sampling point input vector used by the MIP mode needs to be transposed, and then the transposed buffer is determined as the input reference sample value set. In this way, after the input reference sample value set is obtained, it can be used in the process of determining the intra-frame prediction value corresponding to the current block in the MIP mode.

[0092] It should also be noted that, at the encoder side, after determining the value of the MIP transposition indication parameter, it is also necessary to write the determined value of the MIP transposition indication parameter into the bitstream to facilitate subsequent parsing processing at the decoder side.

[0093] In some embodiments, the MIP parameters may also include a MIP mode index number (which can be represented by modeId), wherein the MIP mode index number is used to indicate the MIP mode used by the current block, and the MIP mode is used to indicate the calculation and derivation method of using MIP to determine the intra-frame prediction value of the current block.

[0094] That is to say, in the MIP mode, since there are many types of MIP modes, these multiple MIP modes can be distinguished by the MIP mode index number, that is, different MIP modes have different MIP mode index numbers; in this way, according to the calculation and derivation method of using MIP to determine the intra-frame prediction value of the current block, the specific MIP mode can be determined, and thus the corresponding MIP mode index number can be obtained; in the embodiment of the present application, the value of the MIP mode index number can be 0, 1, 2, 3, 4 or 5.

[0095] In some embodiments, the MIP parameters may also include parameters such as the size and aspect ratio of the current block; wherein, according to the size of the current block (ie, the width and height of the current block), the category of the current block (which can be represented by mipSizeId) may also be determined.

[0096] In one implementation, determining the category of the current block according to the size of the current block may include:

[0097] If the width and height of the current block are both equal to 4, the value of mipSizeId can be set to 0;

[0098] On the contrary, if one of the width and height of the current block is equal to 4, or the width and height of the current block are both equal to 8, then the value of mipSizeId can be set to 1;

[0099] On the contrary, if the current block is a block of another size, the value of mipSizeId can be set to 2.

[0100] In another implementation, determining the category of the current block according to the size of the current block may include:

[0101] If the width and height of the current block are both equal to 4, the value of mipSizeId can be set to 0;

[0102] Conversely, if one of the width and height of the current block is equal to 4, the value of mipSizeId can be set to 1;

[0103] On the contrary, if the current block is a block of another size, the value of mipSizeId can be set to 2.

[0104] In this way, in the process of using MIP to determine the intra-frame prediction value, the MIP parameters can also be determined, so that the LFNST transformation kernel (which can be represented by kernel) used by the current block can be determined according to the determined MIP parameters.

[0105] S304: when the current block uses LFNST, determine the LFNST transformation core used by the current block according to the MIP parameters, set the LFNST index number and write it into the video bitstream;

[0106] It should be noted that not every current block can perform LFNST. LFNST can be performed on the current block only when the current block satisfies the following conditions at the same time. Among them, these conditions include: (a) the width and height of the current block are both greater than or equal to 4; (b) the width and height of the current block are both less than or equal to the maximum size of the transform block; (c) the prediction mode of the current block or the current coding block is an intra-frame prediction mode; (d) the single transform of the current block is a two-dimensional forward single transform (DCT2) in both the horizontal and vertical directions; (e) the intra-frame prediction mode of the current block or the coding block where the current block is located is a non-MIP mode or the prediction mode of the transform block is a MIP mode and the width and height of the transform block are both greater than or equal to 16. In other words, for the current block in the embodiment of the present application, the above five conditions need to be met at the same time.

[0107] Furthermore, when it is determined that the current block can perform LFNST, it is also necessary to determine the LFNST transform kernel (which can be represented by kernel) used by the current block. There are 4 transform kernel candidate sets in LFNST, and these four transform kernel candidate sets may include set0, set1, set2 and set3. According to the coding parameters of the current block or the coding block where the current block is located, the selected transform kernel candidate set can be implicitly derived; for example, in the current H.266 / VVC, according to the intra-frame prediction mode of the current block, it can be determined which of the four transform kernel candidate sets to use.

[0108] Specifically, after obtaining the intra prediction mode of the current block, the value of the intra prediction mode indicator (which can be represented by predModeIntra) can be determined. The calculation formula is as follows:

[0109]

[0110] Among them, the image component indicator (which can be represented by cIdx) is used to indicate the luminance component or chrominance component of the current block; here, if the current block is predicted as a luminance component, then cIdx is equal to 0; if the current block is predicted as a chrominance component, then cIdx is equal to 1. In addition, (xTbY, yTbY) is the coordinate of the upper left corner sampling point of the current block, IntraPredModeY[xTbY][yTbY] is the intra-frame prediction mode of the luminance component, and IntraPredModeC[xTbY][yTbY] is the intra-frame prediction mode of the chrominance component.

[0111] In the current H.266 / VVC, intra prediction modes can be divided into traditional intra prediction modes and non-traditional intra prediction modes. For non-traditional intra prediction modes, the information indicated by the value of predModeIntra is as follows:

[0112] If the prediction mode of the current block is CCLM mode, the value of predModeIntra can be INTRA_LT_CCLM, INTRA_L_CCLM or INTRA_T_CCLM (81, 82, 83 in VVC respectively);

[0113] If the prediction mode of the current block is the MIP mode, the value of predModeIntra may be the MIP mode index number used;

[0114] If the prediction mode of the current block is the traditional intra prediction mode, the value of predModeIntra can be in [0,66].

[0115] Furthermore, if the prediction mode of the current block is CCLM mode or MIP mode, the value of predModeIntra can also be set in the following way:

[0116] (1) When the prediction mode of the current block is CCLM mode,

[0117] If the mode of the central luminance block at the luminance position corresponding to the current block (such as the chrominance block) is the MIP mode, that is, intra_mip_flag[xTbY+nTbW / 2][yTbY+nTbH / 2] is 1, the value of predMode Intra is set to the index number indicating the PLANAR mode (that is, 0);

[0118] Otherwise, if the mode of the central luminance block at the luminance position corresponding to the current block (such as the chrominance block) is the IBC mode or the PLT mode, the value of predModeIntra is set to the index number indicating the DC mode (i.e., 1);

[0119] Otherwise, the value of predModeIntra is set to the value of the mode index number of the central luminance block corresponding to the luminance position of the current block (such as the chrominance block) IntraPredModeY[xTbY+nTbW / 2][yTbY+nTbH / 2];

[0120] (2) When the prediction mode of the current block is MIP mode,

[0121] The value of predModeIntra may be directly set to the index number indicating the PLANAR mode (ie, 0).

[0122] For the traditional intra prediction mode (such as wide angle mapping), wide angle mapping can be performed according to the size of the current block, and the traditional intra prediction mode [0, 66] is extended to [-14, 80]. The specific mapping process is as follows:

[0123] First, calculate the width-to-height ratio factor (which can be expressed as whRatio), as shown below:

[0124] whRatio=Abs(Log2(nTbW / nTbH)) (2)

[0125] For a non-square current block (i.e., nTbW is not equal to nTbH), predModeIntra can be modified as follows: if nTbW is greater than nTbH, and predModeIntra is greater than or equal to 2, and predModeIntra is less than ((whRatio>1?(8+2×whRatio):8), then predModeIntra=(predModeIntra+65); otherwise, if nTbW is less than nTbH, and predModeIntra is less than or equal to 66, and predModeIntra is greater than ((whRatio>1?(60-2×whRatio):60), then predModeIntra=(predModeIntra-67).

[0126] In the current H.266 / VVC, according to the value of predModeIntra and Table 1, the value of the LFNST index number (which can be represented by SetIdx) can be determined, and the specific values ​​are shown in Table 1. Here, the value of the LFNST index number is set to indicate that the current block uses LFNST and the index number of the LFNST transform core in the LFNST transform core candidate set. Generally speaking, the LFNST transform set includes four transform core candidate sets (set0, set1, set2, set3), corresponding to the values ​​of SetIdx of 0, 1, 2, and 3, respectively.

[0127] Table 1

[0128] predModeIntra SetIdx predModeIntra<0 1 0<=predModeIntra<=1 0 2<=predModeIntra<=12 1 13<=predModeIntra<=23 2 24<=predModeIntra<=44 3 45<=predModeIntra<=55 2 56<=predModeIntra<=80 1

[0129] In the current H.266 / VVC, for the MIP mode, since the value of predModeIntra is set to the index number indicating the PLANAR mode (i.e. 0), the transform set used by the current block in the MIP mode can only select the transform set with the LFNST index number equal to 0, resulting in a lack of variability when performing LFNST in the MIP mode, thereby reducing the coding efficiency.

[0130] In the embodiment of the present application, the LFNST transform core candidate set can be first determined according to the MIP parameters, and then the LFNST transform core used by the current block is determined from the LFNST transform core candidate set, and the LFNST index number is set and written into the video bitstream. Here, the transformation matrix of LFNST is a plurality of fixed coefficient matrices obtained through training, and the LFNST transform core candidate set includes 2 groups of transformation matrices (also referred to as LFNST transform cores). After the LFNST transform core candidate set is determined, a group of LFNST transform cores needs to be selected from the LFNST transform core candidate set, that is, the transformation matrix used when determining the LFNST of the current block.

[0131] Here, the MIP parameters may include parameters such as a MIP transposition indication parameter (which can be represented by isTransposed), a MIP mode index number (which can be represented by modeId), the size of the current block, the category of the current block (which can be represented by mipSizeId), etc.; the following will describe in detail how to select the LFNST transform kernel used for the current block based on the MIP parameters.

[0132] Optionally, in some embodiments, when the MIP parameter is a MIP transposition indication parameter, for S304, when the current block uses LFNST, determining the LFNST transformation kernel used by the current block according to the MIP parameter, setting the LFNST index sequence number and writing it into the video bitstream may include:

[0133] Selecting a transform kernel used by the current block from a LFNST transform kernel candidate set;

[0134] When the value of the MIP transposition indication parameter indicates that a transposition process is performed on the sampling point input vector used in the MIP mode, a matrix transposition process is performed on the selected transformation kernel to obtain a LFNST transformation kernel used by the current block;

[0135] The value of the LFNST index number is set to indicate that the current block uses LFNST and the index number of the LFNST transform core in a LFNST transform core candidate set; wherein the LFNST transform core candidate set includes two or more transform cores preset for MIP.

[0136] It should be noted that, since the LFNST transform core candidate set includes two or more transform cores preset for MIP, the transform core used for the current block can be selected in a rate-distortion optimization manner. Specifically, the rate-distortion cost (Rate Distortion Cost, RDCost) can be calculated for each transform core in a rate-distortion optimization manner, and then the transform core with the smallest rate-distortion cost is selected as the transform core used for the current block.

[0137] That is to say, on the encoder side, a group of LFNST transform cores can be selected through RDCost, and the index number corresponding to the LFNST transform core (which can be represented by lfnst_idx) is written into the video bitstream and transmitted to the decoder side. When the first group of LFNST transform cores (i.e., the first group of transform matrices) in the LFNST transform core candidate set is selected, lfnst_idx is set to 1; when the second group of LFNST transform cores (i.e., the second group of transform matrices) in the LFNST transform core candidate set is selected, lfnst_idx is set to 2.

[0138] It should also be noted that since the value of the MIP transposition indication parameter is used to indicate whether to transpose the sampling point input vector used by the MIP mode, when the value of the MIP transposition indication parameter is equal to 1, that is, the value of the MIP transposition indication parameter indicates that the sampling point input vector used by the MIP mode is transposed, then the selected transform kernel needs to be matrix transposed to obtain the LFNST transform kernel used by the current block.

[0139] Here, for the value of the LFNST index number (i.e., lfnst_idx), when the value of the LFNST index number is equal to 0, LFNST will not be used; and when the value of the LFNST index number is greater than 0, LFNST will be used, and the index number of the transform core is equal to the value of the LFNST index number, or the value of the LFNST index number minus 1. In this way, after determining the LFNST transform core used by the current block, the LFNST index number needs to be set and written into the video bitstream, so that the subsequent decoder side can obtain the LFNST index number by parsing the bitstream.

[0140] Optionally, in some embodiments, when the MIP parameter is a MIP mode index number, for S304, when the current block uses LFNST, determining, according to the MIP parameter, a LFNST transform kernel used by the current block, setting the LFNST index number and writing it into the video bitstream may include:

[0141] Determine the value of the LFNST intra prediction mode index number according to the value of the MIP mode index number;

[0142] Selecting an LFNST transform core candidate set from multiple LFNST transform core candidate sets according to the value of the LFNST intra prediction mode index sequence number;

[0143] Selecting a transformation core indicated by the LFNST index number from the selected LFNST transformation core candidate set, and setting it as the LFNST transformation core used by the current block;

[0144] The value of the LFNST index number is set to indicate that the current block uses LFNST and the index number of the LFNST transform core in a LFNST transform core candidate set; wherein the LFNST transform core candidate set includes two or more preset LFNST transform cores.

[0145] It should be noted that the MIP mode index number is used to indicate the MIP mode used by the current block, and the MIP mode is used to indicate the calculation and derivation method of using MIP to determine the intra-frame prediction value of the current block; that is, according to the MIP mode index number, the LFNST transform kernel can also be determined.

[0146] It should also be noted that after determining the MIP mode index number, the MIP mode index number can also be converted into the value of the LFNST intra-frame prediction mode index number (which can be represented by predModeIntra); then, based on the value of predModeIntra, an LFNST transform core candidate set is selected from multiple LFNST transform core candidate sets to determine the transform core candidate set; and in the selected LFNST transform core candidate set, the transform core indicated by the LFNST index number is selected and set as the LFNST transform core used by the current block.

[0147] Here, for the value of the LFNST index number, when the value of the LFNST index number is equal to 0, the LFNST will not be used; and when the value of the LFNST index number is greater than 0, the LFNST will be used, and the index number of the transform core is equal to the value of the LFNST index number, or the value of the LFNST index number minus 1. In this way, after determining the LFNST transform core used by the current block, the LFNST index number needs to be set and written into the video bitstream, so that the subsequent decoder side can obtain the LFNST index number by parsing the bitstream.

[0148] Optionally, in some embodiments, when the MIP parameters are the MIP mode index number and the MIP transposition indication parameter, for S304, when the current block uses LFNST, determining the LFNST transform kernel used by the current block according to the MIP parameters, setting the LFNST index number and writing it into the video bitstream may include:

[0149] Determine the value of the LFNST intra prediction mode index number according to the value of the MIP mode index number;

[0150] Selecting an LFNST transform core candidate set from multiple LFNST transform core candidate sets according to the value of the LFNST intra prediction mode index sequence number;

[0151] Selecting a transformation core indicated by the LFNST index number from the selected LFNST transformation core candidate set, and setting it as the LFNST transformation core used by the current block;

[0152] When the value of the MIP transposition indication parameter indicates that a sampling point input vector used in the MIP mode is to be transposed, a matrix transposition process is performed on the LFNST transform kernel used in the current block, and the transform kernel obtained after the transposition process is set as the LFNST transform kernel used in the current block;

[0153] The value of the LFNST index number is set to indicate that the current block uses LFNST and the index number of the LFNST transform core in the LFNST transform core candidate set;

[0154] The LFNST transform kernel candidate set includes two or more preset LFNST transform kernels.

[0155] It should be noted that the MIP mode index number is used to indicate the MIP mode used by the current block, and the MIP mode is used to indicate the calculation and derivation method of using the MIP to determine the intra-frame prediction value of the current block; the value of the MIP transposition indication parameter is used to indicate whether to transpose the sampling point input vector used by the MIP mode; that is, the LFNST transform kernel can also be determined based on the combination of the MIP transposition indication parameter and the MIP mode index number.

[0156] It should also be noted that after determining the MIP mode index number, the MIP mode index number can be converted into the value of the LFNST intra-frame prediction mode index number (which can be represented by predModeIntra), and then, according to the value of predModeIntra, an LFNST transform core candidate set is selected from multiple LFNST transform core candidate sets to determine the transform core candidate set; and in the selected LFNST transform core candidate set, the transform core indicated by the LFNST index number is selected; and when it is indicated that transposition processing is required, the LFNST transform core used by the current block needs to be matrix transposed, and then the transform core obtained after the transposition processing is set as the LFNST transform core used by the current block.

[0157] Here, for the value of the LFNST index number, when the value of the LFNST index number is equal to 0, the LFNST will not be used; and when the value of the LFNST index number is greater than 0, the LFNST will be used, and the index number of the transform core is equal to the value of the LFNST index number, or the value of the LFNST index number minus 1. In this way, after determining the LFNST transform core used by the current block, the LFNST index number needs to be set and written into the video bitstream, so that the subsequent decoder side can obtain the LFNST index number by parsing the bitstream.

[0158] It should also be noted that, for the above-mentioned methods of selecting the LFNST transform kernel used for the current block, experimental results show that the performance of the LFNST transform kernel determined according to the combination of the MIP transposition indication parameter and the MIP mode index number is the best.

[0159] Furthermore, when the MIP parameters include at least a MIP mode index number (modeId), in the process of selecting the LFNST transform kernel used for the current block, the value of the LFNST intra prediction mode index number can also be obtained by looking up a table.

[0160] Specifically, in some embodiments, determining the value of the LFNST intra prediction mode index number according to the value of the MIP mode index number may include:

[0161] A first lookup table is used to determine the value of the LFNST intra-frame prediction mode index number corresponding to the value of the MIP mode index number, wherein the first lookup table contains one or more different MIP mode index numbers corresponding to at least two LFNST intra-frame prediction mode index numbers with different values.

[0162] Here, the first look-up table (Look-Up Table 1, LUT1) is used to reflect the correspondence between the MIP mode index number and the LFNST intra-frame prediction mode index number, that is, the first look-up table contains at least one or more different MIP mode index numbers corresponding to two LFNST intra-frame prediction mode index numbers with different values.

[0163] That is to say, different MIP modes may correspond to different values ​​of predModeIntra; thus, the MIP mode index number is determined according to the MIP mode, and then the value of predModeIntra is determined according to the first lookup table; then, according to the value of predModeIntra, an LFNST transform core candidate set may be selected from multiple LFNST transform core candidate sets, thereby determining the LFNST transform core used for the current block.

[0164] It can be understood that the value of predModeIntra can be determined according to the value of the MIP mode index number (modeId); then according to the value of predModeIntra, the value of SetIdx can be directly determined according to Table 2, that is, the LFNST transform core candidate set selected by the current block is determined. Here, the value of SetIdx indicates the transform core candidate set used in LFNST; since the value of modeId can include 0, 1, 2, 3, 4, 5, the value of predModeIntra is also 0, 1, 2, 3, 4, 5; its corresponding relationship with SetIdx is as follows,

[0165] Table 2

[0166] predModeIntra SetIdx 0 2 1 2 2 0 3 0 4 1 5 3

[0167] Furthermore, the LFNST transform kernel candidate set can be directly determined according to the value of the MIP mode index number. In this case, there is no need to use the value of predModeIntra, that is, there is no need to determine the value of the LFNST intra prediction mode index number according to the MIP mode index number.

[0168] Optionally, in some embodiments, when the current block uses LFNST, determining the LFNST transform kernel used by the current block according to the MIP parameter, setting the LFNST index sequence number and writing it into the video bitstream may include:

[0169] Selecting an LFNST transform core candidate set from multiple LFNST transform core candidate sets according to the value of the MIP mode index sequence number;

[0170] Selecting a transformation core indicated by the LFNST index number from the selected LFNST transformation core candidate set, and setting it as the LFNST transformation core used by the current block;

[0171] The value of the LFNST index number is set to indicate that the current block uses LFNST and the index number of the LFNST transform core in the LFNST transform core candidate set;

[0172] The LFNST transform kernel candidate set includes two or more preset LFNST transform kernels.

[0173] Further, selecting an LFNST transform core candidate set from multiple LFNST transform core candidate sets according to the value of the MIP mode index sequence number may include:

[0174] Determine the value of the LFNST transform core candidate set index number corresponding to the value of the MIP mode index number using a second lookup table, and select the LFNST transform core candidate set indicated by the value of the LFNST transform core candidate set index number as the selected LFNST transform core candidate set;

[0175] The second look-up table (Look-Up Table 2, LUT2) contains at least one or more different MIP mode index numbers corresponding to two different values ​​of the LFNST transform core candidate set index numbers.

[0176] It should be noted that if the prediction mode used by the current block is the MIP mode, the selected LFNST transform core candidate set can be determined according to the value of the MIP mode index number (modeId). As shown in Table 3, each MIP mode index number corresponds to a LFNST transform core candidate set.

[0177] Table 3

[0178] modeId SetIdx 0 0 1 0 2 0 3 1 4 2 5 3

[0179] In this way, according to the value of the MIP mode index number, the value of the corresponding LFNST transform core candidate set index number can be determined to select the LFNST transform core candidate set indicated by the value of the LFNST transform core candidate set index number as the selected LFNST transform core candidate set. For example, as can be seen from Table 3, when the value of the MIP mode index number (modeId) is 0, it can be determined that the value of the LFNST transform core candidate set index number (SetIdx) is 0, that is, the transform core candidate set indicated by 0 is used as the selected LFNST transform core candidate set; or, when modeId is 3, it can be determined that the value of SetIdx is 1, that is, the transform core candidate set indicated by 1 is used as the selected LFNST transform core candidate set, and so on.

[0180] Optionally, in some embodiments, selecting an LFNST transform core candidate set from multiple LFNST transform core candidate sets according to the value of the MIP mode index sequence number may include:

[0181] According to the value of the MIP mode index number, determine the index number of the LFNST transform kernel candidate set using a first calculation method;

[0182] The LFNST transform core candidate set indicated by the value of the LFNST transform core candidate set index sequence number is selected as the selected LFNST transform core candidate set.

[0183] Further, the determining the index number of the LFNST transform kernel candidate set by using the first calculation method according to the value of the MIP mode index number may include:

[0184] When the value of the MIP mode index number is less than a first preset value, setting the value of the LFNST transform core candidate set index number to be equal to the value of the MIP mode index number;

[0185] When the value of the MIP mode index number is greater than or equal to a first preset value, the value of the LFNST transform core candidate set index number is set to be equal to the difference between the value of the MIP mode index number and the second preset value; wherein the first preset value and the second preset value are both integer values.

[0186] In an embodiment of the present application, the value of the first preset value may be equal to 3, and the value of the second preset value may be equal to 2.

[0187] That is to say, according to the value of the MIP mode index number, the index number of the LFNST transform kernel candidate set can be determined using the first calculation method, wherein the first calculation formula is as follows:

[0188] SetIdx=modeId <x?0:(modeId-y) (3)

[0189] Wherein, x represents a first preset value, y represents a second preset value; SetIdx represents the value of the LFNST transform core candidate set index number, and modeId represents the value of the MIP mode index number.

[0190] Specifically, assuming that x is equal to 3 and y is equal to 2; then when modeId<3, it means that SetIdx can be 0 at this time; when modeId≥3, it means that SetIdx can be modeId-2 at this time; that is, when modeId=3, SetIdx can be 1; when modeId=4, SetIdx can be 2; when modeId=5, SetIdx can be 3; it can be seen that according to formula (3), the value of SetIdx obtained is the same as the content of Table 3.

[0191] In addition, if the prediction mode of the current block is the MIP mode, for determining the LFNST transform core candidate set, one of the transform sets (such as set1 or set2 or set3) can be fixedly selected as an LFNST transform core candidate set selected from multiple LFNST transform core candidate sets.

[0192] In addition, if the prediction mode of the current block is the MIP mode, the LFNST transform core candidate set can also be selected according to the value of the MIP mode index number (modeId); as shown in Table 4, each MIP mode corresponds to an LFNST transform core candidate set; here, the specific number of the LFNST transform core candidate set is limited; wherein a, b, c, d, e can be any value from {0, 1, 2, 3}.

[0193] Table 4

[0194]

[0195]

[0196] It should also be noted that if the prediction mode of the current block is the MIP mode, the LFNST transform kernel candidate set can also be selected according to one or more information combinations in the MIP parameters; it can also be mapped with the traditional intra-frame prediction mode according to the MIP parameters, and then the LFNST transform kernel candidate set can be selected according to the mapped angle; it can even be determined according to one or more information combinations in the MIP parameters to determine the transform matrix (i.e., transform kernel) to be used, and determining the transform matrix includes the selection of a transform set and a transform matrix group; here, a transform matrix group in a transform set can be fixedly selected according to the MIP parameters without making a selection on the encoder side, and lfnst_idx does not need to be transmitted at this time.

[0197] Further, in some embodiments, when the prediction mode parameter indicates that the current block uses a non-MIP mode, the method may further include:

[0198] Based on the intra prediction mode, determine the value of predModeIntra;

[0199] According to the value of predModeIntra, an LFNST transform core candidate set is selected from multiple LFNST transform core candidate sets.

[0200] That is to say, if the prediction mode of the current block is a non-MIP mode, the value of predModeIntra can be determined according to the intra-frame prediction mode; then according to the value of predModeIntra, in combination with the above Table 1, an LFNST transform core candidate set can be selected from multiple LFNST transform core candidate sets, and then the transform core indicated by the LFNST index number is selected from the selected LFNST transform core candidate set, and set as the LFNST transform core used by the current block; the value of the LFNST index number is set to indicate that the current block uses LFNST and the index number of the LFNST transform core in the LFNST transform core candidate set.

[0201] Here, for the value of the LFNST index number, when the value of the LFNST index number is equal to 0, the LFNST will not be used; and when the value of the LFNST index number is greater than 0, the LFNST will be used, and the index number of the transform core is equal to the value of the LFNST index number, or the value of the LFNST index number minus 1. In this way, after determining the LFNST transform core used by the current block, the LFNST index number needs to be set and written into the video bitstream, so that the subsequent decoder side can obtain the LFNST index number by parsing the bitstream.

[0202] S305: Use the LFNST transformation kernel to transform the prediction difference.

[0203] It should be noted that after determining the FNST transform kernel, the transform matrix selected for the current block can be obtained, and at this time, the prediction difference can be transformed.

[0204] Among them, each group of transform matrices can also include basic transform matrices T of two sizes, such as 16x16 and 16x48. For the transform matrices selected for the 4 types of TUs of different sizes, specifically, for the 4x4-sized TU, an 8x16 transform matrix will be used, and this 8x16 transform matrix comes from the first 8x16 of the 16x16 basic transform matrix; for the 4xN or Nx4 (N>4)-sized TU, a 16x16 basic transform matrix will be used; for the 8x8-sized TU, an 8x48 transform matrix will be used, and this 8x48 transform matrix comes from the first 8x48 of the 16x48 basic transform matrix; for the TU larger than 8x8 size, a 16x48 basic transform matrix will be used. It should be noted that in the current H.266 / VVC, only the transform matrix of the decoder-side LFNST (which can be represented by T T ) is stored, and the transform matrix used on the encoder side is the transpose matrix of the transform matrix of the LFNST (which can be represented by T).

[0205] It should also be noted that LFNST applies an inseparable transform based on the direct matrix multiplication method. In order to minimize the computational complexity and storage space as much as possible, a simplified inseparable transform technology is used in the LFNST transform. Among them, the main idea of the simplified inseparable transform technology is to map an N-dimensional vector to an R-dimensional vector in a different space. Here, N / R (R<N) is the scaling factor; at this time, the transform matrix corresponding to the simplified inseparable transform technology is an R×N matrix, as shown below,

[0206]

[0207] Here, the transform matrices used for the forward LFNST transform and the reverse LFNST transform are transpose relations with each other. Refer to Figure 5 , which shows a schematic structural diagram of the calculation process of a matrix multiplication of an LFNST technology provided by an embodiment of the present application. As Figure 5 shown, (a) shows the calculation process of the forward LFNST transform. After the primary transform coefficients pass through the transform matrix T, the secondary transform coefficients can be obtained; (b) shows the calculation process of the reverse LFNST transform. After the anti-secondary transform coefficients pass through the transpose transform matrix T T , the anti-primary transform coefficients can be obtained.

[0208] Furthermore, in the LFNST technology, it can be determined whether to use 4×4 inseparable transform or 8×8 inseparable transform according to the size of the current block; here, "4×4 inseparable transform" can be collectively referred to as "4×4LFNST" and "8×8 inseparable transform" can be collectively referred to as "8×8LFNST". Assuming that the width of the current block is nTbW and the height is nTbH, it can be concluded that: if min(nTbW, nTbH) <= 4, then 4×4LFNST can be used for the current block; otherwise, 8×8LFNST can be used for the current block. It should be noted that the return value of min(A, B) is the smaller value of A and B.

[0209] In one embodiment, for 4×4 LFNST, on the encoder side, 16 coefficients will be input, and after forward LFNST, 16 or 8 coefficients will be output; on the decoder side, 16 or 8 coefficients will be input, and 16 coefficients will be output; that is, the encoder and decoder are exactly opposite in terms of the number of inputs and outputs.

[0210] Assume that the size of a transform unit (TU) can be expressed as nTbW×nTbH, where the transform block is a prediction residual block obtained based on the prediction difference. That is, the size of a TU can be equal to 4×4, or 4×N or N×4 (N>4). The following describes them in detail.

[0211] When the TU size is equal to 4×4, the forward LFNST process corresponding to the 4×4 transform block is as follows Fig. 6A As shown. Fig. 6A In the example, the white blocks are prediction differences, the gray blocks are primary transform coefficients, and the black blocks are secondary transform coefficients; here, at the position of the "0" example, the encoder sets the transform coefficient to 0. For a 4×4 transform block, during forward LFNST, the transform matrix size used is 8×16, and all 4×4 primary transform coefficients in the current transform block are used as input, and the output is 4×2 secondary transform coefficients.

[0212] When the size of TU is equal to 4×N or N×4 (N>4), the forward LFNST process corresponding to the 4×N or N×4 transform block is as follows: Figure 6B As shown. Figure 6BIn the example, the white block is the prediction difference, the gray block is the primary transform coefficient, and the black block is the secondary transform coefficient; here, for a 4×N or N×4 transform block, during forward LFNST, the transform matrix size used is 16×16, and the primary transform coefficient in the first 4×4 sub-block in the current transform block (specifically, the uppermost sub-block for a 4×N transform block and the leftmost sub-block for an N×4 transform block) is used as input, and the output is 4×4 secondary transform coefficients. Here, at the position of the "0" example, the encoder still sets the transform coefficient to 0.

[0213] In another embodiment, for 8×8 LFNST, on the encoder side, 48 coefficients will be input, and after forward LFNST, 16 or 8 coefficients will be output; on the decoder side, 16 or 8 coefficients will be input, and 48 coefficients will be output; that is, the encoder and decoder are exactly opposite in terms of the number of inputs and outputs.

[0214] When the TU size is equal to 8×8, the forward LFNST process corresponding to the 8×8 transform block is as follows Figure 6C As shown. Figure 6C In the example, the white block is the prediction difference, the gray block is the primary transform coefficient, and the black block is the secondary transform coefficient. For an 8×8 transform block, the transform matrix size used in the forward LFNST is 8×48, and the primary transform coefficients of the first three 4×4 sub-blocks (i.e., the three sub-blocks in the upper left corner) in the current transform block are used as input, and the output is 4×2 secondary transform coefficients. Here, at the position of the "0" example, the encoder still sets the transform coefficient to 0.

[0215] When the TU size is larger than 8×8, the forward LFNST process corresponding to the transform block larger than 8×8 is as follows Fig.6D As shown. Fig.6D In the example, the white block is the prediction difference, the gray block is the primary transform coefficient, and the black block is the secondary transform coefficient. For transform blocks larger than 8×8, the transform matrix size used in the forward LFNST is 48×16, and the primary transform coefficients of the first three 4×4 sub-blocks (i.e., the three sub-blocks in the upper left corner) in the current transform block are used as input, and the output is 4×4 secondary transform coefficients. Here, at the position of the "0" example, the encoder still sets the transform coefficient to 0.

[0216] In this way, the TU corresponding to the prediction difference can be 4×4, 4×N, N×4 (N>4), or 8×8, or even larger than 8×8. Fig. 6A or Figure 6B or Figure 6C or Fig.6D To realize the transformation processing of the prediction difference.

[0217] In the embodiment of the present application, the applicability of the LFNST technology to the current block using the MIP mode can be improved, making the selection of the transform set (or transform kernel) more flexible. By introducing the relevant information of the MIP parameters in the process of performing LFNST on the current block using the MIP mode, and judging the characteristics of the current block based on these MIP information, the transform set (or transform kernel) is selected. For example, the transformation method is applied to VTM7.0, and tested under All Intra conditions at 24 frame intervals. Based on the average bit rate change under the same peak signal-to-noise ratio (PSNR), the BD-rate changes of -0.03%, 0.00% and -0.01% can be obtained on the three image components (ie, Y, Cb and Cr) respectively; especially in the sequence of large resolution, there will be better performance. Specifically, on Class A1, Y can achieve a BD-rate change of -0.10%, thereby improving the coding efficiency.

[0218] This embodiment provides a transformation method, which is applied to an encoder. Determine the prediction mode parameters of the current block; when the prediction mode parameters indicate that the current block uses a matrix-based intra-frame prediction MIP to determine an intra-frame prediction value, determine the MIP parameters; determine the intra-frame prediction value of the current block according to the MIP parameters, and calculate the prediction difference between the current block and the intra-frame prediction value; when the current block uses a low-frequency inseparable secondary transform LFNST, determine the LFNST transformation core used by the current block according to the MIP parameters, set the LFNST index number and write it into the video bitstream; use the LFNST transformation core to transform the prediction difference. In this way, for the current block using the MIP mode, since the MIP parameters are introduced when performing the LFNST transformation, the selection of the LFNST transformation core is more flexible, which not only improves the applicability of the LFNST technology to non-traditional intra-frame prediction modes, but also improves the coding efficiency, and can also improve the video image quality.

[0219] Based on the above Figure 2B For an example of an application scenario, see Figure 7 , which shows a flow chart of another transformation method provided in an embodiment of the present application. Figure 7 As shown, the method may include:

[0220] S701: Parse the bitstream and determine the prediction mode parameters of the current block;

[0221] It should be noted that the prediction mode parameter indicates the encoding mode of the current block and the parameters related to the mode. Among them, the prediction mode generally includes the traditional intra-frame prediction mode and the non-traditional intra-frame prediction mode, and the traditional intra-frame prediction mode can include the DC mode, the PLANAR mode and the angle mode, and the non-traditional intra-frame prediction mode can include the MIP mode, the CCLM mode, the IBC mode and the PLT mode.

[0222] It should also be noted that on the encoder side, predictive coding will be performed on the current block. During this process, the prediction mode of the current block can be determined, and the corresponding prediction mode parameters are written into the bitstream and transmitted from the encoder to the decoder.

[0223] On the decoder side, the intra prediction mode of the luminance or chrominance component of the current block or the coding block where the current block is located can be obtained by parsing the bitstream. At this time, the value of predModeIntra can be determined, and the calculation formula is as shown in the above formula (1).

[0224] In formula (1), the image component indicator (which can be represented by cIdx) is used to indicate the luminance component or chrominance component of the current block; here, if the current block is predicted as a luminance component, then cIdx is equal to 0; if the current block is predicted as a chrominance component, then cIdx is equal to 1. In addition, (xTbY, yTbY) is the coordinate of the upper left corner sampling point of the current block, IntraPredModeY[xTbY][yTbY] is the intra-frame prediction mode of the luminance component, and IntraPredModeC[xTbY][yTbY] is the intra-frame prediction mode of the chrominance component.

[0225] S702: When the prediction mode parameter indicates that the current block uses MIP to determine the intra-frame prediction value, parse the bitstream to determine the MIP parameter;

[0226] It should be noted that the MIP parameters may include parameters such as the MIP transposition indication parameter (which can be represented by isTransposed), the MIP mode index number (which can be represented by modeId), the size of the current block, the category of the current block (which can be represented by mipSizeId), etc. The values ​​of these parameters can be obtained by parsing the bitstream.

[0227] In some embodiments, the value of isTransposed can be determined by parsing the bitstream; when the value of isTransposed is equal to 1, the sampling point input vector used by the MIP mode needs to be transposed; when the value of isTransposed is equal to 0, the sampling point input vector used by the MIP mode does not need to be transposed; that is, the MIP transposition indication parameter can be used to indicate whether to transpose the sampling point input vector used by the MIP mode.

[0228] In some embodiments, by parsing the bitstream, the MIP mode index number (which can be represented by modeId) can also be determined; wherein the MIP mode index number can be used to indicate the MIP mode used by the current block, and the MIP mode can be used to indicate the calculation and derivation method of using the MIP to determine the intra-frame prediction value of the current block. That is to say, different MIP modes have different corresponding MIP mode index numbers; here, the MIP mode index number can be 0, 1, 2, 3, 4 or 5.

[0229] In some embodiments, by parsing the bitstream, the size, aspect ratio, category (which can be represented by mipSizeId) and other parameter information of the current block can also be determined. In this way, after the MIP parameters are determined, it is convenient to select the LFNST transformation kernel (which can be represented by kernel) used for the current block according to the determined MIP parameters.

[0230] S703: Parse the bitstream to determine the transform coefficient and LFNST index number of the current block;

[0231] It should be noted that the value of the LFNST index number can be used to indicate whether the current block uses LFNST and the index number of the LFNST transform core in the LFNST transform core candidate set. Specifically, after parsing the LFNST index number, when the value of the LFNST index number is equal to 0, it indicates that the current block does not use LFNST; and when the value of the LFNST index number is greater than 0, it indicates that the current block uses LFNST, and the index number of the transform core is equal to the value of the LFNST index number, or the value of the LFNST index number minus 1.

[0232] It should also be noted that, on the decoder side, the input data of LFNST may include: the brightness position of the current transform block (xTbY, yTbY), the width of the current block nTbW, the height of the current block nTbH, whether the current block is a brightness component or a chrominance component cIdx, and the coefficients of the current transform block after inverse quantization (scaling) d[x][y], x=0,1,…,nTbW-1, y=0,1,…,nTbH-1; the output data of LFNST may include: the primary transform coefficients d'[x][y] generated by the secondary transform coefficients through LFNST, x=0,1,…,nLfnstSize-1, y=0,1,…,nLfnstSize-1.

[0233] Specifically, Figure 8As shown, the specific process of LFNST can be divided into five steps: configuring core parameters 81, intra prediction mode mapping 82, selecting transformation matrix 83, calculating matrix multiplication 84 and constructing a transformation coefficient matrix 85. Among them, for intra prediction mode mapping 82, this step is used to determine the value of predModeIntra, which can mainly include: non-traditional intra prediction mode mapping 821 and wide angle mapping 822; for selecting transformation matrix 83, this step is used to select a transformation set and a transformation matrix, which can mainly include: selecting a transformation set 831, selecting a transformation matrix group 832 and selecting a transformation matrix size 833.

[0234] For configuring the core parameter 81, first, it is necessary to configure the length of the input secondary transform coefficient vector for LFNST calculation (which can be represented by nonZeroSize) and the length of the output primary transform coefficient vector (which can be represented by nLfnstOutSzie). The values ​​of nonZeroSize and nLfnstOutSzie are shown in Table 5.

[0235] Table 5

[0236] Change block size nonZeroSize nLfnstOutSzie 4×4 8 16 4×N or N×4 (N>4) 16 16 8×8 8 48 >8×8 16 48

[0237] In the current H.266 / VVC, the nonZeroSize and nLfnstOutSzie parameters are calculated using the following formula:

[0238] nLfnstOutSzie=(nTbW>=8&&nTbH>=8)? 48:16 (5)

[0239] nonZeroSize=(nTbW==4&&nTbH==4)||(nTbW==8&&nTbH==8)? 8:16 (6)

[0240] In addition, the parameter nLfnstSize needs to be configured, which means that there will be only one transform coefficient within the first nLfnstSize×nLfnstSize range in the current block. The value of nLfnstSize is as follows:

[0241] Log2LfnstSize=(nTbW>=8&&nTbH>=8)? 3:2 (7)

[0242] nLfnstSize=1<<Log2LfnstSize (8)

[0243] At this time, by parsing the bitstream, the intra-frame prediction mode of the brightness or chrominance component of the current block or the coding block where the current block is located can be obtained. At this time, the value of predModeIntra can be determined, and the calculation formula is as shown in the above formula (1).

[0244] Further, obtain the vector u[i] of the secondary change coefficient, i=0,1,…,nonZeroSize-1. When it is determined that the current transform block uses LFNST, the inverse quantized coefficient d[x][y] at this time is the secondary transform coefficient. Obtain its first nonZeroSize values ​​in diagonal scanning order, which is the vector u[i] of the secondary change coefficient, i=0,1,…,nonZeroSize-1; in the following formula, xC and yC are expressed as the horizontal and vertical coordinates of the coefficient numbered x in the current block relative to the upper left corner in diagonal order), xC and yC are as follows,

[0245] xC=DiagScanOrder[2][2][x][0] (9)

[0246] yC=DiagScanOrder[2][2][x][1] (10)

[0247] u[i]=d[xC][yC] (11)

[0248] Further, for the intra prediction mode mapping 82, the intra prediction mode can be divided into a traditional intra prediction mode and a non-traditional intra prediction mode. For the non-traditional intra prediction mode, the information indicated by the value of predModeIntra is as follows:

[0249] If the value of predModeIntra can be INTRA_LT_CCLM, INTRA_L_CCLM or INTRA_T_CCLM (81, 82, 83 in VVC respectively), it indicates that the prediction mode of the current block is CCLM mode;

[0250] If intra_mip_flag[xTbY][yTbY] is equal to 1 and cIdx is equal to 0, it indicates that the prediction mode of the current block is MIP mode. At this time, the value of predModeIntra indicates the MIP mode index number modeId used;

[0251] If it is not the above two cases, the value of predModeIntra can be [0,66], indicating that the prediction mode of the current block is the traditional intra-frame prediction mode.

[0252] Furthermore, the LFNST transform kernel candidate set index number is determined by parsing the bitstream according to the number of the traditional intra-frame prediction mode. At this time, if the prediction mode of the current block is CCLM mode or MIP mode, the value of predModeIntra can also be set in the following way:

[0253] (1) When the value of predModeIntra indicates INTRA_LT_CCLM, INTRA_L_CCLM, or INTRA_T_CCLM (81, 82, 83 in VVC, respectively),

[0254] If the mode of the central luminance block at the luminance position corresponding to the current block (such as the chrominance block) is the MIP mode, that is, intra_mip_flag[xTbY+nTbW / 2][yTbY+nTbH / 2] is 1, the value of predMode Intra is set to the index number indicating the PLANAR mode (that is, 0);

[0255] Otherwise, if the mode of the central luminance block at the luminance position corresponding to the current block (such as the chrominance block) is the IBC mode or the PLT mode, the value of predModeIntra is set to the index number indicating the DC mode (i.e. 1);

[0256] Otherwise, the value of predModeIntra is set to the value of the mode index number of the central luminance block corresponding to the luminance position of the current block (such as the chrominance block) IntraPredModeY[xTbY+nTbW / 2][yTbY+nTbH / 2];

[0257] (2) When intra_mip_flag[xTbY][yTbY] is equal to 1 and cIdx is equal to 0, that is, the prediction mode of the current block is the MIP mode, the value of predModeIntra can be directly set to the index number indicating the PLANAR mode (that is, 0).

[0258] For the traditional intra prediction mode (such as wide angle mapping), during the code stream parsing process, wide angle mapping can also be performed according to the size of the current block, and the traditional intra prediction mode [0, 66] is extended to [-14, 80]. The specific mapping process is as follows:

[0259] First, the width-to-height ratio factor (which can be expressed as whRatio) is calculated, as shown in the above formula (2).

[0260] For a non-square current block (i.e., nTbW is not equal to nTbH), the value of predModeIntra can be modified as follows: if nTbW is greater than nTbH, and predModeIntra is greater than or equal to 2, and predModeIntra is less than ((whRatio>1?(8+2×whRatio):8), then predModeIntra=(predModeIntra+65); otherwise, if nTbW is less than nTbH, and predModeIntra is less than or equal to 66, and predModeIntra is greater than ((whRatio>1?(60-2×whRatio):60), then predModeIntra=(predModeIntra-67).

[0261] In the current H.266 / VVC, according to the value of predModeIntra and Table 1, the value of the LFNST index number (which can be represented by SetIdx) can be determined, and the specific values ​​are shown in Table 1. Here, the value of the LFNST index number is set to indicate that the current block uses LFNST and the index number of the LFNST transform core in the LFNST transform core candidate set. Generally speaking, the LFNST transform set includes four transform core candidate sets (set0, set1, set2, set3), corresponding to the values ​​of SetIdx of 0, 1, 2, and 3, respectively.

[0262] In the current H.266 / VVC, for the MIP mode, since the value of predModeIntra is set to the index number indicating the PLANAR mode (i.e., 0), the transform set used by the current block in the MIP mode can only select the transform set with the LFNST index number equal to 0, resulting in a lack of variability when performing LFNST in the MIP mode, thereby reducing the decoding efficiency. In the embodiment of the present application, after determining the LFNST index number, the LFNST transform core candidate set can be determined according to the MIP parameters, and then the LFNST transform core used by the current block can be determined from the LFNST transform core candidate set.

[0263] S704: When the LFNST index number indicates that the current block uses LFNST, determine the LFNST transformation kernel used by the current block according to the MIP parameter;

[0264] Here, the MIP parameters may include parameters such as a MIP transposition indication parameter (which can be represented by isTransposed), a MIP mode index number (which can be represented by modeId), the size of the current block, the category of the current block (which can be represented by mipSizeId), etc.; the following will describe in detail how to select the LFNST transform kernel used for the current block based on the MIP parameters.

[0265] Optionally, in some embodiments, when the MIP parameter is a MIP transposition indication parameter, for S704, when the LFNST index number indicates that the current block uses LFNST, determining, according to the MIP parameter, a LFNST transformation kernel used by the current block may include:

[0266] Selecting a transformation core indicated by the LFNST index number from the LFNST transformation core candidate set;

[0267] When the value of the MIP transposition indication parameter indicates that the sampling point input vector used by the MIP mode is transposed, the selected transform kernel is matrix transposed to obtain the LFNST transform kernel used by the current block; wherein the LFNST transform kernel candidate set includes two or more transform kernels preset for MIP.

[0268] It should be noted that, since the LFNST transform core candidate set includes two or more transform cores preset for MIP, after the LFNST index number is obtained by parsing the bitstream, the transform core indicated by the LFNST index number can be selected from the LFNST transform core candidate set according to the value of the obtained LFNST index number. For example, when the value of the LFNST index number is 1, the first group of LFNST transform cores (i.e., the first group of transform matrices) in the LFNST transform core candidate set will be selected; and when the value of the LFNST index number is 2, the second group of LFNST transform cores (i.e., the second group of transform matrices) in the LFNST transform core candidate set will be selected.

[0269] It should also be noted that since the value of the MIP transposition indication parameter is used to indicate whether to transpose the sampling point input vector used by the MIP mode, when the value of the MIP transposition indication parameter is equal to 1, that is, the value of the MIP transposition indication parameter indicates that the sampling point input vector used by the MIP mode is transposed, then the selected transform kernel needs to be matrix transposed to obtain the LFNST transform kernel used by the current block.

[0270] Here, for the value of the LFNST index number (i.e., lfnst_idx), when the value of the LFNST index number is equal to 0, LFNST will not be used; and when the value of the LFNST index number is greater than 0, LFNST will be used, and the index number of the transform core is equal to the value of the LFNST index number, or the value of the LFNST index number minus 1. In this way, according to the LFNST index number, the LFNST transform core used by the current block can be determined.

[0271] Optionally, in some embodiments, when the MIP parameter is a MIP mode index number, for S704, when the LFNST index number indicates that the current block uses LFNST, determining, according to the MIP parameter, a LFNST transform kernel used by the current block may include:

[0272] Determine the value of the LFNST intra prediction mode index number according to the value of the MIP mode index number;

[0273] Selecting an LFNST transform core candidate set from multiple LFNST transform core candidate sets according to the value of the LFNST intra prediction mode index sequence number;

[0274] A transformation core indicated by the LFNST index number is selected from the selected LFNST transformation core candidate set, and is set as the LFNST transformation core used by the current block; wherein the LFNST transformation core candidate set includes two or more preset LFNST transformation cores.

[0275] It should be noted that the MIP mode index number is used to indicate the MIP mode used by the current block, and the MIP mode is used to indicate the calculation and derivation method of using MIP to determine the intra-frame prediction value of the current block; that is, according to the MIP mode index number, the LFNST transform kernel can also be determined.

[0276] It should also be noted that after determining the MIP mode index number, the MIP mode index number can also be converted into the value of the LFNST intra-frame prediction mode index number (which can be represented by predModeIntra); then, based on the value of predModeIntra, an LFNST transform core candidate set is selected from multiple LFNST transform core candidate sets to determine the transform core candidate set; and in the selected LFNST transform core candidate set, the transform core indicated by the LFNST index number is selected and set as the LFNST transform core used by the current block.

[0277] Optionally, in some embodiments, when the MIP parameters are the MIP mode index number and the MIP transposition indication parameter, for S704, when the LFNST index number indicates that the current block uses LFNST, determining, according to the MIP parameters, the LFNST transform kernel used by the current block may include:

[0278] Determine the value of the LFNST intra prediction mode index number according to the value of the MIP mode index number;

[0279] Selecting an LFNST transform core candidate set from multiple LFNST transform core candidate sets according to the value of the LFNST intra prediction mode index sequence number;

[0280] Selecting a transformation core indicated by the LFNST index number from the selected LFNST transformation core candidate set, and setting it as the LFNST transformation core used by the current block;

[0281] When the value of the MIP transposition indication parameter indicates that a sampling point input vector used by the MIP mode is to be transposed, a matrix transposition is performed on the LFNST transform kernel used by the current block, and the transform kernel obtained after the transposition is set as the LFNST transform kernel used by the current block; wherein the LFNST transform kernel candidate set includes two or more preset LFNST transform kernels.

[0282] It should be noted that the MIP mode index number is used to indicate the MIP mode used by the current block, and the MIP mode is used to indicate the calculation and derivation method of using the MIP to determine the intra-frame prediction value of the current block; the value of the MIP transposition indication parameter is used to indicate whether to transpose the sampling point input vector used by the MIP mode; that is, the LFNST transform kernel can also be determined based on the combination of the MIP transposition indication parameter and the MIP mode index number.

[0283] It should also be noted that after determining the MIP mode index number, the MIP mode index number can be converted into the value of the LFNST intra-frame prediction mode index number (which can be represented by predModeIntra), and then, according to the value of predModeIntra, an LFNST transform core candidate set is selected from multiple LFNST transform core candidate sets to determine the transform core candidate set; and in the selected LFNST transform core candidate set, the transform core indicated by the LFNST index number is selected; and when it is indicated that transposition processing is required, the LFNST transform core used by the current block needs to be matrix transposed, and then the transform core obtained after the transposition processing is set as the LFNST transform core used by the current block.

[0284] It should also be noted that, for the above-mentioned methods of selecting the LFNST transform kernel used for the current block, experimental results show that the performance of the LFNST transform kernel determined according to the combination of the MIP transposition indication parameter and the MIP mode index number is the best.

[0285] Furthermore, when the MIP parameters include at least a MIP mode index number (modeId), in the process of selecting the LFNST transform kernel used for the current block, the value of the LFNST intra prediction mode index number can also be obtained by looking up a table.

[0286] Specifically, in some embodiments, determining the value of the LFNST intra prediction mode index number according to the value of the MIP mode index number may include:

[0287] A first lookup table is used to determine the value of the LFNST intra-frame prediction mode index number corresponding to the value of the MIP mode index number, wherein the first lookup table contains one or more different MIP mode index numbers corresponding to at least two LFNST intra-frame prediction mode index numbers with different values.

[0288] Here, the first look-up table (Look-Up Table 1, LUT1) is used to reflect the correspondence between the MIP mode index number and the LFNST intra-frame prediction mode index number, that is, the first look-up table contains at least one or more different MIP mode index numbers corresponding to two LFNST intra-frame prediction mode index numbers with different values.

[0289] That is to say, different MIP modes may correspond to different values ​​of predModeIntra; thus, the MIP mode index number is determined according to the MIP mode, and then the value of predModeIntra is determined according to the first lookup table; then, according to the value of predModeIntra, an LFNST transform core candidate set may be selected from multiple LFNST transform core candidate sets, thereby determining the LFNST transform core used for the current block.

[0290] It can be understood that the value of predModeIntra can be determined according to the value of the MIP mode index number (modeId); then according to the value of predModeIntra, the value of SetIdx can be directly determined according to Table 2, that is, the LFNST transform core candidate set selected by the current block is determined. Here, the value of SetIdx indicates the transform core candidate set used in LFNST; since the value of modeId can include 0, 1, 2, 3, 4, 5, the value of predModeIntra is also 0, 1, 2, 3, 4, 5; the corresponding relationship between predModeIntra and SetIdx can be shown in Table 2.

[0291] Furthermore, the LFNST transform kernel candidate set can be directly determined according to the value of the MIP mode index number. In this case, there is no need to use the value of predModeIntra, that is, there is no need to determine the value of the LFNST intra prediction mode index number according to the MIP mode index number.

[0292] Optionally, in some embodiments, when the LFNST index number indicates that the current block uses LFNST, determining, according to the MIP parameter, a LFNST transform kernel used by the current block may include:

[0293] Selecting an LFNST transform core candidate set from multiple LFNST transform core candidate sets according to the value of the MIP mode index sequence number;

[0294] A transformation core indicated by the LFNST index number is selected from the selected LFNST transformation core candidate set, and is set as the LFNST transformation core used by the current block; wherein the LFNST transformation core candidate set includes two or more preset LFNST transformation cores.

[0295] Further, selecting an LFNST transform core candidate set from multiple LFNST transform core candidate sets according to the value of the MIP mode index sequence number may include:

[0296] Determine the value of the LFNST transform core candidate set index number corresponding to the value of the MIP mode index number using a second lookup table, and select the LFNST transform core candidate set indicated by the value of the LFNST transform core candidate set index number as the selected LFNST transform core candidate set;

[0297] The second look-up table (Look-Up Table 2, LUT2) contains at least one or more different MIP mode index numbers corresponding to two different values ​​of the LFNST transform core candidate set index numbers.

[0298] It should be noted that if the prediction mode used by the current block is the MIP mode, the selected LFNST transform core candidate set can be determined according to the value of the MIP mode index number (modeId). As shown in Table 3, each MIP mode index number corresponds to a LFNST transform core candidate set.

[0299] In this way, according to the value of the MIP mode index number, the value of the corresponding LFNST transform core candidate set index number can be determined to select the LFNST transform core candidate set indicated by the value of the LFNST transform core candidate set index number as the selected LFNST transform core candidate set. For example, as can be seen from Table 3, when the value of the MIP mode index number (modeId) is 0, it can be determined that the value of the LFNST transform core candidate set index number (SetIdx) is 0, that is, the transform core candidate set indicated by 0 is used as the selected LFNST transform core candidate set; or, when modeId is 3, it can be determined that the value of SetIdx is 1, that is, the transform core candidate set indicated by 1 is used as the selected LFNST transform core candidate set, and so on.

[0300] Optionally, in some embodiments, selecting an LFNST transform core candidate set from multiple LFNST transform core candidate sets according to the value of the MIP mode index sequence number may include:

[0301] According to the value of the MIP mode index number, determine the index number of the LFNST transform kernel candidate set using a first calculation method;

[0302] The LFNST transform core candidate set indicated by the value of the LFNST transform core candidate set index sequence number is selected as the selected LFNST transform core candidate set.

[0303] Further, the determining the index number of the LFNST transform kernel candidate set by using the first calculation method according to the value of the MIP mode index number may include:

[0304] When the value of the MIP mode index number is less than a first preset value, setting the value of the LFNST transform core candidate set index number to be equal to the value of the MIP mode index number;

[0305] When the value of the MIP mode index number is greater than or equal to a first preset value, the value of the LFNST transform core candidate set index number is set to be equal to the difference between the value of the MIP mode index number and the second preset value; wherein the first preset value and the second preset value are both integer values.

[0306] In an embodiment of the present application, the value of the first preset value may be equal to 3, and the value of the second preset value may be equal to 2.

[0307] That is to say, according to the value of the MIP mode index number, the first calculation method can be used to determine the index number of the LFNST transform core candidate set, where the first calculation formula is as shown in the above formula (3), x represents the first preset value, y represents the second preset value; SetIdx represents the value of the LFNST transform core candidate set index number, and modeId represents the value of the MIP mode index number.

[0308] Specifically, assuming that x is equal to 3 and y is equal to 2; then when modeId<3, it means that SetIdx can be 0 at this time; when modeId≥3, it means that SetIdx can be modeId-2 at this time; that is, when modeId=3, SetIdx can be 1; when modeId=4, SetIdx can be 2; when modeId=5, SetIdx can be 3; it can be seen that according to formula (3), the value of SetIdx obtained is the same as the content of Table 3.

[0309] In addition, if the prediction mode of the current block is the MIP mode, for determining the LFNST transform core candidate set, one of the transform sets (such as set1 or set2 or set3) can be fixedly selected as an LFNST transform core candidate set selected from multiple LFNST transform core candidate sets.

[0310] In addition, if the prediction mode of the current block is the MIP mode, the LFNST transform core candidate set can also be selected according to the value of the MIP mode index number (modeId); as shown in Table 4, each MIP mode corresponds to an LFNST transform core candidate set; here, the specific number of the LFNST transform core candidate set is limited; wherein a, b, c, d, e can be any value from {0, 1, 2, 3}.

[0311] It should also be noted that if the prediction mode of the current block is the MIP mode, the LFNST transform kernel candidate set can also be selected according to one or more information combinations in the MIP parameters; it can also be mapped with the traditional intra-frame prediction mode according to the MIP parameters, and then the LFNST transform kernel candidate set can be selected according to the mapped angle; it can even be determined according to one or more information combinations in the MIP parameters to determine the transform matrix (i.e., transform kernel) to be used, and determining the transform matrix includes the selection of a transform set and a transform matrix group; here, a transform matrix group in a transform set can be fixedly selected according to the MIP parameters, and there is no need to select it on the encoder side, and there is no need to transmit lfnst_idx, and the decoder side does not need to parse the lfnst_idx bitstream.

[0312] Further, in some embodiments, when the prediction mode parameter indicates that the current block uses a non-MIP mode, the method may further include:

[0313] Based on the intra prediction mode, determine the value of predModeIntra;

[0314] According to the value of predModeIntra, an LFNST transform core candidate set is selected from multiple LFNST transform core candidate sets.

[0315] That is to say, if the prediction mode of the current block is non-MIP mode, the value of predModeIntra can be determined according to the intra prediction mode; then according to the value of predModeIntra, combined with the above Table 1, an LFNST transform core candidate set can be selected from multiple LFNST transform core candidate sets, and then the transform core indicated by the LFNST index number is selected from the selected LFNST transform core candidate set, and set as the LFNST transform core used by the current block.

[0316] S705: Use the LFNST transformation kernel to perform transformation processing on the transformation coefficients.

[0317] It should be noted that after selecting the LFNST transform core candidate set, the value of the LFNST index number (lfnst_idx) is obtained by parsing the bitstream; according to the value of lfnst_idx, the transform matrix (transform core) indicated by lfnst_idx can be selected from the LFNST transform core candidate set. For example, when lfnst_idx is 1, the first set of transform matrices can be used as the LFNST transform core during decoding; when lfnst_idx is 2, the second set of transform matrices can be used as the LFNST transform core during decoding.

[0318] Furthermore, for each set of transformation matrices (transformation kernels), there are two sizes of basic transformation matrices, and the basic transformation matrix sizes used on the decoder side are 16×16 and 48×16. The selection is made according to nLfnstOutSzie. If nLfnstOutSzie is 16, the 16x16 basic transformation matrix is ​​selected; or, if nLfnstOutSzie is 48, the 48x16 basic transformation matrix is ​​selected. Alternatively, if nonZeroSize is 8, only the first 8 rows in the transformation matrix are used for matrix multiplication calculation.

[0319] Furthermore, the secondary transform coefficient vector u[i] is taken as input and multiplied with the transformation matrix to obtain the primary transform coefficient vector v[j], where i = 0, 1, ..., nonZeroSize-1, j = 0, 1, ..., nLfnstOutSzie-1. Assuming that the transformation matrix obtained in the previous step is lowFreqTransMatrix, the specific calculation process of v[j] is as follows:

[0320]

[0321] Here, Clip3 works as a clamping operation, which can limit the value of the coefficient between the following two numbers, as shown below:

[0322] CoeffMin=-(1<<15) (13)

[0323] CoeffMax=(1<<15)-1 (14)

[0324] In this way, after the above matrix calculation, the transformation processing of the transformation coefficients can be realized. Here, for 4×4 LFNST, on the decoder side, 16 or 8 coefficients will be input and 16 coefficients will be output; and for 8×8 LFNST, on the decoder side, 16 or 8 coefficients will be input and 48 coefficients will be output to realize the LFNST transformation processing of the transformation coefficients.

[0325] In the embodiment of the present application, the applicability of the LFNST technology to the current block using the MIP mode can be improved, making the selection of the transform set (or transform kernel) more flexible. By introducing the relevant information of the MIP parameters in the process of performing LFNST on the current block using the MIP mode, and judging the characteristics of the current block based on these MIP information, the transform set (or transform kernel) is selected. For example, the transformation method is applied to VTM7.0, and tested under All Intra conditions at 24 frame intervals. Based on the average bit rate change under the same peak signal-to-noise ratio (PSNR), the BD-rate changes of -0.03%, 0.00% and -0.01% can be obtained on the three image components (ie, Y, Cb and Cr) respectively; especially in the sequence of large resolution, there will be better performance, specifically, on Class A1, Y can reach a BD-rate change of -0.10%, thereby improving the decoding efficiency.

[0326] This embodiment provides a transformation method, which parses a bitstream to determine the prediction mode parameters of the current block; when the prediction mode parameters indicate that the current block uses MIP to determine the intra-frame prediction value, parses the bitstream to determine the MIP parameters; parses the bitstream to determine the transformation coefficient and LFNST index number of the current block; when the LFNST index number indicates that the current block uses LFNST, determines the LFNST transformation kernel used by the current block according to the MIP parameters; and uses the LFNST transformation kernel to transform the transformation coefficient. In this way, for the current block using the MIP mode, since the MIP parameters are introduced when performing the LFNST transformation, the selection of the LFNST transformation kernel is more flexible, which not only improves the applicability of the LFNST technology to non-traditional intra-frame prediction modes, but also improves the decoding efficiency, and can also improve the video image quality.

[0327] Based on the same inventive concept as the above embodiments, see Fig. 9 , which shows a schematic diagram of the structure of an encoder 90 provided in an embodiment of the present application. Fig. 9 As shown, the encoder 90 may include: a first determining unit 901, a first calculating unit 902 and a first transforming unit 903; wherein,

[0328] A first determining unit 901 is configured to determine a prediction mode parameter of a current block;

[0329] The first determining unit 901 is further configured to determine a MIP parameter when the prediction mode parameter indicates that the current block uses a matrix-based intra prediction MIP to determine an intra prediction value;

[0330] A first calculation unit 902 is configured to determine an intra prediction value of the current block according to the MIP parameter, and calculate a prediction difference between the current block and the intra prediction value;

[0331] The first determining unit 901 is further configured to, when the current block uses a low frequency non-separable secondary transform LFNST, determine the LFNST transform kernel used by the current block according to the MIP parameter, set the LFNST index number and write it into the video bitstream;

[0332] The first transform unit 903 is configured to use the LFNST transform kernel to perform transform processing on the prediction difference.

[0333] In the above solution, the MIP parameter includes a MIP transposition indication parameter, wherein the value of the MIP transposition indication parameter is used to indicate whether to perform transposition processing on the sampling point input vector used by the MIP mode.

[0334] In the above scheme, see Fig. 9 , the encoder 90 may further include a first selection unit 904, a first transposition unit 905 and a setting unit 906; wherein,

[0335] A first selection unit 904 is configured to select a transform kernel used by the current block from a LFNST transform kernel candidate set;

[0336] A first transposition unit 905 is configured to perform matrix transposition processing on the selected transformation kernel to obtain the LFNST transformation kernel used by the current block when the value of the MIP transposition indication parameter indicates that the sampling point input vector used by the MIP mode is to be transposed;

[0337] The setting unit 906 is configured to set the value of the LFNST index number to indicate that the current block uses LFNST and the index number of the LFNST transform core in the LFNST transform core candidate set; wherein the LFNST transform core candidate set includes two or more transform cores preset for MIP.

[0338] In the above scheme, the MIP parameter includes a MIP mode index number, wherein the MIP mode index number is used to indicate the MIP mode used by the current block, and the MIP mode is used to indicate a calculation and derivation method for determining an intra-frame prediction value of the current block using the MIP.

[0339] In the above solution, the first determining unit 901 is further configured to determine the value of the LFNST intra prediction mode index number according to the value of the MIP mode index number;

[0340] The first selection unit 904 is further configured to select an LFNST transform core candidate set from multiple LFNST transform core candidate sets according to the value of the LFNST intra prediction mode index number; and select a transform core indicated by the LFNST index number from the selected LFNST transform core candidate set, and set it as the LFNST transform core used by the current block;

[0341] The setting unit 906 is configured to set the value of the LFNST index number to indicate that the current block uses LFNST and the index number of the LFNST transform core in the LFNST transform core candidate set; wherein the LFNST transform core candidate set includes two or more preset LFNST transform cores.

[0342] In the above scheme, the first determination unit 901 is further configured to use a first lookup table to determine the value of the LFNST intra-frame prediction mode index number corresponding to the value of the MIP mode index number, wherein the first lookup table contains at least one or more different MIP mode index numbers corresponding to two LFNST intra-frame prediction mode index numbers with different values.

[0343] In the above solution, the first selection unit 904 is further configured to select an LFNST transform core candidate set from multiple LFNST transform core candidate sets according to the value of the MIP mode index number; and select the transform core indicated by the LFNST index number from the selected LFNST transform core candidate set, and set it as the LFNST transform core used by the current block;

[0344] The setting unit 906 is configured to set the value of the LFNST index number to indicate that the current block uses LFNST and the index number of the LFNST transform core in the LFNST transform core candidate set; wherein the LFNST transform core candidate set includes two or more preset LFNST transform cores.

[0345] In the above scheme, the first selection unit 904 is further configured to use a second lookup table to determine the value of the LFNST transform core candidate set index number corresponding to the value of the MIP mode index number, and select the LFNST transform core candidate set indicated by the value of the LFNST transform core candidate set index number as the selected LFNST transform core candidate set; wherein the second lookup table contains at least one or more different MIP mode index numbers corresponding to two different values ​​of the LFNST transform core candidate set index numbers.

[0346] In the above solution, the first calculation unit 902 is further configured to determine the index number of the LFNST transform kernel candidate set using the first calculation method according to the value of the MIP mode index number;

[0347] The first selection unit 904 is further configured to select the LFNST transform core candidate set indicated by the value of the LFNST transform core candidate set index sequence number as the selected LFNST transform core candidate set.

[0348] In the above scheme, the first calculation unit 902 is specifically configured to set the value of the LFNST transform core candidate set index number to be equal to the value of the MIP mode index number when the value of the MIP mode index number is less than the first preset value; and to set the value of the LFNST transform core candidate set index number to be equal to the difference between the value of the MIP mode index number and the second preset value when the value of the MIP mode index number is greater than or equal to the first preset value; wherein the first preset value and the second preset value are both integer values.

[0349] In the above solution, the value of the first preset value is equal to 3.

[0350] In the above solution, the value of the second preset value is equal to 2.

[0351] In the above solution, the MIP parameters also include a MIP transposition indication parameter, wherein the value of the MIP transposition indication parameter is used to indicate whether to perform transposition processing on the sampling point input vector used by the MIP mode;

[0352] The first transposition unit 905 is further configured to perform matrix transposition processing on the LFNST transform kernel used by the current block when the value of the MIP transposition indication parameter indicates that the sampling point input vector used by the MIP mode is to be transposed, and set the transform kernel obtained after the transposition processing as the LFNST transform kernel used by the current block.

[0353] It is understandable that in the embodiments of the present application, a "unit" may be a part of a circuit, a part of a processor, a part of a program or software, etc., and of course, it may be a module, or it may be non-modular. Moreover, the components in the present embodiment may be integrated into a processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of a software functional module.

[0354] If the integrated unit is implemented in the form of a software function module and is not sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this embodiment is essentially or the part that contributes to the prior art or the whole or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) or a processor to perform all or part of the steps of the method described in this embodiment. The aforementioned storage medium includes: U disk, mobile hard disk, read only memory (ROM), random access memory (RAM), disk or optical disk, etc., various media that can store program codes.

[0355] Therefore, an embodiment of the present application provides a computer storage medium, which is applied to the encoder 90, and the computer storage medium stores a transformation program, and when the transformation program is executed by the first processor, the method described in any one of the above embodiments is implemented.

[0356] Based on the composition of the encoder 90 and the computer storage medium, see Fig.10 , which shows a specific hardware structure example of the encoder 90 provided in an embodiment of the present application, which may include: a first communication interface 1001, a first memory 1002 and a first processor 1003; each component is coupled together through a first bus system 1004. It can be understood that the first bus system 1004 is used to realize the connection and communication between these components. In addition to the data bus, the first bus system 1004 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, in Fig.10 In the figure, various buses are marked as the first bus system 1004. Among them,

[0357] The first communication interface 1001 is used for receiving and sending signals during the process of sending and receiving information with other external network elements;

[0358] A first memory 1002, used to store a computer program that can be run on the first processor 1003;

[0359] The first processor 1003 is configured to, when running the computer program, execute:

[0360] Determining prediction mode parameters for the current block;

[0361] When the prediction mode parameter indicates that the current block uses a matrix-based intra prediction MIP to determine an intra prediction value, determining a MIP parameter;

[0362] Determine an intra-frame prediction value of a current block according to the MIP parameter, and calculate a prediction difference between the current block and the intra-frame prediction value;

[0363] When the current block uses a low frequency non-separable secondary transform LFNST, determining the LFNST transform kernel used by the current block according to the MIP parameter, setting the LFNST index number and writing it into the video bitstream;

[0364] The prediction difference is transformed using the LFNST transformation kernel.

[0365] It can be understood that the first memory 1002 in the embodiment of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM) and direct memory bus random access memory (DRRAM). The first memory 1002 of the system and method described in the present application is intended to include but is not limited to these and any other suitable types of memory.

[0366] The first processor 1003 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the first processor 1003. The above-mentioned first processor 1003 can be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application can be directly embodied as a hardware decoding processor to execute, or the hardware and software modules in the decoding processor are combined and executed. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the first memory 1002, and the first processor 1003 reads the information in the first memory 1002 and completes the steps of the above method in combination with its hardware.

[0367] It is to be understood that these embodiments described in the present application can be implemented with hardware, software, firmware, middleware, microcode or a combination thereof. For hardware implementation, the processing unit can be implemented in one or more application specific integrated circuits (Application Specific Integrated Circuits, ASIC), digital signal processors (Digital Signal Processing, DSP), digital signal processing equipment (DSP Device, DSPD), programmable logic devices (Programmable Logic Device, PLD), field programmable gate arrays (Field-Programmable Gate Array, FPGA), general-purpose processors, controllers, microcontrollers, microprocessors, other electronic units or combinations thereof for performing functions described in the present application. For software implementation, the technology described in the present application can be implemented by a module (such as a process, function, etc.) that performs functions described in the present application. The software code can be stored in a memory and executed by a processor. The memory can be implemented in a processor or outside a processor.

[0368] Optionally, as another embodiment, the first processor 1003 is further configured to execute the method described in any one of the aforementioned embodiments when running the computer program.

[0369] This embodiment provides an encoder, which may include a first determination unit, a first calculation unit and a first transformation unit; thus, for the current block adopting the MIP mode, since the MIP parameters are introduced when performing the LFNST transformation, the selection of the LFNST transformation kernel is more flexible, thereby not only improving the applicability of the LFNST technology to non-traditional intra-frame prediction modes, but also improving the encoding efficiency, while also being able to improve the video image quality.

[0370] Based on the same inventive concept as the above embodiments, see Fig.11 , which shows a schematic diagram of the structure of a decoder 110 provided in an embodiment of the present application. Fig.11 As shown, the decoder 110 may include: a parsing unit 1101, a second determining unit 1102, and a second transforming unit 1103; wherein,

[0371] The parsing unit 1101 is configured to parse the bitstream to determine the prediction mode parameters of the current block; and is further configured to parse the bitstream to determine the MIP parameters when the prediction mode parameters indicate that the current block uses MIP to determine the intra-frame prediction value; and is further configured to parse the bitstream to determine the transform coefficient and LFNST index number of the current block;

[0372] A second determining unit 1102 is configured to determine, according to the MIP parameter, an LFNST transform kernel used by the current block when the LFNST index number indicates that the current block uses LFNST;

[0373] The second transform unit 1103 is configured to use the LFNST transform kernel to perform transform processing on the transform coefficients.

[0374] In the above solution, the MIP parameter includes a MIP transposition indication parameter, wherein the value of the MIP transposition indication parameter is used to indicate whether to perform transposition processing on the sampling point input vector used by the MIP mode.

[0375] In the above scheme, see Fig.11 , the decoder 110 may further include a second selection unit 1104 and a second transposition unit 1105; wherein,

[0376] The second selection unit 1104 is configured to select a transformation core indicated by the LFNST index number from the LFNST transformation core candidate set;

[0377] The second transposition unit 1105 is configured to perform matrix transposition processing on the selected transform kernel to obtain the LFNST transform kernel used by the current block when the value of the MIP transposition indication parameter indicates that the sampling point input vector used by the MIP mode is transposed; wherein the LFNST transform kernel candidate set includes two or more transform kernels preset for MIP.

[0378] In the above scheme, the MIP parameter includes a MIP mode index number, wherein the MIP mode index number is used to indicate the MIP mode used by the current block, and the MIP mode is used to indicate a calculation and derivation method for determining an intra-frame prediction value of the current block using the MIP.

[0379] In the above solution, the second determining unit 1102 is further configured to determine the value of the LFNST intra prediction mode index number according to the value of the MIP mode index number;

[0380] The second selection unit 1104 is further configured to select an LFNST transform core candidate set from multiple LFNST transform core candidate sets according to the value of the LFNST intra-frame prediction mode index number; and select the transform core indicated by the LFNST index number from the selected LFNST transform core candidate set, and set it as the LFNST transform core used by the current block; wherein the LFNST transform core candidate set includes two or more preset LFNST transform cores.

[0381] In the above scheme, the second determination unit 1102 is further configured to use a first lookup table to determine the value of the LFNST intra-frame prediction mode index number corresponding to the value of the MIP mode index number, wherein the first lookup table contains at least one or more different MIP mode index numbers corresponding to two LFNST intra-frame prediction mode index numbers with different values.

[0382] In the above scheme, the second selection unit 1104 is further configured to select an LFNST transform core candidate set from multiple LFNST transform core candidate sets according to the value of the MIP mode index number; and select the transform core indicated by the LFNST index number from the selected LFNST transform core candidate set, and set it as the LFNST transform core used by the current block; wherein the LFNST transform core candidate set includes two or more preset LFNST transform cores.

[0383] In the above scheme, the second selection unit 1104 is further configured to use a second lookup table to determine the value of the LFNST transform core candidate set index number corresponding to the value of the MIP mode index number, and select the LFNST transform core candidate set indicated by the value of the LFNST transform core candidate set index number as the selected LFNST transform core candidate set; wherein the second lookup table contains at least one or more different MIP mode index numbers corresponding to two different values ​​of the LFNST transform core candidate set index numbers.

[0384] In the above scheme, see Fig.11 The decoder 110 may further include a second calculation unit 1106 configured to determine the index number of the LFNST transform kernel candidate set using a first calculation method according to the value of the MIP mode index number;

[0385] The second selection unit 1104 is further configured to select the LFNST transform core candidate set indicated by the value of the LFNST transform core candidate set index sequence number as the selected LFNST transform core candidate set.

[0386] In the above scheme, the second calculation unit 1106 is specifically configured to set the value of the LFNST transform core candidate set index number to be equal to the value of the MIP mode index number when the value of the MIP mode index number is less than the first preset value; and to set the value of the LFNST transform core candidate set index number to be equal to the difference between the value of the MIP mode index number and the second preset value when the value of the MIP mode index number is greater than or equal to the first preset value; wherein the first preset value and the second preset value are both integer values.

[0387] In the above solution, the value of the first preset value is equal to 3.

[0388] In the above solution, the value of the second preset value is equal to 2.

[0389] In the above solution, the MIP parameter also includes a MIP transposition indication parameter, wherein the value of the MIP transposition indication parameter indicates whether to perform transposition processing on the sampling point input vector used by the MIP mode;

[0390] The second transposition unit 1105 is further configured to perform matrix transposition processing on the LFNST transformation kernel used by the current block when the value of the MIP transposition indication parameter indicates that the sampling point input vector used by the MIP mode is to be transposed, and set the transformation kernel obtained after the transposition processing as the LFNST transformation kernel used by the current block.

[0391] It can be understood that in this embodiment, the "unit" can be a part of the circuit, a part of the processor, a part of the program or software, etc., and of course it can also be a module, or it can be non-modular. Moreover, the various components in this embodiment can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional module. If the integrated unit is implemented in the form of a software functional module and is not sold or used as an independent product, it can be stored in a computer-readable storage medium.

[0392] Therefore, this embodiment provides a computer storage medium, which is applied to the decoder 110. The computer storage medium stores a transformation program. When the transformation program is executed by the second processor, the method described in any one of the above embodiments is implemented.

[0393] Based on the above-mentioned components of the decoder 110 and the computer storage medium, see Fig.12 , which shows a specific hardware structure example of the decoder 110 provided in an embodiment of the present application, which may include: a second communication interface 1201, a second memory 1202, and a second processor 1203; each component is coupled together via a second bus system 1204. It is understandable that the second bus system 1204 is used to realize the connection and communication between these components. In addition to the data bus, the second bus system 1204 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, in Fig.12 In the figure, various buses are labeled as the second bus system 1204. Among them,

[0394] The second communication interface 1201 is used for receiving and sending signals during the process of sending and receiving information with other external network elements;

[0395] The second memory 1202 is used to store a computer program that can be run on the second processor 1203;

[0396] The second processor 1203 is configured to, when running the computer program, execute:

[0397] Parse the bitstream and determine the prediction mode parameters of the current block;

[0398] When the prediction mode parameter indicates that the current block uses MIP to determine the intra-frame prediction value, parsing the bitstream to determine the MIP parameter;

[0399] Parsing the bitstream to determine the transform coefficient and LFNST index number of the current block;

[0400] When the LFNST index number indicates that the current block uses LFNST, the LFNST transformation kernel used by the current block is determined according to the MIP parameters;

[0401] The transform coefficients are transformed using the LFNST transform kernel.

[0402] Optionally, as another embodiment, the second processor 1203 is further configured to execute any one of the methods described in the foregoing embodiments when running the computer program.

[0403] It can be understood that the hardware functions of the second memory 1202 and the first memory 1002 are similar, and the hardware functions of the second processor 1203 and the first processor 1003 are similar; they will not be described in detail here.

[0404] This embodiment provides a decoder, which may include a parsing unit, a second determination unit, and a second transformation unit; thus, for the current block adopting the MIP mode, since the MIP parameters are introduced when performing the LFNST transformation, the selection of the LFNST transformation kernel is more flexible, thereby not only improving the applicability of the LFNST technology to non-traditional intra-frame prediction modes, but also improving the decoding efficiency, and at the same time being able to improve the video image quality.

[0405] It should be noted that, in this application, the terms "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0406] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0407] The methods disclosed in several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0408] The features disclosed in several product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0409] The features disclosed in several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments or device embodiments.

[0410] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0411] Industrial Applicability

[0412] In an embodiment of the present application, the prediction mode parameters of the current block are first determined; when the prediction mode parameters indicate that the current block uses MIP to determine the intra-frame prediction value, the MIP parameters are determined; then, based on the MIP parameters, the intra-frame prediction value of the current block is determined, and the prediction difference between the current block and the intra-frame prediction value is calculated; when the current block uses LFNST, the LFNST transformation core used by the current block is determined based on the MIP parameters, the LFNST index number is set and written into the video bitstream; finally, the LFNST transformation core is used to transform the prediction difference. In this way, for the current block using the MIP mode, since the MIP parameters are introduced when performing the LFNST transformation, the selection of the LFNST transformation core is more flexible, which not only improves the applicability of the LFNST technology to non-traditional intra-frame prediction modes, but also improves the encoding and decoding efficiency, and can also improve the video image quality.

Claims

1. A transformation method, applied to a decoder, comprising: Parse the bitstream and determine the prediction mode parameters of the current block; When the prediction mode parameter indicates that the current block uses MIP to determine the intra-frame prediction value, parsing the bitstream to determine the MIP parameter; Parsing the bitstream to determine the transform coefficient and LFNST index number of the current block; When the LFNST index number indicates that the current block uses LFNST, determining, according to the MIP parameter, an LFNST transformation kernel used by the current block; The transform coefficients are transformed using the LFNST transform kernel.

2. The method according to claim 1, wherein: The MIP parameter includes a MIP mode index number, wherein the MIP mode index number is used to indicate a MIP mode used by the current block, and the MIP mode is used to indicate a calculation and derivation method for determining an intra-frame prediction value of the current block using the MIP.

3. The method according to claim 2, wherein: When the LFNST index number indicates that the current block uses LFNST, determining, according to the MIP parameter, a LFNST transformation kernel used by the current block, comprises: Determine the value of the LFNST intra prediction mode index number according to the value of the MIP mode index number; Selecting an LFNST transform core candidate set from multiple LFNST transform core candidate sets according to the value of the LFNST intra prediction mode index sequence number; Selecting a transform core indicated by the LFNST index number from the selected LFNST transform core candidate set, and setting it as the LFNST transform core used by the current block; The LFNST transform kernel candidate set includes two or more preset LFNST transform kernels.

4. The method according to claim 3, wherein: The determining, according to the value of the MIP mode index number, the value of the LFNST intra prediction mode index number comprises: A first lookup table is used to determine the value of the LFNST intra-frame prediction mode index number corresponding to the value of the MIP mode index number, wherein the first lookup table contains at least one or more different MIP mode index numbers corresponding to two LFNST intra-frame prediction mode index numbers with different values.

5. The method according to claim 2, wherein: When the LFNST index number indicates that the current block uses LFNST, determining, according to the MIP parameter, a LFNST transformation kernel used by the current block, comprises: Selecting an LFNST transform core candidate set from multiple LFNST transform core candidate sets according to the value of the MIP mode index sequence number; Selecting a transformation core indicated by the LFNST index number from the selected LFNST transformation core candidate set, and setting it as the LFNST transformation core used by the current block; The LFNST transform kernel candidate set includes two or more preset LFNST transform kernels.

6. The method according to claim 5, wherein: The step of selecting a LFNST transform core candidate set from a plurality of LFNST transform core candidate sets according to the value of the MIP mode index sequence number comprises: Determine the value of the LFNST transform core candidate set index number corresponding to the value of the MIP mode index number using a second lookup table, and select the LFNST transform core candidate set indicated by the value of the LFNST transform core candidate set index number as the selected LFNST transform core candidate set; The second lookup table contains at least one or more different MIP mode index numbers corresponding to two different values ​​of the LFNST transform core candidate set index numbers.

7. The method according to claim 5, wherein: The step of selecting a LFNST transform core candidate set from a plurality of LFNST transform core candidate sets according to the value of the MIP mode index sequence number comprises: According to the value of the MIP mode index number, determine the index number of the LFNST transform kernel candidate set using a first calculation method; The LFNST transform core candidate set indicated by the value of the LFNST transform core candidate set index sequence number is selected as the selected LFNST transform core candidate set.

8. The method according to claim 7, wherein: The step of determining the index number of the LFNST transform kernel candidate set by using a first calculation method according to the value of the MIP mode index number includes: When the value of the MIP mode index number is less than a first preset value, setting the value of the LFNST transform core candidate set index number to be equal to the value of the MIP mode index number; When the value of the MIP mode index number is greater than or equal to a first preset value, setting the value of the LFNST transform core candidate set index number to be equal to a difference obtained by subtracting a second preset value from the value of the MIP mode index number; The first preset value and the second preset value are both integer values.

9. A transformation method, applied to an encoder, the method comprising: Determining prediction mode parameters for the current block; When the prediction mode parameter indicates that the current block uses a matrix-based intra prediction MIP to determine an intra prediction value, determining a MIP parameter; Determine an intra-frame prediction value of the current block according to the MIP parameter, and calculate a prediction difference between the current block and the intra-frame prediction value; When the current block uses a low frequency non-separable secondary transform LFNST, determining the LFNST transform kernel used by the current block according to the MIP parameter, setting the LFNST index number and writing it into the video bitstream; The prediction difference is transformed using the LFNST transformation kernel.

10. An encoder, comprising a first memory and a first processor; wherein: The first memory is used to store a computer program that can be run on the first processor; The first processor is configured to execute the method according to claim 9 when running the computer program.

11. A decoder, comprising a second memory and a second processor; wherein: The second memory is used to store a computer program that can be run on the second processor; The second processor is configured to execute the method according to any one of claims 1 to 8 when running the computer program.

12. A computer storage medium, wherein: The computer storage medium stores a code stream, which is generated by executing the steps of the encoding method according to claim 9.

Images