Transform method, encoder, decoder, and storage medium

Abstract

This application discloses a transformation method, encoder, decoder, and storage medium. The method includes: determining prediction mode parameters of a current block; when the prediction mode parameters indicate that the current block uses matrix-based intra-frame prediction (MIP) to determine intra-frame prediction values, determining MIP parameters; determining the intra-frame prediction values ​​of the current block based on the MIP parameters, and calculating the prediction difference between the current block and the intra-frame prediction values; when the current block uses low-frequency non-separable quadratic transform (LFNST), determining the LFNST transform kernel used by the current block based on the MIP parameters, setting the LFNST index number and writing it into the video stream; and using the LFNST transform kernel to transform the prediction difference.

Description

[0001] Case Analysis

[0002] This application is a divisional application of Chinese Patent No. 201980103146.X, filed on December 30, 2019, entitled "Transformation Method, Encoder, Decoder and Storage Medium". Technical Field

[0003] This application relates to the field of image processing technology, and in particular to a transformation method, encoder, decoder, and storage medium. Background Technology

[0004] As people's demands for video display quality increase, new video application forms such as high-definition and ultra-high-definition video have emerged. H.265 / High Efficiency Video Coding (HEVC) can no longer meet the needs of the rapidly developing video applications. The Joint Video Exploration Team (JVET) proposed the next-generation video coding standard H.266 / Versatile Video Coding (VVC), and its corresponding test model is the VVC Reference Software Test Model (VTM).

[0005] In H.266 / VVC, the Reduced Second Transform (RST) technique has been adopted and renamed the Low-Frequency Non-Separable Transform (LFNST) technique. Because the selection of the transform set in LFNST is based on the intra-prediction mode, it lacks variability when performing LFNST transformations for non-traditional intra-prediction modes, thus reducing coding efficiency. Summary of the Invention

[0006] This application provides a transform method, encoder, decoder, and storage medium, which can improve the applicability of LFNST technology to non-traditional intra-frame prediction modes, make the selection of transform sets more flexible, and thus improve encoding and decoding efficiency.

[0007] The technical solution of this application embodiment can be implemented as follows:

[0008] In a first aspect, embodiments of this application provide a transformation method applied to an encoder, the method comprising:

[0009] Determine the prediction mode parameters for the current block;

[0010] The MIP parameters are determined when the prediction mode parameters instruct the current block to use matrix-based intra-prediction (MIP) to determine intra-prediction values.

[0011] Based on the MIP parameters, determine the intra-prediction value of the current block, and calculate the prediction difference between the current block and the intra-prediction value;

[0012] When the current block uses the Low Frequency Inseparable Secondary Transform (LFNST), the LFNST transform core used by the current block is determined according to the MIP parameters, the LFNST index number is set and written into the video stream;

[0013] The predicted difference is transformed using the LFNST transform kernel.

[0014] Secondly, embodiments of this application provide a transformation method applied to a decoder, the method comprising:

[0015] Analyze the bitstream to determine the prediction mode parameters for the current block;

[0016] When the prediction mode parameter indicates that the current block uses MIP to determine the intra-frame prediction value, the bitstream is parsed and the MIP parameter is determined.

[0017] Analyze the bitstream to determine the transform coefficients and LFNST index number of the current block;

[0018] When the LFNST index number indicates that the current block uses LFNST, the LFNST transform kernel used by the current block is determined according to the MIP parameter;

[0019] The transformation coefficients are transformed using the LFNST transform kernel.

[0020] Thirdly, embodiments of this application provide an encoder, which includes a first determining unit, a first calculating unit, and a first transforming unit; wherein,

[0021] The first determining unit is configured to determine the prediction mode parameters of the current block;

[0022] The first determining unit is further configured to determine the MIP parameter when the prediction mode parameter indicates that the current block uses matrix-based intra-prediction (MIP) to determine the intra-prediction value;

[0023] The first calculation unit is configured to determine the intra-frame prediction value of the current block based on the MIP parameters, and calculate the prediction difference between the current block and the intra-frame prediction value.

[0024] The first determining unit is further configured to, when the current block uses the low-frequency inseparable secondary transform LFNST, determine the LFNST transform core used by the current block according to the MIP parameters, set the LFNST index number and write it into the video stream;

[0025] The first transformation unit is configured to use the LFNST transformation kernel to transform the predicted difference.

[0026] Fourthly, embodiments of this application provide an encoder, which includes a first memory and a first processor; wherein,

[0027] A first memory for storing computer programs that can run on a first processor;

[0028] A first processor is configured to execute the method described in the first aspect when running the computer program.

[0029] Fifthly, embodiments of this application provide a decoder, which includes a parsing unit, a second determining unit, and a second transforming unit; wherein,

[0030] The parsing unit is configured to parse the bitstream to determine the prediction mode parameters of the current block; and is further configured to parse the bitstream to determine the MIP parameters when the prediction mode parameters indicate that the current block uses MIP to determine the intra-frame prediction value; and is further configured to parse the bitstream to determine the transform coefficients and LFNST index number of the current block.

[0031] The second determining unit is configured to determine the LFNST transform kernel used by the current block according to the MIP parameters when the LFNST index number indicates that the current block uses LFNST.

[0032] The second transformation unit is configured to use the LFNST transformation kernel to perform transformation processing on the transformation coefficients.

[0033] Sixthly, embodiments of this application provide a decoder, which includes a second memory and a second processor; wherein,

[0034] The second memory is used to store computer programs that can run on the second processor;

[0035] A second processor is configured to execute the method described in the second aspect when running the computer program.

[0036] In a seventh aspect, embodiments of this application provide a computer storage medium storing a computer program that, when executed by a first processor, implements the method described in the first aspect, or when executed by a second processor, implements the method described in the second aspect.

[0037] This application provides a transformation method, encoder, decoder, and storage medium. The method involves determining the prediction mode parameters of the current block; when the prediction mode parameters indicate that the current block uses MIP to determine intra-frame prediction values, determining the MIP parameters; determining the intra-frame prediction values ​​of the current block based on the MIP parameters, and calculating the prediction difference between the current block and the intra-frame prediction values; when the current block uses LFNST, determining the LFNST transform kernel used by the current block based on the MIP parameters, setting the LFNST index number, and writing it into the video bitstream; and using the LFNST transform kernel to transform the prediction difference. Thus, for the current block using MIP mode, the introduction of MIP parameters during LFNST transformation makes the selection of the LFNST transform kernel more flexible, thereby not only improving the applicability of LFNST technology to non-traditional intra-frame prediction modes but also improving encoding and decoding efficiency and video image quality. Attached Figure Description

[0038] Figure 1 A schematic diagram illustrating the application location of LFNST technology for related technical solutions;

[0039] Figure 2A A block diagram of a video encoding system provided in an embodiment of this application;

[0040] Figure 2B A block diagram of a video decoding system provided in an embodiment of this application;

[0041] Figure 3 A flowchart illustrating a transformation method provided in an embodiment of this application;

[0042] Figure 4 A flowchart illustrating a MIP prediction process provided in an embodiment of this application;

[0043] Figure 5 A schematic diagram illustrating the calculation process of matrix multiplication using LFNST technology, provided for an embodiment of this application;

[0044] Figure 6A A structural block diagram of the LFNST transform is provided for related technical solutions;

[0045] Figure 6B Another structural block diagram of the LFNST transform provided for related technical solutions;

[0046] Figure 6C Another structural block diagram of the LFNST transform provided for related technical solutions;

[0047] Figure 6DAnother structural block diagram of the LFNST transform provided for related technical solutions;

[0048] Figure 7 A flowchart illustrating another transformation method provided in an embodiment of this application;

[0049] Figure 8 A flowchart illustrating a specific LFNST process provided in this application embodiment;

[0050] Figure 9 A schematic diagram of the composition structure of an encoder provided in an embodiment of this application;

[0051] Figure 10 This is a schematic diagram of the specific hardware structure of an encoder provided in an embodiment of this application;

[0052] Figure 11 A schematic diagram of the composition structure of a decoder provided in an embodiment of this application;

[0053] Figure 12 This is a schematic diagram of the specific hardware structure of a decoder provided in an embodiment of this application. Detailed Implementation

[0054] In order to gain a more detailed understanding of the features and technical content of the embodiments of this application, the implementation of the embodiments of this application will be described in detail below with reference to the accompanying drawings. The accompanying drawings are for reference and illustration only and are not intended to limit the embodiments of this application.

[0055] In video images, a coding block (CB) is generally represented by a first image component, a second image component, and a third image component. These three image components are a luminance component, a blue chrominance component, and a red chrominance component, respectively. Specifically, the luminance component is usually represented by the symbol Y, the blue chrominance component is usually represented by the symbol Cb or U, and the red chrominance component is usually represented by the symbol Cr or V. Thus, video images can be represented in YCbCr format or YUV format.

[0056] In the embodiments of this application, the first image component can be a luminance component, the second image component can be a blue chroma component, and the third image component can be a red chroma component, but the embodiments of this application do not impose specific limitations.

[0057] The following will describe the relevant technical solutions for the current LFNST technology.

[0058] See Figure 1 It illustrates a schematic diagram of the application location of LFNST technology provided by a related technical solution. For example... Figure 1As shown, in the intra-frame prediction mode, for the encoder side, LFNST technology is applied between the positive first transformation unit 11 and the quantization unit 12, and LFNST technology is applied between the inverse quantization unit 13 and the inverse first transformation unit 14.

[0059] Specifically, on the encoder side, the data, such as the prediction residual (which can be represented as residual), is first transformed by the positive first-order transform unit 11 (which can be called "Core Transform", "first-order transform" or "main transform") to obtain the transform coefficient matrix after the first transformation; then, the coefficients in the transform coefficient matrix are subjected to LFNST transformation (which can be called "Secondary Transform") to obtain the LFNST transform coefficient matrix; finally, the LFNST transform coefficient matrix is ​​quantized by the quantization unit 12, and the final quantized value is written into the video bitstream.

[0060] On the decoder side, the quantized values ​​of the LFNST transform coefficient matrix can be obtained by parsing the bitstream. The inverse quantization unit 13 performs inverse quantization on these values ​​(which can be called "scaling") to obtain the recovered values ​​of the LFNST transform coefficient matrix. Performing an inverse LFNST transform on these recovered values ​​yields the coefficient matrix. Then, the inverse first-order transform unit 14 performs an inverse transform on the coefficient matrix corresponding to the encoder-side "Core Transform," ultimately obtaining the residual recovered values. It is important to note that the standard only defines the "inverse transform" operation on the decoder side; therefore, the "inverse LFNST transform" is also referred to as the "LFNST transform" in the standard. Here, to distinguish it from the encoder-side transform, the encoder-side "LFNST transform" can be called the "forward LFNST transform," and the decoder-side "LFNST transform" can be called the "inverse LFNST transform."

[0061] In other words, on the encoder side, the prediction residual of the current transform block is transformed into first-order transform coefficients through a positive first-order transform. Then, some of the first-order transform coefficients are transformed into second-order transform coefficients through matrix multiplication to obtain fewer and more concentrated second-order transform coefficients, which are then quantized. On the decoder side, after parsing the quantized values, they are dequantized. The dequantized coefficients are then transformed into inverse second-order transforms through matrix multiplication, and then transformed into inverse first-order transforms to recover the prediction residual.

[0062] In LFNST technology, the LFNST transform process can include steps such as configuring core parameters, intra-frame prediction mode mapping, selecting the transform matrix, calculating matrix multiplication, and constructing the inverse first-order transform coefficient matrix. After these steps, the LFNST transform is complete. However, in the step of selecting the transform matrix, the transform set must first be selected. Since the transform matrix is ​​related to the directional characteristics of the prediction mode, the transform set is currently selected based on the intra-frame prediction mode. For traditional intra-prediction modes, the value of the intra-prediction mode indicator (represented by predModeIntra) can be determined based on the traditional intra-prediction mode number, and then the transform set index number can be determined based on the value of predModeIntra. However, for non-traditional intra-prediction modes, especially matrix-based intra-prediction (MIP) mode, the value of predModeIntra is directly set to the intra-prediction mode index number corresponding to the PLANAR mode (i.e., 0). This causes the current block in MIP mode to be able to select the transform set with transform set index number 0. This makes the current block lack variability when performing LFNST transform in MIP mode, which makes LFNST technology not well applied to MIP mode and also reduces coding efficiency.

[0063] This application provides a transformation method applied to an encoder. The method involves determining the prediction mode parameters of the current block; when the prediction mode parameters indicate that the current block uses matrix-based intra-prediction (MIP) to determine intra-prediction values, determining the MIP parameters; determining the intra-prediction values ​​of the current block based on the MIP parameters, and calculating the prediction difference between the current block and the intra-prediction values; when the current block uses Low-Frequency Inseparable Secondary Transform (LFNST), determining the LFNST transform kernel used by the current block based on the MIP parameters, setting the LFNST index number, and writing it into the video bitstream; and using the LFNST transform kernel to transform the prediction difference. Thus, for the current block using MIP mode, the introduction of MIP parameters during LFNST transformation makes the selection of the LFNST transform kernel more flexible, thereby not only improving the applicability of LFNST technology to non-traditional intra-prediction modes but also improving encoding and decoding efficiency and enhancing video image quality.

[0064] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0065] See Figure 2A It illustrates an example block diagram of a video encoding system provided in an embodiment of this application; as shown Figure 2AAs shown, the video coding system 10 includes a transform and quantization unit 101, an intra-frame estimation unit 102, an intra-frame prediction unit 103, a motion compensation unit 104, a motion estimation unit 105, an inverse transform and inverse quantization unit 106, a filter control and analysis unit 107, a filtering unit 108, an encoding unit 109, and a decoded image buffer unit 110. The filtering unit 108 can implement deblocking filtering and Sample Adaptive Offset (SAO) filtering, while the encoding unit 109 can implement header information encoding and Context-based Adaptive Binary Arithmetic Coding (CABAC). For the input raw video signal, the system uses coding tree blocks (Coding Tree Blocks) to perform the encoding of the raw video signal. The partitioning of a TreeUnit (CTU) yields a video coding block. The residual pixel information obtained after intra- or inter-frame prediction is then transformed by the transform and quantization unit 101. This transformation involves converting the residual information from the pixel domain to the transform domain and quantizing the resulting transform coefficients to further reduce the bit rate. Intra-frame estimation unit 102 and intra-frame prediction unit 103 perform intra-frame prediction on the video coding block. Specifically, intra-frame estimation unit 102 and intra-frame prediction unit 103 determine the intra-frame prediction mode to be used to encode the video coding block. Motion compensation unit 104 and motion estimation unit 105 perform inter-frame prediction coding of the received video coding block relative to one or more blocks in one or more reference frames to provide temporal prediction information. The motion estimation performed by motion estimation unit 105 is a process of generating motion vectors, which can estimate the motion of the video coding block. Then, motion compensation unit 104 uses the motion vectors determined by motion estimation unit 105 to generate motion vectors. Motion vectors perform motion compensation; after determining the intra-prediction mode, the intra-prediction unit 103 is also used to provide the selected intra-prediction data to the coding unit 109, and the motion estimation unit 105 also sends the calculated motion vector data to the coding unit 109; in addition, the inverse transform and inverse quantization unit 106 is used to reconstruct the video coding block, reconstructing the residual block in the pixel domain, the reconstructed residual block is removed by the filter control analysis unit 107 and the filtering unit 108 to remove block artifacts, and then the reconstructed residual block is added to a predictive block in the frame of the decoding image buffer unit 110 to generate the reconstructed video coding block; the coding unit 109 is used to encode various coding parameters and quantized transform coefficients. In the CABAC-based coding algorithm, the context content can be based on adjacent coding blocks and can be used to encode information indicating the determined intra-prediction mode, outputting the bitstream of the video signal; and the decoding image buffer unit 110 is used to store the reconstructed video coding block for prediction reference.As video image encoding proceeds, new reconstructed video encoding blocks are continuously generated, and these reconstructed video encoding blocks are stored in the decoding image buffer unit 110.

[0066] See Figure 2B It illustrates an example block diagram of a video decoding system provided in an embodiment of this application; as shown Figure 2B As shown, the video decoding system 20 includes a decoding unit 201, an inverse transform and inverse quantization unit 202, an intra-frame prediction unit 203, a motion compensation unit 204, a filtering unit 205, and a decoding image buffer unit 206. The decoding unit 201 can perform header information decoding and CABAC decoding, while the filtering unit 205 can perform deblocking filtering and SAO filtering. The input video signal is processed... Figure 2A After encoding, the video signal bitstream is output. This bitstream is input into the video decoding system 20, first passing through the decoding unit 201 to obtain the decoded transform coefficients. The transform coefficients are then processed by the inverse transform and inverse quantization unit 202 to generate residual blocks in the pixel domain. The intra-frame prediction unit 203 can generate prediction data for the current video decoding block based on the determined intra-frame prediction mode and data from previously decoded blocks in the current frame or image. The motion compensation unit 204 determines the prediction information for the video decoding block by analyzing motion vectors and other associated syntax elements, and uses this prediction information. The predictive block of the video block being decoded is generated; the decoded video block is formed by summing the residual block from the inverse transform and inverse quantization unit 202 with the corresponding predictive block generated by the intra-prediction unit 203 or the motion compensation unit 204; the decoded video signal is passed through the filtering unit 205 to remove block artifacts, which can improve video quality; then the decoded video block is stored in the decoding image buffer unit 206, which stores reference images for subsequent intra-prediction or motion compensation, and is also used for the output of the video signal, thus obtaining the recovered original video signal.

[0067] The transformation method in the embodiments of this application can be applied to, for example, Figure 2A The transformation and quantization unit 101 shown includes: Figure 1 The positive first-order transformation unit 11 and quantization unit 12 shown indicate that the transformation method is specifically applied in the part between transformation and quantization. Furthermore, the transformation method in this embodiment can also be applied to, for example... Figure 2A The inverse transform and inverse quantization unit 106 shown, or as... Figure 2B The inverse transform and inverse quantization unit 202 shown, whether it is the inverse transform and inverse quantization unit 106 or the inverse transform and inverse quantization unit 202, can include the following: Figure 1The inverse quantization unit 13 and the inverse first-order transform unit 14 shown indicate that the transform method is specifically applied in the part between inverse quantization and inverse transform. That is, the transform method in this embodiment can be applied to both video encoding and video decoding systems, and even simultaneously, but this embodiment does not impose specific limitations. It should also be noted that when the transform method is applied to a video encoding system, "current block" specifically refers to the current coded block in intra-frame prediction; when the transform method is applied to a video decoding system, "current block" specifically refers to the current decoded block in intra-frame prediction.

[0068] Based on the above Figure 2A For application scenario examples, see Figure 3 The diagram illustrates a flowchart of a transformation method provided in an embodiment of this application. Figure 3 As shown, the method may include:

[0069] S301: Determine the prediction mode parameters for the current block;

[0070] It should be noted that a video image can be divided into multiple image blocks, and each image block to be encoded can be called a coding block (CB). Here, each coding block may include a first image component, a second image component, and a third image component; and the current block is the coding block in the video image to be predicted for the first image component, the second image component, or the third image component.

[0071] In this context, if the current block performs prediction of the first image component, and the first image component is the luminance component, that is, the image component to be predicted is the luminance component, then the current block can also be called the luminance block; or, if the current block performs prediction of the second image component, and the second image component is the chrominance component, that is, the image component to be predicted is the chrominance component, then the current block can also be called the chrominance block.

[0072] It should also be noted that the prediction mode parameters indicate the coding mode of the current block and the parameters associated with that mode. Rate Distortion Optimization (RDO) is typically used to determine the prediction mode parameters of the current block.

[0073] Specifically, in some embodiments, for S301, determining the prediction mode parameters of the current block may include:

[0074] Determine the image components to be predicted for the current block;

[0075] Based on the parameters of the current block, the image components to be predicted are predicted and encoded using multiple prediction modes respectively, and the rate-distortion cost result corresponding to each prediction mode is calculated.

[0076] Select the minimum rate distortion cost result from the multiple calculated rate distortion cost results, and determine the prediction mode corresponding to the minimum rate distortion cost result as the prediction mode parameter of the current block.

[0077] In other words, on the encoder side, multiple prediction modes can be used to encode the image components to be predicted for the current block. Here, multiple prediction modes typically include traditional intra-frame prediction modes and non-traditional intra-frame prediction modes. Traditional intra-frame prediction modes can include Direct Current (DC) mode, Planar mode, and Angle mode, while non-traditional intra-frame prediction modes can include MIP mode, Cross-component Linear Model Prediction (CCLM) mode, Intra Block Copy (IBC) mode, and PLT (Palette) mode, etc.

[0078] In this way, after encoding the current block using multiple prediction modes, the rate-distortion cost result corresponding to each prediction mode can be obtained. Then, the minimum rate-distortion cost result is selected from the multiple rate-distortion cost results, and the prediction mode corresponding to the minimum rate-distortion cost result is determined as the prediction mode parameter of the current block. In this way, the current block can be encoded using the determined prediction mode. Moreover, under this prediction mode, the prediction residual can be small, which can improve the coding efficiency.

[0079] S302: When the prediction mode parameter indicates that the current block uses MIP to determine the intra-frame prediction value, determine the MIP parameter;

[0080] S303: Determine the intra-frame prediction value of the current block based on the MIP parameters, and calculate the prediction difference between the current block and the intra-frame prediction value;

[0081] It should be noted that, for MIP mode, the input data for MIP prediction includes: the position of the current block (xTbCmp, yTbCmp), the MIP prediction mode applied to the current block (which can be represented by modeId), the height of the current block (represented by nTbH), the width of the current block (represented by nTbW), and a transpose processing indicator flag indicating whether transpose is required (which can be represented by isTransposed). The output data for MIP prediction includes: the prediction block of the current block, where the intra-frame prediction value corresponding to the pixel coordinates [x][y] in the prediction block is predSamples[x][y]; where x = 0, 1, ..., nTbW-1; y = 0, 1, ..., nTbH-1.

[0082] Specifically, such as Figure 4 As shown, the MIP prediction process can be divided into four steps: configuring core parameters 41, acquiring reference pixels 42, constructing input samples 43, and generating predicted values ​​44. Regarding configuring core parameters 41, the current block can be divided into three categories based on its size within the frame, with mipSizeId recording the type of the current block; moreover, the number of reference sampling points and the number of matrix multiplication output sampling points differ for different types of current blocks. Regarding acquiring reference pixels 42, when predicting the current block, the upper and left blocks are already encoded. The reference pixels for MIP technology are the reconstructed values ​​of the previous row of pixels and the left column of pixels in the current block. The process of acquiring the reference pixels adjacent to the upper side (denoted by refT) and the reference pixels adjacent to the left side (denoted by refL) of the current block is the reference pixel acquisition process. For constructing input samples 43, this step is used for the input of matrix multiplication and mainly includes: obtaining reference samples 431, constructing a reference sampling buffer 432, and deriving matrix multiplication input samples 433. The process of obtaining reference samples is a downsampling process, and constructing the reference sampling buffer 432 can include buffer filling methods 4321 when transposition is not needed and buffer filling methods 4322 when transposition is needed. For generating predicted values ​​44, this step is used to obtain the MIP predicted value of the current block and mainly includes: constructing a matrix multiplication output sampling block 441, matrix multiplication output sampling embedding 442, matrix multiplication output sampling transposition 443, and generating the final MIP predicted value 444. The process of constructing the matrix multiplication output sampling block 441 can include obtaining the weight matrix 4411, obtaining the shift factor and offset factor 4412, and performing matrix multiplication operations 4413. The process of generating the final MIP predicted value 444 can include generating predicted values ​​that do not require upsampling 4441 and generating predicted values ​​that require upsampling 4442. After these four steps, the intra-frame prediction value of the current block can be obtained.

[0083] Thus, after determining the intra-frame predicted value of the current block, the difference between the actual pixel value of the current block and the intra-frame predicted value can be calculated, and the calculated difference can be used as the prediction difference, which is convenient for subsequent transformation processing of the prediction difference.

[0084] Furthermore, the MIP parameters also need to be determined during the MIP prediction process.

[0085] In some embodiments, the MIP parameter may include a MIP transpose indicator parameter (which can be represented by isTransposed); here, the value of the MIP transpose indicator parameter is used to indicate whether the sampling point input vector used in the MIP mode is transposed.

[0086] Specifically, in MIP mode, an adjacent reference sample set can be obtained based on the reference sample values ​​corresponding to the reference pixels on the left and top edges of the current block. After obtaining the adjacent reference sample set, an input reference sample set can be constructed, which is the sampling point input vector used in MIP mode. However, the construction methods for the input reference sample set differ between the encoder and decoder sides, mainly due to the values ​​of the MIP transpose indicator parameter.

[0087] When applied to the encoder side, the value of the MIP transpose indicator parameter can still be determined using rate-distortion optimization. Specifically, this can include:

[0088] Calculate the first-generation value after transposition and the second-generation value without transposition, respectively.

[0089] If the value of the first generation is less than the value of the second generation, then the value of the MIP transpose indicator parameter can be determined to be 1.

[0090] If the value of the first generation is not less than the value of the second generation, then the value of the MIP transpose indicator parameter can be determined to be 0.

[0091] Furthermore, when the MIP transpose indicator parameter is 0, the reference sample value corresponding to the upper edge of the adjacent reference sample set can be stored before the reference sample value corresponding to the left edge in the buffer. In this case, no transpose processing is needed; that is, the sampling point input vector used in MIP mode does not need to be transposed, and the buffer can be directly determined as the input reference sample set. When the MIP transpose indicator parameter is 1, the reference sample value corresponding to the upper edge of the adjacent reference sample set can be stored after the reference sample value corresponding to the left edge in the buffer. In this case, transpose processing is required for the buffer; that is, the sampling point input vector used in MIP mode needs to be transposed, and then the transposed buffer is determined as the input reference sample set. Thus, after obtaining the input reference sample set, it can be used in the process of determining the intra-prediction value corresponding to the current block in MIP mode.

[0092] It should also be noted that, on the encoder side, after determining the value of the MIP transpose indicator parameter, the determined value of the MIP transpose indicator parameter needs to be written into the bitstream for subsequent parsing processing on the decoder side.

[0093] In some embodiments, the MIP parameter may further include a MIP mode index number (which may be represented by modeId), wherein the MIP mode index number is used to indicate the MIP mode used by the current block, and the MIP mode is used to indicate the calculation derivation method for determining the intra-prediction value of the current block using MIP.

[0094] In other words, since there are many types of MIP modes, these MIP modes can be distinguished by their MIP mode index numbers. Different MIP modes have different MIP mode index numbers. Thus, based on the calculation derivation method of using MIP to determine the intra-prediction value of the current block, the specific MIP mode can be determined, and the corresponding MIP mode index number can be obtained. In this embodiment, the value of the MIP mode index number can be 0, 1, 2, 3, 4 or 5.

[0095] In some embodiments, the MIP parameters may also include parameters such as the size of the current block and its aspect ratio; wherein, based on the size of the current block (i.e., the width and height of the current block), the category of the current block (which can be represented by mipSizeId) can also be determined.

[0096] In one implementation, determining the category of the current block based on its size may include:

[0097] If the width and height of the current block are both equal to 4, then the value of mipSizeId can be set to 0;

[0098] Conversely, if either the width or height of the current block is 4, or if both the width and height of the current block are 8, then the value of mipSizeId can be set to 1.

[0099] Conversely, if the current block is of a different size, then the value of mipSizeId can be set to 2.

[0100] In another implementation, determining the category of the current block based on its size may include:

[0101] If the width and height of the current block are both equal to 4, then the value of mipSizeId can be set to 0;

[0102] Conversely, if either the width or height of the current block is 4, then the value of mipSizeId can be set to 1;

[0103] Conversely, if the current block is of a different size, then the value of mipSizeId can be set to 2.

[0104] In this way, during the process of using MIP to determine the intra-prediction value, the MIP parameters can also be determined, which makes it easier to determine the LFNST transform kernel (which can be represented as kernel) used in the current block based on the determined MIP parameters.

[0105] S304: When the current block uses LFNST, determine the LFNST transform core used by the current block according to the MIP parameters, set the LFNST index number and write it into the video stream;

[0106] It should be noted that not every current block can undergo LFNST. LFNST can only be performed on the current block if it simultaneously meets the following conditions: (a) the width and height of the current block are both greater than or equal to 4; (b) the width and height of the current block are both less than or equal to the maximum size of the transform block; (c) the prediction mode of the current block or the current coding block is intra-frame prediction mode; (d) the first transform of the current block is a two-dimensional forward first transform (DCT2) in both the horizontal and vertical directions; (e) the intra-frame prediction mode of the current block or the coding block is non-MIP mode, or the prediction mode of the transform block is MIP mode and the width and height of the transform block are both greater than or equal to 16. In other words, for the current block in this embodiment, all five conditions above must be met simultaneously.

[0107] Furthermore, when it is determined that the current block can execute LFNST, it is also necessary to determine the LFNST transform kernel (which can be represented as a kernel) used by the current block. There are four transform kernel candidate sets in LFNST, which can include set0, set1, set2, and set3. The selected transform kernel candidate set can be implicitly derived based on the encoding parameters of the current block or the coding block to which the current block belongs; for example, in H.266 / VVC, based on the intra-prediction mode of the current block, it can be determined which of the four transform kernel candidate sets to use.

[0108] Specifically, after obtaining the intra-prediction mode of the current block, the value of the intra-prediction mode indicator (which can be represented by predModeIntra) can be determined. The calculation formula is as follows:

[0109]

[0110] The image component indicator (represented by cIdx) indicates whether the current block contains the luma or chroma component. Here, if the current block is predicted to contain the luma component, cIdx equals 0; if the current block is predicted to contain the chroma component, cIdx equals 1. Additionally, (xTbY, yTbY) are the coordinates of the top-left sampling point of the current block, IntraPredModeY[xTbY][yTbY] is the intra-prediction mode for the luma component, and IntraPredModeC[xTbY][yTbY] is the intra-prediction mode for the chroma component.

[0111] In current H.266 / VVC, intra-prediction modes can be further divided into traditional intra-prediction modes and non-traditional intra-prediction modes. For non-traditional intra-prediction modes, the predModeIntra value indicates the following information:

[0112] If the prediction mode of the current block is CCLM mode, the value of predModeIntra can be INTRA_LT_CCLM, INTRA_L_CCLM or INTRA_T_CCLM (81, 82, 83 in VVC respectively).

[0113] If the prediction mode of the current block is MIP mode, then the value of predModeIntra can be the index number of the MIP mode used.

[0114] If the prediction mode of the current block is the traditional intra-frame prediction mode, then the value of predModeIntra can be [0, 66].

[0115] Furthermore, if the prediction mode of the current block is CCLM or MIP, the value of predModeIntra can be set as follows:

[0116] (1) When the prediction mode of the current block is CCLM mode,

[0117] If the mode of the center luminance block at the luminance position corresponding to the current block (such as the chroma block) is MIP mode, that is, intra_mip_flag[xTbY+nTbW / 2][yTbY+nTbH / 2] is 1, then the value of predMode Intra is set to the index number indicating the PLANAR mode (i.e., 0).

[0118] Otherwise, if the mode of the center luminance block at the luminance position corresponding to the current block (such as the chroma block) is IBC mode or PLT mode, then the value of predModeIntra is set to the index number indicating the DC mode (i.e., 1).

[0119] Otherwise, set the value of predModeIntra to the mode index number of the center luminance block at the luminance position of the current block (e.g., chroma block) IntraPredModeY[xTbY+nTbW / 2][yTbY+nTbH / 2];

[0120] (2) When the prediction mode of the current block is MIP mode,

[0121] You can directly set the value of predModeIntra to the index number indicating the PLANA mode (i.e., 0).

[0122] For traditional intra-frame prediction modes (such as wide-angle mapping), wide-angle mapping can be performed based on the size of the current block, extending the traditional intra-frame prediction mode [0, 66] to [-14, 80]. The specific mapping process is as follows:

[0123] First, calculate the aspect ratio factor (which can be represented by whRatio), as shown below.

[0124] whRatio=Abs(Log2(nTbW / nTbH)) (2)

[0125] For the current block that is not square (i.e., nTbW is not equal to nTbH), the predModeIntra can be corrected as follows: if nTbW is greater than nTbH, and predModeIntra is greater than or equal to 2, and predModeIntra is less than ((whRatio>1?(8+2×whRatio):8), then predModeIntra = (predModeIntra+65); otherwise, if nTbW is less than nTbH, and predModeIntra is less than or equal to 66, and predModeIntra is greater than ((whRatio>1?(60-2×whRatio):60), then predModeIntra = (predModeIntra-67).

[0126] In the current H.266 / VVC, based on the value of predModeIntra and Table 1, the value of the LFNST index number (which can be represented by SetIdx) can be determined, as shown in Table 1. Here, the LFNST index number is set to indicate that the current block uses LFNST and that the LFNST transform kernel is in the LFNST transform kernel candidate set. Typically, the LFNST transform set includes four transform kernel candidate sets (set0, set1, set2, set3), corresponding to SetIdx values ​​of 0, 1, 2, and 3, respectively.

[0127] Table 1

[0128] predModeIntra SetIdx predModeIntra<0 1 0<=predModeIntra<=1 0 2<=predModeIntra<=12 1 13<=predModeIntra<=23 2 24<=predModeIntra<=44 3 45<=predModeIntra<=55 2 56<=predModeIntra<=80 1

[0129] In the current H.266 / VVC, for MIP mode, because the value of predModeIntra is set to the index number indicating the PLANA mode (i.e., 0), the transform set used by the current block in MIP mode can only be selected from the transform set with the LFNST index number equal to 0. This results in a lack of variability when performing LFNST in MIP mode, which reduces coding efficiency.

[0130] In this embodiment, based on the MIP parameters, the LFNST transform kernel candidate set is first determined. Then, the LFNST transform kernel used in the current block is determined from the LFNST transform kernel candidate set, and the LFNST index number is set and written into the video bitstream. Here, the LFNST transform matrix is ​​a set of fixed coefficient matrices obtained through training. The LFNST transform kernel candidate set includes two sets of transform matrices (also referred to as LFNST transform kernels). After determining the LFNST transform kernel candidate set, one set of LFNST transform kernels needs to be selected from the LFNST transform kernel candidate set, that is, the transform matrix used when performing LFNST on the current block is determined.

[0131] Here, the MIP parameters may include the MIP transpose indicator parameter (which can be represented by isTransposed), the MIP mode index number (which can be represented by modeId), the size of the current block, and the class of the current block (which can be represented by mipSizeId). The following section will describe in detail how to select the LFNST transform kernel to be used for the current block based on the MIP parameters.

[0132] Optionally, in some embodiments, when the MIP parameter is a MIP transpose indicator parameter, for S304, the step of determining the LFNST transform kernel used by the current block according to the MIP parameter, setting the LFNST index number, and writing it into the video stream when the current block uses LFNST may include:

[0133] Select the transform kernel to be used by the current block from the LFNST transform kernel candidate set;

[0134] When the value of the MIP transpose indicator parameter indicates that the sampling point input vector used in the MIP mode is transposed, the selected transform kernel is matrix transposed to obtain the LFNST transform kernel used by the current block.

[0135] The value of the LFNST index number is set to indicate that the current block uses LFNST and that the LFNST transform core is the index number in the LFNST transform core candidate set; wherein, the LFNST transform core candidate set contains two or more preset transform cores for MIP.

[0136] It should be noted that since the LFNST transform core candidate set includes two or more pre-defined transform cores for MIP, rate-distortion optimization can be used to select the transform core to be used in the current block. Specifically, the rate-distortion cost (RDCost) can be calculated for each transform core using rate-distortion optimization, and then the transform core with the lowest rate-distortion cost can be selected as the transform core to be used in the current block.

[0137] In other words, on the encoder side, a set of LFNST transform kernels can be selected using RDCost, and the index number corresponding to the LFNST transform kernel (which can be represented by lfnst_idx) is written into the video bitstream and transmitted to the decoder side. Specifically, when the first set of LFNST transform kernels (i.e., the first set of transform matrices) in the LFNST transform kernel candidate set is selected, lfnst_idx is set to 1; when the second set of LFNST transform kernels (i.e., the second set of transform matrices) in the LFNST transform kernel candidate set is selected, lfnst_idx is set to 2.

[0138] It should also be noted that since the value of the MIP transpose indicator parameter is used to indicate whether the sample point input vector used in MIP mode is transposed, when the value of the MIP transpose indicator parameter is equal to 1, that is, when the value of the MIP transpose indicator parameter indicates that the sample point input vector used in MIP mode is transposed, the selected transform kernel needs to be matrix transposed to obtain the LFNST transform kernel used in the current block.

[0139] Here, regarding the value of the LFNST index number (i.e., lfnst_idx), when the LFNST index number is equal to 0, LFNST will not be used; while when the LFNST index number is greater than 0, LFNST will be used, and the transform kernel index number will be equal to the LFNST index number, or the LFNST index number minus 1. Thus, after determining the LFNST transform kernel used in the current block, it is also necessary to set the LFNST index number and write it into the video bitstream so that the decoder can obtain the LFNST index number by parsing the bitstream.

[0140] Optionally, in some embodiments, when the MIP parameter is the MIP mode index number, for S304, the step of determining the LFNST transform core used by the current block according to the MIP parameter, setting the LFNST index number, and writing it into the video stream when the current block uses LFNST may include:

[0141] Based on the value of the MIP mode index number, determine the value of the LFNST intra-frame prediction mode index number.

[0142] Based on the value of the LFNST intra-frame prediction mode index number, select one LFNST transform kernel candidate set from multiple LFNST transform kernel candidate sets;

[0143] From the selected LFNST transform core candidate set, select the transform core indicated by the LFNST index number and set it as the LFNST transform core used by the current block;

[0144] The value of the LFNST index number is set to indicate that the current block uses LFNST and that the LFNST transform core is the index number in the LFNST transform core candidate set; wherein, the LFNST transform core candidate set contains two or more preset LFNST transform cores.

[0145] It should be noted that the MIP mode index number is used to indicate the MIP mode used in the current block, and the MIP mode is used to indicate the calculation derivation method for determining the intra-frame prediction value of the current block using MIP; that is, the LFNST transform kernel can also be determined according to the MIP mode index number.

[0146] It should also be noted that after determining the MIP mode index number, the MIP mode index number can be converted into the value of the LFNST intra-prediction mode index number (which can be represented by predModeIntra); then, based on the value of predModeIntra, an LFNST transform kernel candidate set is selected from multiple LFNST transform kernel candidate sets to determine the transform kernel candidate set; and in the selected LFNST transform kernel candidate set, the transform kernel indicated by the LFNST index number is selected and set as the LFNST transform kernel used in the current block.

[0147] Here, regarding the value of the LFNST index number, when the LFNST index number is equal to 0, LFNST will not be used; while when the LFNST index number is greater than 0, LFNST will be used, and the index number of the transform core will be equal to the value of the LFNST index number, or the value of the LFNST index number minus 1. Thus, after determining the LFNST transform core used in the current block, it is also necessary to set the LFNST index number and write it into the video bitstream, so that the decoder can obtain the LFNST index number by parsing the bitstream.

[0148] Optionally, in some embodiments, when the MIP parameters are the MIP mode index number and the MIP transpose indicator parameter, for S304, the step of determining the LFNST transform kernel used by the current block according to the MIP parameters, setting the LFNST index number, and writing it into the video stream when the current block uses LFNST may include:

[0149] Based on the value of the MIP mode index number, determine the value of the LFNST intra-frame prediction mode index number.

[0150] Based on the value of the LFNST intra-frame prediction mode index number, select one LFNST transform kernel candidate set from multiple LFNST transform kernel candidate sets;

[0151] From the selected LFNST transform core candidate set, select the transform core indicated by the LFNST index number and set it as the LFNST transform core used by the current block;

[0152] When the value of the MIP transpose indicator parameter indicates that the sampling point input vector used in the MIP mode is transposed, the LFNST transform kernel used in the current block is matrix transposed, and the transform kernel obtained after transposition is set as the LFNST transform kernel used in the current block.

[0153] Set the value of the LFNST index number to indicate that the current block uses LFNST and that the LFNST transform kernel is the index number in the LFNST transform kernel candidate set;

[0154] The LFNST transform core candidate set includes two or more preset LFNST transform cores.

[0155] It should be noted that the MIP mode index number is used to indicate the MIP mode used in the current block, and the MIP mode is used to indicate the calculation derivation method for determining the intra-frame prediction value of the current block using MIP; the value of the MIP transpose indicator parameter is used to indicate whether the sampling point input vector used in the MIP mode is transposed; that is, the LFNST transform kernel can also be determined by the combination of the MIP transpose indicator parameter and the MIP mode index number.

[0156] It should also be noted that after determining the MIP mode index number, the MIP mode index number can be converted into the LFNST intra-prediction mode index number (which can be represented by predModeIntra). Then, based on the value of predModeIntra, an LFNST transform kernel candidate set is selected from multiple LFNST transform kernel candidate sets to determine the transform kernel candidate set. In the selected LFNST transform kernel candidate set, the transform kernel indicated by the LFNST index number is selected. When the indication requires transpose processing, the LFNST transform kernel used by the current block also needs to be matrix transposed. Then, the transform kernel obtained after transpose processing is set as the LFNST transform kernel used by the current block.

[0157] Here, regarding the value of the LFNST index number, when the LFNST index number is equal to 0, LFNST will not be used; while when the LFNST index number is greater than 0, LFNST will be used, and the index number of the transform core will be equal to the value of the LFNST index number, or the value of the LFNST index number minus 1. Thus, after determining the LFNST transform core used in the current block, it is also necessary to set the LFNST index number and write it into the video bitstream, so that the decoder can obtain the LFNST index number by parsing the bitstream.

[0158] It should also be noted that, regarding the above-mentioned methods for selecting the LFNST transform core to be used in the current block, experimental results show that the LFNST transform core determined by the combination of the MIP transpose indicator parameter and the MIP mode index number has the best performance.

[0159] Furthermore, when the MIP parameters include at least the MIP mode index number (modeId), the value of the LFNST intra-frame prediction mode index number can be obtained by looking up a table during the process of selecting the LFNST transform kernel used in the current block.

[0160] Specifically, in some embodiments, determining the value of the LFNST intra-frame prediction mode index number based on the value of the MIP mode index number may include:

[0161] The first lookup table is used to determine the value of the LFNST intra-prediction mode index corresponding to the value of the MIP mode index, wherein the first lookup table contains at least one or more different MIP mode indexes corresponding to each of two different LFNST intra-prediction mode indexes.

[0162] Here, the first look-up table (LUT1) is used to reflect the correspondence between the MIP mode index number and the LFNST intra-prediction mode index number. That is, the first look-up table contains at least one or more different MIP mode index numbers corresponding to each of the two different LFNST intra-prediction mode index numbers.

[0163] In other words, different MIP modes can correspond to different values ​​of predModeIntra. Thus, the MIP mode index number is determined according to the MIP mode, and then the value of predModeIntra is determined according to the first lookup table. Then, based on the value of predModeIntra, an LFNST transform core candidate set can be selected from multiple LFNST transform core candidate sets, thereby determining the LFNST transform core used by the current block.

[0164] Understandably, the value of `predModeIntra` can be determined based on the MIP mode index number (`modeId`); then, based on the value of `predModeIntra`, the value of `SetIdx` can be directly determined according to Table 2, i.e., the LFNST transform kernel candidate set selected for the current block is determined. Here, the value of `SetIdx` indicates the transform kernel candidate set used during LFNST; since the value of `modeId` can include 0, 1, 2, 3, 4, and 5, the value of `predModeIntra` is also 0, 1, 2, 3, 4, and 5; its correspondence with `SetIdx` is as follows:

[0165] Table 2

[0166] predModeIntra SetIdx 0 2 1 2 2 0 3 0 4 1 5 3

[0167] Furthermore, the candidate set of LFNST transform kernels can be directly determined based on the value of the MIP mode index number. In this case, it is no longer necessary to use the value of predModeIntra, that is, it is no longer necessary to determine the value of the LFNST intra-frame prediction mode index number based on the MIP mode index number.

[0168] Optionally, in some embodiments, when the current block uses LFNST, determining the LFNST transform kernel used by the current block according to the MIP parameters, setting the LFNST index number, and writing it into the video stream may include:

[0169] Based on the value of the MIP mode index number, select one LFNST transform kernel candidate set from multiple LFNST transform kernel candidate sets;

[0170] From the selected LFNST transform core candidate set, select the transform core indicated by the LFNST index number and set it as the LFNST transform core used by the current block;

[0171] Set the value of the LFNST index number to indicate that the current block uses LFNST and that the LFNST transform kernel is the index number in the LFNST transform kernel candidate set;

[0172] The LFNST transform core candidate set includes two or more preset LFNST transform cores.

[0173] Further, the step of selecting an LFNST transform kernel candidate set from multiple LFNST transform kernel candidate sets based on the value of the MIP mode index number may include:

[0174] The second lookup table is used to determine the value of the LFNST transform kernel candidate set index corresponding to the value of the MIP mode index number, and the LFNST transform kernel candidate set indicated by the value of the LFNST transform kernel candidate set index number is selected as the selected LFNST transform kernel candidate set.

[0175] The second look-up table (LUT2) contains at least one or more different MIP mode index numbers corresponding to the LFNST transform kernel candidate set index numbers with two different values.

[0176] It should be noted that if the prediction mode used by the current block is MIP mode, the selected LFNST transform kernel candidate set can be determined based on the value of the MIP mode index number (modeId). As shown in Table 3, each MIP mode index number corresponds to one LFNST transform kernel candidate set.

[0177] Table 3

[0178] modeId SetIdx 0 0 1 0 2 0 3 1 4 2 5 3

[0179] Thus, based on the value of the MIP mode index number, the corresponding LFNST transform kernel candidate set index number can be determined, and the LFNST transform kernel candidate set indicated by the value of the LFNST transform kernel candidate set index number can be selected as the chosen LFNST transform kernel candidate set. For example, as can be seen from Table 3, when the value of the MIP mode index number (modeId) is 0, the value of the LFNST transform kernel candidate set index number (SetIdx) can be determined to be 0, that is, the transform kernel candidate set indicated by 0 is selected as the chosen LFNST transform kernel candidate set; or, when modeId is 3, the value of SetIdx can be determined to be 1, that is, the transform kernel candidate set indicated by 1 is selected as the chosen LFNST transform kernel candidate set, and so on.

[0180] Optionally, in some embodiments, selecting an LFNST transform core candidate set from multiple LFNST transform core candidate sets based on the value of the MIP mode index number may include:

[0181] Based on the value of the MIP mode index number, the index number of the LFNST transform kernel candidate set is determined using the first calculation method;

[0182] The LFNST transform kernel candidate set indicated by the value of the LFNST transform kernel candidate set index is selected as the selected LFNST transform kernel candidate set.

[0183] Further, determining the index number of the LFNST transform kernel candidate set using the first calculation method based on the value of the MIP mode index number may include:

[0184] When the value of the MIP mode index number is less than the first preset value, the value of the LFNST transform kernel candidate set index number is set to be equal to the value of the MIP mode index number.

[0185] When the value of the MIP mode index number is greater than or equal to the first preset value, the value of the LFNST transform kernel candidate set index number is set to be equal to the difference between the value of the MIP mode index number and the second preset value; wherein, the first preset value and the second preset value are both integer values.

[0186] In this embodiment of the application, the first preset value can be equal to 3, and the second preset value can be equal to 2.

[0187] In other words, based on the value of the MIP mode index number, the index number of the LFNST transform kernel candidate set can be determined using the first calculation method, where the first calculation formula is as follows:

[0188] SetIdx = modeId <x?0:(modeId-y) (3)

[0189] Where x represents the first preset value, y represents the second preset value; SetIdx represents the index number of the LFNST transform kernel candidate set, and modeId represents the index number of the MIP mode.

[0190] Specifically, assuming x equals 3 and y equals 2; then when modeId < 3, it means that SetIdx can be 0; when modeId ≥ 3, it means that SetIdx can be modeId - 2; that is, when modeId = 3, SetIdx can be 1; when modeId = 4, SetIdx can be 2; when modeId = 5, SetIdx can be 3; it can be seen that, according to equation (3), the value of SetIdx obtained is the same as the content of Table 3.

[0191] In addition, if the prediction mode of the current block is MIP mode, for the determination of the LFNST transform kernel candidate set, one of the transform sets (such as set1, set2, or set3) can be fixed as the LFNST transform kernel candidate set selected from multiple LFNST transform kernel candidate sets.

[0192] In addition, if the prediction mode of the current block is MIP mode, the LFNST transform kernel candidate set can be selected according to the value of the MIP mode index number (modeId); as shown in Table 4, each MIP mode corresponds to one LFNST transform kernel candidate set; here, the specific number of the LFNST transform kernel candidate set is limited; where a, b, c, d, and e can all be arbitrarily selected from {0, 1, 2, 3}.

[0193] Table 4

[0194]

[0195]

[0196] It is also worth noting that if the prediction mode of the current block is MIP mode, the LFNST transform kernel candidate set can be selected based on one or more combinations of information in the MIP parameters; alternatively, the mapping between the MIP parameters and the traditional intra-frame prediction mode can be performed, and then the LFNST transform kernel candidate set can be selected based on the mapped angle; even the transform matrix (i.e., transform kernel) to be used can be determined based on one or more combinations of information in the MIP parameters, and the determination of the transform matrix includes the selection of the transform set and the transform matrix group; here, a certain transform matrix group in a certain transform set can be fixedly selected based on the MIP parameters, without selection on the encoder side, and in this case, lfnst_idx does not need to be transmitted.

[0197] Furthermore, in some embodiments, when the prediction mode parameter indicates that the current block uses a non-MIP mode, the method may further include:

[0198] Based on the intra-frame prediction mode, determine the value of predModeIntra;

[0199] Based on the value of predModeIntra, select one LFNST transform kernel candidate set from multiple LFNST transform kernel candidate sets.

[0200] In other words, if the prediction mode of the current block is non-MIP mode, the value of predModeIntra can be determined based on the intra-frame prediction mode. Then, based on the value of predModeIntra and in conjunction with Table 1 above, an LFNST transform kernel candidate set can be selected from multiple LFNST transform kernel candidate sets. Subsequently, from the selected LFNST transform kernel candidate set, the transform kernel indicated by the LFNST index number is selected and set as the LFNST transform kernel used by the current block. The value of the LFNST index number is set to the index number of the LFNST transform kernel in the LFNST transform kernel candidate set that indicates the current block uses LFNST.

[0201] Here, regarding the value of the LFNST index number, when the LFNST index number is equal to 0, LFNST will not be used; while when the LFNST index number is greater than 0, LFNST will be used, and the index number of the transform core will be equal to the value of the LFNST index number, or the value of the LFNST index number minus 1. Thus, after determining the LFNST transform core used in the current block, it is also necessary to set the LFNST index number and write it into the video bitstream, so that the decoder can obtain the LFNST index number by parsing the bitstream.

[0202] S305: Use the LFNST transform kernel to transform the predicted difference.

[0203] It should be noted that after determining the FNST transform kernel, the transform matrix selected for the current block can be obtained, and at this time, the prediction difference can be transformed.

[0204] Each group of transform matrices can also include basic transform matrices T of two sizes, such as 16x16 and 16x48. For the transform matrices selected for the 4 types of TUs of different sizes, specifically, for the 4x4-sized TU, an 8x16 transform matrix will be used, and this 8x16 transform matrix comes from the first 8x16 of the 16x16 basic transform matrix; for the 4xN or Nx4 (N>4)-sized TU, a 16x16 basic transform matrix will be used; for the 8x8-sized TU, an 8x48 transform matrix will be used, and this 8x48 transform matrix comes from the first 8x48 of the 16x48 basic transform matrix; for the TU larger than 8x8, a 16x48 basic transform matrix will be used. It should be noted that in the current H.266 / VVC, only the transform matrix of the LFNST on the decoder side (which can be represented by T T is stored), and the transform matrix used on the encoder side is the transpose matrix of the transform matrix of the LFNST (which can be represented by T).

[0205] It should also be noted that LFNST applies an inseparable transform based on the direct matrix multiplication method. In order to minimize the computational complexity and storage space as much as possible, a simplified inseparable transform technology is used in the LFNST transform. Among them, the main idea of the simplified inseparable transform technology is to map an N-dimensional vector to an R-dimensional vector in a different space. Here, N / R (R<N) is the scaling factor; at this time, the transform matrix corresponding to the simplified inseparable transform technology is an R×N matrix, as shown below,

[0206]

[0207] Here, the transform matrices used for the forward LFNST transform and the reverse LFNST transform are transpose relations with each other. Refer to Figure 5 , which shows a schematic structural diagram of the calculation process of a matrix multiplication of an LFNST technology provided by an embodiment of the present application. As Figure 5 shown, (a) shows the calculation process of the forward LFNST transform. After the primary transform coefficients pass through the transform matrix T, the secondary transform coefficients can be obtained; (b) shows the calculation process of the reverse LFNST transform. After the inverse secondary transform coefficients pass through the transpose transform matrix T T then the inverse primary transform coefficients can be obtained.

[0208] Furthermore, in the LFNST technique, the choice between a 4×4 or 8×8 inseparable transformation can be determined based on the size of the current block. Here, the "4×4 inseparable transformation" can be collectively referred to as "4×4 LFNST," and the "8×8 inseparable transformation" as "8×8 LFNST." Assuming the current block has a width of nTbW and a height of nTbH, we can conclude that: if min(nTbW, nTbH) <= 4, then 4×4 LFNST can be used on the current block; otherwise, 8×8 LFNST can be used. It is important to note that the return value of min(A, B) is the smaller of A and B.

[0209] In one implementation, for a 4×4 LFNST, 16 coefficients will be input on the encoder side, and after passing through the forward LFNST, 16 or 8 coefficients will be output; while on the decoder side, 16 or 8 coefficients will be input, and 16 coefficients will be output; that is, the encoder and decoder have exactly opposite numbers of input and output.

[0210] Assume the size of the transform unit (TU) can be expressed as nTbW×nTbH, where the transform unit is the prediction residual block obtained based on the prediction difference. That is, the size of TU can be equal to 4×4, or 4×N or N×4 (N>4). These will be described in detail below.

[0211] When the size of TU is equal to 4×4, the forward LFNST process corresponding to the 4×4 transform block is as follows: Figure 6A As shown. Among them, in Figure 6A In the diagram, white blocks represent prediction differences, gray blocks represent first-order transform coefficients, and black blocks represent second-order transform coefficients. Here, at the "0" example position, the encoder sets the transform coefficients to 0. For a 4×4 transform block, during forward LFNST, the transform matrix size used is 8×16. All 4×4 first-order transform coefficients within the current transform block are used as input, and the output is 4×2 second-order transform coefficients.

[0212] When the size of TU is equal to 4×N or N×4 (N>4), the forward LFNST process corresponding to the 4×N or N×4 transform block is as follows: Figure 6B As shown. Among them, in Figure 6BIn the diagram, white blocks represent prediction differences, gray blocks represent first-order transform coefficients, and black blocks represent second-order transform coefficients. For 4×N or N×4 transform blocks, during forward LFNST, the transform matrix size is 16×16. The first-order transform coefficients within the first 4×4 sub-block of the current transform block (specifically, the topmost sub-block for 4×N transform blocks and the leftmost sub-block for N×4 transform blocks) are used as input, and the output is 4×4 second-order transform coefficients. Here, at the "0" example position, the encoder still sets the transform coefficients to 0.

[0213] In another implementation, for an 8×8 LFNST, 48 coefficients will be input on the encoder side, and after passing through the forward LFNST, 16 or 8 coefficients will be output; while on the decoder side, 16 or 8 coefficients will be input, and 48 coefficients will be output; that is, the encoder and decoder have exactly opposite numbers of input and output.

[0214] When the size of TU is equal to 8×8, the forward LFNST process corresponding to the 8×8 transform block is as follows: Figure 6C As shown. Among them, in Figure 6C In the diagram, white blocks represent prediction differences, gray blocks represent first-order transform coefficients, and black blocks represent second-order transform coefficients. For an 8×8 transform block, during forward LFNST, the transform matrix size used is 8×48. The first-order transform coefficients of the first three 4×4 sub-blocks within the current transform block (i.e., the three sub-blocks located in the upper left corner) are used as input, and the output is 4×2 second-order transform coefficients. Here, at the "0" example position, the encoder still sets the transform coefficients to 0.

[0215] When TU is greater than 8×8, the forward LFNST process corresponding to the transform block greater than 8×8 is as follows: Figure 6D As shown. Among them, in Figure 6D In the diagram, white blocks represent prediction differences, gray blocks represent first-order transform coefficients, and black blocks represent second-order transform coefficients. For transform blocks larger than 8×8, the transform matrix size used in forward LFNST is 48×16. The first-order transform coefficients of the first three 4×4 sub-blocks within the current transform block (i.e., the three sub-blocks located in the upper left corner) are used as input, and the output is 4×4 second-order transform coefficients. Here, at the "0" example position, the encoder still sets the transform coefficients to 0.

[0216] Thus, the TU corresponding to the prediction difference can be 4×4, 4×N, N×4 (N>4), 8×8, or even larger than 8×8. In this case, it can be calculated according to... Figure 6A or Figure 6B or Figure 6C or Figure 6D This is used to transform the predicted difference.

[0217] In this embodiment, the applicability of LFNST technology to the current block using MIP mode can be improved, making the selection of transform set (or transform kernel) more flexible. By introducing relevant information about MIP parameters during LFNST of the current block using MIP mode, and judging the characteristics of the current block based on this MIP information, the transform set (or transform kernel) is selected. For example, this transform method is applied to VTM7.0 and tested at 24-frame intervals under All Intra conditions. Based on the average bitrate change under the same Peak Signal-to-Noise Ratio (PSNR), BD-rate changes of -0.03%, 0.00%, and -0.01% can be obtained on the three image components (Y, Cb, and Cr), respectively. Especially on high-resolution sequences, better performance will be achieved; specifically, on Class A1, a -0.10% BD-rate change in Y can be achieved, thereby improving coding efficiency.

[0218] This embodiment provides a transformation method applied to an encoder. The transformation method includes: determining prediction mode parameters for the current block; when the prediction mode parameters indicate that the current block uses matrix-based intra-prediction (MIP) to determine intra-predicted values, determining MIP parameters; determining the intra-predicted values ​​of the current block based on the MIP parameters, and calculating the prediction difference between the current block and the intra-predicted values; when the current block uses Low-Frequency Inseparable Secondary Transform (LFNST), determining the LFNST transform kernel used by the current block based on the MIP parameters, setting the LFNST index number and writing it into the video bitstream; and using the LFNST transform kernel to transform the prediction difference. Thus, for the current block using MIP mode, the introduction of MIP parameters during LFNST transformation makes the selection of the LFNST transform kernel more flexible, thereby not only improving the applicability of LFNST technology to non-traditional intra-prediction modes, but also improving coding efficiency and video image quality.

[0219] Based on the above Figure 2B For application scenario examples, see Figure 7 This illustrates a flowchart of another transformation method provided in an embodiment of this application. For example... Figure 7 As shown, the method may include:

[0220] S701: Parse the bitstream and determine the prediction mode parameters for the current block;

[0221] It should be noted that the prediction mode parameter indicates the coding mode of the current block and the parameters related to that mode. The prediction mode typically includes traditional intra-frame prediction modes and non-traditional intra-frame prediction modes. Traditional intra-frame prediction modes can include DC mode, PLANA mode, and angle mode, while non-traditional intra-frame prediction modes can include MIP mode, CCLM mode, IBC mode, and PLT mode.

[0222] It should also be noted that on the encoder side, predictive encoding is performed on the current block. During this process, the prediction mode of the current block can be determined, and the corresponding prediction mode parameters are written into the bitstream and transmitted from the encoder to the decoder.

[0223] On the decoder side, the intra-prediction mode of the luminance or chrominance component of the current block or the coding block to which the current block is located can be obtained by parsing the bitstream. At this time, the value of predModeIntra can be determined, and the calculation formula is as described in the previous formula (1).

[0224] In equation (1), the image component indicator (which can be represented by cIdx) is used to indicate the luma component or chroma component of the current block; here, if the current block is predicted to be the luma component, then cIdx equals 0; if the current block is predicted to be the chroma component, then cIdx equals 1. In addition, (xTbY, yTbY) are the coordinates of the top left sampling point of the current block, IntraPredModeY[xTbY][yTbY] is the intra-prediction mode of the luma component, and IntraPredModeC[xTbY][yTbY] is the intra-prediction mode of the chroma component.

[0225] S702: When the prediction mode parameter indicates that the current block uses MIP to determine the intra-frame prediction value, parse the bitstream and determine the MIP parameter;

[0226] It should be noted that MIP parameters may include MIP transpose indicator parameters (which can be represented by isTransposed), MIP mode index number (which can be represented by modeId), current block size, current block class (which can be represented by mipSizeId), etc.; the values ​​of these parameters can be obtained by parsing the bitstream.

[0227] In some embodiments, the value of isTransposed can be determined by parsing the bitstream. When the value of isTransposed is equal to 1, the sampling point input vector used in MIP mode needs to be transposed. When the value of isTransposed is equal to 0, the sampling point input vector used in MIP mode does not need to be transposed. In other words, the MIP transpose indicator parameter can be used to indicate whether the sampling point input vector used in MIP mode needs to be transposed.

[0228] In some embodiments, by parsing the bitstream, the MIP mode index number (which can be represented by modeId) can also be determined. The MIP mode index number can indicate the MIP mode used in the current block, and the MIP mode can indicate the calculation derivation method for determining the intra-frame prediction value of the current block using MIP. That is, different MIP modes have different values ​​for their corresponding MIP mode index numbers; here, the value of the MIP mode index number can be 0, 1, 2, 3, 4, or 5.

[0229] In some embodiments, by parsing the bitstream, parameters such as the size, aspect ratio, and type of the current block (which can be represented by mipSizeId) can also be determined. This allows for the selection of the LFNST transform kernel (which can be represented by kernel) for the current block based on the determined MIP parameters.

[0230] S703: Analyze the bitstream to determine the transform coefficients and LFNST index number of the current block;

[0231] It should be noted that the LFNST index number can be used to indicate whether the current block uses LFNST and the index number of the LFNST transform kernel in the LFNST transform kernel candidate set. Specifically, after resolving the LFNST index number, when the LFNST index number is equal to 0, it indicates that the current block does not use LFNST; while when the LFNST index number is greater than 0, it indicates that the current block uses LFNST, and the index number of the transform kernel is equal to the value of the LFNST index number, or the value of the LFNST index number minus 1.

[0232] It should also be noted that, on the decoder side, the input data of LFNST may include: the luminance position (xTbY, yTbY) of the current transform block, the width of the current block nTbW, the height of the current block nTbH, whether the current block is a luminance component or a chrominance component cIdx, and the coefficients d[x][y] after the current transform block is dequantized (scaled), x = 0, 1, ..., nTbW-1, y = 0, 1, ..., nTbH-1; the output data of LFNST may include: the first transform coefficients d'[x][y] generated by LFNST from the second transform coefficients, x = 0, 1, ..., nLfnstSize-1, y = 0, 1, ..., nLfnstSize-1.

[0233] Specifically, such as Figure 8As shown, the LFNST process can be divided into five steps: configuring core parameters 81, intra-prediction mode mapping 82, selecting the transformation matrix 83, calculating matrix multiplication 84, and constructing the first-order transformation coefficient matrix 85. Specifically, for intra-prediction mode mapping 82, this step determines the value of `predModeIntra`, which mainly includes: non-traditional intra-prediction mode mapping 821 and wide-angle mapping 822. For selecting the transformation matrix 83, this step selects the transformation set and the transformation matrix, which mainly includes: selecting the transformation set 831, selecting the transformation matrix group 832, and selecting the transformation matrix size 833.

[0234] To configure core parameter 81, you first need to configure the length of the input quadratic transform coefficient vector (which can be represented by nonZeroSize) and the length of the output first transform coefficient vector (which can be represented by nLfnstOutSzie) for LFNST calculation. The values ​​for nonZeroSize and nLfnstOutSzie are shown in Table 5.

[0235] Table 5

[0236] Transform block size nonZeroSize nLfnstOutSzie 4×4 8 16 4×N or N×4 (N>4) 16 16 8×8 8 48 >8×8 16 48

[0237] In the current H.266 / VVC, the parameters nonZeroSize and nLfnstOutSzie are calculated using the following formula:

[0238] nLfnstOutSzie=(nTbW>=8&&nTbH>=8)? 48:16 (5)

[0239] nonZeroSize=(nTbW==4&&nTbH==4)||(nTbW==8&&nTbH==8)? 8:16 (6)

[0240] In addition, the parameter nLfnstSize needs to be configured, indicating that there will only be one transformation coefficient within the first nLfnstSize × nLfnstSize range in the current block. The values ​​of nLfnstSize are shown below.

[0241] Log2LfnstSize=(nTbW>=8&&nTbH>=8)? 3:2 (7)

[0242] nLfnstSize=1<<Log2LfnstSize (8)

[0243] At this point, by parsing the bitstream, we can also obtain the intra-prediction mode of the luminance or chrominance components of the current block or the coding block to which the current block is located. At this point, we can determine the value of predModeIntra, and the calculation formula is as described in the previous formula (1).

[0244] Further, obtain the vector u[i] of the quadratic transformation coefficients, i = 0, 1, ..., nonZeroSize-1. When it is determined that the current transform block uses LFNST, the dequantized coefficients d[x][y] are the quadratic transformation coefficients. Obtain the first nonZeroSize values ​​in diagonal scan order, which are the vector u[i] of the quadratic transformation coefficients, i = 0, 1, ..., nonZeroSize-1; in the following formula, xC and yC represent the x and y coordinates of the coefficient numbered x in the current block relative to the top left corner point, in diagonal order. xC and yC are shown below.

[0245] xC=DiagScanOrder[2][2][x][0] (9)

[0246] yC=DiagScanOrder[2][2][x][1] (10)

[0247] u[i]=d[xC][yC] (11)

[0248] Furthermore, for intra-prediction mode mapping 82, intra-prediction modes can be further divided into traditional intra-prediction modes and non-traditional intra-prediction modes. For non-traditional intra-prediction modes, the information indicated by the predModeIntra value is as follows:

[0249] If the value of predModeIntra can be INTRA_LT_CCLM, INTRA_L_CCLM or INTRA_T_CCLM (81, 82, 83 in VVC respectively), it indicates that the prediction mode of the current block is CCLM mode.

[0250] If intra_mip_flag[xTbY][yTbY] equals 1 and cIdx equals 0, it indicates that the prediction mode of the current block is MIP mode. In this case, the value of predModeIntra indicates the MIP mode index number modeId used.

[0251] If neither of the above two cases applies, the value of predModeIntra can be [0, 66], indicating that the prediction mode of the current block is the traditional intra-frame prediction mode.

[0252] Furthermore, the LFNST transform kernel candidate set index number is determined by parsing the bitstream based on the numbering of the traditional intra-frame prediction mode. At this point, if the prediction mode of the current block is CCLM or MIP mode, the value of predModeIntra can be set as follows:

[0253] (1) When the value of predModeIntra indicates INTRA_LT_CCLM, INTRA_L_CCLM, or INTRA_T_CCLM (81, 82, and 83 in VVC respectively),

[0254] If the mode of the center luminance block at the luminance position corresponding to the current block (such as the chroma block) is MIP mode, that is, intra_mip_flag[xTbY+nTbW / 2][yTbY+nTbH / 2] is 1, then the value of predMode Intra is set to the index number indicating the PLANAR mode (i.e., 0).

[0255] Otherwise, if the mode of the center luminance block at the luminance position corresponding to the current block (such as the chroma block) is IBC mode or PLT mode, then the value of predModeIntra is set to the index number indicating the DC mode (i.e., 1).

[0256] Otherwise, set the value of predModeIntra to the mode index number of the center luminance block at the luminance position of the current block (e.g., chroma block) IntraPredModeY[xTbY+nTbW / 2][yTbY+nTbH / 2];

[0257] (2) When intra_mip_flag[xTbY][yTbY] equals 1 and cIdx equals 0, that is, the prediction mode of the current block is MIP mode, the value of predModeIntra can be directly set to the index number indicating the PLANA mode (i.e., 0).

[0258] For traditional intra-frame prediction modes (such as wide-angle mapping), during the bitstream parsing process, wide-angle mapping can be performed based on the size of the current block, extending the traditional intra-frame prediction mode [0, 66] to [-14, 80]. The specific mapping process is as follows:

[0259] First, calculate the width-to-height ratio factor (which can be represented by whRatio), as shown in equation (2) above.

[0260] For the current block that is not square (i.e., nTbW is not equal to nTbH), the value of predModeIntra can be corrected as follows: if nTbW is greater than nTbH, and predModeIntra is greater than or equal to 2, and predModeIntra is less than ((whRatio>1?(8+2×whRatio):8), then predModeIntra = (predModeIntra+65); otherwise, if nTbW is less than nTbH, and predModeIntra is less than or equal to 66, and predModeIntra is greater than ((whRatio>1?(60-2×whRatio):60), then predModeIntra = (predModeIntra-67).

[0261] In the current H.266 / VVC, based on the value of predModeIntra and Table 1, the value of the LFNST index number (which can be represented by SetIdx) can be determined, as shown in Table 1. Here, the LFNST index number is set to indicate that the current block uses LFNST and that the LFNST transform kernel is in the LFNST transform kernel candidate set. Typically, the LFNST transform set includes four transform kernel candidate sets (set0, set1, set2, set3), corresponding to SetIdx values ​​of 0, 1, 2, and 3, respectively.

[0262] In current H.266 / VVC, for MIP mode, because the value of predModeIntra is set to the index number indicating the PLANA mode (i.e., 0), the transform set used by the current block in MIP mode can only be selected from the transform set with an LFNST index number equal to 0. This results in a lack of variability when performing LFNST in MIP mode, reducing decoding efficiency. However, in the embodiments of this application, after determining the LFNST index number, a candidate set of LFNST transform kernels can be determined based on the MIP parameters, and then the LFNST transform kernel used by the current block can be determined from the candidate set.

[0263] S704: When the LFNST index number indicates that the current block uses LFNST, determine the LFNST transform kernel used by the current block according to the MIP parameters;

[0264] Here, the MIP parameters may include the MIP transpose indicator parameter (which can be represented by isTransposed), the MIP mode index number (which can be represented by modeId), the size of the current block, and the class of the current block (which can be represented by mipSizeId). The following section will describe in detail how to select the LFNST transform kernel to be used for the current block based on the MIP parameters.

[0265] Optionally, in some embodiments, when the MIP parameter is a MIP transpose indicator parameter, for S704, determining the LFNST transform kernel used by the current block according to the MIP parameter when the LFNST index number indicates that the current block uses LFNST may include:

[0266] Select the transform kernel indicated by the LFNST index number from the LFNST transform kernel candidate set;

[0267] When the value of the MIP transpose indicator parameter indicates that the sampling point input vector used in the MIP mode is transposed, the selected transform kernel is matrix transposed to obtain the LFNST transform kernel used by the current block; wherein, the LFNST transform kernel candidate set includes two or more preset transform kernels for MIP.

[0268] It should be noted that since the LFNST transform kernel candidate set includes two or more pre-defined transform kernels for MIP, after obtaining the LFNST index number by parsing the bitstream, the transform kernel indicated by the obtained LFNST index number can be selected from the LFNST transform kernel candidate set according to the value of the obtained LFNST index number. For example, when the value of the LFNST index number is 1, the first group of LFNST transform kernels (i.e., the first group of transform matrices) in the LFNST transform kernel candidate set will be selected; while when the value of the LFNST index number is 2, the second group of LFNST transform kernels (i.e., the second group of transform matrices) in the LFNST transform kernel candidate set will be selected.

[0269] It should also be noted that since the value of the MIP transpose indicator parameter is used to indicate whether the sample point input vector used in MIP mode is transposed, when the value of the MIP transpose indicator parameter is equal to 1, that is, when the value of the MIP transpose indicator parameter indicates that the sample point input vector used in MIP mode is transposed, the selected transform kernel needs to be matrix transposed to obtain the LFNST transform kernel used in the current block.

[0270] Here, regarding the value of the LFNST index number (i.e., lfnst_idx), when the LFNST index number is equal to 0, LFNST will not be used; while when the LFNST index number is greater than 0, LFNST will be used, and the transform kernel index number will be equal to the LFNST index number, or the LFNST index number minus 1. Thus, based on the LFNST index number, the LFNST transform kernel used in the current block can be determined.

[0271] Optionally, in some embodiments, when the MIP parameter is a MIP mode index number, for S704, determining the LFNST transform kernel used by the current block according to the MIP parameter when the LFNST index number indicates that the current block uses LFNST may include:

[0272] Based on the value of the MIP mode index number, determine the value of the LFNST intra-frame prediction mode index number.

[0273] Based on the value of the LFNST intra-frame prediction mode index number, select one LFNST transform kernel candidate set from multiple LFNST transform kernel candidate sets;

[0274] From the selected LFNST transform core candidate set, select the transform core indicated by the LFNST index number and set it as the LFNST transform core used by the current block; wherein, the LFNST transform core candidate set contains two or more preset LFNST transform cores.

[0275] It should be noted that the MIP mode index number is used to indicate the MIP mode used in the current block, and the MIP mode is used to indicate the calculation derivation method for determining the intra-frame prediction value of the current block using MIP; that is, the LFNST transform kernel can also be determined according to the MIP mode index number.

[0276] It should also be noted that after determining the MIP mode index number, the MIP mode index number can be converted into the value of the LFNST intra-prediction mode index number (which can be represented by predModeIntra); then, based on the value of predModeIntra, an LFNST transform kernel candidate set is selected from multiple LFNST transform kernel candidate sets to determine the transform kernel candidate set; and in the selected LFNST transform kernel candidate set, the transform kernel indicated by the LFNST index number is selected and set as the LFNST transform kernel used in the current block.

[0277] Optionally, in some embodiments, when the MIP parameters are the MIP mode index number and the MIP transpose indication parameter, for S704, determining the LFNST transform kernel used by the current block according to the MIP parameters when the LFNST index number indicates that the current block uses LFNST may include:

[0278] Based on the value of the MIP mode index number, determine the value of the LFNST intra-frame prediction mode index number.

[0279] Based on the value of the LFNST intra-frame prediction mode index number, select one LFNST transform kernel candidate set from multiple LFNST transform kernel candidate sets;

[0280] From the selected LFNST transform core candidate set, select the transform core indicated by the LFNST index number and set it as the LFNST transform core used by the current block;

[0281] When the value of the MIP transpose indicator parameter indicates that the sampling point input vector used in the MIP mode is transposed, the LFNST transform kernel used in the current block is matrix transposed, and the transform kernel obtained after transposition is set as the LFNST transform kernel used in the current block; wherein, the LFNST transform kernel candidate set includes two or more preset LFNST transform kernels.

[0282] It should be noted that the MIP mode index number is used to indicate the MIP mode used in the current block, and the MIP mode is used to indicate the calculation derivation method for determining the intra-frame prediction value of the current block using MIP; the value of the MIP transpose indicator parameter is used to indicate whether the sampling point input vector used in the MIP mode is transposed; that is, the LFNST transform kernel can also be determined by the combination of the MIP transpose indicator parameter and the MIP mode index number.

[0283] It should also be noted that after determining the MIP mode index number, the MIP mode index number can be converted into the LFNST intra-prediction mode index number (which can be represented by predModeIntra). Then, based on the value of predModeIntra, an LFNST transform kernel candidate set is selected from multiple LFNST transform kernel candidate sets to determine the transform kernel candidate set. In the selected LFNST transform kernel candidate set, the transform kernel indicated by the LFNST index number is selected. When the indication requires transpose processing, the LFNST transform kernel used by the current block also needs to be matrix transposed. Then, the transform kernel obtained after transpose processing is set as the LFNST transform kernel used by the current block.

[0284] It should also be noted that, regarding the above-mentioned methods for selecting the LFNST transform core to be used in the current block, experimental results show that the LFNST transform core determined by the combination of the MIP transpose indicator parameter and the MIP mode index number has the best performance.

[0285] Furthermore, when the MIP parameters include at least the MIP mode index number (modeId), the value of the LFNST intra-frame prediction mode index number can be obtained by looking up a table during the process of selecting the LFNST transform kernel used in the current block.

[0286] Specifically, in some embodiments, determining the value of the LFNST intra-frame prediction mode index number based on the value of the MIP mode index number may include:

[0287] The first lookup table is used to determine the value of the LFNST intra-prediction mode index corresponding to the value of the MIP mode index, wherein the first lookup table contains at least one or more different MIP mode indexes corresponding to each of two different LFNST intra-prediction mode indexes.

[0288] Here, the first look-up table (LUT1) is used to reflect the correspondence between the MIP mode index number and the LFNST intra-prediction mode index number. That is, the first look-up table contains at least one or more different MIP mode index numbers corresponding to each of the two different LFNST intra-prediction mode index numbers.

[0289] In other words, different MIP modes can correspond to different values ​​of predModeIntra. Thus, the MIP mode index number is determined according to the MIP mode, and then the value of predModeIntra is determined according to the first lookup table. Then, based on the value of predModeIntra, an LFNST transform core candidate set can be selected from multiple LFNST transform core candidate sets, thereby determining the LFNST transform core used by the current block.

[0290] Understandably, the value of predModeIntra can be determined based on the value of the MIP mode index number (modeId); then, based on the value of predModeIntra, the value of SetIdx can be directly determined according to Table 2, that is, the LFNST transform kernel candidate set selected for the current block is determined. Here, the value of SetIdx indicates the transform kernel candidate set used during LFNST; since the value of modeId can include 0, 1, 2, 3, 4, 5, the value of predModeIntra is also 0, 1, 2, 3, 4, 5; the correspondence between predModeIntra and SetIdx is shown in Table 2.

[0291] Furthermore, the candidate set of LFNST transform kernels can be directly determined based on the value of the MIP mode index number. In this case, it is no longer necessary to use the value of predModeIntra, that is, it is no longer necessary to determine the value of the LFNST intra-frame prediction mode index number based on the MIP mode index number.

[0292] Optionally, in some embodiments, determining the LFNST transform kernel used by the current block according to the MIP parameters when the LFNST index number indicates that the current block uses LFNST may include:

[0293] Based on the value of the MIP mode index number, select one LFNST transform kernel candidate set from multiple LFNST transform kernel candidate sets;

[0294] From the selected LFNST transform core candidate set, select the transform core indicated by the LFNST index number and set it as the LFNST transform core used by the current block; wherein, the LFNST transform core candidate set contains two or more preset LFNST transform cores.

[0295] Further, the step of selecting an LFNST transform kernel candidate set from multiple LFNST transform kernel candidate sets based on the value of the MIP mode index number may include:

[0296] The second lookup table is used to determine the value of the LFNST transform kernel candidate set index corresponding to the value of the MIP mode index number, and the LFNST transform kernel candidate set indicated by the value of the LFNST transform kernel candidate set index number is selected as the selected LFNST transform kernel candidate set.

[0297] The second look-up table (LUT2) contains at least one or more different MIP mode index numbers corresponding to the LFNST transform kernel candidate set index numbers with two different values.

[0298] It should be noted that if the prediction mode used by the current block is MIP mode, the selected LFNST transform kernel candidate set can be determined based on the value of the MIP mode index number (modeId). As shown in Table 3, each MIP mode index number corresponds to one LFNST transform kernel candidate set.

[0299] Thus, based on the value of the MIP mode index number, the corresponding LFNST transform kernel candidate set index number can be determined, and the LFNST transform kernel candidate set indicated by the value of the LFNST transform kernel candidate set index number can be selected as the chosen LFNST transform kernel candidate set. For example, as can be seen from Table 3, when the value of the MIP mode index number (modeId) is 0, the value of the LFNST transform kernel candidate set index number (SetIdx) can be determined to be 0, that is, the transform kernel candidate set indicated by 0 is selected as the chosen LFNST transform kernel candidate set; or, when modeId is 3, the value of SetIdx can be determined to be 1, that is, the transform kernel candidate set indicated by 1 is selected as the chosen LFNST transform kernel candidate set, and so on.

[0300] Optionally, in some embodiments, selecting an LFNST transform core candidate set from multiple LFNST transform core candidate sets based on the value of the MIP mode index number may include:

[0301] Based on the value of the MIP mode index number, the index number of the LFNST transform kernel candidate set is determined using the first calculation method;

[0302] The LFNST transform kernel candidate set indicated by the value of the LFNST transform kernel candidate set index is selected as the selected LFNST transform kernel candidate set.

[0303] Further, determining the index number of the LFNST transform kernel candidate set using the first calculation method based on the value of the MIP mode index number may include:

[0304] When the value of the MIP mode index number is less than the first preset value, the value of the LFNST transform kernel candidate set index number is set to be equal to the value of the MIP mode index number.

[0305] When the value of the MIP mode index number is greater than or equal to the first preset value, the value of the LFNST transform kernel candidate set index number is set to be equal to the difference between the value of the MIP mode index number and the second preset value; wherein, the first preset value and the second preset value are both integer values.

[0306] In this embodiment of the application, the first preset value can be equal to 3, and the second preset value can be equal to 2.

[0307] In other words, based on the value of the MIP mode index number, the index number of the LFNST transform kernel candidate set can be determined using the first calculation method. The first calculation formula is shown in equation (3) above, where x represents the first preset value, y represents the second preset value, SetIdx represents the value of the LFNST transform kernel candidate set index number, and modeId represents the value of the MIP mode index number.

[0308] Specifically, assuming x equals 3 and y equals 2; then when modeId < 3, it means that SetIdx can be 0; when modeId ≥ 3, it means that SetIdx can be modeId - 2; that is, when modeId = 3, SetIdx can be 1; when modeId = 4, SetIdx can be 2; when modeId = 5, SetIdx can be 3; it can be seen that, according to equation (3), the value of SetIdx obtained is the same as the content of Table 3.

[0309] In addition, if the prediction mode of the current block is MIP mode, for the determination of the LFNST transform kernel candidate set, one of the transform sets (such as set1, set2, or set3) can be fixed as the LFNST transform kernel candidate set selected from multiple LFNST transform kernel candidate sets.

[0310] In addition, if the prediction mode of the current block is MIP mode, the LFNST transform kernel candidate set can be selected according to the value of the MIP mode index number (modeId); as shown in Table 4, each MIP mode corresponds to one LFNST transform kernel candidate set; here, the specific number of the LFNST transform kernel candidate set is limited; where a, b, c, d, and e can all be arbitrarily selected from {0, 1, 2, 3}.

[0311] It is also worth noting that if the prediction mode of the current block is MIP mode, the LFNST transform kernel candidate set can be selected based on one or more combinations of information in the MIP parameters; alternatively, the mapping between the MIP parameters and the traditional intra-frame prediction mode can be performed, and then the LFNST transform kernel candidate set can be selected based on the mapped angle; even the transform matrix (i.e., transform kernel) to be used can be determined based on one or more combinations of information in the MIP parameters, and the determination of the transform matrix includes the selection of the transform set and the transform matrix group; here, a certain transform matrix group in a certain transform set can be fixedly selected based on the MIP parameters, which does not require selection on the encoder side, and therefore does not require transmission of lfnst_idx, and the decoder side does not need to parse the bitstream of lfnst_idx.

[0312] Furthermore, in some embodiments, when the prediction mode parameter indicates that the current block uses a non-MIP mode, the method may further include:

[0313] Based on the intra-frame prediction mode, determine the value of predModeIntra;

[0314] Based on the value of predModeIntra, select one LFNST transform kernel candidate set from multiple LFNST transform kernel candidate sets.

[0315] In other words, if the prediction mode of the current block is non-MIP mode, the value of predModeIntra can be determined according to the intra-frame prediction mode. Then, based on the value of predModeIntra and in conjunction with Table 1 above, an LFNST transform kernel candidate set can be selected from multiple LFNST transform kernel candidate sets. Then, from the selected LFNST transform kernel candidate set, the transform kernel indicated by the LFNST index number is selected and set as the LFNST transform kernel used by the current block.

[0316] S705: Use the LFNST transform kernel to perform transform processing on the transform coefficients.

[0317] It should be noted that after selecting the LFNST transform kernel candidate set, the LFNST index number (lfnst_idx) is obtained by parsing the bitstream. Based on the value of lfnst_idx, the transform matrix (transform kernel) indicated by lfnst_idx can be selected from the LFNST transform kernel candidate set. For example, when lfnst_idx is 1, the first set of transform matrices can be used as the LFNST transform kernel during the decoding process; when lfnst_idx is 2, the second set of transform matrices can be used as the LFNST transform kernel during the decoding process.

[0318] Furthermore, for each set of transformation matrices (transform kernels), there are two sizes of fundamental transformation matrices, with the sizes used on the decoder side being 16×16 and 48×16. The selection is based on nLfnstOutSzie: if nLfnstOutSzie is 16, a 16x16 fundamental transformation matrix is ​​selected; or if nLfnstOutSzie is 48, a 48x16 fundamental transformation matrix is ​​selected. Alternatively, if nonZeroSize is 8, only the first 8 rows of the transformation matrix are used for matrix multiplication calculations.

[0319] Furthermore, the quadratic transformation coefficient vector u[i] is taken as input and multiplied with the transformation matrix to obtain the first-order transformation coefficient vector v[j], where i = 0, 1, ..., nonZeroSize-1, j = 0, 1, ..., nLfnstOutSzie-1. Assuming the transformation matrix obtained in the previous steps is lowFreqTransMatrix, the specific calculation process of v[j] is as follows:

[0320]

[0321] Here, Clip3 is a clipping mechanism that restricts the value of the coefficient to between two numbers, as shown below.

[0322] CoeffMin = -(1 << 15) (13)

[0323] CoeffMax = (1 << 15) - 1 (14)

[0324] Thus, after the matrix calculations described above, the transformation coefficients can be processed. Here, for 4×4 LFNST, the decoder will input either 16 or 8 coefficients and output 16 coefficients; while for 8×8 LFNST, the decoder will input either 16 or 8 coefficients and output 48 coefficients, thereby achieving LFNST transformation processing of the transformation coefficients.

[0325] In this embodiment, the applicability of LFNST technology to the current block using MIP mode can be improved, making the selection of transform set (or transform kernel) more flexible. By introducing relevant information of MIP parameters during LFNST of the current block using MIP mode, and judging the characteristics of the current block based on this MIP information, the transform set (or transform kernel) is selected. For example, this transform method is applied to VTM7.0 and tested under All Intra conditions with a 24-frame interval. Based on the average bitrate change under the same Peak Signal to Noise Ratio (PSNR), BD-rate changes of -0.03%, 0.00%, and -0.01% can be obtained on the three image components (Y, Cb, and Cr), respectively. Especially on high-resolution sequences, better performance will be achieved. Specifically, on Class A1, a BD-rate change of -0.10% for Y can be achieved, thereby improving decoding efficiency.

[0326] This embodiment provides a transformation method, including: parsing the bitstream to determine the prediction mode parameters of the current block; when the prediction mode parameters indicate that the current block uses MIP to determine intra-frame prediction values, parsing the bitstream to determine the MIP parameters; parsing the bitstream to determine the transform coefficients and LFNST index number of the current block; when the LFNST index number indicates that the current block uses LFNST, determining the LFNST transform kernel used by the current block according to the MIP parameters; and using the LFNST transform kernel to transform the transform coefficients. Thus, for the current block using MIP mode, the introduction of MIP parameters during LFNST transformation makes the selection of the LFNST transform kernel more flexible, thereby not only improving the applicability of LFNST technology to non-traditional intra-frame prediction modes, but also improving decoding efficiency and video image quality.

[0327] Based on the same inventive concept as the foregoing embodiments, see [link to previous document]. Figure 9 This illustrates a schematic diagram of the composition structure of an encoder 90 provided in an embodiment of this application. Figure 9 As shown, the encoder 90 may include: a first determining unit 901, a first calculating unit 902, and a first transforming unit 903; wherein,

[0328] The first determining unit 901 is configured to determine the prediction mode parameters of the current block;

[0329] The first determining unit 901 is further configured to determine the MIP parameter when the prediction mode parameter indicates that the current block uses matrix-based intra-prediction (MIP) to determine the intra-prediction value.

[0330] The first calculation unit 902 is configured to determine the intra-frame prediction value of the current block based on the MIP parameters, and calculate the prediction difference between the current block and the intra-frame prediction value.

[0331] The first determining unit 901 is further configured to, when the current block uses the low-frequency inseparable secondary transform LFNST, determine the LFNST transform core used by the current block according to the MIP parameters, set the LFNST index number and write it into the video stream;

[0332] The first transformation unit 903 is configured to use the LFNST transformation kernel to perform transformation processing on the prediction difference.

[0333] In the above scheme, the MIP parameters include the MIP transpose indicator parameter, wherein the value of the MIP transpose indicator parameter is used to indicate whether the sampling point input vector used in the MIP mode is transposed.

[0334] In the above scheme, see [reference] Figure 9 The encoder 90 may further include a first selection unit 904, a first transpose unit 905, and a setting unit 906; wherein,

[0335] The first selection unit 904 is configured to select the transform core used by the current block from the LFNST transform core candidate set;

[0336] The first transpose unit 905 is configured to perform matrix transpose processing on the selected transform kernel when the value of the MIP transpose indication parameter indicates that the sampling point input vector used in the MIP mode is transposed, so as to obtain the LFNST transform kernel used by the current block.

[0337] Setting unit 906 is configured to set the value of the LFNST index number to indicate that the current block uses LFNST and that the LFNST transform core is the index number in the LFNST transform core candidate set; wherein the LFNST transform core candidate set includes two or more preset transform cores for MIP.

[0338] In the above scheme, the MIP parameter includes the MIP mode index number, which is used to indicate the MIP mode used by the current block. The MIP mode is used to indicate the calculation derivation method for determining the intra-prediction value of the current block using MIP.

[0339] In the above scheme, the first determining unit 901 is further configured to determine the value of the LFNST intra-frame prediction mode index number based on the value of the MIP mode index number.

[0340] The first selection unit 904 is further configured to select an LFNST transform kernel candidate set from multiple LFNST transform kernel candidate sets according to the value of the LFNST intra-frame prediction mode index number; and to select the transform kernel indicated by the LFNST index number from the selected LFNST transform kernel candidate set and set it as the LFNST transform kernel used by the current block.

[0341] Setting unit 906 is configured to set the value of the LFNST index number to indicate that the current block uses LFNST and that the LFNST transform core is the index number in the LFNST transform core candidate set; wherein the LFNST transform core candidate set includes two or more preset LFNST transform cores.

[0342] In the above scheme, the first determining unit 901 is further configured to use a first lookup table to determine the value of the LFNST intra-frame prediction mode index number corresponding to the value of the MIP mode index number, wherein the first lookup table contains at least one or more different MIP mode index numbers corresponding to each of two different values ​​of the LFNST intra-frame prediction mode index number.

[0343] In the above scheme, the first selection unit 904 is further configured to select an LFNST transform core candidate set from multiple LFNST transform core candidate sets according to the value of the MIP mode index number; and select the transform core indicated by the LFNST index number from the selected LFNST transform core candidate set and set it as the LFNST transform core used by the current block.

[0344] Setting unit 906 is configured to set the value of the LFNST index number to indicate that the current block uses LFNST and that the LFNST transform core is the index number in the LFNST transform core candidate set; wherein the LFNST transform core candidate set includes two or more preset LFNST transform cores.

[0345] In the above scheme, the first selection unit 904 is further configured to use a second lookup table to determine the value of the LFNST transform kernel candidate set index number corresponding to the value of the MIP mode index number, and select the LFNST transform kernel candidate set indicated by the value of the LFNST transform kernel candidate set index number as the selected LFNST transform kernel candidate set; wherein, the second lookup table contains at least one or more different MIP mode index numbers corresponding to each of the two different values ​​of the LFNST transform kernel candidate set index numbers.

[0346] In the above scheme, the first calculation unit 902 is further configured to determine the index number of the LFNST transform kernel candidate set using the first calculation method according to the value of the MIP mode index number.

[0347] The first selection unit 904 is further configured to select the LFNST transform kernel candidate set indicated by the value of the LFNST transform kernel candidate set index number as the selected LFNST transform kernel candidate set.

[0348] In the above scheme, the first calculation unit 902 is specifically configured to set the value of the LFNST transform kernel candidate set index number to be equal to the value of the MIP mode index number when the value of the MIP mode index number is less than a first preset value; and to set the value of the LFNST transform kernel candidate set index number to be equal to the difference obtained by subtracting a second preset value from the value of the MIP mode index number when the value of the MIP mode index number is greater than or equal to the first preset value; wherein the first preset value and the second preset value are both integer values.

[0349] In the above scheme, the first preset value is equal to 3.

[0350] In the above scheme, the second preset value is equal to 2.

[0351] In the above scheme, the MIP parameters also include MIP transpose indicator parameters, wherein the value of the MIP transpose indicator parameter is used to indicate whether the sampling point input vector used in the MIP mode is transposed.

[0352] The first transpose unit 905 is further configured to perform matrix transpose processing on the LFNST transform kernel used by the current block when the value of the MIP transpose indication parameter indicates that the sampling point input vector used in the MIP mode is transposed, and set the transpose-processed transform kernel as the LFNST transform kernel used by the current block.

[0353] Understandably, in the embodiments of this application, a "unit" can be a portion of a circuit, a portion of a processor, a portion of a program or software, etc., and can also be a module or a non-modular one. Furthermore, the components in this embodiment can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit described above can be implemented in hardware or as a software functional module.

[0354] If the integrated unit is implemented as a software functional module and not sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this embodiment, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the method described in this embodiment. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0355] Therefore, this application provides a computer storage medium applied to an encoder 90, the computer storage medium storing a transformation program, which, when executed by a first processor, implements the method described in any of the foregoing embodiments.

[0356] Based on the composition of the encoder 90 and the computer storage medium described above, see [link to documentation]. Figure 10 This illustrates a specific hardware structure example of the encoder 90 provided in this application embodiment, which may include: a first communication interface 1001, a first memory 1002, and a first processor 1003; the various components are coupled together through a first bus system 1004. It is understood that the first bus system 1004 is used to implement the connection and communication between these components. In addition to a data bus, the first bus system 1004 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in... Figure 10 The general designated all buses as the first bus system 1004. Among them,

[0357] The first communication interface 1001 is used for receiving and sending signals during the process of sending and receiving information with other external network elements;

[0358] The first memory 1002 is used to store computer programs that can run on the first processor 1003;

[0359] The first processor 1003 is configured to, when running the computer program, execute:

[0360] Determine the prediction mode parameters for the current block;

[0361] When the prediction mode parameters indicate that the current block uses matrix-based intra-prediction (MIP) to determine intra-prediction values, the MIP parameters are determined;

[0362] Based on the MIP parameters, determine the intra-prediction value of the current block, and calculate the prediction difference between the current block and the intra-prediction value;

[0363] When the current block uses the Low Frequency Inseparable Secondary Transform (LFNST), the LFNST transform core used by the current block is determined according to the MIP parameters, the LFNST index number is set and written into the video stream;

[0364] The predicted difference is transformed using the LFNST transform kernel.

[0365] It is understood that the first memory 1002 in the embodiments of this application can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate Synchronous DRAM (DDRSDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchlink DRAM (SLDRAM), and Direct Rambus RAM (DRRAM). The first memory 1002 of the system and method described in this application is intended to include, but is not limited to, these and any other suitable types of memory.

[0366] The first processor 1003 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of the first processor 1003 or by instructions in software form. The first processor 1003 may be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules may reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in the first memory 1002. The first processor 1003 reads the information in the first memory 1002 and completes the steps of the above method in conjunction with its hardware.

[0367] It is understood that the embodiments described in this application can be implemented using hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing unit can be implemented in one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), DSP devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions described in this application, or combinations thereof. For software implementation, the technology described in this application can be implemented through modules (e.g., procedures, functions, etc.) that perform the functions described in this application. Software code can be stored in memory and executed by a processor. The memory can be implemented in the processor or external to the processor.

[0368] Alternatively, as another embodiment, the first processor 1003 is further configured to execute the method described in any of the foregoing embodiments when running the computer program.

[0369] This embodiment provides an encoder, which may include a first determining unit, a first calculating unit, and a first transforming unit. Thus, for the current block using the MIP mode, the introduction of MIP parameters during the LFNST transform makes the selection of the LFNST transform kernel more flexible. This not only improves the applicability of LFNST technology to non-traditional intra-prediction modes, but also improves coding efficiency and enhances video image quality.

[0370] Based on the same inventive concept as the foregoing embodiments, see [link to previous document]. Figure 11 This illustrates a schematic diagram of the composition structure of a decoder 110 provided in an embodiment of this application. Figure 11 As shown, the decoder 110 may include: a parsing unit 1101, a second determining unit 1102, and a second transforming unit 1103; wherein,

[0371] The parsing unit 1101 is configured to parse the bitstream and determine the prediction mode parameters of the current block; and is further configured to parse the bitstream and determine the MIP parameters when the prediction mode parameters indicate that the current block uses MIP to determine the intra-frame prediction value; and is further configured to parse the bitstream and determine the transform coefficients and LFNST index number of the current block.

[0372] The second determining unit 1102 is configured to determine the LFNST transform kernel used by the current block according to the MIP parameters when the LFNST index number indicates that the current block uses LFNST.

[0373] The second transformation unit 1103 is configured to use the LFNST transformation kernel to perform transformation processing on the transformation coefficients.

[0374] In the above scheme, the MIP parameters include the MIP transpose indicator parameter, wherein the value of the MIP transpose indicator parameter is used to indicate whether the sampling point input vector used in the MIP mode is transposed.

[0375] In the above scheme, see [reference] Figure 11 The decoder 110 may further include a second selection unit 1104 and a second transpose unit 1105; wherein,

[0376] The second selection unit 1104 is configured to select the transform core indicated by the LFNST index number from the LFNST transform core candidate set;

[0377] The second transpose unit 1105 is configured to perform matrix transpose processing on the selected transform kernel when the value of the MIP transpose indication parameter indicates that the sampling point input vector used in the MIP mode is transposed, so as to obtain the LFNST transform kernel used by the current block; wherein the LFNST transform kernel candidate set includes two or more preset transform kernels for MIP.

[0378] In the above scheme, the MIP parameter includes the MIP mode index number, which is used to indicate the MIP mode used by the current block. The MIP mode is used to indicate the calculation derivation method for determining the intra-prediction value of the current block using MIP.

[0379] In the above scheme, the second determining unit 1102 is further configured to determine the value of the LFNST intra-frame prediction mode index number based on the value of the MIP mode index number.

[0380] The second selection unit 1104 is further configured to select an LFNST transform core candidate set from multiple LFNST transform core candidate sets according to the value of the LFNST intra-prediction mode index number; and to select the transform core indicated by the LFNST index number from the selected LFNST transform core candidate set and set it as the LFNST transform core used by the current block; wherein the LFNST transform core candidate set includes two or more preset LFNST transform cores.

[0381] In the above scheme, the second determining unit 1102 is further configured to use a first lookup table to determine the value of the LFNST intra-frame prediction mode index corresponding to the value of the MIP mode index, wherein the first lookup table contains at least one or more different MIP mode indexes corresponding to each of two different values ​​of the LFNST intra-frame prediction mode index.

[0382] In the above scheme, the second selection unit 1104 is further configured to select an LFNST transform core candidate set from multiple LFNST transform core candidate sets according to the value of the MIP mode index number; and select the transform core indicated by the LFNST index number from the selected LFNST transform core candidate set and set it as the LFNST transform core used by the current block; wherein the LFNST transform core candidate set includes two or more preset LFNST transform cores.

[0383] In the above scheme, the second selection unit 1104 is further configured to use a second lookup table to determine the value of the LFNST transform kernel candidate set index number corresponding to the value of the MIP mode index number, and select the LFNST transform kernel candidate set indicated by the value of the LFNST transform kernel candidate set index number as the selected LFNST transform kernel candidate set; wherein, the second lookup table contains at least one or more different MIP mode index numbers corresponding to each of the two different values ​​of the LFNST transform kernel candidate set index numbers.

[0384] In the above scheme, see [reference] Figure 11 The decoder 110 may also include a second computing unit 1106, configured to determine the index number of the LFNST transform kernel candidate set using a first calculation method based on the value of the MIP mode index number.

[0385] The second selection unit 1104 is further configured to select the LFNST transform kernel candidate set indicated by the value of the LFNST transform kernel candidate set index number as the selected LFNST transform kernel candidate set.

[0386] In the above scheme, the second calculation unit 1106 is specifically configured to set the value of the LFNST transform kernel candidate set index number to be equal to the value of the MIP mode index number when the value of the MIP mode index number is less than the first preset value; and to set the value of the LFNST transform kernel candidate set index number to be equal to the difference obtained by subtracting the second preset value from the value of the MIP mode index number when the value of the MIP mode index number is greater than or equal to the first preset value; wherein the first preset value and the second preset value are both integer values.

[0387] In the above scheme, the first preset value is equal to 3.

[0388] In the above scheme, the second preset value is equal to 2.

[0389] In the above scheme, the MIP parameters also include MIP transpose indicator parameters, wherein the value of the MIP transpose indicator parameter indicates whether the sampling point input vector used in the MIP mode is transposed.

[0390] The second transpose unit 1105 is further configured to perform matrix transpose processing on the LFNST transform kernel used by the current block when the value of the MIP transpose indication parameter indicates that the sampling point input vector used in the MIP mode is transposed, and set the transpose kernel obtained after transpose processing as the LFNST transform kernel used by the current block.

[0391] Understandably, in this embodiment, a "unit" can be a portion of a circuit, a portion of a processor, a portion of a program or software, etc., and can also be a module or a non-modular component. Furthermore, the components in this embodiment can be integrated into a single processing unit, or each unit can exist physically separately, or two or more units can be integrated into a single unit. The integrated unit can be implemented in hardware or as a software functional module. If the integrated unit is implemented as a software functional module and is not sold or used as an independent product, it can be stored in a computer-readable storage medium.

[0392] Therefore, this embodiment provides a computer storage medium applied to a decoder 110, the computer storage medium storing a transformation program, which, when executed by a second processor, implements the method described in any of the foregoing embodiments.

[0393] Based on the composition of the decoder 110 and the computer storage medium described above, see [link to documentation]. Figure 12 This illustrates a specific hardware structure example of the decoder 110 provided in this application embodiment, which may include: a second communication interface 1201, a second memory 1202, and a second processor 1203; the various components are coupled together through a second bus system 1204. It is understood that the second bus system 1204 is used to implement the connection and communication between these components. In addition to a data bus, the second bus system 1204 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in... Figure 12 The various buses are all labeled as the second bus system 1204. Among them,

[0394] The second communication interface 1201 is used for receiving and sending signals during the process of sending and receiving information with other external network elements;

[0395] The second memory 1202 is used to store computer programs that can run on the second processor 1203;

[0396] The second processor 1203 is configured to, when running the computer program, perform:

[0397] Analyze the bitstream to determine the prediction mode parameters for the current block;

[0398] When the prediction mode parameter indicates that the current block uses MIP to determine the intra-frame prediction value, the bitstream is parsed and the MIP parameter is determined.

[0399] Analyze the bitstream to determine the transform coefficients and LFNST index number of the current block;

[0400] When the LFNST index number indicates that the current block uses LFNST, the LFNST transform kernel used by the current block is determined according to the MIP parameters;

[0401] The transformation coefficients are transformed using the LFNST transform kernel.

[0402] Alternatively, as another embodiment, the second processor 1203 is also configured to perform the method described in any of the foregoing embodiments when running the computer program.

[0403] It is understood that the second memory 1202 has similar hardware functions to the first memory 1002, and the second processor 1203 has similar hardware functions to the first processor 1003; these will not be described in detail here.

[0404] This embodiment provides a decoder, which may include a parsing unit, a second determining unit, and a second transforming unit. Thus, for the current block using the MIP mode, the introduction of MIP parameters during the LFNST transform makes the selection of the LFNST transform kernel more flexible. This not only improves the applicability of LFNST technology to non-traditional intra-prediction modes but also improves decoding efficiency and enhances video image quality.

[0405] It should be noted that, in this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0406] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0407] The methods disclosed in the several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0408] The features disclosed in the several product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0409] The features disclosed in the several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method or device embodiments.

[0410] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0411] Industrial applicability

[0412] In this embodiment, the prediction mode parameters of the current block are first determined. When the prediction mode parameters indicate that the current block uses MIP to determine intra-frame prediction values, the MIP parameters are determined. Then, based on the MIP parameters, the intra-frame prediction values ​​of the current block are determined, and the prediction difference between the current block and the intra-frame prediction values ​​is calculated. When the current block uses LFNST, the LFNST transform kernel used by the current block is determined based on the MIP parameters, the LFNST index number is set, and written into the video bitstream. Finally, the LFNST transform kernel is used to transform the prediction difference. Thus, for the current block using MIP mode, the introduction of MIP parameters during LFNST transformation makes the selection of the LFNST transform kernel more flexible. This not only improves the applicability of LFNST technology to non-traditional intra-frame prediction modes but also improves encoding and decoding efficiency and enhances video image quality.

Claims

1. A transformation method applied to a decoder, characterized in that, The method includes: Analyze the bitstream to determine the prediction mode parameters for the current block; When the prediction mode parameter instructs the current block to use matrix-based intra-prediction (MIP) to determine the intra-prediction value, the bitstream is parsed and the MIP parameter is determined. Analyze the bitstream to determine the transform coefficients and the low-frequency inseparable quadratic transform (LFNST) index number of the current block; When the LFNST index number indicates that the current block uses LFNST, the LFNST transform kernel used by the current block is determined according to the MIP parameter; The transformation coefficients are transformed using the LFNST transform kernel. The MIP parameters include a MIP mode index number, which indicates the MIP mode used by the current block. The MIP mode indicates the calculation and derivation method for determining the intra-prediction value of the current block using MIP. When the LFNST index number indicates that the current block uses LFNST, determining the LFNST transform kernel used by the current block based on the MIP parameters includes: Based on the value of the MIP mode index number, determine the value of the LFNST intra-frame prediction mode index number. Based on the value of the LFNST intra-frame prediction mode index number, select one LFNST transform kernel candidate set from multiple LFNST transform kernel candidate sets; From the selected LFNST transform core candidate set, select the transform core indicated by the LFNST index number and set it as the LFNST transform core used by the current block.

2. The method according to claim 1, characterized in that, The LFNST transform kernel candidate set contains a plurality of preset LFNST transform kernels.

3. The method according to claim 1 or 2, characterized in that, Determining the value of the LFNST intra-frame prediction mode index number based on the value of the MIP mode index number includes: A lookup table is used to determine the value of the LFNST intra-prediction mode index corresponding to the value of the MIP mode index, wherein the lookup table contains at least one or more different MIP mode indexes corresponding to each of two different LFNST intra-prediction mode indexes.

4. The method according to claim 1, characterized in that, The method further includes: The MIP parameters also include a MIP transpose indicator parameter, wherein the value of the MIP transpose indicator parameter indicates whether the sampling point input vector used in the MIP mode is transposed. When the value of the MIP transpose indicator parameter indicates that the sampling point input vector used in the MIP mode is transposed, the LFNST transform kernel used in the current block is matrix transposed, and the transform kernel obtained after transposition is set as the LFNST transform kernel used in the current block.

5. A transformation method applied to an encoder, characterized in that, The method includes: Determine the prediction mode parameters for the current block; The MIP parameters are determined when the prediction mode parameters instruct the current block to use matrix-based intra-prediction (MIP) to determine intra-prediction values. Based on the MIP parameters, determine the intra-prediction value of the current block, and calculate the prediction difference between the current block and the intra-prediction value; When the current block uses the Low Frequency Inseparable Secondary Transform (LFNST), the LFNST transform core used by the current block is determined according to the MIP parameters, the LFNST index number is set and written into the video stream; The predicted difference is transformed using the LFNST transform kernel. The MIP parameters include a MIP mode index number, which indicates the MIP mode used by the current block. The MIP mode indicates the calculation and derivation method for determining the intra-prediction value of the current block using MIP. When the current block uses LFNST, the process of determining the LFNST transform kernel used by the current block based on the MIP parameters, setting the LFNST index number, and writing it into the video stream includes: Based on the value of the MIP mode index number, determine the value of the LFNST intra-frame prediction mode index number. Based on the value of the LFNST intra-frame prediction mode index number, select one LFNST transform kernel candidate set from multiple LFNST transform kernel candidate sets; From the selected LFNST transform core candidate set, select the transform core indicated by the LFNST index number and set it as the LFNST transform core used by the current block; The value of the LFNST index number is set to indicate that the current block uses LFNST and that the LFNST transform core is the index number in the selected LFNST transform core candidate set.

6. The method according to claim 5, characterized in that, The LFNST transform kernel candidate set contains a plurality of preset LFNST transform kernels.

7. The method according to claim 5 or 6, characterized in that, Determining the value of the LFNST intra-frame prediction mode index number based on the value of the MIP mode index number includes: A lookup table is used to determine the value of the LFNST intra-prediction mode index corresponding to the value of the MIP mode index, wherein the lookup table contains at least one or more different MIP mode indexes corresponding to each of two different LFNST intra-prediction mode indexes.

8. The method according to claim 5, characterized in that, The method further includes: The MIP parameters also include a MIP transpose indicator parameter, wherein the value of the MIP transpose indicator parameter is used to indicate whether the sampling point input vector used in the MIP mode is transposed. When the value of the MIP transpose indicator parameter indicates that the sampling point input vector used in the MIP mode is transposed, the LFNST transform kernel used in the current block is matrix transposed, and the transform kernel obtained after transposition is set as the LFNST transform kernel used in the current block.

9. An encoder, the encoder comprising a first determining unit, a first calculating unit, and a first transforming unit; wherein: The first determining unit is configured to determine the prediction mode parameters of the current block; when the prediction mode parameters indicate that the current block uses matrix-based intra-prediction (MIP) to determine intra-prediction values, the MIP parameters are determined. The first calculation unit is configured to determine the intra-frame prediction value of the current block based on the MIP parameters, and calculate the prediction difference between the current block and the intra-frame prediction value; The first determining unit is further configured to, when the current block uses the low-frequency inseparable secondary transform LFNST, determine the LFNST transform core used by the current block according to the MIP parameters, set the LFNST index number and write it into the video bitstream; The first transformation unit is configured to use the LFNST transformation kernel to perform transformation processing on the prediction difference; The MIP parameters include a MIP mode index number, which indicates the MIP mode used by the current block. The MIP mode indicates the calculation and derivation method for determining the intra-prediction value of the current block using MIP. The first determining unit is further configured to: Based on the value of the MIP mode index number, determine the value of the LFNST intra-frame prediction mode index number. Based on the value of the LFNST intra-frame prediction mode index number, select one LFNST transform kernel candidate set from multiple LFNST transform kernel candidate sets; From the selected LFNST transform core candidate set, select the transform core indicated by the LFNST index number and set it as the LFNST transform core used by the current block; The value of the LFNST index number is set to indicate that the current block uses LFNST and that the LFNST transform core is the index number in the selected LFNST transform core candidate set.

10. A decoder, the decoder comprising a parsing unit, a second determining unit, and a second transforming unit; wherein: The parsing unit is configured to parse the bitstream to determine the prediction mode parameters of the current block; and is further configured to parse the bitstream to determine the MIP parameters when the prediction mode parameters indicate that the current block uses MIP to determine the intra-frame prediction value; and is further configured to parse the bitstream to determine the transform coefficients and LFNST index number of the current block. The second determining unit is configured to determine the LFNST transform kernel used by the current block according to the MIP parameters when the LFNST index number indicates that the current block uses LFNST; The second transformation unit is configured to use the LFNST transformation kernel to perform transformation processing on the transformation coefficients; The MIP parameters include a MIP mode index number, which indicates the MIP mode used by the current block. The MIP mode indicates the calculation derivation method for determining the intra-prediction value of the current block using MIP. The second determining unit is further configured to: Based on the value of the MIP mode index number, determine the value of the LFNST intra-frame prediction mode index number. Based on the value of the LFNST intra-frame prediction mode index number, select one LFNST transform kernel candidate set from multiple LFNST transform kernel candidate sets; From the selected LFNST transform core candidate set, select the transform core indicated by the LFNST index number and set it as the LFNST transform core used by the current block.

11. A method for transmitting a code stream, characterized in that, The transformation method described in any one of claims 5-8 is performed to generate a bitstream; and the bitstream is transmitted.

Images