Encoding method, decoding method, encoder, decoder, and storage medium

Abstract

The embodiment of the application discloses a kind of encoding method, decoding method, encoder, decoder and storage medium, the method comprises: determining the prediction parameter of current block, the prediction parameter includes prediction mode parameter;When prediction mode parameter indicates that the intra prediction value of current block is determined using matrix-based intra prediction MIP mode for the chroma component of current block, the adjacent sample value of current block is obtained, and the MIP input sample value of current block is determined according to the adjacent sample value of current block;MIP prediction value of the chroma component of current block is determined according to MIP input sample value, MIP weighting matrix and shift parameter;The MIP prediction value is filtered, and the intra prediction value of the chroma component of current block is determined;According to the intra prediction value of the chroma component of current block, the prediction residual value of the chroma component of current block is determined;LFNST parameter is determined by performing low frequency non-separable secondary transform LFNST to the prediction residual value;The LFNST parameter is encoded, and write in code stream.

Description

[0001] Cross-references to related applications

[0002] This application claims priority to U.S. Provisional Patent Application No. 62 / 958,582, filed on January 8, 2020, entitled "VIDEO ENCODING AND DECODING METHOD, APPARATUS AND COMMUNICATION SYSTEM," filed in the names of Junyan Huo, Shuai Wan, and Yanzhuo Ma, the entire contents of which are incorporated herein by reference. Technical Field

[0003] This application relates to the field of video encoding and decoding technology, and in particular to an encoding method, a decoding method, an encoder, a decoder, and a storage medium. Background Technology

[0004] As people's demands for video display quality increase, new video application forms such as high-definition and ultra-high-definition video have emerged. H.265 / High Efficiency Video Coding (HEVC) can no longer meet the needs of the rapidly developing video applications. The Joint Video Exploration Team (JVET) proposed the next-generation video coding standard H.266 / Versatile Video Coding (VVC).

[0005] In H.266 / VVC, Matrix-based Intra Prediction (MIP) is an intra prediction mode used to obtain the intra prediction block for the current block; then, Low-Frequency Non-Separable Transform (LFNST) is used to determine the prediction residual for the current block. However, when applied to MIP mode prediction, LFNST negatively impacts the encoding and decoding efficiency of VVC. Summary of the Invention

[0006] This application provides an encoding method, a decoding method, an encoder, a decoder, and a storage medium, which can improve encoding and decoding efficiency.

[0007] The technical solution of this application embodiment can be implemented as follows:

[0008] In a first aspect, embodiments of this application provide an encoding method applied to an encoder, the method comprising:

[0009] Determine the prediction parameters for the current block, wherein the prediction parameters include prediction mode parameters;

[0010] When the prediction mode parameter indicates that the matrix-based intra-prediction (MIP) mode is used to determine the intra-prediction value for the chrominance component of the current block, the adjacent sample values ​​of the current block are obtained, and the MIP input sample value of the current block is determined based on the adjacent sample values ​​of the current block.

[0011] Based on the MIP input sample value, the MIP weighting matrix, and the shift parameter, the MIP prediction value of the chromaticity component of the current block is determined; wherein, the MIP prediction value is the prediction value of a portion of the sample points in the chromaticity component of the current block;

[0012] The MIP prediction values ​​are filtered to determine the intra-frame prediction values ​​of the chroma components of the current block.

[0013] Based on the intra-frame prediction values ​​of the chrominance components of the current block, determine the prediction residual values ​​of the chrominance components of the current block;

[0014] Perform a low-frequency inseparable quadratic transform (LFNST) on the predicted residual values ​​to determine the LFNST parameters;

[0015] The LFNST parameters are encoded and written into the bitstream.

[0016] Secondly, embodiments of this application provide a decoding method applied to a decoder, the method comprising:

[0017] Parse the bitstream to obtain the prediction parameters and LFNST parameters of the current block, wherein the prediction parameters include prediction mode parameters;

[0018] When the prediction mode parameter indicates that the matrix-based intra-prediction (MIP) mode is used to determine the intra-prediction value for the chrominance component of the current block, the adjacent sample values ​​of the current block are obtained, and the MIP input sample value of the current block is determined based on the adjacent sample values ​​of the current block.

[0019] Based on the MIP input sample value, the MIP weighting matrix, and the shift parameter, the MIP prediction value of the chromaticity component of the current block is determined; wherein, the MIP prediction value is the prediction value of a portion of the sample points in the chromaticity component of the current block;

[0020] The MIP prediction values ​​are filtered to determine the intra-frame prediction values ​​of the chroma components of the current block.

[0021] When the LFNST parameter indicates that LFNST is performed on the current block, the reconstructed transform coefficient block of the current block is determined, and LFNST is performed on at least a portion of the reconstructed transform coefficients in the reconstructed transform coefficient block to obtain the second transform coefficient block;

[0022] The second transform coefficient block is transformed for the first time to obtain the reconstructed residual block of the chromaticity components of the current block;

[0023] The reconstruction block of the chroma component of the current block is determined based on the intra-frame prediction value of the chroma component of the current block and the reconstruction residual block.

[0024] Thirdly, embodiments of this application provide an encoder, which includes a first determining unit, a first predicting unit, a first transforming unit, and an encoding unit; wherein,

[0025] The first determining unit is configured to determine the prediction parameters of the current block, wherein the prediction parameters include prediction mode parameters;

[0026] The first determining unit is further configured to, when the prediction mode parameter indicates that the intra-prediction value is determined by using a matrix-based intra-prediction (MIP) mode for the chroma components of the current block, acquire the adjacent sample values ​​of the current block and determine the MIP input sample value of the current block based on the adjacent sample values ​​of the current block;

[0027] The first prediction unit is configured to determine the MIP prediction value of the chroma component of the current block based on the MIP input sample value, the MIP weighting matrix, and the shift parameter; wherein the MIP prediction value is the prediction value of a portion of the sample points in the chroma component of the current block; and to filter the MIP prediction value to determine the intra-frame prediction value of the chroma component of the current block.

[0028] The first transformation unit is configured to determine the prediction residual value of the chrominance component of the current block based on the intra-frame prediction value of the chrominance component of the current block; and to perform a low-frequency non-separable quadratic transform (LFNST) on the prediction residual value to determine the LFNST parameters.

[0029] The encoding unit is configured to encode the LFNST parameters and write them into the bitstream.

[0030] Fourthly, embodiments of this application provide an encoder, which includes a first memory and a first processor; wherein,

[0031] The first memory is used to store computer programs that can run on the first processor;

[0032] The first processor is configured to execute the method as described in the first aspect when running the computer program.

[0033] Fifthly, embodiments of this application provide a decoder, which includes a parsing unit, a second determining unit, a second predicting unit, and a second transforming unit; wherein,

[0034] The parsing unit is configured to parse the bitstream and obtain the prediction parameters and LFNST parameters of the current block, wherein the prediction parameters include prediction mode parameters;

[0035] The second determining unit is configured to, when the prediction mode parameter indicates that the intra-prediction value is determined by using a matrix-based intra-prediction (MIP) mode for the chroma components of the current block, acquire the adjacent sample values ​​of the current block and determine the MIP input sample value of the current block based on the adjacent sample values ​​of the current block;

[0036] The second prediction unit is configured to determine the MIP prediction value of the chroma component of the current block based on the MIP input sample value, the MIP weighting matrix, and the shift parameter; wherein the MIP prediction value is the prediction value of a portion of the sample points in the chroma component of the current block; and to determine the intra-frame prediction value of the chroma component of the current block by filtering the MIP prediction value.

[0037] The second transformation unit is configured to, when the LFNST parameter indicates that LFNST is performed on the current block, determine the reconstruction transformation coefficient block of the current block, perform LFNST on at least a portion of the reconstruction transformation coefficients in the reconstruction transformation coefficient block to obtain a second transformation coefficient block; and perform a first transformation on the second transformation coefficient block to obtain the reconstruction residual block of the chromaticity components of the current block.

[0038] The second determining unit is further configured to determine the reconstructed block of the chroma component of the current block based on the intra-frame prediction value of the chroma component of the current block and the reconstructed residual block.

[0039] Sixthly, embodiments of this application provide a decoder, which includes a second memory and a second processor; wherein,

[0040] The second memory is used to store computer programs that can run on the second processor;

[0041] The second processor is configured to execute the method described in the second aspect when running the computer program.

[0042] In a seventh aspect, embodiments of this application provide a computer storage medium storing a computer program that, when executed by a first processor, implements the method described in the first aspect, or when executed by a second processor, implements the method described in the second aspect.

[0043] This application provides an encoding method, a decoding method, an encoder, a decoder, and a storage medium. On the encoder side, after determining the prediction parameters of the current block, when the prediction mode parameters indicate that the matrix-based intra-frame prediction (MIP) mode is used to determine the intra-frame prediction value for the chrominance component of the current block, the adjacent sample values ​​of the current block are acquired, and the MIP input sample value of the current block is determined based on the adjacent sample values. The MIP prediction value of the chrominance component of the current block is determined based on the MIP input sample value, the MIP weighting matrix, and the shift parameter. The MIP prediction value is filtered to determine the intra-frame prediction value of the chrominance component of the current block. The prediction residual value of the chrominance component of the current block is determined based on the intra-frame prediction value of the chrominance component. The low-frequency non-separable quadratic transform (LFNST) is performed on the prediction residual value to determine the LFNST parameters. The LFNST parameters are encoded and written into the bitstream. On the decoder side, after parsing the bitstream and obtaining the prediction parameters and LFNST parameters of the current block, when the prediction mode parameter indicates that the MIP mode is used to determine the intra-frame prediction value of the chroma component of the current block, the adjacent sample values ​​of the current block are obtained, and the MIP input sample value of the current block is determined based on the adjacent sample values ​​of the current block; the MIP prediction value of the chroma component of the current block is determined based on the MIP input sample value, the MIP weighting matrix, and the shift parameter; the MIP prediction value is filtered to determine the intra-frame prediction value of the chroma component of the current block; when the LFNST parameter indicates that LFNST is performed on the current block, the reconstruction transform coefficient block of the current block is determined, and LFNST is performed on at least some of the reconstruction transform coefficients in the reconstruction transform coefficient block to obtain the second transform coefficient block; the second transform coefficient block is subjected to a first transform to obtain the reconstruction residual block of the chroma component of the current block; the reconstruction block of the chroma component of the current block is determined based on the intra-frame prediction value of the chroma component of the current block and the reconstruction residual block. In this way, for MIP mode, the complexity can be reduced while ensuring encoding and decoding performance, and the storage space required during the decoding process can be reduced, effectively improving encoding and decoding efficiency. In addition, when LFNST technology is applied to MIP mode prediction, the introduction of MIP parameters makes the LFNST transform more flexible, further improving encoding and decoding efficiency. Attached Figure Description

[0044] Figure 1 A schematic diagram of the composition structure of an encoder provided in an embodiment of this application;

[0045] Figure 2A schematic diagram of the composition structure of a decoder provided in an embodiment of this application;

[0046] Figure 3 A flowchart illustrating an encoding method provided in an embodiment of this application;

[0047] Figure 4 A flowchart illustrating another encoding method provided in an embodiment of this application;

[0048] Figure 5 A flowchart illustrating an MPI prediction process provided in an embodiment of this application;

[0049] Figure 6 This is a schematic diagram of a process for obtaining intra-frame prediction blocks using MIP mode in an embodiment of this application;

[0050] Figure 7 A schematic diagram of a process framework for executing LFNST is provided for an embodiment of this application;

[0051] Figure 8 A schematic diagram illustrating the process of encoding LFNST parameters provided in this application embodiment;

[0052] Figure 9 A flowchart illustrating a decoding method provided in an embodiment of this application;

[0053] Figure 10 A schematic diagram illustrating the process of parsing LFNST parameters provided in this application embodiment;

[0054] Figure 11 A schematic diagram of another process framework for executing LFNST provided for an embodiment of this application;

[0055] Figure 12 A schematic diagram of the composition structure of another encoder provided in an embodiment of this application;

[0056] Figure 13 This is a schematic diagram of the specific hardware structure of an encoder provided in an embodiment of this application;

[0057] Figure 14 A schematic diagram illustrating the composition structure of another decoder provided in an embodiment of this application;

[0058] Figure 15 This is a schematic diagram of the specific hardware structure of a decoder provided in an embodiment of this application;

[0059] Figure 16 This is a schematic diagram of the composition structure of a transmitting device provided in an embodiment of this application;

[0060] Figure 17This is a schematic diagram of the composition structure of a target device provided in an embodiment of this application;

[0061] Figure 18 This is a schematic diagram of the composition structure of a communication system provided in an embodiment of this application. Detailed Implementation

[0062] In order to gain a more detailed understanding of the features and technical content of the embodiments of this application, the implementation of the embodiments of this application will be described in detail below with reference to the accompanying drawings. The accompanying drawings are for reference and illustration only and are not intended to limit the embodiments of this application.

[0063] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0064] In the following description, references to "some embodiments" refer to a subset of all possible embodiments. It is understood that "some embodiments" may be the same or different subsets of all possible embodiments and may be combined with each other without conflict. It should also be noted that the terms "first," "second," and "third" used in the embodiments of this application are merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first," "second," and "third" may be interchanged in a specific order or sequence where permissible, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0065] With the development of video compression technology, the International Telecommunication Union-Telecommunication (ITU-T) and the International Organization for Standardization (ISO) / International Electrotechnical Commission (IEC) launched a standardization project called VVC to develop a new generation of video coding standards. The aim of VVC is to improve performance by approximately 50% compared to the H.265 / HEVC standard when encoding high-quality video with one or more of the following characteristics: high resolution, high frame rate, high bit depth, high dynamic range, wide color gamut, and omnidirectional viewpoint. The standardization project is led by the Joint Video Exploration Team (JVET) of ITU-T SG 16WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11. Various intra-frame prediction modes and inter-frame prediction modes have been verified to achieve high compression efficiency when encoding high-quality video and have therefore been adopted in the VVC Working Draft (WD).

[0066] Here, Matrix-based Intra Prediction (MIP) mode is an intra prediction mode. In VVC, the encoder or decoder can invoke the MIP mode to obtain the intra prediction block for the current block. Then, the Low Frequency Non-Separable Transform (LFNST) as a secondary transform (or second transform) can be applied to the prediction residual of the current block.

[0067] In related technologies, although LFNST can be applied to MIP mode prediction with a fixed configuration of LFNST transform kernel candidate set and LFNST transpose indicator parameters, it has a negative impact on the encoding and decoding efficiency of VVC.

[0068] Based on this, one embodiment of this application provides an encoding method. The basic idea of ​​this encoding method is as follows: determine the prediction parameters of the current block, wherein the prediction parameters include prediction mode parameters; when the prediction mode parameters indicate that the matrix-based intra-prediction (MIP) mode is used to determine the intra-prediction value of the chrominance component of the current block, obtain the adjacent sample values ​​of the current block, and determine the MIP input sample value of the current block based on the adjacent sample values ​​of the current block; determine the MIP prediction value of the chrominance component of the current block based on the MIP input sample value, the MIP weighting matrix, and the shift parameter; filter the MIP prediction value to determine the intra-prediction value of the chrominance component of the current block; determine the prediction residual value of the chrominance component of the current block based on the intra-prediction value of the chrominance component of the current block; perform a low-frequency non-separable quadratic transform (LFNST) on the prediction residual value to determine the LFNST parameters; encode the LFNST parameters and write them into the bitstream. Another embodiment of this application provides a decoding method. The basic idea of ​​this decoding method is as follows: parse the bitstream to obtain the prediction parameters and LFNST parameters of the current block, wherein the prediction parameters include prediction mode parameters; when the prediction mode parameters indicate that matrix-based intra-prediction (MIP) mode is used to determine the intra-prediction value for the chrominance components of the current block, obtain the adjacent sample values ​​of the current block, and determine the MIP input sample value of the current block based on the adjacent sample values; determine the chrominance components of the current block based on the MIP input sample value, the MIP weighting matrix, and the shift parameters. The process involves: obtaining the MIP prediction value; filtering the MIP prediction value to determine the intra-frame prediction value of the chroma component of the current block; when the LFNST parameter indicates that LFNST should be performed on the current block, determining the reconstruction transform coefficient block of the current block; performing LFNST on at least some of the reconstruction transform coefficients in the reconstruction transform coefficient block to obtain a second transform coefficient block; performing a first transform on the second transform coefficient block to obtain the reconstruction residual block of the chroma component of the current block; and determining the reconstruction block of the chroma component of the current block based on the intra-frame prediction value of the chroma component of the current block and the reconstruction residual block. In this way, for MIP mode, complexity can be reduced while ensuring encoding and decoding performance, and the storage space required during decoding can be reduced, effectively improving encoding and decoding efficiency. Furthermore, when applied to MIP mode prediction, the introduction of the MIP parameter makes the LFNST transform more flexible, further improving encoding and decoding efficiency.

[0069] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0070] See Figure 1 This illustrates a system composition block diagram example of an encoder provided in an embodiment of this application. Figure 1As shown, the encoder 100 may include: a segmentation unit 101, a prediction unit 102, a first adder 107, a transform unit 108, a quantization unit 109, an inverse quantization unit 110, an inverse transform unit 111, a second adder 112, a filtering unit 113, a Decoded Picture Buffer (DPB) unit 114, and an entropy coding unit 115. Here, the input of the encoder 100 can be a video consisting of a series of images or a single still image, and the output of the encoder 100 can be a bitstream (also called a "bitstream") representing a compressed version of the input video.

[0071] The segmentation unit 101 divides the images in the input video into one or more coding tree units (CTUs). The segmentation unit 101 divides the images into multiple tiles, and can further divide a tile into one or more bricks. Here, a tile or a brick can include one or more complete and / or partial CTUs. Additionally, the segmentation unit 101 can form one or more slices, where a slice can include one or more tiles arranged in raster order in the image, or one or more tiles covering a rectangular area of ​​the image. The segmentation unit 101 can also form one or more sub-images, where a sub-image can include one or more slices, tiles, or bricks.

[0072] During the encoding process of encoder 100, segmentation unit 101 transmits the CTU to prediction unit 102. Typically, prediction unit 102 may consist of block segmentation unit 103, motion estimation (ME) unit 104, motion compensation (MC) unit 105, and intra-prediction unit 106. Specifically, block segmentation unit 103 iteratively uses quadtree segmentation, binary tree segmentation, and ternary tree segmentation to further divide the input CTU into smaller coding units (CUs). Prediction unit 102 can use ME unit 104 and MC unit 105 to obtain inter-frame prediction blocks of the CUs. Intra-prediction unit 106 can use various intra-prediction modes, including MIP modes, to obtain intra-frame prediction blocks of the CUs. In the example, rate-distortion optimized motion estimation can be invoked by ME unit 104 and MC unit 105 to obtain inter-frame prediction blocks, and rate-distortion optimized mode determination can be invoked by intra-prediction unit 106 to obtain intra-frame prediction blocks.

[0073] Prediction unit 102 outputs the predicted block of the CU. First adder 107 calculates the difference between the CU in the output of segmentation unit 101 and the predicted block of the CU, i.e., the residual CU. Transform unit 108 reads the residual CU and performs one or more transform operations on the residual CU to obtain coefficients. Quantization unit 109 quantizes the coefficients and outputs quantization coefficients (i.e., levels). Inverse quantization unit 110 performs a scaling operation on the quantization coefficients to output reconstructed coefficients. Inverse transform unit 111 performs one or more inverse transforms corresponding to the transforms in transform unit 108 and outputs the reconstructed residual. Second adder 112 calculates the reconstructed CU by adding the reconstructed residual to the predicted block of the CU from prediction unit 102. Second adder 112 also sends its output to prediction unit 102 as an intra-frame prediction reference. After all CUs in the image or sub-image are reconstructed, filtering unit 113 performs loop filtering on the reconstructed image or sub-image. Here, the filtering unit 113 includes one or more filters, such as a deblocking filter, a Sample Adaptive Offset (SAO) filter, an Adaptive Loop Filter (ALF), a Luma Mapping with Chroma Scaling (LMCS) filter, and a neural network-based filter. Alternatively, when the filtering unit 113 determines that the CU is not used as a reference for encoding other CUs, the filtering unit 113 performs loop filtering on one or more target pixels in the CU.

[0074] The output of filtering unit 113 is a decoded image or sub-image, which is buffered in DPB unit 114. DPB unit 114 outputs the decoded image or sub-image according to timing and control information. Here, the image stored in DPB unit 114 can also be used as a reference for prediction unit 102 to perform inter-frame prediction or intra-frame prediction. Finally, entropy coding unit 115 converts the parameters (such as control parameters and supplementary information) necessary for decoding the image from encoder 100 into binary form, and writes such binary form into the bitstream according to the syntax structure of each data unit, which is the final output bitstream of encoder 100.

[0075] Furthermore, the encoder 100 may be a first memory having a first processor and a computer program for recording. When the first processor reads and runs the computer program, the encoder 100 reads the input video and generates a corresponding bitstream. Additionally, the encoder 100 may also be a computing device having one or more chips. These units, implemented as integrated circuits on the chips, have... Figure 1 Similar connection and data exchange functions to the corresponding units in the middle.

[0076] See Figure 2This illustrates a system block diagram example of a decoder provided in an embodiment of this application. Figure 2 As shown, the decoder 200 may include: a parsing unit 201, a prediction unit 202, a scaling unit 205, a transformation unit 206, an adder 207, a filtering unit 208, and a decoded image buffer unit 209. Here, the input of the decoder 200 is a bitstream representing a compressed version of a video or a still image, and the output of the decoder 200 may be a decoded video consisting of a series of images or a decoded still image.

[0077] The input bitstream to the decoder 200 can be the bitstream generated by the encoder 100. The parsing unit 201 parses the input bitstream and obtains the values ​​of the syntax elements. The parsing unit 201 converts the binary representation of the syntax elements into numerical values ​​and sends these values ​​to units in the decoder 200 to obtain one or more decoded images. The parsing unit 201 can also parse one or more syntax elements from the input bitstream to display the decoded images.

[0078] During the decoding process of decoder 200, parsing unit 201 sends the value of the syntax element and one or more variables set or determined according to the value of the syntax element for obtaining one or more decoded images to the unit in decoder 200.

[0079] Prediction unit 202 determines the prediction block of the current decoded block (e.g., CU). Here, prediction unit 202 may include motion compensation unit 203 and intra-prediction unit 204. Specifically, when an inter-frame decoding mode is indicated for decoding the current decoded block, prediction unit 202 passes relevant parameters from parsing unit 201 to motion compensation unit 203 to obtain inter-frame prediction blocks; when an intra-frame prediction mode (including MIP mode indicated by MIP mode index value) is indicated for decoding the current decoded block, prediction unit 202 passes relevant parameters from parsing unit 201 to intra-prediction unit 204 to obtain intra-frame prediction blocks.

[0080] Scaling unit 205 has the same function as dequantization unit 110 in encoder 100. Scaling unit 205 performs a scaling operation on the quantization coefficients (i.e., levels) from parsing unit 201 to obtain reconstruction coefficients.

[0081] Transformation unit 206 has the same function as inverse transformation unit 111 in encoder 100. Transformation unit 206 performs one or more transformation operations (i.e., the inverse operation of one or more transformation operations performed by inverse transformation unit 111 in encoder 100) to obtain the reconstructed residual.

[0082] Adder 207 performs an addition operation on its inputs (the predicted block from prediction unit 202 and the reconstructed residual from transform unit 206) to obtain the reconstructed block of the current decoded block. The reconstructed block is also sent to prediction unit 202 as a reference for other blocks encoded in intra-prediction mode.

[0083] After all CUs in an image or sub-image are reconstructed, filtering unit 208 performs loop filtering on the reconstructed image or sub-image. Filtering unit 208 includes one or more filters, such as deblocking filters, sampling adaptive compensation filters, adaptive loop filters, luminance mapping and chroma scaling filters, and neural network-based filters. Alternatively, when filtering unit 208 determines that a reconstructed block is not used as a reference for decoding other blocks, filtering unit 208 performs loop filtering on one or more target pixels in the reconstructed block. Here, the output of filtering unit 208 is a decoded image or sub-image, which is buffered in DPB unit 209. DPB unit 209 outputs the decoded image or sub-image based on timing and control information. The image stored in DPB unit 209 can also be used as a reference for performing inter-frame prediction or intra-frame prediction by prediction unit 202.

[0084] Furthermore, the decoder 200 can be a second memory having a second processor and a computer program for recording. When the first processor reads and runs the computer program, the decoder 200 reads the input bitstream and generates the corresponding decoded video. Additionally, the decoder 200 can also be a computing device having one or more chips. These units, implemented as integrated circuits on the chip, have... Figure 2 Similar connection and data exchange functions to the corresponding units in the middle.

[0085] It should be noted that the encoding method provided in this application embodiment mainly operates on the intra-frame prediction unit 106 and transform unit 108 of the encoder 100, and the decoding method provided in this application embodiment mainly operates on the intra-frame prediction unit 204 and transform unit 206 of the decoder 200. That is to say, the embodiments of this application can be applied to the encoder, the decoder, or even to both the encoder and the decoder simultaneously, but no limitation is made here.

[0086] It should also be noted that when the embodiments of this application are applied to the encoder 100, the "current block" specifically refers to the current image block to be encoded in intra-frame prediction (also known as the "encoded block"); when the embodiments of this application are applied to the decoder 200, the "current block" specifically refers to the current image block to be decoded in intra-frame prediction (also known as the "decoded block").

[0087] In addition, if the encoder 100 can obtain a better prediction effect through the encoding method provided in the embodiments of this application to improve the encoding performance, then correspondingly, the decoder 200 can also improve the video decoding and recovery quality through the decoding method provided in the embodiments of this application, thereby improving the decoding performance.

[0088] In one embodiment of this application, see [link to embodiment]. Figure 3 This illustrates a flowchart of an encoding method provided in an embodiment of this application. Figure 3 As shown, the method may include:

[0089] S301: Determine the prediction parameters for the current block, wherein the prediction parameters include prediction mode parameters.

[0090] It should be noted that the image to be encoded can be divided into multiple image blocks, and each image block to be encoded can be called a coding block. Here, each coding block may include a first image component, a second image component, and a third image component. The current block is the coding block in the video image for which the prediction of the first image component, the second image component, or the third image component is currently to be performed.

[0091] In this scenario, assuming the first image component prediction is performed on the current block, and the first image component is the luminance component (i.e., the image component to be predicted is the luminance component), then the current block can also be called the luminance block, and the predicted value of the luminance component of the current block can be obtained. Alternatively, assuming the second image component prediction is performed on the current block, and the second image component is the chrominance component (i.e., the image component to be predicted is the chrominance component), then the current block can also be called the chrominance block, and the predicted value of the luminance component of the current block can be obtained.

[0092] It should also be noted that the prediction parameters may include prediction mode parameters, which indicate the prediction mode adopted by the current block, and different prediction modes correspond to different prediction mode parameters. Here, the determination of the prediction mode parameters can employ a simple decision-making strategy, such as determining them based on the magnitude of the distortion value; or it can employ a complex decision-making strategy, such as determining them based on the results of Rate Distortion Optimization (RDO). This application embodiment does not impose any limitations. Generally speaking, the prediction mode parameters of the current block can be determined using the RDO method, that is, the prediction parameters of the current block are determined.

[0093] Specifically, in some embodiments, determining the prediction parameters for the current block may include:

[0094] The current block is pre-encoded using multiple prediction modes to obtain the rate-distortion cost corresponding to the multiple prediction modes;

[0095] Select the optimal rate distortion value from the multiple rate distortion values ​​obtained, and determine the prediction parameters of the current block as the prediction parameters of the prediction mode corresponding to the optimal rate distortion value.

[0096] In other words, on the encoder side, multiple prediction modes can be used to pre-encode the current block. These multiple prediction modes typically include inter-frame prediction modes, traditional intra-frame prediction modes, and non-traditional intra-frame prediction modes. Traditional intra-frame prediction modes can include Direct Current (DC) mode, Planar mode, and angular intra-frame prediction mode, etc. Non-traditional intra-frame prediction modes can include MIP mode, Cross-component Linear Model Prediction (CCLM) mode, IntraBlock Copy (IBC) mode, and PLT (Palette) mode, etc. Inter-frame prediction modes can include traditional inter-frame prediction modes and Geometrical Partitioning for Inter Blocks (GEO) mode, etc.

[0097] Thus, after precoding the current block using multiple prediction modes, the rate-distortion values ​​corresponding to these multiple prediction modes can be obtained. Then, the optimal rate-distortion value is selected from the multiple obtained rate-distortion values ​​(usually, the minimum rate-distortion value is the optimal rate-distortion value), and the prediction parameters in the prediction mode corresponding to the optimal rate-distortion value are determined as the prediction parameters for the current block.

[0098] In addition, multiple prediction modes can be used to pre-encode the current block separately, obtaining the distortion values ​​corresponding to each prediction mode. Then, the optimal distortion value is selected from the obtained distortion values, and the prediction parameters of the prediction mode corresponding to this optimal distortion value are determined as the prediction parameters for the current block. In this way, the encoder ultimately uses the prediction mode indicated by the prediction mode parameters among the determined prediction parameters to predict and encode the current block, resulting in smaller prediction residuals and improved coding efficiency.

[0099] S302: When the prediction mode parameter indicates that the intra-prediction value of the chroma component of the current block is determined using the matrix-based intra-prediction (MIP) mode, the adjacent sample values ​​of the current block are obtained, and the MIP input sample value of the current block is determined based on the adjacent sample values ​​of the current block.

[0100] It should be noted that, for the current block, this embodiment of the application utilizes the MIP mode to perform intra-frame prediction of the chroma components of the current block. In this process, it is first necessary to obtain the neighboring sample values ​​of the current block; then, based on the neighboring sample values ​​of the current block, the MIP input sample values ​​of the current block are determined.

[0101] It should be understood that for MIP mode, the MIP core parameters need to be configured first. These parameters can include the type of the current block (represented by mipSizeId), the number of reference samples per edge (represented by boundySize), the number of MIP input samples (represented by inSize), and the size of the MIP prediction block output by matrix multiplication (arranged as predSize × predSize). The MIP mode can classify the current block into three categories based on its width and height. mipSizeId can be equal to 0, 1, or 2, where mipSizeId represents the type of the current block, i.e., the "block size index value of the current block" shown in this embodiment. For different mipSizeId values, the number of reference samples (boundySize reference sample points per edge), the number of MIP input samples (inSize), and the size of the MIP prediction block output by matrix multiplication (arranged as predSize × predSize) also differ.

[0102] It should also be noted that, in addition to the prediction mode parameters, the prediction parameters may also include the size parameter of the current block. The size parameter of the current block may include the width (denoted by nTbW) and height (denoted by nTbH) of the current block. Furthermore, based on the size parameter of the current block, the block size index value (i.e., mipSizeId) can be determined.

[0103] In one possible implementation, determining the block size index value of the current block based on the current block's size parameters may include:

[0104] If the width and height of the current block are both equal to 4, then the block size index value of the current block can be set to 0;

[0105] Conversely, if the width and height of the current block are both equal to 8, or if one of the width and height of the current block is equal to 4, then the block size index value of the current block can be set to 1;

[0106] Conversely, if the current block is of a different size, the block size index can be set to 2.

[0107] In another possible implementation, determining the block size index value of the current block based on the current block's size parameters may include:

[0108] If the width and height of the current block are both equal to 4, then the block size index value of the current block can be set to 0;

[0109] Conversely, if either the width or height of the current block is 4, then the block size index value of the current block can be set to 1;

[0110] Conversely, if the current block is of a different size, the block size index can be set to 2.

[0111] Thus, based on the block size index of the current block, the number of adjacent boundary reference samples (variable: boundarySize) and the size of the MIP prediction block (variable: predSize, where the size of the MIP prediction block is predSize × predSize) can be determined using the look-up table (LUT) shown in Table 1. The number of MIP input sample values ​​used in the MIP matrix multiplication process (denoted by inSize) can then be calculated using the following formula.

[0112] inSize=(2×boundarySize)-(mipSizeId==2)? 1:0 (1)

[0113] The operation rules of the operators in equation (1) are the same as those of the operators defined in the ITU-TH.265 standard. For example, “==” is the logical “equal to” operator.

[0114] Table 1

[0115] mipSizeId boundarySize predSize 0 2 4 1 4 4 2 4 8

[0116] Thus, according to Table 1, when mipSizeId is 0, boundarySize can be 2 and predSize can be 4. That is, in this case, the reference pixel selects two pixels for each edge, and the matrix multiplication output is a 4×4 MIP prediction block.

[0117] When mipSizeId is 1, boundarySize and predSize can both be 4. This means that in this case, the reference pixel selects four pixels for each edge, and the matrix multiplication output is a 4×4 MIP prediction block.

[0118] When mipSizeId is 2, boundarySize can be 4 and predSize can be 8; that is, the reference pixel selects four pixels for each edge, and the matrix multiplication output is an 8×8 MIP prediction block.

[0119] In addition, based on the current block size index value and the lookup table shown in Table 2, the values ​​of boundarySize, inSize, and predSize can be determined simultaneously.

[0120] Table 2

[0121] mipSizeId boundarySize inSize predSize 0 2 4 4 1 4 8 4 2 4 7 8

[0122] Thus, according to Table 2, when mipSizeId is 0, boundarySize can be 2, inSize can be 4, and predSize can be 4; that is, at this time, the reference pixel selects two pixels for each edge, the number of sampling points for matrix multiplication is four, and the output of matrix multiplication is a 4×4 MIP prediction block.

[0123] When mipSizeId is 1, boundarySize can be 4, inSize can be 8, and predSize can be 4; that is, the reference pixel selects four pixels for each edge, the number of sampling points for matrix multiplication is eight, and the output of matrix multiplication is a 4×4 MIP prediction block.

[0124] When mipSizeId is 2, boundarySize can be 4, inSize can be 7, and predSize can be 8; that is, the reference pixel selects four pixels for each edge, the number of sampling points for matrix multiplication is seven, and the output of matrix multiplication is an 8×8 MIP prediction block.

[0125] Furthermore, after configuring the MIP core parameters, it is also necessary to obtain reference pixels; then, based on the reference pixels and the MIP core parameters, the MIP input sample values ​​are constructed. Here, the reference pixels are the adjacent sample values ​​of the current block, which can include: the left adjacent sample value and the upper adjacent sample value of the current block. In other words, the MIP input sample values ​​of the current block can be determined based on the left and upper adjacent sample values ​​of the current block.

[0126] In some embodiments, determining the MIP input sample value of the current block based on the adjacent sample values ​​of the current block may include:

[0127] Determine the block size index value of the current block based on the current block's size parameters;

[0128] The adjacent sampled values ​​of the current block are downsampled and filtered to obtain a first temporary reference value;

[0129] When the block size index value of the current block is within a preset range, a second constant value is determined based on the bit depth of the adjacent sample values ​​of the current block; the value corresponding to index number 0 in the MIP input sample value is set to be equal to the difference between the second constant value and the value corresponding to index number 0 in the first temporary reference value; the value corresponding to index number i in the MIP input sample value is set to be equal to the difference between the value corresponding to index number i in the first temporary reference value and the value corresponding to index number 0 in the first temporary reference value, where i is an integer greater than 0;

[0130] When the block size index value of the current block is outside the preset range, the value corresponding to the index number j in the MIP input sample value is set to the difference between the value corresponding to the index number j+1 in the first temporary reference value and the value corresponding to the index number 0 in the first temporary reference value, where j is an integer greater than or equal to 0.

[0131] Furthermore, when the block size index value of the current block is within a preset range, the method may further include:

[0132] Based on the bit depth of the adjacent sampled values ​​of the current block, a second constant value is determined; the value corresponding to index 0 in the MIP input sample value is set to be equal to the difference between the value corresponding to index 0 in the first temporary reference value and the second constant value; the value corresponding to index i in the MIP input sample value is set to be equal to the difference between the value corresponding to index i in the first temporary reference value and the value corresponding to index 0 in the first temporary reference value, where i is an integer greater than 0.

[0133] It should be noted that after downsampling and filtering the adjacent sampled values ​​of the current block, the first temporary reference value can be obtained. Specifically, for the first temporary reference value, after downsampling and filtering the adjacent sampled values ​​of the current block, the filtered adjacent sampled values ​​can be cached in a buffer (represented by pTemp). Among them, the value corresponding to index 0 in the first temporary reference value refers to pTemp[0], and the value corresponding to index i in the first temporary reference value refers to pTemp[i].

[0134] It should also be noted that whether the current block's size parameter value is within the preset range can be determined by whether the current block's block size index value (represented by mipSizeId) is within the preset range. Specifically, when mipSizeId = 0 or 1, it indicates that the current block's block size index value is within the preset range, meaning the current block's size parameter value is within the preset range; when mipSizeId = 2, it indicates that the current block's block size index value is outside the preset range, meaning the current block's size parameter value is not within the preset range.

[0135] In other words, the MIP input sample value is determined by the buffer (represented by pTemp), the block size index of the current block (represented by MipSizeId), and the bit depth of the adjacent sample values ​​of the current block (represented by BitDepth). Moreover, the number of input samples contained in the MIP input sample value is only related to the block size index of the current block. Finally, the value corresponding to index number x in the MIP input sample value can be obtained (represented by p[x]).

[0136] Furthermore, in some embodiments, determining the second constant value based on the bit depth of adjacent sampled values ​​of the current block may include:

[0137] Set the second constant value to an integer power equal to 2, where the power is equal to the bit depth of the adjacent sample value of the current block minus 1.

[0138] Alternatively, in some embodiments, determining the second constant value based on the bit depth of adjacent sampled values ​​of the current block may include:

[0139] The second constant value is obtained by left-shifting the binary bits of "1", wherein the number of bits shifted left is equal to the bit depth of the adjacent sample values ​​of the current block minus 1.

[0140] In other words, after obtaining the bit depth (represented by BitDepth) of the adjacent sample values ​​of the current block, the second constant value can be expressed as 1<<(BitDepth-1) or 2^(BitDepth-1). Thus, if the size parameter of the current block is within a preset range, the MIP input sample value of the current block can be determined by combining the second constant value.

[0141] It should also be noted that the MIP input sample is a matrix vector used for matrix multiplication operations. Current related technical solutions are determined by the buffer (represented by pTemp), the type of the current block (i.e., the block size index value of the current block, represented by mipSizeId), the bit depth of the adjacent sample values ​​of the current block (represented by BitDepth), and the number of MIP input samples, and finally obtain the value corresponding to the index x in the MIP input sample value (represented by p[x]).

[0142] Specifically, the construction process of the x-th input sample value p[x] is as follows:

[0143] In one possible implementation, when the block size index value of the current block is within a preset range, the construction process may include:

[0144] Calculate the difference between the second constant value and the value corresponding to index number 0 in the first temporary reference value to obtain the value corresponding to index number 0 in the MIP input sample value;

[0145] The value corresponding to index i in the first temporary reference value is subtracted from the value corresponding to index 0 in the first temporary reference value to obtain the value corresponding to index i in the MIP input sample value; where i is a positive integer greater than 0 and less than N, and N represents the number of elements contained in the input sampling matrix.

[0146] In this embodiment of the application, the minuend in the calculation of the difference is set to be equal to the second constant value, and the subtrahend in the calculation of the difference is set to be equal to the value corresponding to index number 0 in the first temporary reference value.

[0147] In other words, when mipSizeId = 0 or 1, p[0] can be obtained by subtracting pTemp[0] from 1 << (BitDepth-1); and when x is not equal to 0, p[x] is obtained by subtracting pTemp[0] from pTemp[x]. The specifics are as follows:

[0148]

[0149] In another possible implementation, when the block size index value of the current block is within a preset range, the construction process may include:

[0150] Calculate the difference between the value corresponding to index number 0 in the first temporary reference value and the second constant value to obtain the value corresponding to index number 0 in the MIP input sample value;

[0151] The value corresponding to index i in the first temporary reference value is subtracted from the value corresponding to index 0 in the first temporary reference value to obtain the value corresponding to index i in the MIP input sample value; where i is a positive integer greater than 0 and less than N, and N represents the number of elements contained in the input sampling matrix.

[0152] In this embodiment of the application, the minuend in the calculation of the difference is set to be equal to the value corresponding to index number 0 in the first temporary reference value, and the subtrahend in the calculation of the difference is set to be equal to the second constant value.

[0153] In other words, when mipSizeId = 0 or 1, when x equals 0, the value corresponding to index 0 in the MIP input sample value (represented by p[0]) can be obtained by subtracting the second constant value (i.e., 1 << (BitDepth-1)) from the value corresponding to index 0 in the first temporary reference value (i.e., pTemp[0]). When x is not equal to 0, the value corresponding to index x in the MIP input sample value (represented by p[x]) needs to be obtained by subtracting the value corresponding to index 0 in the first temporary reference value (i.e., pTemp[0]) from the value corresponding to index x in the first temporary reference value (i.e., pTemp[x]).

[0154] The details are as follows:

[0155]

[0156] In another possible implementation, when the block size index value of the current block is outside a preset range, the construction process may include:

[0157] The value corresponding to index i+1 in the first temporary reference value is subtracted from the value corresponding to index 0 in the first temporary reference value to obtain the value corresponding to index i in the MIP input sample value; where i is a positive integer greater than or equal to 0 and less than N, and N represents the number of elements contained in the input sampling matrix.

[0158] In other words, when mipSizeId = 2, the value corresponding to index 0 in the first temporary reference value, i.e., pTemp[0], can be ignored. Then, the value corresponding to index x in the MIP input sample value (represented by p[x]) can be obtained by subtracting the value corresponding to index 0 in the first temporary reference value (i.e., pTemp[x+1]) from the value corresponding to index x+1 in the first temporary reference value. Here, x is a positive integer greater than or equal to 0. The specifics are as follows:

[0159] p[x]=pTemp[x+1]-pTemp[0] for x=0,...,inSize-1 (4)

[0160] Thus, taking the current 4×4 block as an example, the buffer pTemp stores four values, namely the first temporary reference values, which include: the value corresponding to index number 0 (i.e., pTemp[0]), the value corresponding to index number 1 (i.e., pTemp[1]), the value corresponding to index number 2 (i.e., pTemp[2]), and the value corresponding to index number 3 (i.e., pTemp[3]). At this time, according to equation (2), equation (3), or equation (4), four MIP input sampling values ​​can be determined, denoted by p[x], where x = 0, 1, 2, 3. Here, for these four MIP input sampling values, they can also be combined into a 1×4 MIP input sampling matrix.

[0161] In addition, embodiments of this application can also utilize a unified calculation method to obtain the value of p[x] without needing to determine the value of mipSizeId. In some embodiments, determining the MIP input sample value of the current block based on the adjacent sample values ​​of the current block may include:

[0162] The adjacent sampled values ​​of the current block are downsampled and filtered to obtain a first temporary reference value;

[0163] Based on the bit depth of the adjacent sampled values ​​of the current block, a second constant value is determined, and the second constant value is cached in a data unit after the first temporary reference value to obtain the second temporary reference value.

[0164] The value corresponding to index j in the MIP input sample value is set to the difference between the value corresponding to index j+1 in the second temporary reference value and the value corresponding to index 0 in the second temporary reference value, where j is an integer greater than or equal to 0.

[0165] In other words, in this embodiment of the application, a second constant value (i.e., (1<<(BitDepth-1))) can be appended to the end of the buffer pTemp as an additional element. At this time, the size parameter of the current block is no longer considered (i.e., the value of mipSizeId does not need to be considered), and p[x] can be directly set to equal pTemp[x+1]-pTemp[0], x=0,…,inSize-1.

[0166] In this way, after determining the MIP input sample value of the current block based on the adjacent sample values ​​of the current block, the MIP prediction value of the current block can be further determined.

[0167] S303: Determine the MIP prediction value of the chromaticity component of the current block based on the MIP input sample value, the MIP weighting matrix, and the shift parameter.

[0168] The MIP prediction value is the prediction value of a portion of the sampled points in the chromaticity component of the current block.

[0169] It should be noted that shift parameters can include shift offset parameters and shift amount parameters. In this case, such as... Figure 4 As shown, S303 may include the following steps:

[0170] S303-1: Determine the product of the sum of the MIP input sample values ​​and the shift offset parameter based on the value of the shift offset parameter. The value of the shift offset parameter is a fixed constant.

[0171] It should be noted that the shift offset parameter can also be called the shift compensation parameter or the offset factor, and can be represented by fO. In the embodiments of this application, the value of the shift offset parameter can be set to a fixed constant, such as 32, 46, 56, or 66, etc. Alternatively, the value of the shift offset parameter can be set to be related to the shift offset parameter table, and the value of the shift offset parameter can be determined by looking up the table; no limitations are imposed here.

[0172] S303-2: Determine the first constant value based on the value of the shift quantity parameter.

[0173] S303-3: Set the value of the first offset to the difference between the first constant value and the product.

[0174] It should be noted that the shift quantity parameter can also be called the shift factor, shift bits, or weight shift value, and can be represented by sW, shift, or weight shift. In the embodiments of this application, the shift quantity parameter is represented by sW. In addition, the first offset can be represented by oW, and the first offset is related to both the shift quantity parameter and the shift offset parameter.

[0175] In this embodiment, the value of the shift quantity parameter can be set to a fixed constant, such as 5, 6, or 7. Alternatively, the value of the shift quantity parameter can be set to be related to a shift quantity parameter table, and the value of the shift quantity parameter can be determined by looking up the table; no limitations are imposed here.

[0176] S303-4: Determine the MIP weighting matrix of the current block based on the prediction parameters.

[0177] It should be noted that the prediction parameters can include prediction mode parameters and the size parameters of the current block. In the prediction parameters, when the prediction mode parameters indicate that the chroma components of the current block use the MIP mode to determine the intra-frame prediction value, a weight matrix table can be pre-established and stored in memory or a storage unit. This memory or storage unit can be integrated into the encoder or set up separately. Thus, based on the block size index (mipSizeId) and MIP mode index (modeId) of the current block, the MIP weighting matrix (or MIP weight matrix, or simply MIP matrix) required for the current block can be determined by looking up the table, denoted by mWeight[x][y]. The block size index (mipSizeId) of the current block is determined by the size parameters of the current block, while the size of the MIP weighting matrix mWeight[x][y] is only related to the block size index of the current block, as shown in Table 3.

[0178] In the MIP weighting matrix shown in Table 3, the number of columns is the number of matrix multiplication input samples inSize, and the number of rows is the number of matrix multiplication output samples predSized×predSized, thus enabling the determination of the MIP weighting matrix of the current block.

[0179] Table 3

[0180] mipSizeId Number of columns Number of lines 0 4 16 1 8 16 2 7 64

[0181] It is understood that a shift quantity parameter table can be pre-established in the encoder, and this shift quantity parameter table is also stored in a memory or storage unit. This memory or storage unit can be integrated into the encoder or can be set up separately. In the embodiments of this application, the determination of the shift quantity parameter (sW) can include the following methods:

[0182] In one possible implementation, the shift amount parameter can be different for different block size index values ​​and different MIP mode index values. In some embodiments, the method may further include:

[0183] When the chroma components of the current block are used to determine the intra-prediction value using the MIP mode, the MIP mode index value of the current block is determined.

[0184] Based on the MIP mode index value, the value corresponding to the MIP mode index value is queried from the first preset lookup table, wherein the first preset lookup table is used to record the correspondence between the MIP mode index value and the value of the shift amount parameter;

[0185] The retrieved value is determined as the value of the shift quantity parameter.

[0186] It should be noted that the value of the shift amount parameter can be looked up based on the current block size index (represented by mipSizeId) and the MIP mode index (modeId). As shown in the first preset lookup table in Table 4, the shift amount parameter required for matrix multiplication can be determined by looking up different mipSizeId and modeId.

[0187] Table 4

[0188]

[0189] However, on the encoder side, Table 4 needs to be stored in memory or storage unit in the form of a lookup table; however, storage comes at a cost, and the lookup process also comes at a cost; since the value of the shift amount parameter in Table 4 is related to the block size of the current block and the MIP mode index value, it increases memory usage and computational complexity.

[0190] To reduce memory usage and computational complexity, the method for determining the shift quantity parameter can be simplified in this embodiment.

[0191] In another possible implementation, the shift amount parameter can be set to a fixed constant, independent of the block size index value and the MIP mode index value. For example, the shift amount parameter can be set to 5 for different block size index values ​​and different MIP mode index values; or, the shift amount parameter can be set to 6 for different block size index values ​​and different MIP mode index values; or, the shift amount parameter can be set to 7 for different block size index values ​​and different MIP mode index values. In this embodiment, preferably, the shift amount parameter is set to 6, but this is not a limitation.

[0192] In yet another possible implementation, the method may further include, for the value of the shift amount parameter:

[0193] Determine the block size index value of the current block based on the current block's size parameters;

[0194] The value of the shift quantity parameter is determined based on the block size index value of the current block.

[0195] It should be noted that the block size index value of the current block can be determined based on the size parameter of the current block. In some embodiments, determining the block size index value of the current block based on the size parameter of the current block may include:

[0196] When the width and height of the current block are both equal to 4, the block size index value of the current block is set to 0;

[0197] When the width and height of the current block are both equal to 8, or when one of the width and height of the current block is equal to 4, the block size index value of the current block is set to 1;

[0198] If the width and height of the current block do not meet the aforementioned conditions, the block size index value of the current block is set to 2.

[0199] Thus, after determining the block size index value of the current block, the value of the shift amount parameter can be further determined based on the block size index value of the current block.

[0200] Optionally, in some embodiments, determining the value of the shift amount parameter based on the block size index value of the current block may include:

[0201] When the block size index value is equal to 0, 1 and 2 respectively, the value of the shift quantity parameter corresponding to the block size index value of the current block is determined to be equal to 5, 6 and 5 respectively.

[0202] Optionally, in some embodiments, determining the value of the shift amount parameter based on the block size index value of the current block may include:

[0203] The value of the shift quantity parameter is set to be equal to the ratio between the width or height of the current block and a first preset value corresponding to the block size index value of the current block.

[0204] Here, the first preset value represents the number of MIP input sample values ​​obtained from the boundary of the current block. In this case, the method may further include:

[0205] When the block size index value of the current block is equal to 0, 1 and 2 respectively, the first preset value corresponding to the block size index value of the current block is determined to be equal to 2, 4 and 4 respectively.

[0206] In other words, when the first preset value represents the number of MIP input sample values ​​obtained from the boundary of the current block, if the block size index of the current block is equal to 0, then the corresponding first preset value is equal to 2; if the block size index of the current block is equal to 1, then the corresponding first preset value is equal to 4; if the block size index of the current block is equal to 2, then the corresponding first preset value is equal to 4. Thus, the value of the shift quantity parameter can be determined according to the ratio between the width or height of the current block and the corresponding first preset value.

[0207] Optionally, in some embodiments, determining the value of the shift amount parameter based on the block size index value of the current block may include:

[0208] The value of the shift quantity parameter is set to be equal to the ratio between the width or height of the current block and a second preset value corresponding to the block size index value of the current block.

[0209] Here, the second preset value represents the size of the MIP predicted block of the current block, obtained by directly calculating using the MIP weighting matrix. In this case, the method may further include:

[0210] When the block size index value of the current block is equal to 0, 1 and 2 respectively, the second preset value corresponding to the block size index value of the current block is determined to be equal to 4, 4 and 8 respectively.

[0211] In other words, when the second preset value represents the size of the MIP predicted block of the current block obtained by directly calculating using the MIP weighted matrix, if the block size index value of the current block is equal to 0, then the corresponding second preset value is equal to 4; if the block size index value of the current block is equal to 1, then the corresponding second preset value is equal to 4; if the block size index value of the current block is equal to 2, then the corresponding second preset value is equal to 8. Thus, the value of the shift amount parameter can be determined based on the ratio between the width or height of the current block and the corresponding second preset value.

[0212] In another possible implementation, the shift amount parameter table can be minimized, and a lookup table is still used to determine the value of the shift amount parameter. Optionally, in some embodiments, determining the value of the shift amount parameter based on the block size index of the current block may include:

[0213] Based on the block size index value, the value corresponding to the block size index value is retrieved from the second preset lookup table, wherein the second preset lookup table is used to record the correspondence between the block size index value and the value of the shift quantity parameter;

[0214] The retrieved value is determined as the value of the shift quantity parameter.

[0215] It should be noted that the value of the shift amount parameter can be queried based solely on the block size index value of the current block (represented by mipSizeId). As shown in the second preset lookup table in Table 5, each block size index value corresponds to a fixed value. In other words, the size of each block or the set of sizes of each block can have a fixed value for the shift amount parameter as shown in Table 5.

[0216] Table 5

[0217] mipSizeId sW 0 5 1 6 2 5

[0218] According to Table 5, when the block size index value of the current block is equal to 0, 1 and 2 respectively, it can be determined that the value of the shift quantity parameter corresponding to the block size index value is equal to 5, 6 and 5 respectively.

[0219] In the above implementation, by simplifying the method of determining the shift quantity parameter, especially by minimizing the shift quantity parameter table or fixing the value of the shift quantity parameter, the storage of the lookup table can be minimized, thereby reducing the memory occupied by the shift quantity parameter table in MIP mode without increasing computational complexity.

[0220] In the encoder, a shift offset parameter table can be pre-established, and this shift offset parameter table is also stored in a memory or storage unit. This memory or storage unit can be integrated into the encoder or can be set up separately. In this embodiment, the determination of the shift offset parameter (fO) can include the following methods:

[0221] In one possible implementation, the shift offset parameter may be different for different block sizes and different MIP mode index values. In some embodiments, the method may further include:

[0222] When the chroma components of the current block are used to determine the intra-prediction value using the MIP mode, the MIP mode index value of the current block is determined.

[0223] Based on the MIP mode index value, the value corresponding to the MIP mode index value is retrieved from the third preset lookup table, wherein the third preset lookup table is used to record the correspondence between the MIP mode index value and the value of the shift offset parameter;

[0224] The retrieved value is determined as the value of the shift offset parameter.

[0225] It should be noted that the shift offset parameter value can be looked up based on the current block size index (represented by MipSizeId) and MIP mode index (modeId). As shown in the third preset lookup table in Table 6, the shift offset parameter required for matrix multiplication can be determined by looking up the table for different mipSizeId and modeId.

[0226] Table 6

[0227]

[0228] However, on the encoder side, Table 6 also needs to be stored in memory or storage unit in the form of a lookup table; however, storage comes at a cost, and the lookup process also comes at a cost; since the value of the shift offset parameter in Table 6 is related to the block size of the current block and the MIP mode index value, it increases memory usage and computational complexity.

[0229] To reduce memory usage and computational complexity, the method for determining the shift offset parameters can also be simplified in the embodiments of this application.

[0230] In another possible implementation, the shift offset parameter can be set to a fixed constant, independent of the block size index and the MIP mode index. Typically, this fixed constant ranges from 0 to 100. For example, the shift offset parameter can be set to 32 for different block size indexes and different MIP mode indexes; or, it can be set to 46 for different block size indexes and different MIP mode indexes; or, it can be set to 56 for different block size indexes and different MIP mode indexes; or, it can be set to 66 for different block size indexes and different MIP mode indexes. In this embodiment, preferably, the shift offset parameter is set to 32, but this is not a limitation.

[0231] In yet another possible implementation, the method may further include, for the value of the shift offset parameter:

[0232] Determine the block size index value of the current block based on the current block's size parameters;

[0233] The value of the shift offset parameter is determined based on the block size index value of the current block.

[0234] It should be noted that the block size index value of the current block can be determined based on the current block size parameter; then, the value of the shift offset parameter can be further determined based on the current block size index value.

[0235] Optionally, in some embodiments, determining the value of the shift offset parameter based on the block size index value of the current block may include:

[0236] When the block size index value is equal to 0, 1 and 2 respectively, the shift offset parameter corresponding to the block size index value of the current block is determined to be equal to 34, 23 and 46 respectively.

[0237] Optionally, in some embodiments, the shift offset parameter table can be minimized, and a lookup table method can still be used to determine the offset factor. Optionally, in some embodiments, determining the value of the shift offset parameter based on the block size index value of the current block may include:

[0238] Based on the block size index value, the value corresponding to the block size index value is queried from the fourth preset lookup table, wherein the fourth preset lookup table is used to record the correspondence between the block size index value and the value of the shift offset parameter;

[0239] The retrieved value is determined as the value of the shift offset parameter.

[0240] It should be noted that the value of the shift offset parameter can be queried based solely on the block size index value of the current block (represented by mipSizeId). As shown in the fourth preset lookup table in Table 7, each type of block size index value corresponds to a fixed value. In other words, the size of each block or the set of sizes of each block can have a fixed value of the shift offset parameter as shown in Table 7.

[0241] Table 7

[0242] mipSizeId fO 0 34 1 23 2 46

[0243] According to Table 7, when the block size index value of the current block is equal to 0, 1 and 2 respectively, the shift offset parameter corresponding to the block size index value can be determined to be equal to 34, 23 and 46 respectively.

[0244] In the above implementation, by simplifying the determination of shift offset parameters, especially by minimizing the shift offset parameter table or fixing the values ​​of shift offset parameters, the storage of the lookup table can be minimized, thereby reducing the memory occupied by the shift offset parameter table in MIP mode without increasing computational complexity.

[0245] Thus, after determining the shift offset parameter (fO) and the shift amount parameter (sW), for the first constant value, optionally, in some embodiments, determining the first constant value based on the value of the shift amount parameter may include:

[0246] The first constant value is set to an integer power equal to 2, wherein the exponent of the power is equal to the value of the shift quantity parameter minus 1.

[0247] Optionally, in some embodiments, determining the first constant value based on the value of the shift amount parameter may include:

[0248] The first constant value is obtained by left-shifting the binary bits of "1", wherein the number of bits shifted left is equal to the value of the shift quantity parameter minus 1.

[0249] In other words, after obtaining the shift amount parameter (sW), the first constant value can be expressed as 1<<(sW-1) or 2^(sW-1). When the shift amount parameter is set to 6, the first constant value becomes 32.

[0250] Here, if the first offset is represented by oW, the product of the sum of the MIP input sample values ​​and the shift offset parameter (fO) can be calculated based on the shift offset parameter, which is: Then the value of oW can be set to

[0251] In this way, under MIP mode, the MIP weighting matrix, MIP input sample value, shift quantity parameter, and first offset can be obtained to subsequently determine the MIP prediction value of the current block.

[0252] S303-5: Determine the MIP prediction value of the chromaticity component of the current block based on the MIP weighting matrix, the MIP input sample value, the shift quantity parameter, and the first offset.

[0253] Here, the MIP prediction value represents the prediction value of a portion of the sampled points in the chromaticity component of the current block.

[0254] It should be noted that determining the MIP prediction value of the current block based on the MIP weighting matrix, the MIP input sample value, the shift quantity parameter, and the first offset may include:

[0255] Calculate the first weighted sum of the MIP weighting matrix and the MIP input sample values;

[0256] Calculate the first sum of the first weighted sum and the first offset;

[0257] The first sum is right-shifted by binary bits to obtain a first right-shifted value, wherein the number of bits shifted is equal to the value of the shift quantity parameter;

[0258] The MIP prediction value of the current block is set to be equal to the sum of the first right shift value and the value corresponding to index number 0 in the first temporary reference value; wherein the first temporary reference value is obtained by downsampling and filtering the adjacent sampled values ​​of the current block.

[0259] Specifically, in MIP mode, the MIP weighting matrix (mWeight), shift amount parameter (sW), and shift offset parameter (fO) are determined based on the current block size index (mipSizeId) and MIP mode index (modeId). Then, the MIP input sample values ​​(p[x]), mWeight, sW, and fO are multiplied into the input matrix to obtain the MIP prediction value (predMip[x][y]). The sample points in predMip[x][y] are then arranged in a matrix / array format according to predSize × predSize. The calculation formula is as follows:

[0260]

[0261] Where [x][y] represents the position coordinates of the pixel, x represents the horizontal direction and y represents the vertical direction; inSize represents the number of input samples, p[i] represents the value with index i in the MIP input sample value; pTemp[0] represents the value with index 0 in the first temporary reference value; ">>" represents the binary right shift operator and "<<" represents the binary left shift operator; x = 0, ..., predSize-1, y = 0, ..., predSize-1. Thus, according to the above equation (5), predMip[x][y] can be calculated to obtain the MIP prediction block.

[0262] In some embodiments, if sW is fixed at 6 and fO is fixed at 32, then the calculation formula is as follows:

[0263]

[0264] Furthermore, it is necessary to perform embedding processing on the predicted values ​​of the sampling points in the MIP prediction block to obtain the MIP prediction block of the current block; then it is determined whether to perform transpose processing on the MIP prediction block; if the determination result is yes, then it is necessary to perform transpose processing on the predicted sampling values ​​in the MIP prediction block, and determine the transposed MIP prediction block as the MIP prediction block of the current block; if the determination result is no, then it is not necessary to perform transpose processing on the predicted sampling values ​​in the MIP prediction block, and the MIP prediction block can be directly determined as the MIP prediction block of the current block to obtain the MIP prediction value of the current block, and the MIP prediction value is the predicted value of some sampling points in the current block.

[0265] S304: Filter the MIP prediction value to determine the intra-frame prediction value of the chroma component of the current block.

[0266] It should be noted that the filtering process performed on the MIP prediction value to determine the intra-frame prediction value of the current block may include:

[0267] Determine whether the size parameters of the MIP predicted block are the same as the size parameters of the current block;

[0268] When the size parameter of the MIP prediction block is the same as the size parameter of the current block, the intra-prediction block of the current block is set to be equal to the MIP prediction block; wherein, the MIP prediction block contains the predicted sample values ​​of all pixel positions in the current block;

[0269] When the size parameters of the MIP prediction block are different from the size parameters of the current block, the MIP prediction block is filtered to obtain a filtered prediction block, and the filtered prediction block is set as the intra-prediction block of the current block.

[0270] Here, filtering can include upsampling filtering or low-pass filtering.

[0271] It should be noted that the MIP prediction block is composed of MIP prediction values. After obtaining the MIP prediction block, since there are only two possible size parameters for the MIP prediction block: 4×4 and 8×8, the size parameters of the current block may or may not be the same as those of the MIP prediction block. In other words, the sampled values ​​corresponding to the MIP prediction block may not be able to fill the current block completely. This means that generating the final prediction value may require upsampling the MIP prediction block. Specifically, this is determined by checking whether the size parameters of the MIP prediction block are the same as those of the current block.

[0272] It should also be noted that when the size parameters of the MIP prediction block are the same as those of the current block (i.e., the width and height of the MIP prediction block are the same), it indicates that upsampling of the MIP prediction block is not required. In this case, the MIP prediction block can be directly filled into the current block, meaning there are no empty pixels in the filled current block. The intra-frame prediction value of each pixel in the current block can then be directly set to the prediction value of each pixel in the MIP prediction block, as shown below.

[0273] predSamples[x][y]=predMip[x][y] (7)

[0274] Where [x][y] represents the position coordinates of a pixel, x represents the horizontal direction, and y represents the vertical direction; predSamples[x][y] represents the intra-predicted value corresponding to the pixel at position coordinates [x][y] in the current block, and predMip[x][y] represents the predicted value corresponding to the pixel at position coordinates [x][y] in the MIP prediction block. Thus, according to equation (7), the MIP prediction block predMip[x][y] can be directly used as the intra-predicted block predSamples[x][y] of the current block, that is, the intra-predicted value of at least one pixel in the current block.

[0275] It should also be noted that when the size parameters of the MIP prediction block are different from those of the current block, that is, at least one of the width and height of the MIP prediction block is different from that of the current block, the MIP prediction block cannot fill the current block. In other words, there are empty pixels in the current block after filling, indicating that filtering processing of the MIP prediction block is required. In other words, if upsampling processing is required in both the horizontal and vertical directions, the MIP prediction block can be sampled horizontally first and then vertically to obtain the first upsampled block, which can be represented by predSamples[x][y]. Then, the MIP prediction block can be sampled vertically and then horizontally to obtain the second upsampled block, which can be represented by predSamplesTemp[x][y]. Finally, the weighted average of predSamples[x][y] and predSamplesTemp[x][y] is calculated to obtain the intra-frame prediction value of the chroma component of the current block, that is, the intra-frame prediction block of the current block.

[0276] For example, if the side length nTbS of the current block (where S can be replaced by W and H respectively) is equal to the side length predSize of predMip (where predSize is only related to the blocksizeIdx of the current block), then the MIP prediction block can be directly set as the intra-prediction block of the current block; otherwise, the MIP prediction block needs to be filtered to obtain the intra-prediction block of the current block. Here, taking a 4×4 current block as an example, if the size parameters of the current block and the MIP prediction block are the same, then there is no need to filter the MIP prediction block, and the MIP prediction block can be directly set as the intra-prediction block of the current block, thereby obtaining the intra-prediction value of at least one pixel in the current block.

[0277] In short, the prediction process of MIP mode includes the following input data: the position of the current block (xTbCmp, yTbCmp), the MIP mode index value applied to the current block (represented by modeId), the height of the current block (represented by nTbH), the width of the current block (represented by nTbW), and a transpose indicator (i.e., the MIP transpose indicator parameter, represented by isTransposed) indicating whether transpose processing is required. The output data of MIP mode includes the intra-predicted block of the current block. The predicted value corresponding to the pixel coordinates [x][y] in this intra-predicted block is predSamples[x][y]; where x = 0, 1, ..., nTbW-1; y = 0, 1, ..., nTbH-1.

[0278] like Figure 5 As shown, the MIP prediction process specifically includes the following steps:

[0279] S501: Configure core parameters.

[0280] For S501, the core parameters of MIP can include the type of the current block (represented by mipSizeId), the number of reference samples for each edge (represented by boundySize), the number of MIP input samples (represented by inSize), and the size of the MIP prediction block output by matrix multiplication (arranged as predSize × predSize). Based on the size of the current block, it can be divided into three categories, with mipSizeId recording the type. Here, mipSizeId can be equal to 0, 1, or 2. Furthermore, the number of reference samples and the size of the MIP prediction block output by matrix multiplication differ for different types of current blocks.

[0281] S502: Obtain reference pixels.

[0282] For S502, when predicting the current block, the upper and left adjacent blocks are already encoded. The reference pixels in MIP mode are the reconstructed values ​​of the pixels in the row above and column to the left of the current block. In other words, the process of obtaining the reference pixels (denoted by refT) adjacent to the upper side and the reference pixels (denoted by refL) adjacent to the left side of the current block is the process of obtaining reference pixels.

[0283] S503: Construct input sampling.

[0284] For S503, this step is used as input for matrix multiplication and may specifically include:

[0285] S5031: Obtain reference sample;

[0286] S5032: Construct a reference sampling buffer;

[0287] S5033: Derivation of matrix multiplication input sampling.

[0288] Specifically, for S5031, the process of acquiring the reference sample involves downsampling the reference pixel. For S5032, there are two filling methods in the construction of the reference sampling buffer, which may include:

[0289] S5032-1: Buffer padding method when transpose is not required;

[0290] S5032-2: Buffer filling method when transposition is required.

[0291] S504: Generate intra-frame prediction values.

[0292] For S504, this step is used to obtain the MIP prediction value for the current block, and may specifically include:

[0293] S5041: Construct a MIP prediction block for matrix multiplication output sampling;

[0294] S5042: Embedding processing is performed on the MIP prediction block sampled from the matrix multiplication output;

[0295] S5043: Transpose the MIP prediction block sampled from matrix multiplication output;

[0296] S5044: Generate MIP prediction values.

[0297] Specifically, for S5041, the process of constructing the MIP prediction block may include:

[0298] S5041-1: Obtain the weight matrix;

[0299] S5041-2: Obtain shift parameters;

[0300] S5041-3: Matrix multiplication operation.

[0301] In other words, the process of constructing a MIP prediction block requires obtaining not only the weight matrix but also the shift parameters (including shift offset and shift amount parameters); then, matrix multiplication is performed to obtain the MIP prediction block sampled from the matrix multiplication output. For the S5044, generating the final MIP prediction value involves two scenarios, specifically:

[0302] S5044-1: Generate predicted values ​​that do not require upsampling filtering;

[0303] S5044-2: Generate predicted values ​​that require upsampling and filtering.

[0304] Thus, after going through the four steps S501 to S504 described above, the embodiment of this application can obtain the intra-frame prediction value of at least one pixel in the chromaticity component of the current block.

[0305] S305: Determine the prediction residual value of the chroma component of the current block based on the intra-frame prediction value of the chroma component of the current block.

[0306] It should be noted that after obtaining the intra-frame prediction value of the chrominance component of the current block, the difference between the true value of the chrominance component of the current block and the intra-frame prediction value of the chrominance component of the current block can be calculated to obtain the prediction residual value of the chrominance component of the current block, that is, to obtain the residual block of the current block.

[0307] S306: Perform a low-frequency non-separable quadratic transform (LFNST) on the predicted residual value to determine the LFNST parameters.

[0308] It should be noted that, for Figure 1 Regarding the transformation unit 108 shown, the transformation unit 108 can perform a first transformation on the residual block of the current block, such as an integer transformation based on the initial design of Discrete Cosine Transform (DCT); it can also perform a second transformation on the residual block of the current block, such as a Low-Frequency Non-Separable Transform (LFNST). Specifically, in some embodiments, performing the Low-Frequency Non-Separable Transform (LFNST) on the predicted residual value and determining the LFNST parameters may include:

[0309] The residual block of the current block is determined based on the predicted residual value, and the residual block is transformed for the first time to obtain the first transformation coefficient block;

[0310] Determine whether to perform LFNST on at least a portion of the transform coefficients in the first transform coefficient block;

[0311] If the result is yes, LFNST is performed on at least a portion of the transform coefficients in the first transform coefficient block to determine the LFNST parameters.

[0312] In this embodiment, the encoder can perform a first transformation (also known as a "Core Transform," "first transformation," or "main transformation") on the predicted residual values ​​in the residual block to obtain a first transformed coefficient block after the first transformation; then, an LFNST transformation (also known as a "Secondary Transform") is performed on some or all of the transformed coefficients in the first transformed coefficient block to determine the LFNST parameters. It should be noted that the first transformation here is different from the LFNST transformation.

[0313] It should also be noted that not every current block can undergo LFNST. After the first transform, certain conditions need to be checked (such as the minimum value in the current block's size parameters, the current block's block size index, etc.) to determine whether to perform LFNST on at least a portion of the transform coefficients in the first transform coefficient block. Optionally, in some embodiments, determining whether to perform LFNST on at least a portion of the transform coefficients in the first transform coefficient block may include:

[0314] When using MIP mode to determine the intra-prediction value of the chroma component of the current block, determine the minimum value among the size parameters of the current block;

[0315] Based on the minimum value, determine whether to perform LFNST on at least a portion of the transform coefficients in the first transform coefficient block.

[0316] Further, determining whether to perform LFNST on at least a portion of the transform coefficients in the first transform coefficient block based on the minimum value may include:

[0317] If the minimum value is greater than or equal to the first preset threshold, then it is determined that LFNST will be performed on at least a portion of the transformation coefficients in the first transformation coefficient block.

[0318] Here, the first preset threshold can be used to represent a pre-set threshold value for measuring whether LFNST should be performed. In the embodiments of this application, the first preset threshold can be set to 8, but is not specifically limited to this.

[0319] In other words, when intra-frame prediction is performed on the chroma components of the current block using MIP mode, the minimum value of the size parameters of the current block can be determined; for example, when the minimum value of the width and height is greater than or equal to 8, it can be determined that LFNST is performed on at least some of the transform coefficients in the first transform coefficient block.

[0320] Optionally, in some embodiments, determining whether to perform LFNST on at least a portion of the transform coefficients in the first transform coefficient block may include:

[0321] When the chroma components of the current block are used to determine the intra-prediction value using the MIP mode, the block size index value of the current block is determined according to the size parameter of the current block;

[0322] Based on the block size index value, determine whether to perform LFNST on at least a portion of the transform coefficients in the first transform coefficient block.

[0323] Further, determining whether to perform LFNST on at least a portion of the transform coefficients in the first transform coefficient block based on the block size index value may include:

[0324] If the block size index value is equal to the second preset threshold, then it is determined that LFNST will be performed on at least a portion of the transform coefficients in the first transform coefficient block.

[0325] Here, the second preset threshold can also be used to represent a pre-set threshold value for measuring whether LFNST is performed. In the embodiments of this application, the second preset threshold can be set to a preset value (e.g., 2), or one of multiple preset values ​​(e.g., 1, 2), but is not specifically limited.

[0326] In other words, when performing intra-frame prediction using MIP mode on the chrominance components of the current block, the block size index value (mipSizeId) of the current block can be determined based on the size parameter of the current block. For example, when the value of mipSizeId is equal to 2, it can be determined that at least some of the transform coefficients in the first transform coefficient block will be subjected to LFNST.

[0327] Optionally, in some embodiments, determining whether to perform LFNST on at least a portion of the transform coefficients in the first transform coefficient block may include:

[0328] The first cost result is calculated without performing LFNST on at least some of the transform coefficients in the first transform coefficient block;

[0329] If LFNST is performed on at least a portion of the transform coefficients in the first transform coefficient block, the second cost result is calculated;

[0330] Based on the first cost result and the second cost result, determine whether to perform LFNST on at least some of the transform coefficients in the first transform coefficient block.

[0331] Further, determining whether to perform LFNST on at least a portion of the transform coefficients in the first transform coefficient block based on the first cost result and the second cost result may include:

[0332] If the first cost result is greater than the second cost result, then it is determined that LFNST will be performed on at least a portion of the transform coefficients in the first transform coefficient block.

[0333] It should be noted that the embodiments of this application can adopt the traditional rate-distortion optimization method. First, without performing LFNST on at least some of the transform coefficients in the first transform coefficient block, a first cost result is calculated. Then, with LFNST performed on at least some of the transform coefficients in the first transform coefficient block, a second cost result is calculated. The first cost result is compared with the second cost result. When the first cost result is greater than the second cost result, it can be determined that LFNST will be performed on at least some of the transform coefficients in the first transform coefficient block.

[0334] Thus, when the result is determined to be yes, that is, when it is determined that at least some of the transform coefficients in the first transform coefficient block need to be performed LFNST, then at least some of the transform coefficients in the first transform coefficient block can be performed LFNST, and the LFNST parameters can be determined.

[0335] S307: Encode the LFNST parameters and write them into the bitstream.

[0336] It should be noted that the LFNST parameter includes the LFNST index number, which can be represented by lfnst_index. In some embodiments, determining the LFNST parameter may include:

[0337] If it is determined that LFNST is performed on at least some of the transform coefficients in the first transform coefficient block, then the value of the LFNST index number is set to be greater than zero;

[0338] If it is determined that LFNST will not be performed on at least some of the transform coefficients in the first transform coefficient block, then the value of the LFNST index number is set to zero.

[0339] Furthermore, for S307, encoding the LFNST parameters and writing them into the bitstream may include:

[0340] The LFNST index number is encoded and written into the bitstream.

[0341] In other words, after determining the LFNST parameter, i.e., lfnst_index, if lfnst_index equals 0, it means LFNST is not executed; if lfnst_index is greater than 0, it means LFNST needs to be executed. During the encoding process, lfnst_index can be encoded using Context-based Adaptive Binary Arithmetic Coding (CABAC) with the descriptor "ae(v)".

[0342] For example, see Table 8, which shows an example of the syntax structure description of an LFNST parameter provided by the related art. The condition for determining whether to perform LFNST on at least a portion of the transform coefficient blocks in the first transform coefficient block may include at least one of the following: treeType == DUAL_TREE_CHROMA, ! IntraMipFlag[x0][y0], Min(lfnstWidth,lfnstHeight) >= 16.

[0343] Table 8

[0344]

[0345] Refer to Tables 9A and 9B, which respectively illustrate an example of the syntax structure description of an LFNST parameter provided in an embodiment of this application. The condition for determining whether to perform LFNST on at least a portion of the transform coefficient blocks in the first transform coefficient block may include at least one of the following: intra_mip_flag[x0][y0] == 1 && Min(lfnstWidth, lfnstHeight) >= 8, intra_mip_flag[x0][y0] == 1 && mipSizeId == 2.

[0346] Specifically, as shown in Table 9A, when the minimum value of the current block's size parameters is greater than or equal to 8 (i.e., both width and height are greater than or equal to 8), the MIP mode (i.e., when intra_mip_flag[x0][y0] == 1) can be used to encode the current block's lfnst_index. Alternatively, this condition can be set to: both width and height are greater than or equal to the first value (e.g., 8), and the sum of width and height is greater than or equal to the second value (e.g., 16). As shown in Table 9B, when the current block's block size index value (mipSizeId) is equal to 2, the MIP mode (i.e., when intra_mip_flag[x0][y0] == 1) can be used to encode the current block's lfnst_index. Alternatively, this condition can be set to: mipSizeId equals one of several preset values ​​(e.g., 1, 2).

[0347] Table 9A

[0348] coding_unit(x0,y0,cbWidth,cbHeight,cqtDepth,treeType,modeType){ Descriptor …… if(…&&(intra_mip_flag[x0][y0]===1&&Min(lfnstWidth,lfnstHeight)>=8)&&…){ …… lfnst_idx ae(v) } …… }

[0349] Table 9B

[0350] coding_unit(x0,y0,cbWidth,cbHeight,cqtDepth,treeType,modeType){ Descriptor …… if(…&&(intra_mip_flag[x0][y0]==1&&mipSizeId==2)&&…){ …… lfnst_idx ae(v) } …… }

[0351] It should also be noted that when it is determined that the current block can perform LFNST, it is also necessary to determine the LFNST transform kernel (represented by kernel) used by the current block. In some embodiments, performing LFNST on the predicted residual value and determining the LFNST parameters may include: determining the LFNST transform kernel used by the current block; performing LFNST on the predicted residual value using the LFNST transform kernel and determining the LFNST parameters.

[0352] In this embodiment, there are four candidate transform cores in the LFNST, which may include set0, set1, set2, and set3. Here, this embodiment can utilize MIP parameters to determine the LFNST candidate transform cores, and then select the LFNST transform core to be used by the current block from these candidate sets. Therefore, in some embodiments, the method may further include:

[0353] When using MIP mode to determine intra-prediction values ​​for the chrominance components of the current block, the MIP parameters are determined from the prediction parameters of the current block;

[0354] Based on the MIP parameters, determine the LFNST transform kernel candidate set;

[0355] In the LFNST transform core candidate set, the transform core indicated by the LFNST index number is set as the LFNST transform core used by the current block;

[0356] Set the value of the LFNST index number to indicate that LFNST is performed on the current block, and set the index number of the LFNST transform kernel in the LFNST transform kernel candidate set;

[0357] The LFNST transform core candidate set includes two or more preset LFNST transform cores.

[0358] It should be noted that the MIP parameters may include at least one of the following: the MIP mode index value (represented by modeId) and the MIP transpose indicator parameter (represented by isTransposed). Specifically, for the LFNST transform kernel used by the current block, when the value of the MIP transpose indicator parameter indicates that the sampling point input vector used by the MIP mode is to be transposed, the selected transform kernel also needs to be matrix transposed to obtain the LFNST transform kernel used by the current block.

[0359] Furthermore, in some embodiments, determining the LFNST transform kernel candidate set based on the MIP parameters may include:

[0360] Based on the MIP mode index value, determine the value of the LFNST intra-frame prediction mode index number;

[0361] Based on the value of the LFNST intra-frame prediction mode index number, the index value of the LFNST transform kernel candidate set is determined through the fifth preset lookup table;

[0362] The LFNST transform kernel candidate set is selected from multiple candidate LFNST transform kernel candidate sets based on the index value of the LFNST transform kernel candidate set.

[0363] Here, the index number of the LFNST intra-prediction mode can be represented by predModeIntra, and the index value of the LFNST transform kernel candidate set can be represented by lfnstTrSetIdx.

[0364] It should be noted that the fifth preset lookup table is shown in Table 10. Based on the MIP mode index value (i.e., the value of modeId), the value of predModeIntra can be determined; then, based on the value of predModeIntra, the value of lfnstTrSetIdx can be directly determined in conjunction with Table 10, thus determining the LFNST transform kernel candidate set selected for the current block. Here, the value of lfnstTrSetIdx indicates the transform kernel candidate set used during LFNST; since the value of modeId can include 0, 1, 2, 3, 4, and 5, the value of predModeIntra is also 0, 1, 2, 3, 4, and 5; its correspondence with lfnstTrSetIdx is as follows:

[0365] Table 10

[0366]

[0367]

[0368] In one specific embodiment, determining the value of the LFNST intra-frame prediction mode index number based on the MIP mode index value may include: setting the value of the LFNST intra-frame prediction mode index number to be equal to the MIP mode index value.

[0369] In other words, in this embodiment, the value of predModeIntra can be set to be equal to the value of modeId. Then, based on the value of predModeIntra and in conjunction with Table 10, the value of lfnstTrSetIdx can be directly determined, thus determining the LFNST transform kernel candidate set selected for the current block. It should also be noted that, in another specific embodiment, this embodiment can also directly map the value of modeId to the value of the PLANA mode, and then use the value of predModeIntra corresponding to the PLANA mode to determine the value of lfnstTrSetIdx, thus determining the LFNST transform kernel candidate set selected for the current block.

[0370] In some embodiments, the LFNST parameter may further include LFNST coefficients. Determining the LFNST parameter may include:

[0371] Perform LFNST on at least a portion of the transform coefficients in the first transform coefficient block to obtain the LFNST coefficients;

[0372] Accordingly, encoding the LFNST parameters and writing them into the bitstream includes:

[0373] The LFNST coefficients are quantized to obtain quantized coefficients;

[0374] The quantization coefficients are encoded and written into the bitstream.

[0375] It should be noted that when lfnst_index is not equal to 0, that is, when it is determined that LFNST will be performed on the current block, the LFNST coefficients can be obtained by performing LFNST on at least some of the transform coefficients in the first transform coefficient block; then the LFNST coefficients are quantized and encoded and written into the bitstream.

[0376] It should be noted that d[x][y] is assumed to be the first transform coefficient block, where x = 0, ..., nTbW-1, y = 0, ..., nTbH-1, and nTbW and nTbH represent the width and height of the current block, respectively. Assume v[x] represents the block containing the LFNST coefficients, where x = 0, ..., nLfnstOutSize-1, and nLfnstOutSize = (nTbW>=8 && nTbH>=8)? 48:16.

[0377] In one possible implementation, LFNST can be directly performed on at least a portion of the transform coefficients in the first transform coefficient block using the horizontal scan sequence to obtain the LFNST coefficients. Specifically, as follows...

[0378]

[0379] The value of log2LfnstSize is determined as follows:

[0380] Log2LfnstSize=(nTbW>=8&&nTbH>=8)? 3:2 (9)

[0381] In another possible implementation, the scanning order may include a horizontal scanning order and a vertical scanning order. Here, the determination of the scanning order is related to the value of the MIP transpose indication parameter (denoted by isTransposed).

[0382] Optionally, in some embodiments, performing LFNST on at least a portion of the transform coefficients in the first transform coefficient block to obtain the LFNST coefficients may include:

[0383] When the value of the MIP transpose indicator parameter indicates that the sampling point input vector used in the MIP mode is transposed, LFNST is performed on at least some of the transform coefficients in the first transform coefficient block according to the vertical scan order to obtain the LFNST coefficients.

[0384] When the value of the MIP transpose indicator parameter indicates that the sampling point input vector used in the MIP mode is not transposed, LFNST is performed on at least some of the transform coefficients in the first transform coefficient block according to the horizontal scan order to obtain the LFNST coefficients.

[0385] Specifically, when isTransposed is equal to 0, the input vector of the sampling points used in MIP mode is not transposed. Therefore, LFNST is performed on at least a portion of the transform coefficients in the first transform coefficient block according to the horizontal scan order, as follows:

[0386]

[0387] When isTransposed equals 1, the input vector of the sampling points used in MIP mode is transposed. Then, LFNST can be performed on at least some of the transform coefficients in the first transform coefficient block according to the vertical scan order, as follows.

[0388]

[0389] The determination of log2LfnstSize is shown in equation (9) above.

[0390] Optionally, in some embodiments, performing LFNST on at least a portion of the transform coefficients in the first transform coefficient block to obtain the LFNST coefficients may include:

[0391] When the value of the MIP transpose indicator parameter indicates that the sampling point input vector used in the MIP mode is transposed, LFNST is performed on at least some of the transform coefficients in the first transform coefficient block according to the horizontal scan order to obtain the LFNST coefficients.

[0392] When the value of the MIP transpose indicator parameter indicates that the sampling point input vector used in the MIP mode is not transposed, LFNST is performed on at least some of the transform coefficients in the first transform coefficient block according to the vertical scan order to obtain the LFNST coefficients.

[0393] Specifically, when isTransposed is equal to 1, the sampling point input vector used in MIP mode is transposed. Then, LFNST is performed on at least some of the transformation coefficients in the first transformation coefficient block according to the horizontal scanning order, as shown in the above equation (10).

[0394] When isTransposed is equal to 0, the sampling point input vector used in MIP mode is not transposed. Then, LFNST is performed on at least some of the transformation coefficients in the first transformation coefficient block according to the vertical scanning order, as shown in equation (11) above.

[0395] The determination of log2LfnstSize is shown in equation (9) above.

[0396] In addition to the above, the LFNST in this application embodiment can be applied not only to MIP mode, but also to traditional intra-frame prediction modes (e.g., angle intra-frame prediction mode, DC mode, and PLANA mode). Its specific implementation is similar to that of MIP mode.

[0397] In some embodiments, determining the LFNST transform kernel candidate set based on the MIP parameters may include:

[0398] The value of the LFNST intra-prediction mode index number is determined based on the index value of the traditional intra-prediction mode.

[0399] Based on the value of the LFNST intra-frame prediction mode index number, the index value of the LFNST transform kernel candidate set is determined through the sixth preset lookup table;

[0400] The LFNST transform kernel candidate set is selected from multiple candidate LFNST transform kernel candidate sets based on the index value of the LFNST transform kernel candidate set.

[0401] It should be noted that the sixth preset lookup table is shown in Table 11. Based on the index value of the traditional intra-prediction mode, the value of `predModeIntra` can be determined; then, based on the value of `predModeIntra`, the value of `lfnstTrSetIdx` can be directly determined in conjunction with Table 11, thus determining the LFNST transform kernel candidate set selected for the current block. Here, the index value of the traditional intra-prediction mode can be assigned to `predModeIntra` to determine the value of `lfnstTrSetIdx`.

[0402] Table 11

[0403] predModeIntra lfnstTrSetIdx predModeIntra<0 1 0<=predModeIntra<=1 0 2<=predModeIntra<=12 1 13<=predModeIntra<=23 2 24<=predModeIntra<=44 3 45<=predModeIntra<=55 2 56<=predModeIntra<=80 1

[0404] The following will combine Figure 1 The encoder 100 shown here provides a detailed description of the encoding method of this application embodiment.

[0405] It should be understood that the current block (or "coded block") in the embodiments of this application can be a CU or a partition of the CU (e.g., a transform block). Specifically, when the intra-prediction unit 106 determines that the MIP mode is used to encode the current block, or when the intra-prediction unit 106 evaluates the MIP mode used to encode the current block using a rate-distortion optimization method, the intra-prediction unit 106 will obtain the intra-prediction block of the current block; wherein, the intra-prediction block of the current block includes the intra-prediction values ​​of at least one pixel (sample) within the current block.

[0406] Specifically, for the intra-prediction unit 106, it uses the MIP mode to obtain the intra-prediction block, and the steps are as follows:

[0407] First, the intra-prediction unit 106 obtains one or more reference pixels from the neighboring pixels of the current block. For example, it obtains them by downsampling the neighboring pixels, or by extracting them directly from the neighboring pixels.

[0408] Then, the intra-prediction unit 106 uses the acquired reference pixels, the MIP matrix, and shift parameters to determine one or more partial predicted pixels corresponding to the pixel positions within the current block. Here, the pixel positions can be preset pixel positions within the current block. For example, the pixel positions have uniform horizontal and vertical coordinate values. The shift parameters include shift amount parameters and shift offset parameters, and the shift parameters can be used for offset operations during the acquisition of the intra-prediction values ​​of partial pixels.

[0409] Finally, if the predicted pixel points corresponding to some pixels in the current block are obtained, the intra-prediction unit 106 also needs to obtain the predicted pixel points corresponding to the remaining pixels in the current block, excluding the some pixels. For example, the intra-prediction unit 106 can use an interpolation filter to obtain the predicted pixel points corresponding to the remaining pixels. Here, the input of the interpolation filter can be some pixels and adjacent pixels.

[0410] like Figure 6 The diagram illustrates a flowchart example of obtaining an intra-prediction block using MIP mode in an embodiment of this application. This process can be implemented on encoder 100, where "current block" refers to "coded block".

[0411] Step 601, the intra-frame prediction unit 106 obtains the neighboring pixels of the current block, for example, neighboring pixels are marked as... Figure 6 The gray-filled squares adjacent to the current block in step 601 are shown. The intra-frame prediction unit 106 obtains one or more reference pixels from neighboring pixels. Figure 6 In the example of step 601 shown, optionally, the intra-prediction unit 106 can calculate the average value of two adjacent pixels and use this average value as a reference pixel. Optionally, the intra-prediction unit 106 selects one adjacent pixel as a reference pixel every other adjacent pixel. For example, in... Figure 6 In the example of step 601 shown, the intra-prediction unit 106 selects 4 reference pixels from the 8 upper adjacent pixels of the current block, and selects another 4 reference pixels from the 8 left adjacent pixels of the current block.

[0412] The specific process of step 601 is as follows:

[0413] Intra-prediction unit 106 obtains the width and height of the current block from block segmentation unit 103, where the width and height are represented by variables cbWidth and cbHeight, respectively. As an example, intra-prediction unit 106 invokes a rate-distortion optimization mode determination method to determine the intra-prediction mode, where the current block is divided into one or more transform blocks. Let variables nTbW and nTbH represent the width and height of the transform block, respectively. When the MIP mode is used as the intra-prediction mode to obtain the intra-prediction value of the current block, intra-prediction unit 106 determines the block size index value of the MIP, i.e., the variable mipSizeId.

[0414] Optionally, the intra-prediction unit 106 determines the value of mipSizeId as follows:

[0415] - If both nTbW and nTbH are equal to 4, then mipSizeId is set to 0.

[0416] Otherwise, if cbWidth or cbHeight equals 4, then mipSizeId is set to equal to 1;

[0417] Otherwise, mipSizeId is set to equal to 2.

[0418] Specifically, when the current block size parameter is 8×8 (i.e., both cbWidth and cbHeight are equal to 8), mipSizeId is set to equal to 2.

[0419] Optionally, the intra-prediction unit 106 determines the value of mipSizeId as follows:

[0420] - If both nTbW and nTbH are equal to 4, then mipSizeId is set to 0.

[0421] Otherwise, if cbWidth or cbHeight equals 4, or if both cbWidth and cbHeight equal 8, then mipSizeId is set to 1.

[0422] Otherwise, mipSizeId is set to equal to 2.

[0423] Specifically, when the current block size parameter is 8×8, mipSizeId is set to equal to 1.

[0424] Furthermore, the intra-prediction unit 106 obtains the values ​​of boundarySize and predSize based on mipSizeId, as follows:

[0425] - If mipSizeId equals 0, then boundarySize is set to equal 2 and predSize is set to equal 4;

[0426] Otherwise, if mipSizeId equals 1, then boundarySize is set to equal 4 and predSize is set to equal 4;

[0427] Otherwise (if mipSizeId equals 2), boundarySize is set to equal to 4, and predSize is set to equal to 8;

[0428] Where boundarySize represents the number of reference pixels obtained from each of the reference pixels adjacent to the upper side and the reference pixels adjacent to the left side of the current block.

[0429] Intra-prediction unit 106 can also obtain the variable isTransposed to indicate the order of reference pixels stored in the buffer pTemp. For example, isTransposed equal to 0 indicates that intra-prediction unit 106 first renders the reference pixel obtained from the reference pixel adjacent to the top edge of the current block, and then renders the reference pixel obtained from the reference pixel adjacent to the left edge; otherwise, isTransposed equal to 1 indicates that intra-prediction unit 106 first renders the reference pixel obtained from the reference pixel adjacent to the left edge of the current block, and then renders the reference pixel obtained from the reference pixel adjacent to the top edge. Here, the value of isTransposed is sent to entropy coding unit 115 as one of the parameters of the MIP mode that needs to be encoded and written into the bitstream.

[0430] Intra-prediction unit 106 obtains the value of inSize to indicate the number of reference pixels used when using MIP mode, as shown in equation (1) above.

[0431] Intra-prediction unit 106 calls the following procedure to obtain a set of reference pixels (stored as an array p[x], where x is 0,…,inSize-1) using the reference pixels adjacent to the current block.

[0432] Intra-prediction unit 106 obtains nTbW reference pixels from the reference pixels adjacent to the upper side of the current block (e.g., stored in array refT), and nTbH reference pixels from the reference pixels adjacent to the left side of the current block (e.g., stored in array refL). Intra-prediction unit 106 performs downsampling processing on refT to obtain boundarySize reference pixels and stores the boundarySize reference pixels in refT. Intra-prediction unit 106 performs downsampling processing on refL to obtain boundarySize reference pixels and stores the boundarySize reference pixels in refL.

[0433] The intra-prediction unit 106 can also obtain the variable `isTransposed` to indicate the order of reference pixels stored in the buffer `pTemp`. For example, `isTransposed` equal to 0 (or `FALSE`) means that the intra-prediction unit 106 first renders the reference pixels obtained from the reference pixels adjacent to the top edge of the current block, and then renders the reference pixels obtained from the reference pixels adjacent to the left edge; otherwise, `isTransposed` equal to 1 (or `TRUE`) means that the intra-prediction unit 106 first renders the reference pixels obtained from the reference pixels adjacent to the left edge of the current block, and then renders the reference pixels obtained from the reference pixels adjacent to the top edge. The intra-prediction unit 106 can determine the value of `isTransposed` using rate-distortion optimization or based on a correlation comparison between adjacent reference pixels and the current block. The value of `isTransposed` is sent to the entropy coding unit 115 as one of the parameters of the MIP mode to be encoded and written into the bitstream.

[0434] Intra-frame prediction unit 106 places elements from refT and refL into buffer pTemp according to the order indicated by isTransposed.

[0435] Optionally, the intra-prediction unit 106 obtains p[x], where x = 0, ..., inSize-1, as follows:

[0436] - If mipSizeId equals 2, then p[x] = pTemp[x+1] - pTemp[0];

[0437] - Otherwise (mipSizeId is less than 2), p[0] = pTemp[0] - (1 << (BitDepth-1)), p[x] = pTemp[x] - pTemp[0];

[0438] BitDepth is the bit depth of the color component of the pixel in the current block. Here, the color component can be one of the RGB components, one of the YUV components, or one of the YCbCr components, for example, the Y component, etc.

[0439] Optionally, the intra-prediction unit 106 can obtain p[x], where x = 0, ..., inSize-1, as follows:

[0440] - If mipSizeId equals 2, then p[x] = pTemp[x+1] - pTemp[0];

[0441] - Otherwise (mipSizeId is less than 2), p[0] = (1 << (BitDepth-1)) - pTemp[0], p[x] = pTemp[x] - pTemp[0].

[0442] Optionally, the intra-prediction unit 106 can use a uniform calculation method to obtain the value of p[x] without having to determine the value of mipSizeId. For example, if (1<<(BitDepth-1)) is appended to the buffer pTemp as an additional element, the intra-prediction unit 106 will calculate p[x] as pTemp[x+1]-pTemp[0].

[0443] In step 602, the intra-prediction unit 106 obtains the MIP prediction value of the current block using a set of reference pixels and a MIP weighting matrix. The MIP weighting matrix is ​​selected from a set of predefined MIP weighting matrices based on the MIP mode index value (denoted as ModeId) corresponding to the MIP mode and the MIP block size index value (denoted as mipSizeId).

[0444] Intra-prediction unit 106 acquires the MIP prediction values ​​(denoted as predMip[x][y]) of a subset of predicted pixels corresponding to one or more pixel positions within the current block. Figure 6 In the example of step 602 shown, some predicted pixels are the pixels marked as gray-filled squares in the current block. The input to prediction module 601 is the reference pixel p[x] obtained in step 601. Prediction module 601 calculates some predicted pixels using the MIP weighting matrix and shift parameters; here, the shift parameters include shift amount parameters and shift offset parameters.

[0445] In one possible implementation, the prediction module 601 can make its coordinates equal to (x,y) and obtain the predicted pixel point represented by predMip[x][y]. The calculation formula of predMip[x][y] is shown in Equation (5) or Equation (6).

[0446] In equation (5) or (6), mWeight[i][j] is the MIP weighting matrix; here, the matrix elements can be predetermined constant values; or, they can be adaptively updated using, for example, a training method whose input is one or more coded images or blocks, or images from other bitstreams provided to encoder 100 by an external device; fO is the shift offset parameter used to determine oW; sW is the shift amount parameter; p[i] is the MIP input sample value calculated using reference pixels; pTemp[0] represents the value corresponding to index 0 in the first temporary reference value; ">>" is the binary right shift operator as defined in VVC. When the MIP mode is finally determined for intra-frame prediction of the current block, intra-frame prediction unit 106 can also send mWeight[i][j] to entropy coding unit 115. Entropy coding unit 115 can write mWeight[i][j] into one or more data units in the bitstream.

[0447] The prediction module 601 can determine the values ​​of sW and fO based on the size of the current block and the MIP mode used for the current block. In one example, the prediction module 601 uses a lookup table to retrieve the values ​​of sW and fO.

[0448] Optionally, the prediction module 601 can use Table 4 above to determine sW based on the current block size parameters and MIP mode.

[0449] Optionally, the prediction module 601 can also use Table 5 above to determine sW based on the size parameters of the current block.

[0450] Optionally, the prediction module 601 can also directly set sW to a constant value. For example, for blocks with various size parameters and different MIP modes, the prediction module 601 can set sW to 5; or, for blocks with various size parameters and different MIP modes, the prediction module 601 can set sW to 6; or, for blocks with various size parameters and different MIP modes, the prediction module 601 can set sW to 7.

[0451] Optionally, the prediction module 601 can use Table 6 above to determine fO based on the current block size parameters and MIP mode.

[0452] Optionally, the prediction module 601 can also use Table 7 above to determine fO based on the size parameters of the current block.

[0453] Optionally, the prediction module 601 can also directly set fO to a constant value (such as 0 to 100). For example, for blocks with various size parameters and different MIP modes, the prediction module 601 sets fO to 32; or, the prediction module 601 sets fO to 46; or, the prediction module 601 sets fO to 56; or, the prediction module 601 sets fO to 66.

[0454] Intra-prediction unit 106 can perform a clipping operation on the MIP prediction values ​​in predMip. When isTransposed equals 1 (or TRUE), the array predMip[x][y] of predSize×predSize (where x = 0,...,predSize-1, y = 0,...,predSize-1) is transformed into predTemp[y][x] = predMip[x][y], and then predMip = predTemp.

[0455] For example, optionally, when the size parameter of the current block is 8×8 (i.e., both cbWidth and cbHeight are equal to 8), the intra-prediction unit 106 determines that mipSizeId is equal to 2 and obtains an 8×8 predMip.

[0456] Optionally, when the size parameter of the current block is 8×8 (i.e., both cbWidth and cbHeight are equal to 8), the intra-prediction unit 106 determines that mipSizeId is equal to 1 and obtains a 4×4 predMip.

[0457] Step 603, the intra-prediction unit 106 obtains the intra-prediction value of the current block (stored in the array as predSamples[x][y], where x = 0,...,nTbW-1, y = 0,...,nTbH-1), as follows:

[0458] If the intra-prediction unit 106 determines that nTbW is greater than predSize or nTbH is greater than predSize, then the intra-prediction unit 106 calls the upsampling procedure to obtain predSamples using predMip. The intra-prediction unit 106 obtains the predicted pixel points corresponding to the remaining pixels in the current block, excluding some pixels. Figure 6 In this process, the intra-prediction unit 106 can use the filtering module 602 to obtain the predicted pixels corresponding to the remaining pixels in the current block, excluding some pixels. The input to the filtering module 602 is the pixel marked as a gray-filled square in step 602. The filtering module 602 can use one or more interpolation filters to obtain the predicted pixels corresponding to the remaining pixels in the current block, excluding some pixels, using this input. For example, the input may include a reference pixel and partial predicted pixels at the positions of one or more pixels in the current block; or, the input may include a neighboring pixel and partial predicted pixels at the positions of one or more pixels in the current block; or, the input may include a reference pixel, a neighboring pixel, and partial predicted pixels at the positions of one or more pixels in the current block. For example, optionally, when the size of the current block is 8×8 (i.e., cbWidth and cbHeight are both equal to 8), the intra-prediction unit 106 determines that mipSizeId is equal to 1, and the intra-prediction unit 106 applies an upsampling process to 4×4predMip to obtain an 8×8 intra-prediction block of the current block.

[0459] Otherwise, the intra-prediction unit 106 sets the intra-prediction block of the current block to be equal to the MIP prediction block of the current block, that is, sets predSamples[x][y] (where x = 0,...,nTbW-1, y = 0,...,nTbH-1) to predMip[x][y]. For example, specifically, when the size parameter of the current block is 8×8 (i.e., cbWidth and cbHeight are both equal to 8), the intra-prediction unit 106 determines that mipSizeId is equal to 2, and the intra-prediction unit 106 obtains the predSamples of the current block, whose size is equal to 8×8 (i.e., cbWidth and cbHeight are both equal to 8).

[0460] Thus, according to Figure 6 As shown in the flowchart, after step 603, the intra-prediction unit 106 can obtain the intra-prediction block of the current block (i.e., CU), that is, determine the intra-prediction value of at least one pixel in the current block.

[0461] Prediction unit 102 outputs the intra-predicted block of the current block. First adder 107 calculates the difference between the current block and the intra-predicted block of the current block in the output of segmentation unit 101, i.e., the residual block (i.e., residual CU). Transform unit 108 reads the residual block and performs one or more transform operations on the residual block to obtain coefficients. Quantization unit 109 quantizes the coefficients and outputs the quantization coefficients (i.e., levels).

[0462] Transform unit 108 performs a first transformation on the residual block, for example, an integer transformation initially designed based on DCT. Transform unit 108 determines whether a second transformation is allowed to be applied to the block. If it is determined that application is allowed, transform unit 108 further determines whether to apply the second transformation to the coefficients obtained after performing the first transformation. LFNST is an example of a second transformation.

[0463] See Figure 7 This illustrates a schematic diagram of a process framework for executing LFNST according to an embodiment of this application. Figure 7 In the diagram, block 701 represents the residual block.

[0464] Step 701, the transformation unit 108 performs the first transformation (i.e., the core transformation) on block 701, for example, an integer transformation initially designed based on DCT, to obtain block 702 containing the transformation coefficients (called "first transformation coefficients") corresponding to the first transformation.

[0465] In step 702, the transformation unit 108 determines whether a secondary transformation, such as LFNST, is allowed on all or part of the first transformation coefficients in block 702.

[0466] For example, optionally, if the intra-prediction mode of the transform block containing LFNST is MIP mode, and the minimum values ​​of the width and height of the transform block are greater than or equal to 8, then the transform unit 108 is allowed to perform LFNST on the transform block.

[0467] Optionally, if the intra-prediction mode of the transform block containing LFNST is MIP mode, and the value of mipSizeId is equal to a preset value (e.g., 2) or one of multiple preset values ​​(e.g., 1, 2), then the transform unit 108 is allowed to perform LFNST on the transform block.

[0468] When transform unit 108 is allowed to perform LFNST on a transform block, transform unit 108 determines LFNST parameters, such as lfnst_index. When lfnst_index equals 0, it means that transform unit 108 does not perform LFNST on the transform block. When lfnst_index is greater than 0, it means that LFNST is performed on the transform block using a transform kernel from the LFNST transform kernel candidate set.

[0469] Transform unit 108 can use a conventional rate-distortion optimization method to determine the value of lfnst_index. First, transform unit 108 obtains the first cost result without performing LFNST on the transform block.

[0470] Transform unit 108 selects a candidate set of LFNST transform kernels based on MIP parameters. MIP parameters may include at least one of the following: MIP mode index value (i.e., modeId) and MIP transpose indicator parameter (i.e., isTransposed).

[0471] Transform unit 108 sets the value of variable predModeIntra to be equal to the value of modeId. Transform unit 108 selects a subset of the first transform coefficients in block 702, for example, the coefficients in sub-blocks 7001, 7002, and 7003, and determines the transform kernel used to perform LFNST on the coefficients in the sub-blocks. The size of the sub-block can be a preset value, such as 8×8. Transform unit 108 determines the index value (i.e., lfnstTrSetIdx) of the LFNST transform kernel candidate set based on the value of predModeIntra. For block coding in MIP mode, transform unit 108 can use a lookup table as shown in Table 10 to obtain the value of lfnstTrSetIdx.

[0472] Transform unit 108 can use a conventional rate-distortion optimization method to determine the value of lfnst_index (i.e., a value greater than 0 indicates that the transform kernel used in the LFNST transform kernel candidate set is indicated by lfnstTrSetIdx). Transform unit 108 sets the value of lfnst_index to the index number of the transform kernel in the LFNST transform kernel candidate set, which causes the cost function to have a minimum value and also obtains a second cost result.

[0473] If the first cost result is greater than the second cost result, then the transform unit 108 determines that LFNST will be performed on the transform block and sends the lfnst_index indicating the index number of the transform kernel to the entropy coding unit 115. Otherwise, if the transform unit 108 determines that LFNST will not be performed on the transform block, then the lfnst_value is set to 0 and the lfnst_index indicating that LFNST will not be performed is sent to the entropy coding unit 115.

[0474] When lfnst_index is not equal to 0, transform unit 108 performs LFNST on the transform block using coefficients from sub-blocks 7001, 7002, and 7003, obtains LFNST coefficients, and places the LFNST coefficients into sub-blocks 7011, 7012, and 7013 of block 703. Assume d[x][y] is the block of the first transform coefficients, where x = 0,...,nTbW-1, y = 0,...,nTbH-1, where nTbW and nTbH are the width and height of block 703, respectively. Assume v[x] is the block of LFNST coefficients, where x = 0,...,nLfnstOutSize-1, and nLfnstOutSize = (nTbW>=8 && nTbH>=8)? 48:16.

[0475] Optionally, the transformation unit 108 uses the following instructions to obtain the block 703 of transformation coefficients, and transmits the block 703 of LFNST coefficients in d[x][y] to the quantization unit 109, as shown in equation (8) above. Wherein, log2LfnstSize is as shown in equation (9) above.

[0476] Optionally, when isTransposed equals 0, the transformation unit 108 uses the following instructions to obtain the block 703 of LFNST coefficients and transmits the block 703 of LFNST coefficients in d[x][y] to the quantization unit 109, as shown in the above formula (10);

[0477] When isTransposed equals 1, the transformation unit 108 uses the following instruction to obtain the block 703 of LFNST coefficients and transmits the block 703 of LFNST coefficients in d[x][y] to the quantization unit 109, as shown in the above formula (11).

[0478] Optionally, when isTransposed equals 1, the transformation unit 108 uses the following instructions to obtain the block 703 of LFNST coefficients and transmits the block 703 of LFNST coefficients in d[x][y] to the quantization unit 109, as shown in the above formula (10);

[0479] When isTransposed equals 0, the transformation unit 108 uses the following instruction to obtain the block 703 of LFNST coefficients and transmits the block 703 of LFNST coefficients in d[x][y] to the quantization unit 109, as shown in the above formula (11).

[0480] The determination of log2LfnstSize is shown in equation (9) above.

[0481] In addition, the transform unit 108 can also determine the value of lfnst_index for transform blocks encoded in conventional intra-prediction modes (i.e., angle intra-prediction mode, DC mode, and PLANA mode, etc.). Similar to the rate-distortion optimization method that determines the value of lfnst_index for transform blocks in MIP mode, the first cost result is determined without performing LFNST on the transform block.

[0482] Transform unit 108 uses a lookup table to determine the value of lfnstTrSetIdx based on the mode index value of the traditional intra-prediction mode (assigned to predModeIntra), that is, to determine the LFNST transform kernel candidate set selected for the current block, as shown in Table 11 above.

[0483] Then, the transform unit 108 can determine the value of lfnst_index using a conventional rate-distortion optimization method (i.e., a value greater than 0 indicates that the transform kernel used in the LFNST transform kernel candidate set is indicated by lfnstTrSetIdx). The transform unit 108 sets the value of lfnst_index to the index number of the transform kernel in the LFNST transform kernel candidate set, which causes the cost function to have a minimum value and also obtains a second cost result.

[0484] If the first cost result is greater than the second cost result, then the transform unit 108 determines that LFNST will be performed on the transform block and sends the lfnst_index indicating the index number of the transform kernel to the entropy coding unit 115. Otherwise, if the transform unit 108 determines that LFNST will not be performed on the transform block, then the lfnst_value is set to 0 and the lfnst_index indicating that LFNST will not be performed is sent to the entropy coding unit 115.

[0485] When lfnst_index is not equal to 0, transform unit 108 performs LFNST on the transform block using coefficients from sub-blocks 7001, 7002, and 7003, obtains LFNST coefficients, and places the LFNST coefficients into sub-blocks 7011, 7012, and 7013 of block 703. Assume d[x][y] is the block of the first transform coefficients, where x = 0,...,nTbW-1, y = 0,...,nTbH-1, where nTbW and nTbH are the width and height of block 703, respectively. Assume v[x] is the block of LFNST coefficients, where x = 0,...,nLfnstOutSize-1, and nLfnstOutSize = (nTbW>=8 && nTbH>=8)? 48:16.

[0486] Optionally, when predModeIntra is less than or equal to 34, the conversion unit 108 uses the following instructions to obtain the block 703 of LFNST coefficients and transmits the block 703 of LFNST coefficients in d[x][y] to the quantization unit 109, as shown in the above formula (10);

[0487] When predModeIntra is greater than 34, the conversion unit 108 uses the following instruction to obtain the block 703 of LFNST coefficients and transmits the block 703 of LFNST coefficients in d[x][y] to the quantization unit 109, as shown in the above formula (11).

[0488] Optionally, when predModeIntra is greater than 34, the conversion unit 108 uses the following instructions to obtain the block 703 of LFNST coefficients and transmits the block 703 of LFNST coefficients in d[x][y] to the quantization unit 109, as shown in the above formula (10);

[0489] When predModeIntra is less than or equal to 34, the conversion unit 108 uses the following instruction to obtain the block 703 of LFNST coefficients and transmits the block 703 of LFNST coefficients in d[x][y] to the quantization unit 109, as shown in the above formula (11).

[0490] The determination of log2LfnstSize is shown in equation (9) above.

[0491] Further, the dequantization unit 110 performs a scaling operation on the quantization coefficients to output reconstructed coefficients. The inverse transform unit 111 performs one or more inverse transforms corresponding to the transforms in the transform unit 108 and outputs the reconstructed residuals. The second adder 112 calculates the reconstructed CU by adding the reconstructed residuals to the intra-predicted block from the current block in the prediction unit 102. The second adder 112 also sends its output to the prediction unit 102 as an intra-predicted reference.

[0492] The output of the filtering unit 113 is a decoded image or sub-image, which is sent to the DPB unit 114. The DPB unit 114 outputs the decoded image according to timing and control information. The images stored in the DPB unit 114 can also be used as a reference for the prediction unit 102 to perform inter-frame prediction or intra-frame prediction.

[0493] The entropy coding unit 115 converts the parameters (necessary for obtaining the decoded image) from the units in the encoder 100, along with control parameters and supplementary information, into a binary representation, and writes such a binary representation into the generated bitstream (or “video bitstream”) according to the syntax structure of each data unit, as shown in Tables 9A and 9B.

[0494] Using the syntax structure in Table 9A, when the minimum value of the width and height of the transform block in the coding unit is greater than or equal to 8 (i.e., both width and height are greater than or equal to 8), the entropy coding unit 115 encodes the lfnst_index of the coding unit using the MIP mode. Another optional condition can be set as follows: the width and height are greater than or equal to a first value (e.g., 8), and the sum of the width and height is greater than or equal to a second value (e.g., 16).

[0495] Using the syntax structure in Table 9B, when the mipSizeId of the coding unit is equal to 2, the entropy coding unit 115 encodes the lfnst_index of the coding unit using the MIP mode. Another optional condition can be set: the mipSizeId of the coding unit is equal to one of several values ​​(e.g., 1 and 2).

[0496] See Figure 8 This illustrates a flowchart of an embodiment of this application for encoding LFNST parameters. Figure 8 As shown, the process may include:

[0497] S801: Entropy coding unit 115 checks the above conditions to determine whether to encode the LFNST parameter.

[0498] S802: Entropy coding unit 115 uses context-based adaptive binary arithmetic coding to encode the LFNST parameters.

[0499] It should be noted that in step S801, the entropy coding unit 115 checks and determines the LFNST parameter, such as the value of lfnst_index; then in step S802, the value of lfnst_index can be encoded using CABAC with the descriptor "ae(v)" as shown in Tables 9A and 9B.

[0500] In this embodiment, the encoder 100 can be a computing device having a processor and a storage medium for recording encoding programs. When the processor reads and runs the encoding program, the encoder 100 reads the input video and generates a corresponding bitstream. Alternatively, the encoder 100 can also be a computing device having one or more chips. Units implemented as integrated circuits on the chips communicate with... Figure 1 The corresponding units in the text have similar connections and data exchange capabilities, thus possessing similar functions.

[0501] This embodiment provides an encoding method that determines prediction parameters for the current block, including prediction mode parameters. When the prediction mode parameters indicate that the matrix-based intra-prediction (MIP) mode is used to determine the intra-prediction value of the chrominance component of the current block, the method acquires the adjacent sample values ​​of the current block and determines the MIP input sample value of the current block based on the adjacent sample values. Based on the MIP input sample value, the MIP weighting matrix, and shift parameters, the method determines the MIP prediction value of the chrominance component of the current block. The method filters the MIP prediction value to determine the intra-prediction value of the chrominance component of the current block. Based on the intra-prediction value of the chrominance component of the current block, the method determines the prediction residual value of the chrominance component of the current block. The method performs a low-frequency non-separable quadratic transform (LFNST) on the prediction residual value to determine the LFNST parameters. The method encodes the LFNST parameters and writes them into the bitstream. In this way, for MIP mode, the complexity can be reduced while ensuring coding performance, and the storage space required during the coding process can be reduced, effectively improving coding efficiency. In addition, when LFNST technology is applied to MIP mode prediction, the introduction of MIP parameters makes the LFNST transform more flexible, further improving coding efficiency.

[0502] In another embodiment of this application, see Figure 9 The diagram illustrates a flowchart of a decoding method provided in an embodiment of this application. Figure 9 As shown, the method may include:

[0503] S901: Parse the bitstream to obtain the prediction parameters and LFNST parameters of the current block, wherein the prediction parameters include prediction mode parameters.

[0504] It should be noted that the image to be decoded can be divided into multiple image blocks, and each image block to be decoded can be called a decoding block. Here, each decoding block may include a first image component, a second image component, and a third image component; and the current block is the decoding block in the video image for which the prediction of the first image component, the second image component, or the third image component is currently to be performed.

[0505] It should also be noted that the prediction parameters may include prediction mode parameters, which indicate the prediction mode used for the current block. Different prediction modes correspond to different prediction mode parameters. Prediction modes typically include inter-frame prediction modes, traditional intra-frame prediction modes, and non-traditional intra-frame prediction modes. Traditional intra-frame prediction modes may include DC mode, PLANA mode, and angle intra-frame prediction mode, while non-traditional intra-frame prediction modes may include MIP mode, CCLM mode, IBC mode, and PLT mode. In other words, the encoder selects the optimal prediction mode to encode the current block. During this process, the prediction mode of the current block is determined, thus obtaining the corresponding prediction mode parameters. These prediction parameters, including the prediction mode parameters, are then written into the bitstream and transmitted by the encoder to the decoder.

[0506] In this way, the decoder can obtain the prediction parameters of the current block by parsing the bitstream, and the prediction mode parameters included in the obtained prediction parameters can be used to determine whether the current block uses MIP mode.

[0507] S902: When the prediction mode parameter indicates that the intra-prediction value of the chroma component of the current block is determined using the matrix-based intra-prediction (MIP) mode, the adjacent sample values ​​of the current block are obtained, and the MIP input sample value of the current block is determined based on the adjacent sample values ​​of the current block.

[0508] It should be noted that, for the current block, this embodiment of the application utilizes the MIP mode to perform intra-frame prediction of the chroma components of the current block. In this process, it is first necessary to obtain the neighboring sample values ​​of the current block; then, based on the neighboring sample values ​​of the current block, the MIP input sample values ​​of the current block are determined.

[0509] The adjacent sample values ​​of the current block can include the left adjacent sample value and the upper adjacent sample value of the current block. In other words, the MIP input sample value of the current block can be determined based on the left adjacent sample value and the upper adjacent sample value of the current block.

[0510] In some embodiments, determining the MIP input sample value of the current block based on the adjacent sample values ​​of the current block may include:

[0511] Determine the block size index value of the current block based on the current block's size parameters;

[0512] The adjacent sampled values ​​of the current block are downsampled and filtered to obtain a first temporary reference value;

[0513] When the block size index value of the current block is within a preset range, a second constant value is determined based on the bit depth of the adjacent sample values ​​of the current block; the value corresponding to index number 0 in the MIP input sample value is set to be equal to the difference between the second constant value and the value corresponding to index number 0 in the first temporary reference value; the value corresponding to index number i in the MIP input sample value is set to be equal to the difference between the value corresponding to index number i in the first temporary reference value and the value corresponding to index number 0 in the first temporary reference value, where i is an integer greater than 0;

[0514] When the block size index value of the current block is outside the preset range, the value corresponding to the index number j in the MIP input sample value is set to the difference between the value corresponding to the index number j+1 in the first temporary reference value and the value corresponding to the index number 0 in the first temporary reference value, where j is an integer greater than or equal to 0.

[0515] Furthermore, when the block size index value of the current block is within a preset range, the method may further include:

[0516] Based on the bit depth of the adjacent sampled values ​​of the current block, a second constant value is determined; the value corresponding to index 0 in the MIP input sample value is set to be equal to the difference between the value corresponding to index 0 in the first temporary reference value and the second constant value; the value corresponding to index i in the MIP input sample value is set to be equal to the difference between the value corresponding to index i in the first temporary reference value and the value corresponding to index 0 in the first temporary reference value, where i is an integer greater than 0.

[0517] It should be noted that after downsampling and filtering the adjacent sampled values ​​of the current block, the first temporary reference value can be obtained. Specifically, for the first temporary reference value, the filtered adjacent sampled values ​​can be cached in a buffer (represented by pTemp). Among them, the value corresponding to index 0 in the first temporary reference value refers to pTemp[0], and the value corresponding to index i in the first temporary reference value refers to pTemp[i].

[0518] In this embodiment, the prediction parameters, in addition to the prediction mode parameters, may also include: the size parameter of the current block. The size parameter of the current block may include: the width (represented by nTbW) and the height (represented by nTbH) of the current block. Furthermore, based on the size parameter of the current block, the block size index value (i.e., mipSizeId) of the current block can be determined.

[0519] It should also be noted that whether the current block's size parameter value is within the preset range can be determined by whether the current block's block size index value (represented by mipSizeId) is within the preset range. Specifically, when mipSizeId = 0 or 1, it indicates that the current block's block size index value is within the preset range, meaning the current block's size parameter value is within the preset range; when mipSizeId = 2, it indicates that the current block's block size index value is outside the preset range, meaning the current block's size parameter value is not within the preset range.

[0520] In other words, the MIP input sample value is determined by the buffer (represented by pTemp), the block size index of the current block (represented by MipSizeId), and the bit depth of the adjacent sample values ​​of the current block (represented by BitDepth). Moreover, the number of input samples contained in the MIP input sample value is only related to the block size index of the current block. Finally, the value corresponding to index number x in the MIP input sample value can be obtained (represented by p[x]).

[0521] Furthermore, in some embodiments, determining the second constant value based on the bit depth of adjacent sampled values ​​of the current block may include:

[0522] Set the second constant value to an integer power equal to 2, where the power is equal to the bit depth of the adjacent sample value of the current block minus 1.

[0523] Alternatively, in some embodiments, determining the second constant value based on the bit depth of adjacent sampled values ​​of the current block may include:

[0524] The second constant value is obtained by left-shifting the binary bits of "1", wherein the number of bits shifted left is equal to the bit depth of the adjacent sample values ​​of the current block minus 1.

[0525] In other words, after obtaining the bit depth (represented by BitDepth) of the adjacent sample values ​​of the current block, the second constant value can be expressed as 1<<(BitDepth-1) or 2^(BitDepth-1). Thus, if the size parameter of the current block is within a preset range, the MIP input sample value of the current block can be determined by combining the second constant value.

[0526] It should also be noted that the MIP input sample is a matrix vector used for matrix multiplication operations. Current related technical solutions are determined by the buffer (represented by pTemp), the type of the current block (i.e., the block size index value of the current block, represented by mipSizeId), the bit depth of the adjacent sample values ​​of the current block (represented by BitDepth), and the number of MIP input samples, and finally obtain the value corresponding to the index x in the MIP input sample value (represented by p[x]).

[0527] In one possible implementation, when mipSizeId = 0 or 1, pTemp[0] can be obtained by subtracting pTemp[0] from 1 << (BitDepth-1); then when x is not equal to 0, pTemp[0] is obtained by subtracting pTemp[x] from pTemp[x]. Specifically, as shown in equation (2) above.

[0528] In another possible implementation, when mipSizeId = 0 or 1, p[0] can be obtained by subtracting 1 << (BitDepth-1) from pTemp[0]; then when x is not equal to 0, p[x] can be obtained by subtracting pTemp[0] from pTemp[x]. Specifically, as shown in equation (3) above.

[0529] In another possible implementation, when mipSizeId = 2, p[x] is obtained by subtracting pTemp[0] from pTemp[x+1]. Specifically, as shown in equation (4) above.

[0530] In addition, embodiments of this application can also use a unified calculation method to obtain the value of p[x] without having to determine the value of mipSizeId. In this case, embodiments of this application can directly set p[x] to be equal to pTemp[x+1]-pTemp[0], where x=0,…,inSize-1, by appending 1<<(BitDepth-1) as an additional element to the end of the buffer pTemp.

[0531] In this way, after determining the MIP input sample value of the current block based on the adjacent sample values ​​of the current block, the MIP prediction value of the current block can be further determined.

[0532] S903: Determine the MIP prediction value of the chromaticity component of the current block based on the MIP input sample value, the MIP weighting matrix, and the shift parameter.

[0533] The MIP prediction value is the prediction value of a portion of the sampling points in the chromaticity component of the current block.

[0534] It should be noted that the shift parameters may include shift offset parameters and shift amount parameters. In this case, for S903, determining the MIP prediction value of the chromaticity component of the current block based on the MIP input sample value, the MIP weighting matrix, and the shift parameters may include:

[0535] Based on the value of the shift offset parameter, determine the product of the sum of the MIP input sample values ​​and the shift offset parameter, wherein the value of the shift offset parameter is a fixed constant;

[0536] Based on the value of the shift quantity parameter, a first constant value is determined, wherein the value of the shift quantity parameter is a fixed constant;

[0537] Set the value of the first offset to the difference between the first constant value and the product;

[0538] Based on the prediction parameters, determine the MIP weighting matrix of the current block;

[0539] The MIP prediction value of the chromaticity component of the current block is determined based on the MIP weighting matrix, the MIP input sample value, the shift quantity parameter, and the first offset.

[0540] It should be noted that the shift offset parameter can also be called the shift compensation parameter or the offset factor, and can be represented by fO. In the embodiments of this application, the value of the shift offset parameter can be set to a fixed constant, such as 32, 46, 56, 66, or 32, etc.; or the value of the shift offset parameter can be set to be related to the shift offset parameter table, and the value of the shift offset parameter can be determined by looking up the table. No limitation is made here.

[0541] It should also be noted that the shift quantity parameter can also be called shift factor, shift bit number, or weight shift value, and can be represented by sW, shift, or weight shift. In the embodiments of this application, the shift quantity parameter is usually represented by sW. In the embodiments of this application, the value of the shift quantity parameter can be set to a fixed constant, such as 5, 6, or 7. Alternatively, the value of the shift quantity parameter can be set to be related to a shift quantity parameter table, and the value of the shift quantity parameter can be determined by looking up the table; no limitations are imposed here.

[0542] In addition, the first offset can be represented by oW, and the first offset is related to both the shift quantity parameter and the shift offset parameter.

[0543] It should also be understood that prediction parameters can include prediction mode parameters and the size parameter of the current block. When the prediction mode parameters indicate that the chroma components of the current block use the MIP mode to determine the intra-frame prediction value, a weight matrix table can be pre-built and stored in memory or a storage unit. This memory or storage unit can be integrated into the decoder or set separately. In this way, based on the block size index value (mipSizeId) and the MIP mode index value (modeId) of the current block, the MIP weight matrix (or MIP weight matrix, or simply MIP matrix) to be used by the current block can be determined by looking up the table, denoted by mWeight[x][y].

[0544] It should also be noted that a shift quantity parameter table can be pre-established in the decoder, and this shift quantity parameter table is also stored in memory or a storage unit. This memory or storage unit can be integrated into the decoder or can be set up separately. In the embodiments of this application, the determination of the shift quantity parameter (sW) can include the following methods:

[0545] In one possible implementation, the shift amount parameter can be different for different block sizes and different MIP mode index values. In some embodiments, the method may further include:

[0546] When the chroma component of the current block uses MIP mode to determine the intra-prediction value, the bitstream is parsed to obtain the MIP mode index value of the current block;

[0547] Based on the MIP mode index value, the value corresponding to the MIP mode index value is queried from the first preset lookup table, wherein the first preset lookup table is used to record the correspondence between the MIP mode index value and the value of the shift amount parameter;

[0548] The retrieved value is determined as the value of the shift quantity parameter.

[0549] It should be noted that the value of the shift amount parameter can be looked up based on the current block size index (represented by mipSizeId) and the MIP mode index (modeId). As shown in the first preset lookup table in Table 4 above, the shift amount parameter required for matrix multiplication can be determined by looking up different mipSizeId and modeId.

[0550] However, on the decoder side, Table 4 needs to be stored in memory or a storage unit in the form of a lookup table; however, storage comes at a cost, and the lookup process also comes at a cost; since the shift factor in Table 4 is related to the block size of the current block and the MIP mode index value, it increases memory usage and computational complexity.

[0551] To reduce memory usage and computational complexity, the present application simplifies the method for determining the shift factor.

[0552] In another possible implementation, the shift amount parameter can be set to a fixed constant, independent of the block size index value and the MIP mode index value. For example, the shift amount parameter can be set to 5 for different block size index values ​​and different MIP mode index values; or, the shift amount parameter can be set to 6 for different block size index values ​​and different MIP mode index values; or, the shift amount parameter can be set to 7 for different block size index values ​​and different MIP mode index values. In this embodiment, preferably, the shift amount parameter is equal to 6, but this is not a limitation.

[0553] In yet another possible implementation, the method may further include, for the value of the shift amount parameter:

[0554] Determine the block size index value of the current block based on the current block's size parameters;

[0555] The value of the shift quantity parameter is determined based on the block size index value of the current block.

[0556] It should be noted that the block size index value of the current block can be determined based on the size parameter of the current block. In one possible implementation, determining the block size index value of the current block based on the size parameter of the current block may include:

[0557] When the width and height of the current block are both equal to 4, the block size index value of the current block is set to 0;

[0558] When the width and height of the current block are both equal to 8, or when one of the width and height of the current block is equal to 4, the block size index value of the current block is set to 1;

[0559] If the width and height of the current block do not meet the aforementioned conditions, the block size index value of the current block is set to 2.

[0560] Thus, after determining the block size index value of the current block, the value of the shift amount parameter can be further determined based on the block size index value of the current block.

[0561] Optionally, in some embodiments, determining the value of the shift amount parameter based on the block size index value of the current block may include:

[0562] When the block size index value is equal to 0, 1 and 2 respectively, the value of the shift quantity parameter corresponding to the block size index value of the current block is determined to be equal to 5, 6 and 5 respectively.

[0563] Optionally, in some embodiments, determining the value of the shift amount parameter based on the block size index value of the current block may include:

[0564] The value of the shift quantity parameter is set to be equal to the ratio between the width or height of the current block and a first preset value corresponding to the block size index value of the current block.

[0565] Here, the first preset value represents the number of MIP input sample values ​​obtained from the boundary of the current block. In this case, the method may further include:

[0566] When the block size index value of the current block is equal to 0, 1 and 2 respectively, the first preset value corresponding to the block size index value of the current block is determined to be equal to 2, 4 and 4 respectively.

[0567] In other words, when the first preset value represents the number of MIP input sample values ​​obtained from the boundary of the current block, if the block size index of the current block is equal to 0, then the corresponding first preset value is equal to 2; if the block size index of the current block is equal to 1, then the corresponding first preset value is equal to 4; if the block size index of the current block is equal to 2, then the corresponding first preset value is equal to 4. Thus, the value of the shift quantity parameter can be determined according to the ratio between the width or height of the current block and the corresponding first preset value.

[0568] Optionally, in some embodiments, determining the value of the shift amount parameter based on the block size index value of the current block may include:

[0569] The value of the shift quantity parameter is set to be equal to the ratio between the width or height of the current block and a second preset value corresponding to the block size index value of the current block.

[0570] Here, the second preset value represents the size of the MIP predicted block of the current block, obtained by directly calculating using the MIP weighting matrix. In this case, the method may further include:

[0571] When the block size index value of the current block is equal to 0, 1 and 2 respectively, the second preset value corresponding to the block size index value of the current block is determined to be equal to 4, 4 and 8 respectively.

[0572] In other words, when the second preset value represents the size of the MIP predicted block of the current block obtained by directly calculating using the MIP weighted matrix, if the block size index value of the current block is equal to 0, then the corresponding second preset value is equal to 4; if the block size index value of the current block is equal to 1, then the corresponding second preset value is equal to 4; if the block size index value of the current block is equal to 2, then the corresponding second preset value is equal to 8. Thus, the value of the shift amount parameter can be determined based on the ratio between the width or height of the current block and the corresponding second preset value.

[0573] In another possible implementation, the shift amount parameter table can be minimized, and a lookup table is still used to determine the value of the shift amount parameter. Optionally, in some embodiments, determining the value of the shift amount parameter based on the block size index of the current block may include:

[0574] Based on the block size index value, the value corresponding to the block size index value is retrieved from the second preset lookup table, wherein the second preset lookup table is used to record the correspondence between the block size index value and the value of the shift quantity parameter;

[0575] The retrieved value is determined as the value of the shift quantity parameter.

[0576] It should be noted that the value of the shift amount parameter can be queried based solely on the block size index value of the current block (represented by mipSizeId). As shown in the second preset lookup table in Table 5 above, each type of block size index value can correspond to a fixed value. In other words, the size of each block or the set of sizes of each block can have a fixed value for the shift amount parameter as shown in Table 5.

[0577] According to Table 5 above, when the block size index value of the current block is equal to 0, 1 and 2 respectively, the shift quantity parameter corresponding to the block size index value can be determined to be equal to 5, 6 and 5 respectively.

[0578] In this way, by simplifying the determination of the shift quantity parameter, especially by minimizing the shift quantity parameter table or fixing the value of the shift quantity parameter, the storage of the lookup table can be minimized, and the memory occupied by the shift factor table in MIP mode can be reduced without increasing the computational complexity.

[0579] Furthermore, a shift offset parameter table can be pre-established in the decoder, and this shift offset parameter table is also stored in memory or a storage unit. This memory or storage unit can be integrated into the decoder or can be set up separately. In this embodiment, the determination of the shift offset parameter (fO) can include the following methods:

[0580] In one possible implementation, the shift offset parameter may be different for different block sizes and different MIP mode index values. In some embodiments, the method may further include:

[0581] When the chroma components of the current block are used to determine the intra-prediction value using MIP mode, the bitstream is parsed to determine the MIP mode index value of the current block.

[0582] Based on the MIP mode index value, the value corresponding to the MIP mode index value is retrieved from the third preset lookup table, wherein the third preset lookup table is used to record the correspondence between the MIP mode index value and the value of the shift offset parameter;

[0583] The retrieved value is determined as the value of the shift offset parameter.

[0584] It should be noted that the value of the shift offset parameter can be looked up based on the current block size index (represented by mipSizeId) and the MIP mode index (modeId). As shown in the third preset lookup table in Table 6 above, the shift offset parameter required for matrix multiplication can be determined by looking up the table for different mipSizeId and modeId.

[0585] However, on the decoder side, Table 6 also needs to be stored in memory or storage unit in the form of a lookup table; however, storage comes at a cost, and the lookup process also comes at a cost; since the value of the shift offset parameter in Table 6 is related to the block size of the current block and the MIP mode index value, it increases memory usage and computational complexity.

[0586] To reduce memory usage and computational complexity, the method for determining the shift offset parameters is also simplified in this embodiment.

[0587] In another possible implementation, the shift offset parameter can be set to a fixed constant, independent of the block size index and the MIP mode index. Typically, this fixed constant ranges from 0 to 100. For example, the shift offset parameter can be set to 32 for different block size indexes and different MIP mode indexes; or, it can be set to 46 for different block size indexes and different MIP mode indexes; or, it can be set to 56 for different block size indexes and different MIP mode indexes; or, it can be set to 66 for different block size indexes and different MIP mode indexes. In this embodiment, preferably, the shift offset parameter is equal to 32, but this is not a limitation.

[0588] In yet another possible implementation, the method may further include, for the value of the shift offset parameter:

[0589] Determine the block size index value of the current block based on the current block's size parameters;

[0590] The value of the shift offset parameter is determined based on the block size index value of the current block.

[0591] It should be noted that the block size index value of the current block can be determined based on the current block size parameter; then, the value of the shift offset parameter can be further determined based on the current block size index value.

[0592] Optionally, in some embodiments, determining the value of the shift offset parameter based on the block size index value of the current block may include:

[0593] When the block size index value is equal to 0, 1 and 2 respectively, the shift offset parameter corresponding to the block size index value of the current block is determined to be equal to 34, 23 and 46 respectively.

[0594] Optionally, in some embodiments, the shift offset parameter table can be minimized, and a lookup table method can still be used to determine the offset factor. Optionally, in some embodiments, determining the value of the shift offset parameter based on the block size index value of the current block may include:

[0595] Based on the block size index value, the value corresponding to the block size index value is queried from the fourth preset lookup table, wherein the fourth preset lookup table is used to record the correspondence between the block size index value and the value of the shift offset parameter;

[0596] The retrieved value is determined as the value of the shift offset parameter.

[0597] It should be noted that the value of the shift offset parameter can be queried based solely on the block size index value of the current block (represented by mipSizeId). As shown in the fourth preset lookup table in Table 7 above, each type of block size index value can correspond to a fixed value. In other words, the size of each block or the set of sizes of each block can have a fixed value of the shift offset parameter as shown in Table 7.

[0598] According to Table 7 above, when the block size index value of the current block is equal to 0, 1 and 2 respectively, the shift offset parameter corresponding to the block size index value can be determined to be equal to 34, 23 and 46 respectively.

[0599] In this way, by simplifying the determination of shift offset parameters, especially by minimizing the shift offset parameter table or fixing the values ​​of shift offset parameters, the storage of the lookup table can be minimized, thereby reducing the memory occupied by the shift offset parameter table in MIP mode without increasing computational complexity.

[0600] Thus, after determining the shift offset parameter (fO) and the shift amount parameter (sW), for the first constant value, optionally, in some embodiments, determining the first constant value based on the value of the shift amount parameter may include:

[0601] Set the first constant value to an integer power equal to 2, wherein the exponent of the power is equal to the value of the shift quantity parameter minus 1.

[0602] Optionally, in some embodiments, determining the first constant value based on the value of the shift amount parameter may include:

[0603] The first constant value is obtained by left-shifting the binary bits of "1", wherein the number of bits shifted left is equal to the value of the shift quantity parameter minus 1.

[0604] In other words, after obtaining the shift amount parameter (sW), the first constant value can be expressed as 1<<(sW-1) or 2^(sW-1). When the shift amount parameter is set to 6, the first constant value becomes 32.

[0605] Here, if the first offset is represented by oW, the product of the sum of the MIP input sample values ​​and the shift offset parameter (fO) can be calculated based on the shift offset parameter, which is: Then the value of oW can be set to

[0606] Thus, in MIP mode, the MIP weighting matrix, MIP input sample values, shift quantity parameter, and first offset can be obtained for subsequent determination of the MIP prediction value of the current block. Specifically, in some embodiments, determining the MIP prediction value of the current block based on the MIP weighting matrix, the MIP input sample values, the shift quantity parameter, and the first offset may include:

[0607] Calculate the first weighted sum of the MIP weighting matrix and the MIP input sample values;

[0608] Calculate the first sum of the first weighted sum and the first offset;

[0609] The first sum is right-shifted by binary bits to obtain a first right-shifted value, wherein the number of bits shifted is equal to the value of the shift quantity parameter;

[0610] The MIP prediction value of the current block is set to be equal to the sum of the first right shift value and the value corresponding to index number 0 in the first temporary reference value; wherein the first temporary reference value is obtained by downsampling and filtering the adjacent sampled values ​​of the current block.

[0611] In other words, in MIP mode, the MIP weighting matrix (mWeight), shift quantity parameter (sW), and shift offset parameter (fO) can be determined based on the current block size index (mipSizeId) and MIP mode index (modeId). Then, the MIP input sample value (p[x]), mWeight, sW, and fO are multiplied into the input matrix to obtain the MIP prediction value (predMip[x][y]) output by matrix multiplication. The sample points in predMip[x][y] are arranged into a matrix / array form according to predSize×predSize. The calculation formula is shown in equation (5) or equation (6) above.

[0612] S904: Filter the MIP prediction value to determine the intra-frame prediction value of the chroma component of the current block.

[0613] It should be noted that the MIP prediction block is composed of MIP prediction values. After obtaining the MIP prediction block, the intra-frame prediction value of the current block can be further determined by checking whether the size parameters of the MIP prediction block are the same as those of the current block. Specifically, based on the determination result, when the size parameters of the MIP prediction block are the same as those of the current block, the intra-frame prediction block of the current block is set to be equal to the MIP prediction block; at this time, the MIP prediction block contains the prediction values ​​of all pixel positions in the current block; when the size parameters of the MIP prediction block are not the same as those of the current block, the MIP prediction block is filtered to obtain a filtered prediction block, and the filtered prediction block is set as the intra-frame prediction block of the current block. Here, the filtering process can include upsampling filtering or low-pass filtering.

[0614] S905: When the LFNST parameter indicates that LFNST is performed on the current block, the reconstruction transform coefficient block of the current block is determined, and LFNST is performed on at least a portion of the reconstruction transform coefficients in the reconstruction transform coefficient block to obtain a second transform coefficient block.

[0615] It should be noted that not every current block can be executed with LFNST, meaning that the LFNST parameter may not necessarily exist in the bitstream. In this case, it is necessary to check certain conditions (such as the minimum value of the current block's size parameter, the block size index value of the current block, etc.) to determine whether the LFNST parameter exists in the bitstream. Therefore, in some embodiments, parsing the bitstream to obtain the LFNST parameter may include:

[0616] Determine whether the LFNST parameter exists in the bitstream;

[0617] If the presence of the LFNST parameter is confirmed in the bitstream, the bitstream is parsed to obtain the LFNST parameter.

[0618] In other words, it is first necessary to determine whether the LFNST parameter exists in the bitstream. Only if the result is yes, that is, if the LFNST parameter exists in the bitstream, can the step of parsing the bitstream and obtaining the LFNST parameter be performed.

[0619] Optionally, in some embodiments, determining whether the LFNST parameter exists in the bitstream may include:

[0620] When the chroma components of the current block are used to determine the intra-prediction value using MIP mode, the minimum value among the size parameters of the current block is determined;

[0621] Based on the minimum value, determine whether the LFNST parameter exists in the bitstream.

[0622] Further, determining whether the LFNST parameter exists in the bitstream based on the minimum value may include:

[0623] If the minimum value is greater than or equal to the first preset threshold, then it is determined that the LFNST parameter exists in the bitstream.

[0624] Here, the first preset threshold can be used to represent a pre-set threshold value for measuring whether LFNST should be performed. In the embodiments of this application, the first preset threshold can be set to 8, but is not specifically limited to this.

[0625] In other words, when performing intra-frame prediction using MIP mode on the chroma components of the current block, the minimum value of the size parameters of the current block can be determined. For example, when the minimum value of the width and height is greater than or equal to 8, it is determined that there is an LFNST parameter in the bitstream. The LFNST parameter, such as the value of lfnst_index, can be obtained by parsing the bitstream.

[0626] Optionally, in some embodiments, determining whether the LFNST parameter exists in the bitstream may include:

[0627] When the chroma components of the current block are used to determine the intra-prediction value using the MIP mode, the block size index value of the current block is determined according to the size parameter of the current block;

[0628] Based on the block size index value, determine whether the LFNST parameter exists in the bitstream.

[0629] Further, determining whether the LFNST parameter exists in the bitstream based on the block size index value may include:

[0630] If the block size index value is equal to the second preset threshold, then it is determined that the LFNST parameter exists in the bitstream.

[0631] Here, the second preset threshold can also be used to represent a pre-set threshold value for measuring whether to perform LFNST. In the embodiments of this application, the second preset threshold can be set to a preset value (e.g., 2), or one of multiple preset values ​​(e.g., 1, 2), but is not specifically limited.

[0632] In other words, when performing intra-frame prediction using MIP mode on the chroma components of the current block, the block size index value (mipSizeId) of the current block can be determined based on the size parameter of the current block. For example, when the value of mipSizeId is equal to 2, it is determined that there is an LFNST parameter in the bitstream. The LFNST parameter, such as the value of lfnst_index, can be obtained by parsing the bitstream.

[0633] In some embodiments, parsing the bitstream and obtaining the LFNST parameters may include:

[0634] Parse the bitstream to obtain the value of the LFNST index number.

[0635] It should be noted that the LFNST index number can be represented by lfnst_index, and the value of the LFNST index number is used to determine whether to perform LFNST on the current block. Specifically, in some embodiments, the method may further include:

[0636] If the value of the LFNST index number is greater than zero, then it is determined that LFNST will be performed on the current block;

[0637] If the value of the LFNST index number is equal to zero, then it is determined that LFNST will not be performed on the current block.

[0638] In other words, after obtaining the value of lfnst_index, if the value of lfnst_index is greater than zero, then it is determined that LFNST will be performed on the current block; if the value of lfnst_index is equal to zero, then it is determined that LFNST will not be performed on the current block.

[0639] It should also be noted that after obtaining the value of the LFNST parameter, such as lfnst_index, if lfnst_index is greater than zero, it indicates that the LFNST parameter indicates that LFNST should be performed on the current block. In some embodiments, when the LFNST parameter indicates that LFNST should be performed on the current block, determining the reconstruction transform coefficient block of the current block may include:

[0640] Parse the bitstream to obtain the quantization coefficients of the current block;

[0641] The quantization coefficients are dequantized to obtain the reconstruction transform coefficient block of the current block.

[0642] Specifically, dequantization is equivalent to scaling. In one specific example, dequantizing the quantization coefficients to obtain the reconstructed transform coefficient block of the current block may include: scaling the quantization coefficients to obtain the reconstructed transform coefficient block of the current block.

[0643] Thus, when the LFNST parameter indicates that LFNST should be performed on the current block, LFNST can be performed on all or part of the reconstructed transform coefficients in the reconstructed transform coefficient block to obtain the second transform coefficient block.

[0644] It should also be noted that when it is determined that the current block can be executed with LFNST, it is also necessary to determine the LFNST transform kernel (represented by kernel) used by the current block. In some embodiments, the step of performing LFNST on at least a portion of the reconstructed transform coefficients in the reconstructed transform coefficient block to obtain a second transform coefficient block may include: performing LFNST on at least a portion of the reconstructed transform coefficients in the reconstructed transform coefficient block using an LFNST transform kernel to obtain the second transform coefficient block.

[0645] In this embodiment, there are four candidate transform cores in the LFNST, which may include set0, set1, set2, and set3. Here, this embodiment can utilize MIP parameters to determine the LFNST candidate transform cores, and then select the LFNST transform core to be used by the current block from these candidate sets. Therefore, in some embodiments, the method may further include:

[0646] When the chroma components of the current block are used to determine the intra-prediction value using the MIP mode, the MIP parameters are determined from the prediction mode parameters of the current block;

[0647] Based on the MIP parameters, determine the LFNST transform kernel candidate set;

[0648] In the LFNST transform core candidate set, the transform core indicated by the value of the LFNST index number is determined as the LFNST transform core used by the current block; wherein, the LFNST transform core candidate set contains two or more preset LFNST transform cores.

[0649] It should be noted that the MIP parameters may include at least one of the following: the MIP mode index value (represented by modeId) and the MIP transpose indicator parameter (represented by isTransposed). Specifically, for the LFNST transform kernel used by the current block, when the value of the MIP transpose indicator parameter indicates that the sampling point input vector used by the MIP mode is to be transposed, the selected transform kernel also needs to be matrix transposed to obtain the LFNST transform kernel used by the current block.

[0650] Furthermore, in some embodiments, determining the LFNST transform kernel candidate set based on the MIP parameters may include:

[0651] Based on the MIP mode index value, determine the value of the LFNST intra-frame prediction mode index number;

[0652] Based on the value of the LFNST intra-frame prediction mode index number, the index value of the LFNST transform kernel candidate set is determined through the fifth preset lookup table;

[0653] The LFNST transform kernel candidate set is selected from multiple candidate LFNST transform kernel candidate sets based on the index value of the LFNST transform kernel candidate set.

[0654] Here, the index number of the LFNST intra-prediction mode can be represented by predModeIntra, and the index value of the LFNST transform kernel candidate set can be represented by lfnstTrSetIdx.

[0655] It should be noted that the fifth preset lookup table is shown in Table 10 above. Based on the MIP mode index value (i.e., modeId), the value of predModeIntra can be determined; then, based on the value of predModeIntra, the value of lfnstTrSetIdx can be directly determined in conjunction with Table 10, that is, the LFNST transform kernel candidate set selected for the current block is determined. Here, the value of lfnstTrSetIdx indicates the transform kernel candidate set used during LFNST; since the value of modeId can include 0, 1, 2, 3, 4, and 5, the value of predModeIntra is also 0, 1, 2, 3, 4, and 5; its correspondence with lfnstTrSetIdx is specifically shown in Table 10.

[0656] In one specific embodiment, determining the value of the LFNST intra-frame prediction mode index number based on the MIP mode index value may include: setting the value of the LFNST intra-frame prediction mode index number to be equal to the MIP mode index value.

[0657] In other words, in this embodiment, the value of predModeIntra can be set to be equal to the value of modeId. Then, based on the value of predModeIntra and in conjunction with Table 10, the value of lfnstTrSetIdx can be directly determined, thus determining the LFNST transform kernel candidate set selected for the current block. It should also be noted that, in another specific embodiment, this embodiment can also directly map the MIP mode index value to the value of the PLANA mode, and then use the value of predModeIntra corresponding to the PLANA mode to determine the value of lfnstTrSetIdx, thus determining the LFNST transform kernel candidate set selected for the current block.

[0658] Furthermore, during the execution of LFNST, the scan order can include a horizontal scan order and a vertical scan order. Here, the determination of the scan order is related to the value of the MIP transpose indicator parameter (denoted by isTransposed).

[0659] Optionally, in some embodiments, performing LFNST on at least a portion of the reconstructed transform coefficients in the reconstructed transform coefficient block to obtain a second transform coefficient block may include:

[0660] When the value of the MIP transpose indicator parameter indicates that the sampling point input vector used in the MIP mode is transposed, LFNST is performed on at least a portion of the reconstructed transform coefficients in the reconstructed transform coefficient block according to the vertical scan order to obtain the second transform coefficient block;

[0661] When the value of the MIP transpose indicator parameter indicates that the sampling point input vector used in the MIP mode is not transposed, LFNST is performed on at least a portion of the reconstructed transform coefficients in the reconstructed transform coefficient block according to the horizontal scan order to obtain the second transform coefficient block.

[0662] Specifically, when the value of isTransposed is equal to 0, the sampling point input vector used in MIP mode is transposed. Then, LFNST is performed on at least some of the transformation coefficients in the first transformation coefficient block according to the horizontal scanning order, as shown in the above equation (10).

[0663] When isTransposed is equal to 1, the sampling point input vector used in MIP mode is not transposed. Then, LFNST is performed on at least some of the transformation coefficients in the first transformation coefficient block according to the vertical scanning order, as shown in equation (11) above.

[0664] The determination of log2LfnstSize is shown in equation (9) above.

[0665] Optionally, in some embodiments, performing LFNST on at least a portion of the reconstructed transform coefficients in the reconstructed transform coefficient block to obtain a second transform coefficient block may include:

[0666] When the value of the MIP transpose indicator parameter indicates that the sampling point input vector used in the MIP mode is transposed, LFNST is performed on at least a portion of the reconstructed transform coefficients in the reconstructed transform coefficient block according to the horizontal scan order to obtain the second transform coefficient block;

[0667] When the value of the MIP transpose indicator parameter indicates that the sampling point input vector used in the MIP mode is not transposed, LFNST is performed on at least a portion of the reconstructed transform coefficients in the reconstructed transform coefficient block according to the vertical scan order to obtain the second transform coefficient block.

[0668] Specifically, when isTransposed is equal to 1, the sampling point input vector used in MIP mode is transposed. Then, LFNST is performed on at least some of the transformation coefficients in the first transformation coefficient block according to the horizontal scanning order, as shown in the above equation (10).

[0669] When isTransposed is equal to 0, the sampling point input vector used in MIP mode is not transposed. Then, LFNST is performed on at least some of the transformation coefficients in the first transformation coefficient block according to the vertical scanning order, as shown in equation (11) above.

[0670] The determination of log2LfnstSize is shown in equation (9) above.

[0671] S906: Perform a first transformation on the second transform coefficient block to obtain the reconstructed residual block of the chromaticity component of the current block.

[0672] S907: Determine the reconstruction block of the chroma component of the current block based on the intra-frame prediction value of the chroma component of the current block and the reconstruction residual block.

[0673] It should be noted that in the decoder, the quantized values ​​of the LFNST coefficients can be obtained by parsing the bitstream. These quantized values ​​are then dequantized (a process known as "scaling") by the dequantization unit to obtain a reconstructed transform coefficient block. Performing an inverse LFNST transform on at least a portion of the reconstructed transform coefficients in this block yields a second transform coefficient block. Finally, a reverse first transform is performed to obtain the reconstructed residual block that recovers the chroma components of the current block. Here, the reverse first transform corresponds to the inverse transform of the "core transform" in the encoder. It's important to note that the standard only defines the "inverse transform" operation in the decoder; therefore, in the standard, the "reverse LFNST transform" can also be referred to as the "LFNST transform."

[0674] It should also be noted that after recovering the reconstructed residual block of the chroma component of the current block, the intra-frame prediction value of the chroma component of the current block and the reconstructed residual block can be added together to determine the reconstructed block of the chroma component of the current block.

[0675] The following will combine Figure 2 The decoder 200 shown here provides a detailed description of the decoding method of the embodiments of this application.

[0676] The input bitstream of decoder 200 can be a bitstream generated by encoder 100. Parsing unit 201 parses the input bitstream and obtains the values ​​of syntax elements from it. Parsing unit 201 converts the binary representation of the syntax elements into digital values ​​and sends the digital values ​​to units in decoder 200 to obtain one or more decoded images. Parsing unit 201 can also parse one or more syntax elements from the input bitstream to render the decoded images.

[0677] See Figure 10 This illustrates a flowchart of an embodiment of the present application for parsing LFNST parameters. Figure 10 As shown, the process may include:

[0678] S1001: Parsing unit 201 checks to determine whether the LFNST parameter exists in the input bitstream.

[0679] It should be noted that the LFNST parameter can refer to the value of lfnst_index. Here, the example of the syntax structure used to represent the LFNST parameter can be the same as the example of the syntax structure shown in Table 8 above, or it can be the same as the example of the syntax structure shown in Tables 9A and 9B above.

[0680] Specifically, when the intra-prediction mode of the current block is MIP mode, the parsing unit 201 obtains the encoded bits of lfnst_index from the input bitstream in the following manner. Optionally, using the syntax structure in Table 9A, when the minimum value of the width and height of the transform block in the current block is greater than or equal to 8 (i.e., both width and height are greater than or equal to 8), the parsing unit 201 reads the lfnst_index bit using MIP mode. Another optional condition can be set as follows: the width and height are greater than or equal to a first value (e.g., 8), and the sum of the width and height is greater than or equal to a second value (e.g., 16).

[0681] Optionally, using the syntax structure in Table 9B, parsing unit 201 reads the lfnst_index bit using MIP mode when the mipSizeId of the current block is equal to 2. Another optional condition can be set to: the mipSizeId of the current block is equal to one of multiple values ​​(e.g., 1 and 2).

[0682] S1002: The parsing unit 201 performs adaptive binary arithmetic decoding on the LFNST parameters to obtain the LFNST parameters.

[0683] Here, since the entropy coding unit 115 on the encoder 100 side uses CABAC with the descriptor "ae(v)" as shown in Tables 9A and 9B to encode the LFNST parameter (e.g., the value of lfnst_index), the value of lfnst_index can be obtained from the lfnst_index bit by performing adaptive binary arithmetic decoding through the parsing unit 201 on the decoder 200 side.

[0684] Parsing unit 201 sends the values ​​of syntax elements and one or more variables, set or determined based on the values ​​of the syntax elements, used to obtain one or more decoded pictures to units in decoder 200. Prediction unit 202 determines the intra-prediction block for the current block (e.g., CU). When an inter-frame decoding mode is indicated for decoding the current decoded block, prediction unit 202 transmits relative parameters from parsing unit 201 to MC unit 203 to obtain an inter-frame prediction block. When an intra-frame prediction mode (including MIP mode indicated based on MIP mode index value) is indicated for decoding the current block, prediction unit 202 transmits relative parameters from parsing unit 201 to intra-frame prediction unit 204 to obtain an intra-frame prediction block.

[0685] It should be understood that the current block (or "decoding block") in the embodiments of this application can be a CU or a partition of a CU (e.g., a transform block). Specifically, when the MIP mode is indicated for decoding the current block, the intra-prediction unit 204 will obtain the intra-prediction block of the current block, wherein the intra-prediction block of the current block includes the intra-prediction values ​​of at least one pixel (sample) within the current block.

[0686] Specifically, for the intra-prediction unit 204, it uses the MIP mode to obtain the intra-prediction block, and the steps are as follows:

[0687] First, the intra-prediction unit 204 obtains one or more reference pixels from the neighboring pixels of the current block, for example by downsampling the neighboring pixels or by directly extracting them from the neighboring pixels.

[0688] Then, the intra-prediction unit 204 uses the acquired reference pixels, the MIP matrix, and shift parameters to determine one or more partial predicted pixels corresponding to the pixel positions within the current block. Here, the pixel positions can be preset pixel positions within the current block. For example, the pixel positions have uniform horizontal and vertical coordinate values. The shift parameters include shift amount parameters and shift offset parameters, and these shift parameters can be used for offset operations during the acquisition of the intra-prediction values ​​of partial pixels.

[0689] Finally, if the predicted pixels corresponding to a subset of pixels in the current block are obtained, the intra-prediction unit 204 also needs to obtain the predicted pixels corresponding to the remaining pixels in the current block, excluding the subset of pixels. For example, the intra-prediction unit 204 can use an interpolation filter to obtain the predicted pixels corresponding to the remaining pixels. Here, the input to the interpolation filter can be a subset of pixels and adjacent pixels.

[0690] like Figure 6 The flowchart example shown, illustrating the use of MIP mode to obtain intra-predictive blocks, can also be implemented on decoder 200, where "current block" refers to "decoded block".

[0691] Step 601, the intra-frame prediction unit 204 obtains the neighboring pixels of the current block, for example, neighboring pixels are marked as... Figure 6 The gray-filled squares adjacent to the current block in step 601 are shown. Intra-prediction unit 204 obtains one or more reference pixels from neighboring pixels. Figure 6 In the example of step 601 shown, optionally, the intra-prediction unit 204 can calculate the average value of two adjacent pixels and use this average value as a reference pixel. Optionally, the intra-prediction unit 204 selects an adjacent pixel as a reference pixel every other adjacent pixel. For example, in... Figure 6 In the example shown in step 601, the intra-prediction unit 204 selects 4 reference pixels from the 8 upper adjacent pixels of the current block and selects another 4 reference pixels from the 8 left adjacent pixels of the current block.

[0692] The specific process of step 601 is as follows:

[0693] Intra-prediction unit 204 obtains the width and height of the current block from parsing unit 201, where the width and height are represented by variables cbWidth and cbHeight, respectively. Based on the parameters from parsing unit 201, the current block is divided into one or more transform blocks. Let variables nTbW and nTbH represent the width and height of the transform block, respectively. When MIP mode is used as the intra-prediction mode to obtain the prediction of the current block, intra-prediction unit 204 determines the block size index value of the MIP, i.e., the variable mipSizeId.

[0694] Optionally, the intra-prediction unit 204 determines the value of mipSizeId as follows:

[0695] - If both nTbW and nTbH are equal to 4, then mipSizeId is set to 0.

[0696] Otherwise, if cbWidth or cbHeight equals 4, then mipSizeId is set to equal to 1;

[0697] Otherwise, mipSizeId is set to equal to 2.

[0698] Specifically, when the current block size parameter is 8×8 (i.e., both cbWidth and cbHeight are equal to 8), mipSizeId is set to equal to 2.

[0699] Optionally, the intra-prediction unit 204 determines the value of mipSizeId as follows:

[0700] - If both nTbW and nTbH are equal to 4, then mipSizeId is set to 0.

[0701] Otherwise, if cbWidth or cbHeight equals 4, or if both cbWidth and cbHeight equal 8, then mipSizeId is set to 1.

[0702] Otherwise, mipSizeId is set to equal to 2.

[0703] Specifically, when the current block size parameter is 8×8, mipSizeId is set to equal to 1.

[0704] Furthermore, the intra-prediction unit 204 obtains the values ​​of boundarySize and predSize based on mipSizeId, as follows:

[0705] - If mipSizeId equals 0, then boundarySize is set to equal 2 and predSize is set to equal 4;

[0706] Otherwise, if mipSizeId equals 1, then boundarySize is set to equal 4 and predSize is set to equal 4;

[0707] Otherwise (if mipSizeId equals 2), boundarySize is set to equal to 4, and predSize is set to equal to 8;

[0708] Where boundarySize represents the number of reference pixels obtained from each of the reference pixels adjacent to the upper side and the reference pixels adjacent to the left side of the current block.

[0709] The intra-prediction unit 204 may also use a parameter from the parsing unit 201, indicating the order of reference pixels stored in the buffer pTemp, to obtain reference pixels for calculating the MIP prediction value. The intra-prediction unit 204 sets the variable isTransposed to be equal to the parameter from the parsing unit 201 indicating the order of reference pixels stored in the buffer pTemp. For example, isTransposed equal to 0 (or FALSE) means that the intra-prediction unit 204 first renders the reference pixel obtained from the reference pixel adjacent to the top edge of the current block, and then renders the reference pixel obtained from the reference pixel adjacent to the left edge; otherwise, isTransposed equal to 1 (or TRUE) means that the intra-prediction unit 204 first renders the reference pixel obtained from the reference pixel adjacent to the left edge of the current block, and then renders the reference pixel obtained from the reference pixel adjacent to the top edge.

[0710] Intra-prediction unit 204 obtains the value of variable inSize to indicate the number of reference pixels used when using MIP mode, as shown in equation (1) above.

[0711] Intra-prediction unit 204 invokes the following procedure to obtain a set of reference pixels (stored as an array p[x], where x is 0,…,inSize-1) using the reference pixels adjacent to the current block.

[0712] Intra-prediction unit 204 obtains nTbW reference pixels from the reference pixels adjacent to the upper side of the current block (e.g., stored in array refT), and nTbH reference pixels from the reference pixels adjacent to the left side of the current block (e.g., stored in array refL). Intra-prediction unit 204 performs downsampling processing on refT to obtain boundarySize reference pixels and stores the boundarySize reference pixels in refT. Intra-prediction unit 204 performs downsampling processing on refL to obtain boundarySize reference pixels and stores the boundarySize reference pixels in refL.

[0713] Intra-prediction unit 204 can also obtain the order indicated by isTransposed to put elements in refT and refL into buffer pTemp.

[0714] Optionally, the intra-prediction unit 204 obtains p[x], where x = 0, ..., inSize-1, as follows:

[0715] - If mipSizeId equals 2, then p[x] = pTemp[x+1] - pTemp[0];

[0716] - Otherwise (mipSizeId is less than 2), p[0] = pTemp[0] - (1 << (BitDepth-1)), p[x] = pTemp[x] - pTemp[0];

[0717] BitDepth is the bit depth of the color component of the pixel in the current block. Here, the color component can be one of the RGB components, one of the YUV components, or one of the YCbCr components, for example, the Y component, etc.

[0718] Optionally, the intra-prediction unit 204 can obtain p[x], where x = 0, ..., inSize-1, as follows:

[0719] - If mipSizeId equals 2, then p[x] = pTemp[x+1] - pTemp[0];

[0720] - Otherwise (mipSizeId is less than 2), p[0] = (1 << (BitDepth-1)) - pTemp[0], p[x] = pTemp[x] - pTemp[0].

[0721] Optionally, the intra-prediction unit 204 can use a uniform calculation method to obtain the value of p[x] without having to determine the value of mipSizeId. For example, if (1<<(BitDepth-1)) is appended to the buffer pTemp as an additional element, the intra-prediction unit 204 will calculate p[x] as pTemp[x+1]-pTemp[0].

[0722] In step 602, the intra-prediction unit 204 obtains the MIP prediction value of the current block using a set of reference pixels and a MIP weighting matrix. The MIP weighting matrix is ​​selected from a set of predefined MIP weighting matrices based on the MIP mode index value (denoted as ModeId) corresponding to the MIP mode and the MIP block size index value (denoted as mipSizeId). The intra-prediction unit 204 obtains the value of ModeId from the parsing unit 201.

[0723] Intra-prediction unit 204 acquires the MIP prediction values ​​(denoted as predMip[x][y]) of a subset of predicted pixels corresponding to one or more pixel positions within the current block. Figure 6 In the example of step 602 shown, some predicted pixels are the pixels marked as gray-filled squares in the current block. The input to prediction module 601 is the reference pixel p[x] obtained in step 601. Prediction module 601 calculates some predicted pixels using the MIP weighting matrix and shift parameters; here, the shift parameters include a shift amount parameter and a shift offset parameter.

[0724] In one possible implementation, the prediction module 601 can represent the pixel point with coordinates (x,y), and the predicted pixel point is represented by predMip[x][y]. The calculation formula of predMip[x][y] is shown in Equation (5) or Equation (6).

[0725] In equation (5) or (6), mWeight[i][j] is the MIP weighting matrix; here, the matrix elements can be predetermined constant values; or, they can be updated adaptively using a training method, the input of which is one or more coded images or blocks, or images from other bitstreams provided to the decoder 200 by an external device, or images obtained from the parsing unit 201 by parsing special data units in the input bitstream containing the MIP weighting matrix; mWeight[i][j] can be determined according to the MIP mode indicated by one or more parameters from the parsing unit 201; fO is the shift offset parameter used to determine oW; sW is the shift amount parameter; p[i] is calculated using reference pixels, i.e., the MIP input sample value in this embodiment; pTemp[0] represents the first pixel in the reference pixels, i.e., the value corresponding to index number 0 in the first temporary reference value in this embodiment; ">>" is the binary right shift operator as defined in VVC.

[0726] The prediction module 601 can determine the values ​​of sW and fO based on the size of the current block and the MIP mode used for the current block. In one example, the prediction module 601 uses a lookup table to retrieve the values ​​of sW and fO.

[0727] Optionally, the prediction module 601 can use Table 4 above to determine sW based on the current block size parameters and MIP mode.

[0728] Optionally, the prediction module 601 can also use Table 5 above to determine sW based on the size parameters of the current block.

[0729] Optionally, the prediction module 601 can also directly set sW to a constant value. For example, for blocks with various size parameters and different MIP modes, the prediction module 601 can set sW to 5; or, for blocks with various size parameters and different MIP modes, the prediction module 601 can set sW to 6; or, for blocks with various size parameters and different MIP modes, the prediction module 601 can set sW to 7.

[0730] Optionally, the prediction module 601 can use Table 6 above to determine fO based on the current block size parameters and MIP mode.

[0731] Optionally, the prediction module 601 can also use Table 7 above to determine fO based on the size parameters of the current block.

[0732] Optionally, the prediction module 601 can also directly set fO to a constant value (such as 0 to 100). For example, for blocks with various size parameters and different MIP modes, the prediction module 601 sets fO to 32; or, the prediction module 601 sets fO to 46; or, the prediction module 601 sets fO to 56; or, the prediction module 601 sets fO to 66.

[0733] Intra-prediction unit 204 can perform a clipping operation on the MIP prediction values ​​in predMip. When isTransposed equals 1 (or TRUE), the array predMip[x][y] of predSize×predSize (where x=0,...,predSize-1, y=0,...,predSize-1) is transformed into predTemp[y][x]=predMip[x][y], and then predMip=predTemp.

[0734] For example, optionally, when the size parameter of the current block is 8×8 (i.e., both cbWidth and cbHeight are equal to 8), the intra-prediction unit 204 determines that mipSizeId is equal to 2 and obtains an 8×8 predMip.

[0735] Optionally, when the size parameter of the current block is 8×8 (i.e., both cbWidth and cbHeight are equal to 8), the intra-prediction unit 204 determines that mipSizeId is equal to 1 and obtains a 4×4 predMip.

[0736] Step 603, the intra-prediction unit 204 obtains the intra-prediction value of the current block (stored in the array as predSamples[x][y], where x = 0,...,nTbW-1, y = 0,...,nTbH-1), as follows:

[0737] If the intra-prediction unit 204 determines that nTbW is greater than predSize or nTbH is greater than predSize, then the intra-prediction unit 204 calls the upsampling procedure to obtain predSamples using predMip. The intra-prediction unit 204 obtains the predicted pixel points corresponding to the remaining pixels in the current block, excluding some pixels. Figure 6In this process, the intra-prediction unit 204 can use the filtering module 602 to obtain the predicted pixels corresponding to the remaining pixels in the current block, excluding some pixels. The input to the filtering module 602 is the pixel marked as a gray-filled square in step 602. The filtering module 602 can use one or more interpolation filters to obtain the predicted pixels corresponding to the remaining pixels in the current block, excluding some pixels, using this input. For example, the input may include a reference pixel and partial predicted pixels at the positions of one or more pixels in the current block; or, the input may include a neighboring pixel and partial predicted pixels at the positions of one or more pixels in the current block; or, the input may include a reference pixel, a neighboring pixel, and partial predicted pixels at the positions of one or more pixels in the current block. For example, optionally, when the size of the current block is 8×8 (i.e., cbWidth and cbHeight are both equal to 8), the intra-prediction unit 204 determines that mipSizeId is equal to 1, and the intra-prediction unit 204 applies an upsampling process to 4×4predMip to obtain an 8×8 intra-prediction block for the current block.

[0738] Otherwise, the intra-prediction unit 204 sets the intra-prediction block of the current block to be equal to the MIP prediction block of the current block, that is, sets predSamples[x][y] (where x = 0,...,nTbW-1, y = 0,...,nTbH-1) to predMip[x][y]. For example, when the size parameter of the current block is 8×8 (i.e., cbWidth and cbHeight are both equal to 8), the intra-prediction unit 204 obtains the predSamples of the current block, whose size is equal to 8×8 (i.e., cbWidth and cbHeight are both equal to 8).

[0739] Thus, according to Figure 6 As shown in the flowchart, after step 603, the intra-prediction unit 204 can obtain the intra-prediction block of the current block (i.e., CU), that is, determine the intra-prediction value of at least one pixel in the current block.

[0740] Scaling unit 205 has the same function as dequantization unit 110 in encoder 100. Scaling unit 205 performs a scaling operation on the quantization coefficients (i.e., levels) from parsing unit 201 to obtain reconstruction coefficients.

[0741] Transformation unit 206 has the same function as inverse transformation unit 111 in encoder 100. Transformation unit 206 performs one or more transformation operations (i.e., the inverse operation of one or more transformation operations performed by inverse transformation unit 111 in encoder 100) to obtain reconstructed residuals.

[0742] When lfnst_index is not equal to 0, the transform unit 206 performs a quadratic transform on the reconstructed coefficients to obtain a quadratic transform coefficient block, and then performs a first transform on the quadratic transform coefficient block, for example, an integer transform initially designed based on DCT, to obtain a reconstructed residual block. LFNST is an example of a quadratic transform.

[0743] See Figure 11 This illustrates a schematic diagram of another process framework for executing LFNST provided in an embodiment of this application. Figure 11 In this context, block 1101 is a reconstruction transform coefficient block containing the reconstruction transform coefficients.

[0744] Step 1101: Transformation unit 206 performs a secondary transformation LFNST on all or part of the reconstructed transformation coefficients in block 1101.

[0745] When the intra-prediction mode of the current block is MIP mode, the transform unit 206 selects the LFNST transform kernel candidate set according to the MIP parameters. The MIP parameters may include at least one of the following: the MIP mode index value (i.e., modeId) and the MIP transpose indicator parameter (i.e., isTransposed).

[0746] Transform unit 206 sets the value of variable predModeIntra to be equal to the value of modeId. Transform unit 206 determines the transform kernel used to perform LFNST on the coefficients of subblocks 11001, 11002, and 11003. The size of the subblock can be a preset value, such as 8×8. Transform unit 206 determines the index value (i.e., lfnstTrSetIdx) of the LFNST transform kernel candidate set based on the value of predModeIntra. For block decoding in MIP mode, transform unit 206 uses a lookup table as shown in Table 10 to obtain the value of lfnstTrSetIdx. Transform unit 206 determines the transform kernel used to perform LFNST as the transform kernel indicated by lfnst_index in the LFNST transform kernel candidate set indicated by lfnstTrSetIdx.

[0747] When lfnst_index is not equal to 0, transform unit 206 performs LFNST on the transform block using coefficients from sub-blocks 11001, 11002, and 11003, obtains LFNST coefficients, and places the LFNST coefficients into sub-blocks 11011, 11012, and 11013 of block 1102. Assume d[x][y] is the block of the first transform coefficients, where x = 0,...,nTbW-1, y = 0,...,nTbH-1, where nTbW and nTbH are the width and height of block 1102, respectively. Assume v[x] is the block of LFNST coefficients, where x = 0,...,nLfnstOutSize-1, and nLfnstOutSize = (nTbW>=8 && nTbH>=8)? 48:16.

[0748] Optionally, the transformation unit 206 uses the following instructions to obtain the transformation coefficient block 1102, as shown in equation (8) above. Wherein, log2LfnstSize is as shown in equation (9) above.

[0749] Optionally, when isTransposed equals 0, the transformation unit 206 uses the following instructions to obtain the block 1102 of the transformation coefficients, as shown in the above formula (10);

[0750] When isTransposed equals 1, the transformation unit 206 uses the following instructions to obtain the transformation coefficient block 1102, as shown in the above formula (11).

[0751] Optionally, when isTransposed equals 1, the transformation unit 206 uses the following instructions to obtain the block 1102 of the transformation coefficients, as shown in the above formula (10);

[0752] When isTransposed equals 0, the transformation unit 206 uses the following instructions to obtain the transformation coefficient block 1102, as shown in the above formula (11).

[0753] The determination of log2LfnstSize is shown in equation (9) above.

[0754] In addition, when the intra-prediction mode of the current block is the traditional intra-prediction mode, i.e., angle intra-prediction mode, DC mode, and PLANA mode, the transform unit 206 uses the lookup table shown in Table 11 to determine the value of lfnstTrSetIdx based on the mode index value of the traditional intra-prediction mode (assigned to predModeIntra), thus determining the LFNST transform kernel candidate set selected for the current block. The transform unit 206 determines the transform kernel used to perform LFNST as the transform kernel indicated by lfnst_index in the LFNST transform kernel candidate set indicated by lfnstTrSetIdx.

[0755] When lfnst_index is not equal to 0, transform unit 206 performs LFNST on the transform block using coefficients from sub-blocks 11001, 11002, and 11003, obtains LFNST coefficients, and places the LFNST coefficients into sub-blocks 11011, 11012, and 11013 of block 1102. Assume d[x][y] is the block of the first transform coefficients, where x = 0,...,nTbW-1, y = 0,...,nTbH-1, where nTbW and nTbH are the width and height of block 1102, respectively. Assume v[x] is the block of LFNST coefficients, where x = 0,...,nLfnstOutSize-1, and nLfnstOutSize = (nTbW>=8 && nTbH>=8)? 48:16.

[0756] Optionally, when predModeIntra is less than or equal to 34, the transformation unit 206 uses the following instructions to obtain the block 1102 of transformation coefficients, as shown in the above formula (10);

[0757] When predModeIntra is greater than 34, the transformation unit 206 uses the following instructions to obtain the transformation coefficient block 1102, as shown in the above formula (11).

[0758] Optionally, when predModeIntra is greater than 34, the transformation unit 206 uses the following instructions to obtain the block 1102 of transformation coefficients, as shown in the above formula (10);

[0759] When predModeIntra is less than or equal to 34, the transformation unit 206 uses the following instructions to obtain the transformation coefficient block 1102, as shown in the above formula (11).

[0760] The determination of log2LfnstSize is shown in equation (9) above.

[0761] Transformation unit 206 performs the first transformation (i.e., core transformation) on the transformation coefficients in block 1102, for example, an integer transformation initially based on DCT design, to obtain the reconstruction residual block 1103 of the current block.

[0762] Furthermore, adder 207 performs an addition operation on its inputs (the intra-predicted block from prediction unit 202 and the reconstructed residual block from transform unit 206) to obtain the reconstructed block of the current block. The reconstructed block is also sent to prediction unit 202 as a reference for other blocks encoded in intra-prediction mode.

[0763] After all CUs in an image or sub-image have been reconstructed, filtering unit 208 performs loop filtering on the reconstructed image or sub-image. Filtering unit 208 includes one or more filters, such as deblocking filters, sampling adaptive offset filters, adaptive loop filters, luminance mapping and chroma scaling filters, and neural network-based filters. Alternatively, when filtering unit 208 determines that a reconstructed block is not to be used as a reference for decoding other blocks, filtering unit 208 performs loop filtering on one or more target pixels in the reconstructed block.

[0764] The output of the filtering unit 208 is a decoded image or sub-image, which is sent to the DPB unit 209. The DPB unit 209 outputs the decoded image based on timing and control information. The image stored in the DPB unit 209 can also be used as a reference for performing inter-frame prediction or intra-frame prediction by the prediction unit 202.

[0765] In this embodiment, the decoder 200 can be a computing device having a processor and a storage medium for recording the decoding program. When the processor reads and runs the decoding program, the decoder 200 reads the input bitstream and generates the corresponding decoded video. Alternatively, the decoder 200 can be a computing device having one or more chips. Units implemented as integrated circuits on the chip communicate with... Figure 2 The corresponding units in the text have similar connections and data exchange capabilities, thus possessing similar functions.

[0766] This embodiment provides an encoding method that obtains prediction parameters and LFNST parameters for the current block by parsing the bitstream. The prediction parameters include prediction mode parameters. When the prediction mode parameters indicate that matrix-based intra-prediction (MIP) mode is used to determine intra-prediction values ​​for the chrominance components of the current block, the adjacent sample values ​​of the current block are obtained, and the MIP input sample values ​​of the current block are determined based on these adjacent sample values. The MIP prediction values ​​for the chrominance components of the current block are determined based on the MIP input sample values, the MIP weighting matrix, and shift parameters. The MIP prediction values ​​are filtered to determine the intra-frame prediction values ​​of the chroma components of the current block. When the LFNST parameter indicates that LFNST should be performed on the current block, the reconstruction transform coefficient block of the current block is determined. LFNST is performed on at least a portion of the reconstruction transform coefficients in the reconstruction transform coefficient block to obtain a second transform coefficient block. A first transform is performed on the second transform coefficient block to obtain the reconstruction residual block of the chroma components of the current block. Based on the intra-frame prediction values ​​of the chroma components of the current block and the reconstruction residual block, the reconstruction block of the chroma components of the current block is determined. In this way, for MIP mode, the complexity can be reduced while ensuring decoding performance, and the storage space required during decoding can be reduced, effectively improving decoding efficiency. In addition, when LFNST technology is applied to MIP mode prediction, the introduction of MIP parameters makes the LFNST transform more flexible, further improving decoding efficiency.

[0767] In another embodiment of this application, based on the same inventive concept as the foregoing embodiments, see [link to previous embodiment]. Figure 12 This illustrates a schematic diagram of the composition structure of an encoder 120 provided in an embodiment of this application. Figure 12 As shown, the encoder 120 may include: a first determining unit 1201, a first predicting unit 1202, a first transforming unit 1203, and an encoding unit 1204; wherein,

[0768] The first determining unit 1201 is configured to determine the prediction parameters of the current block, wherein the prediction parameters include prediction mode parameters;

[0769] The first determining unit 1201 is further configured to, when the prediction mode parameter indicates that the intra-prediction value is determined by using a matrix-based intra-prediction MIP mode for the chroma components of the current block, acquire the adjacent sample values ​​of the current block and determine the MIP input sample value of the current block based on the adjacent sample values ​​of the current block.

[0770] The first prediction unit 1202 is configured to determine the MIP prediction value of the chroma component of the current block based on the MIP input sample value, the MIP weighting matrix and the shift parameter; wherein the MIP prediction value is the prediction value of a portion of the sample points in the chroma component of the current block; and to filter the MIP prediction value to determine the intra-frame prediction value of the chroma component of the current block.

[0771] The first transformation unit 1203 is configured to determine the prediction residual value of the chrominance component of the current block based on the intra-frame prediction value of the chrominance component of the current block; and to perform a low-frequency non-separable quadratic transform (LFNST) on the prediction residual value to determine the LFNST parameters.

[0772] The encoding unit 1204 is configured to encode the LFNST parameters and write them into the bitstream.

[0773] In some embodiments, the adjacent sample values ​​of the current block include the left adjacent sample value and the top adjacent sample value of the current block.

[0774] In some embodiments, the shift parameters include shift offset parameters and shift quantity parameters;

[0775] The first determining unit 1201 is specifically configured to: determine the product of the sum of the MIP input sample values ​​and the shift offset parameter based on the value of the shift offset parameter, wherein the value of the shift offset parameter is a fixed constant; determine a first constant value based on the value of the shift quantity parameter, wherein the value of the shift quantity parameter is a fixed constant; set the value of the first offset to be the difference between the first constant value and the product; determine the MIP weighting matrix of the current block based on the prediction parameter; and determine the MIP prediction value of the chroma component of the current block based on the MIP weighting matrix, the MIP input sample values, the shift quantity parameter, and the first offset.

[0776] In some embodiments, the prediction parameters further include: the size parameter of the current block.

[0777] In some embodiments, the first determining unit 1201 is further configured to: determine the block size index value of the current block according to the size parameters of the current block; perform downsampling filtering on the adjacent sampled values ​​of the current block to obtain a first temporary reference value; and when the block size index value of the current block is within a preset range, determine a second constant value according to the bit depth of the adjacent sampled values ​​of the current block; set the value corresponding to index number 0 in the MIP input sampled values ​​to be equal to the difference between the second constant value and the value corresponding to index number 0 in the first temporary reference value; and set the... The value corresponding to index i in the MIP input sample value is set to the difference between the value corresponding to index i in the first temporary reference value and the value corresponding to index 0 in the first temporary reference value, where i is an integer greater than 0; when the block size index value of the current block is outside the preset range, the value corresponding to index j in the MIP input sample value is set to the difference between the value corresponding to index j+1 in the first temporary reference value and the value corresponding to index 0 in the first temporary reference value, where j is an integer greater than or equal to 0.

[0778] In some embodiments, the first determining unit 1201 is further configured to: when the block size index value of the current block is within a preset range, determine a second constant value based on the bit depth of the adjacent sample values ​​of the current block; set the value corresponding to index number 0 in the MIP input sample value to be equal to the difference between the value corresponding to index number 0 in the first temporary reference value and the second constant value; set the value corresponding to index number i in the MIP input sample value to be equal to the difference between the value corresponding to index number i in the first temporary reference value and the value corresponding to index number 0 in the first temporary reference value, where i is an integer greater than 0.

[0779] In some embodiments, the first determining unit 1201 is further configured to set the block size index value of the current block to 0 when the width and height of the current block are both equal to 4; set the block size index value of the current block to 1 when the width and height of the current block are both equal to 8, or when one of the width and height of the current block is equal to 4; and set the block size index value of the current block to 2 when the width and height of the current block do not meet the aforementioned conditions.

[0780] In some embodiments, the first determining unit 1201 is further configured to perform downsampling filtering on the adjacent sampled values ​​of the current block to obtain a first temporary reference value; and to determine a second constant value based on the bit depth of the adjacent sampled values ​​of the current block, cache the second constant value in a data unit after the first temporary reference value to obtain a second temporary reference value; and to set the value corresponding to index j in the MIP input sampled value to be equal to the difference between the value corresponding to index j+1 in the second temporary reference value and the value corresponding to index 0 in the second temporary reference value, wherein j is an integer greater than or equal to 0.

[0781] In some embodiments, the first determining unit 1201 is further configured to set the second constant value to an integer power equal to 2, wherein the exponent of the power is equal to the bit depth of the adjacent sample value of the current block minus 1.

[0782] In some embodiments, the first determining unit 1201 is further configured to perform a binary bit left shift on "1" to obtain the second constant value, wherein the number of bits shifted left is equal to the bit depth of the adjacent sample values ​​of the current block minus 1.

[0783] In some embodiments, the value of the shift quantity parameter is a fixed constant that is independent of the block size index value and the MIP mode index value.

[0784] In some embodiments, the value of the shift amount parameter is set to equal to 6.

[0785] In some embodiments, the value of the shift offset parameter is a fixed constant that is independent of the block size index value and the MIP mode index value.

[0786] In some embodiments, the value of the shift offset parameter is set to 32.

[0787] In some embodiments, the first determining unit 1201 is further configured to set the first constant value to an integer power equal to 2, wherein the exponent of the power is equal to the value of the shift quantity parameter minus 1.

[0788] In some embodiments, the first determining unit 1201 is further configured to perform a binary bit left shift on "1" to obtain the first constant value, wherein the number of bits shifted left is equal to the value of the shift quantity parameter minus 1.

[0789] In some embodiments, see Figure 12The encoder 120 may further include a first calculation unit 1205, configured to calculate a first weighted sum of the MIP weighting matrix and the MIP input sample value; calculate a first sum of the first weighted sum and the first offset; and perform a binary bit right shift on the first sum to obtain a first right shift value, wherein the number of bits shifted is equal to the value of the shift quantity parameter.

[0790] The first determining unit 1201 is further configured to set the MIP prediction value of the chromaticity component of the current block to be equal to the sum of the first right shift value and the value corresponding to index number 0 in the first temporary reference value; wherein the first temporary reference value is obtained based on downsampling filtering of the adjacent sampled values ​​of the current block.

[0791] In some embodiments, the first transformation unit 1203 is configured to determine the residual block of the current block based on the predicted residual value, and perform a first transformation on the residual block to obtain a first transformation coefficient block;

[0792] The first determining unit 1201 is further configured to determine whether to perform LFNST on at least a portion of the transform coefficients in the first transform coefficient block;

[0793] The first transformation unit 1203 is further configured to perform LFNST on at least a portion of the transformation coefficients in the first transformation coefficient block to determine the LFNST parameters if the determination result is yes.

[0794] In some embodiments, the first determining unit 1201 is further configured to determine the minimum value among the size parameters of the current block when the chroma components of the current block are determined using the MIP mode to determine the intra-prediction value; and if the minimum value is greater than or equal to a first preset threshold, determine to perform LFNST on at least a portion of the transform coefficients in the first transform coefficient block.

[0795] In some embodiments, the LFNST parameter includes an LFNST index number. The first determining unit 1201 is further configured to set the value of the LFNST index number to be greater than zero if it is determined that LFNST will be performed on at least some of the transform coefficients in the first transform coefficient block; and to set the value of the LFNST index number to be equal to zero if it is determined that LFNST will not be performed on at least some of the transform coefficients in the first transform coefficient block.

[0796] The encoding unit 1204 is also configured to encode the LFNST index number and write it into the code stream.

[0797] In some embodiments, the LFNST parameter further includes LFNST coefficients, see [link to relevant documentation]. Figure 12 The encoder 120 may further include a quantization unit 1206; wherein,

[0798] The first transformation unit 1203 is configured to perform LFNST on at least a portion of the transformation coefficients in the first transformation coefficient block to obtain the LFNST coefficients;

[0799] The quantization unit 1206 is configured to quantize the LFNST coefficients to obtain quantized coefficients;

[0800] The encoding unit 1204 is further configured to encode the quantization coefficients and write them into the bitstream.

[0801] Understandably, in the embodiments of this application, a "unit" can be a portion of a circuit, a portion of a processor, a portion of a program or software, etc., and can also be a module or a non-modular one. Furthermore, the components in this embodiment can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit described above can be implemented in hardware or as a software functional module.

[0802] If the integrated unit is implemented as a software functional module and not sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this embodiment, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the method described in this embodiment. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0803] Therefore, this application provides a computer storage medium applied to an encoder 120, the computer storage medium storing a computer program that, when executed by a first processor, implements the method described in any of the foregoing embodiments.

[0804] Based on the composition of the encoder 120 and the computer storage medium described above, see [link to documentation]. Figure 13 This illustrates a schematic diagram of the specific hardware structure of the encoder 120 provided in an embodiment of this application. Figure 13As shown, it may include: a first communication interface 1301, a first memory 1302, and a first processor 1303; the various components are coupled together through a first bus system 1304. It is understood that the first bus system 1304 is used to realize the connection and communication between these components. In addition to a data bus, the first bus system 1304 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in... Figure 13 The general designated all buses as the first bus system 1304. Among them,

[0805] The first communication interface 1301 is used for receiving and sending signals during the process of sending and receiving information with other external network elements;

[0806] The first memory 1302 is used to store computer programs that can run on the first processor 1303;

[0807] The first processor 1303 is configured to, when running the computer program, execute:

[0808] Determine the prediction parameters for the current block, wherein the prediction parameters include prediction mode parameters;

[0809] When the prediction mode parameter indicates that the matrix-based intra-prediction (MIP) mode is used to determine the intra-prediction value for the chrominance component of the current block, the adjacent sample values ​​of the current block are obtained, and the MIP input sample value of the current block is determined based on the adjacent sample values ​​of the current block.

[0810] Based on the MIP input sample value, the MIP weighting matrix, and the shift parameter, the MIP prediction value of the chromaticity component of the current block is determined; wherein, the MIP prediction value is the prediction value of a portion of the sample points in the chromaticity component of the current block;

[0811] The MIP prediction values ​​are filtered to determine the intra-frame prediction values ​​of the chroma components of the current block.

[0812] Based on the intra-frame prediction values ​​of the chrominance components of the current block, determine the prediction residual values ​​of the chrominance components of the current block;

[0813] Perform a low-frequency inseparable quadratic transform (LFNST) on the predicted residual values ​​to determine the LFNST parameters;

[0814] The LFNST parameters are encoded and written into the bitstream.

[0815] It is understood that the first memory 1302 in this embodiment can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate Synchronous DRAM (DDRSDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchlink DRAM (SLDRAM), and Direct Rambus RAM (DRRAM). The first memory 1302 of the system and method described in this application is intended to include, but is not limited to, these and any other suitable types of memory.

[0816] The first processor 1303 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of the first processor 1303 or by instructions in software form. The first processor 1303 may be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules may reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in the first memory 1302. The first processor 1303 reads the information in the first memory 1302 and completes the steps of the above method in conjunction with its hardware.

[0817] It is understood that the embodiments described in this application can be implemented using hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing unit can be implemented in one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), DSP devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions described in this application, or combinations thereof. For software implementation, the technology described in this application can be implemented through modules (e.g., procedures, functions, etc.) that perform the functions described in this application. Software code can be stored in memory and executed by a processor. The memory can be implemented in the processor or external to the processor.

[0818] Alternatively, as another embodiment, the first processor 1303 is further configured to perform the method described in any of the foregoing embodiments when running the computer program.

[0819] This embodiment provides an encoder that may include a first determining unit, a first predicting unit, a first transforming unit, and an encoding unit. In this way, for MIP modes, it can reduce complexity while maintaining encoding performance, and simultaneously reduce the storage space required during the encoding process, effectively improving encoding efficiency. Furthermore, when applied to MIP mode prediction, the LFNST technique, by introducing MIP parameters, makes the LFNST transform more flexible, further improving encoding efficiency.

[0820] In another embodiment of this application, based on the same inventive concept as the foregoing embodiments, see [link to previous embodiment]. Figure 14 This illustrates a schematic diagram of the composition structure of a decoder 140 provided in an embodiment of this application. Figure 14 As shown, the decoder 140 may include: a parsing unit 1401, a second determining unit 1402, a second predicting unit 1403, and a second transforming unit 1404; wherein,

[0821] The parsing unit 1401 is configured to parse the bitstream and obtain the prediction parameters and LFNST parameters of the current block, wherein the prediction parameters include prediction mode parameters;

[0822] The second determining unit 1402 is configured to, when the prediction mode parameter indicates that the intra-prediction value is determined by using a matrix-based intra-prediction (MIP) mode for the chroma components of the current block, acquire the adjacent sample values ​​of the current block and determine the MIP input sample value of the current block based on the adjacent sample values ​​of the current block.

[0823] The second prediction unit 1403 is configured to determine the MIP prediction value of the chroma component of the current block based on the MIP input sample value, the MIP weighting matrix and the shift parameter; wherein the MIP prediction value is the prediction value of a portion of the sample points in the chroma component of the current block; and to determine the intra-frame prediction value of the chroma component of the current block by filtering the MIP prediction value.

[0824] The second transformation unit 1404 is configured to, when the LFNST parameter indicates that LFNST is to be performed on the current block, determine the reconstruction transformation coefficient block of the current block, perform LFNST on at least a portion of the reconstruction transformation coefficients in the reconstruction transformation coefficient block to obtain a second transformation coefficient block; and perform a first transformation on the second transformation coefficient block to obtain the reconstruction residual block of the chromaticity components of the current block.

[0825] The second determining unit 1402 is further configured to determine the reconstructed block of the chroma component of the current block based on the intra-frame prediction value of the chroma component of the current block and the reconstructed residual block.

[0826] In some embodiments, the adjacent sample values ​​of the current block include the left adjacent sample value and the top adjacent sample value of the current block.

[0827] In some embodiments, the shift parameters include shift offset parameters and shift quantity parameters;

[0828] The second determining unit 1402 is specifically configured to: determine the product of the sum of the MIP input sample values ​​and the shift offset parameter based on the value of the shift offset parameter, wherein the value of the shift offset parameter is a fixed constant; determine a first constant value based on the value of the shift quantity parameter, wherein the value of the shift quantity parameter is a fixed constant; set the value of the first offset to the difference between the first constant value and the product; determine the MIP weighting matrix of the current block based on the prediction parameter; and determine the MIP prediction value of the chroma component of the current block based on the MIP weighting matrix, the MIP input sample values, the shift quantity parameter, and the first offset.

[0829] In some embodiments, the prediction parameters further include: the size parameter of the current block.

[0830] In some embodiments, the second determining unit 1402 is further configured to: determine the block size index value of the current block according to the size parameters of the current block; perform downsampling filtering on the adjacent sampled values ​​of the current block to obtain a first temporary reference value; and when the block size index value of the current block is within a preset range, determine a second constant value according to the bit depth of the adjacent sampled values ​​of the current block; set the value corresponding to index number 0 in the MIP input sampled values ​​to be equal to the difference between the second constant value and the value corresponding to index number 0 in the first temporary reference value; and set the... The value corresponding to index i in the MIP input sample value is set to the difference between the value corresponding to index i in the first temporary reference value and the value corresponding to index 0 in the first temporary reference value, where i is an integer greater than 0; when the block size index value of the current block is outside the preset range, the value corresponding to index j in the MIP input sample value is set to the difference between the value corresponding to index j+1 in the first temporary reference value and the value corresponding to index 0 in the first temporary reference value, where j is an integer greater than or equal to 0.

[0831] In some embodiments, the second determining unit 1402 is further configured to: when the block size index value of the current block is within a preset range, determine a second constant value based on the bit depth of the adjacent sample values ​​of the current block; set the value corresponding to index number 0 in the MIP input sample value to be equal to the difference between the value corresponding to index number 0 in the first temporary reference value and the second constant value; set the value corresponding to index number i in the MIP input sample value to be equal to the difference between the value corresponding to index number i in the first temporary reference value and the value corresponding to index number 0 in the first temporary reference value, where i is an integer greater than 0.

[0832] In some embodiments, the second determining unit 1402 is further configured to set the block size index value of the current block to 0 when the width and height of the current block are both equal to 4; set the block size index value of the current block to 1 when the width and height of the current block are both equal to 8, or when one of the width and height of the current block is equal to 4; and set the block size index value of the current block to 2 when the width and height of the current block do not meet the aforementioned conditions.

[0833] In some embodiments, the second determining unit 1402 is further configured to perform downsampling filtering on the adjacent sampled values ​​of the current block to obtain a first temporary reference value; and to determine a second constant value based on the bit depth of the adjacent sampled values ​​of the current block, cache the second constant value in a data unit after the first temporary reference value to obtain a second temporary reference value; and to set the value corresponding to index j in the MIP input sampled value to be equal to the difference between the value corresponding to index j+1 in the second temporary reference value and the value corresponding to index 0 in the second temporary reference value, wherein j is an integer greater than or equal to 0.

[0834] In some embodiments, the second determining unit 1402 is further configured to set the second constant value to an integer power equal to 2, wherein the exponent of the power is equal to the bit depth of the adjacent sample value of the current block minus 1.

[0835] In some embodiments, the second determining unit 1402 is further configured to perform a binary bit left shift on "1" to obtain the second constant value, wherein the number of bits shifted left is equal to the bit depth of the adjacent sample values ​​of the current block minus 1.

[0836] In some embodiments, the value of the shift quantity parameter is a fixed constant that is independent of the block size index value and the MIP mode index value.

[0837] In some embodiments, the value of the shift amount parameter is set to equal to 6.

[0838] In some embodiments, the value of the shift offset parameter is a fixed constant that is independent of the block size index value and the MIP mode index value.

[0839] In some embodiments, the value of the shift offset parameter is set to 32.

[0840] In some embodiments, the second determining unit 1402 is further configured to set the first constant value to an integer power equal to 2, wherein the exponent of the power is equal to the value of the shift quantity parameter minus 1.

[0841] In some embodiments, the second determining unit 1402 is further configured to perform a binary bit left shift on "1" to obtain the first constant value, wherein the number of bits shifted left is equal to the value of the shift quantity parameter minus 1.

[0842] In some embodiments, see Figure 14 The decoder 140 may further include a second computing unit 1405, configured to calculate a first weighted sum of the MIP weighting matrix and the MIP input sample value; calculate a first sum of the first weighted sum and the first offset; and perform a binary bit right shift on the first sum to obtain a first right shift value, wherein the number of bits shifted is equal to the value of the shift quantity parameter.

[0843] The second determining unit 1402 is further configured to set the MIP prediction value of the chromaticity component of the current block to be equal to the sum of the first right shift value and the value corresponding to index number 0 in the first temporary reference value; wherein the first temporary reference value is obtained based on downsampling filtering of the adjacent sampled values ​​of the current block.

[0844] In some embodiments, the second determining unit 1402 is further configured to determine whether the LFNST parameter exists in the bitstream;

[0845] The parsing unit 1401 is further configured to parse the bitstream and obtain the LFNST parameter if it is determined that the bitstream contains the LFNST parameter.

[0846] In some embodiments, the second determining unit 1402 is further configured to determine the minimum value among the size parameters of the current block when the chroma components of the current block are determined using the MIP mode to determine the intra-prediction value; and if the minimum value is greater than or equal to a first preset threshold, determine that the bitstream contains the LFNST parameter.

[0847] In some embodiments, see Figure 14 The decoder 140 may also include an inverse quantization unit 1406;

[0848] The parsing unit 1401 is further configured to parse the bitstream and obtain the quantization coefficients of the current block when the LFNST parameter indicates that LFNST is performed on the current block;

[0849] The dequantization unit 1406 is configured to dequantize the quantization coefficients to obtain the reconstruction transform coefficient block of the current block.

[0850] In some embodiments, the dequantization unit 1406 is specifically configured to perform a scaling operation on the quantization coefficients to obtain the reconstruction transform coefficient block of the current block.

[0851] In some embodiments, the parsing unit 1401 is specifically configured to parse the bitstream to obtain the value of the LFNST index number; and if the value of the LFNST index number is greater than zero, then determine to perform LFNST on the current block; if the value of the LFNST index number is equal to zero, then determine not to perform LFNST on the current block.

[0852] Understandably, in this embodiment, a "unit" can be a portion of a circuit, a portion of a processor, a portion of a program or software, etc., and can also be a module or a non-modular component. Furthermore, the components in this embodiment can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional module.

[0853] If the integrated unit is implemented as a software functional module and is not sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, this embodiment provides a computer storage medium applied to the decoder 140. This computer storage medium stores a computer program, which, when executed by a second processor, implements the method described in any of the foregoing embodiments.

[0854] Based on the composition of the decoder 140 and the computer storage medium described above, see [link to documentation]. Figure 15 This illustrates a schematic diagram of the specific hardware structure of the decoder 140 provided in an embodiment of this application. Figure 15 As shown, it may include: a second communication interface 1501, a second memory 1502, and a second processor 1503; the various components are coupled together via a second bus system 1504. It is understood that the second bus system 1504 is used to implement communication between these components. In addition to a data bus, the second bus system 1504 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in... Figure 15 The various buses are all labeled as the second bus system 1504. Among them,

[0855] The second communication interface 1501 is used for receiving and sending signals during the process of sending and receiving information with other external network elements;

[0856] The second memory 1502 is used to store computer programs that can run on the second processor 1503;

[0857] The second processor 1503 is configured to, when running the computer program, perform:

[0858] Parse the bitstream to obtain the prediction parameters and LFNST parameters of the current block, wherein the prediction parameters include prediction mode parameters;

[0859] When the prediction mode parameter indicates that the matrix-based intra-prediction (MIP) mode is used to determine the intra-prediction value for the chrominance component of the current block, the adjacent sample values ​​of the current block are obtained, and the MIP input sample value of the current block is determined based on the adjacent sample values ​​of the current block.

[0860] Based on the MIP input sample value, the MIP weighting matrix, and the shift parameter, the MIP prediction value of the chromaticity component of the current block is determined; wherein, the MIP prediction value is the prediction value of a portion of the sample points in the chromaticity component of the current block;

[0861] The MIP prediction values ​​are filtered to determine the intra-frame prediction values ​​of the chroma components of the current block.

[0862] When the LFNST parameter indicates that LFNST is performed on the current block, the reconstructed transform coefficient block of the current block is determined, and LFNST is performed on at least a portion of the reconstructed transform coefficients in the reconstructed transform coefficient block to obtain the second transform coefficient block;

[0863] The second transform coefficient block is transformed for the first time to obtain the reconstructed residual block of the chromaticity components of the current block;

[0864] The reconstruction block of the chroma component of the current block is determined based on the intra-frame prediction value of the chroma component of the current block and the reconstruction residual block.

[0865] Alternatively, as another embodiment, the second processor 1503 is also configured to perform the method described in any of the foregoing embodiments when running the computer program.

[0866] It is understood that the second memory 1502 has similar hardware functions to the first memory 1302, and the second processor 1503 has similar hardware functions to the first processor 1303; these will not be described in detail here.

[0867] This embodiment provides a decoder, which may include a parsing unit, a second determining unit, a second predicting unit, and a second transforming unit. In this way, for MIP modes, it can reduce complexity while maintaining decoding performance, and simultaneously reduce the storage space required during the decoding process, effectively improving decoding efficiency. Furthermore, when applied to MIP mode prediction, the LFNST technique, by introducing MIP parameters, makes the LFNST transform more flexible, further improving decoding efficiency.

[0868] In another embodiment of this application, see [reference needed]. Figure 16 This illustrates a schematic diagram of the composition of a transmitting device provided in an embodiment of this application. For example... Figure 16 As shown, an exemplary transmitting device 1600 is illustrated. Acquisition unit 1601 acquires a video signal and transmits it to encoder 1602. Acquisition unit 1601 may be a device including one or more cameras (including depth cameras). Acquisition unit 1601 may be a device that partially or completely decodes the bitstream to obtain video. Acquisition unit 1601 may also include one or more elements to capture audio signals. Encoder 1602 is specifically implemented as encoder 100 or encoder 120 as described in the foregoing embodiments, encoding the video signal from acquisition unit 1601 as its input video and generating a video bitstream. Encoder 1602 may also include one or more audio encoders to encode audio signals to generate an audio bitstream. Storage / transmission unit 1603 receives the video bitstream from encoder 1602. Storage / transmission unit 1603 may also receive the audio bitstream from encoder 1602 and compress the video and audio bitstreams together to form a media file (e.g., an ISO-based media file format) or transport stream.

[0869] Optionally, the storage / transmission unit 1603 writes media files or transmission streams to a storage unit, such as a hard drive, DVD, cloud, or portable storage device.

[0870] Optionally, the storage / transmission unit 1603 transmits the bit stream to a transmission network, such as the Internet, wired network, cellular network, wireless LAN, etc.

[0871] In another embodiment of this application, see [reference needed]. Figure 17 This illustrates a schematic diagram of the composition structure of a target device provided in an embodiment of this application. For example... Figure 17As shown, an exemplary target device 1700 is illustrated. The receiving unit 1701 receives media files or transport streams from a network, or reads media files or transport streams from a storage device. The receiving unit 1701 separates the video and audio bitstreams from the media file or transport stream. The receiving unit 1701 can also generate a new video bitstream by extracting the video bitstream. The receiving unit 1701 can also generate a new audio bitstream by extracting the audio bitstream. The decoder 1702 includes one or more video decoders, such as decoder 200 or decoder 140 as described in the foregoing embodiments.

[0872] Decoder 1702 may also include one or more audio decoders. Decoder 1702 decodes the video and audio streams from receiving unit 1701 to obtain decoded video and one or more decoded audio streams corresponding to one or more channels. Rendering unit 1703 performs operations on the reconstructed video to make it suitable for display. Such operations may include one or more of the following operations to improve perceptual quality: noise reduction, synthesis, color space conversion, upsampling, downsampling, etc. Rendering unit 1703 may also perform operations on the decoded audio to improve the perceptual quality of the displayed audio signal.

[0873] In another embodiment of this application, see [reference needed]. Figure 18 This illustrates a schematic diagram of the composition structure of a communication system provided in an embodiment of this application. For example... Figure 18 As shown, an exemplary communication system 1800 is illustrated. The source device 1801 can be... Figure 16 The transmitting device 1600 is shown. The output of the storage / transmission unit 1603 is processed by the storage medium / transmission network 1802 to store or transmit the bit stream. The target device 1803 may be... Figure 17 The target device 1700 is shown. Here, the receiving unit 1701 can acquire a bitstream from the storage medium / transmission network 1802. The receiving unit 1701 can extract a new video bitstream from a media file or transport stream. The receiving unit 1701 can also extract a new audio bitstream from a media file or transport stream.

[0874] It should be noted that, in this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0875] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0876] The methods disclosed in the several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0877] The features disclosed in the several product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0878] The features disclosed in the several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method or device embodiments.

[0879] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0880] Industrial applicability

[0881] In this embodiment of the application, on the encoder side, after determining the prediction parameters of the current block, when the prediction mode parameters indicate that the matrix-based intra-prediction (MIP) mode is used to determine the intra-prediction value of the chrominance component of the current block, the adjacent sample values ​​of the current block are obtained, and the MIP input sample value of the current block is determined based on the adjacent sample values ​​of the current block; the MIP prediction value of the chrominance component of the current block is determined based on the MIP input sample value, the MIP weighting matrix, and the shift parameter; the MIP prediction value is filtered to determine the intra-prediction value of the chrominance component of the current block; the prediction residual value of the chrominance component of the current block is determined based on the intra-prediction value of the chrominance component of the current block; the low-frequency non-separable quadratic transform (LFNST) is performed on the prediction residual value to determine the LFNST parameters; the LFNST parameters are encoded and written into the bitstream. On the decoder side, after parsing the bitstream and obtaining the prediction parameters and LFNST parameters of the current block, when the prediction mode parameter indicates that the matrix-based intra-prediction (MIP) mode is used to determine the intra-prediction value of the chrominance component of the current block, the adjacent sample values ​​of the current block are obtained, and the MIP input sample value of the current block is determined based on the adjacent sample values ​​of the current block; the MIP prediction value of the chrominance component of the current block is determined based on the MIP input sample value, the MIP weighting matrix, and the shift parameter; the MIP prediction value is filtered to determine the intra-prediction value of the chrominance component of the current block; when the LFNST parameter indicates that LFNST is performed on the current block, the reconstruction transform coefficient block of the current block is determined, and LFNST is performed on at least some of the reconstruction transform coefficients in the reconstruction transform coefficient block to obtain the second transform coefficient block; the second transform coefficient block is subjected to a first transform to obtain the reconstruction residual block of the chrominance component of the current block; the reconstruction block of the chrominance component of the current block is determined based on the intra-prediction value of the chrominance component of the current block and the reconstruction residual block. In this way, for MIP mode, the complexity can be reduced while ensuring encoding and decoding performance, and the storage space required during the decoding process can be reduced, effectively improving encoding and decoding efficiency. In addition, when LFNST technology is applied to MIP mode prediction, the introduction of MIP parameters makes the LFNST transform more flexible, further improving encoding and decoding efficiency.

Claims

1. An encoding method applied to an encoder, the method comprising: Determine the prediction parameters for the current block, wherein the prediction parameters include prediction mode parameters; When the prediction mode parameter indicates that matrix-based intra-prediction (MIP) is used to determine the intra-prediction value for the chrominance component of the current block, the adjacent sample values ​​of the current block are determined, and the MIP input sample value of the current block is determined based on the adjacent sample values ​​of the current block. Based on the MIP input sample value, MIP weighting matrix, shift offset parameter, and shift number parameter, the MIP prediction value of the chroma component of the current block is determined; wherein, the MIP prediction value is the prediction value of a portion of the sample points of the chroma component of the current block, the shift offset parameter is a first fixed value, and the shift number parameter is a second fixed value; The MIP prediction values ​​are filtered to determine the intra-frame prediction values ​​of the chroma components of the current block. Based on the intra-frame prediction values ​​of the chrominance components of the current block, determine the prediction residual values ​​of the chrominance components of the current block; The predicted residual value of the current block is transformed for the first time to obtain the first transformation coefficient; If it is determined that at least a portion of the first transform coefficients are subjected to low-frequency non-separable quadratic transform (LFNST), then at least a portion of the first transform coefficients are subjected to LFNST, and the LFNST coefficients are determined. Determine the value of the LFNST index number; The LFNST coefficients and the LFNST index number are encoded, and the encoded LFNST coefficients and the LFNST index number are written into the bitstream.

2. The method according to claim 1, wherein determining the MIP prediction value of the chromaticity component of the current block based on the MIP input sample value, the MIP weighting matrix, the shift offset parameter, and the shift amount parameter comprises: Determine the product of the sum of the MIP input sample values ​​and the value of the shift offset parameter; Set the value of the first offset to the difference between the first constant value and the product; Based on the prediction parameters, determine the MIP weighting matrix of the current block; Calculate the first weighted sum of the MIP weighting matrix and the MIP input sample values; Calculate the first sum of the first weighted sum and the first offset; The first sum is right-shifted by binary bits to obtain a first right-shifted value, wherein the number of bits shifted is equal to the value of the shift quantity parameter; The MIP prediction value of the chromaticity component of the current block is set to be equal to the sum of the first right shift value and the value corresponding to index number 0 in the first temporary reference value; wherein, the first temporary reference value is obtained by downsampling and filtering the adjacent sampled values ​​of the current block.

3. The method according to claim 1, characterized in that, The prediction parameters further include: the size parameter of the current block, and the step of determining the MIP input sample value of the current block based on the adjacent sample values ​​of the current block includes: Determine the block size index value of the current block based on the current block's size parameters; The adjacent sampled values ​​of the current block are downsampled and filtered to obtain a first temporary reference value; When the block size index value of the current block is within a preset range, the value corresponding to index number 0 in the MIP input sample value is set to be equal to the difference between the second constant value and the value corresponding to index number 0 in the first temporary reference value; the value corresponding to index number i in the MIP input sample value is set to be equal to the difference between the value corresponding to index number i in the first temporary reference value and the value corresponding to index number 0 in the first temporary reference value, where i is an integer greater than 0; When the block size index value of the current block is outside the preset range, the value corresponding to the index number j in the MIP input sample value is set to the difference between the value corresponding to the index number j+1 in the first temporary reference value and the value corresponding to the index number 0 in the first temporary reference value, where j is an integer greater than or equal to 0.

4. The method according to claim 3, wherein, The second constant value is equal to 1 << (BitDepth-1), where BitDepth represents the bit depth of the chroma component of the current block.

5. The method according to claim 2, wherein The value of the shift quantity parameter is equal to 6; The first constant value is equal to 32; and The value of the shift offset parameter is 32.

6. The method according to claim 1, wherein, Encoding the LFNST coefficients and writing the encoded LFNST coefficients into the bitstream includes: The LFNST coefficients are quantized to obtain quantized coefficients; The quantization coefficients are encoded, and the encoded quantization coefficients are written into the bitstream.

7. The method according to claim 6, wherein, The determination of the LFNST index number also includes: If it is determined that LFNST is performed on at least some of the first transform coefficients, then the value of the LFNST index number is set to be greater than zero; If it is determined that LFNST will not be performed on at least some of the first transform coefficients, then the value of the LFNST index number is set to zero.

8. A decoding method applied to a decoder, the method comprising: Parse the bitstream to obtain the prediction parameters and LFNST index number of the current block, wherein the prediction parameters include prediction mode parameters; When the prediction mode parameter indicates that matrix-based intra-prediction (MIP) is used to determine the intra-prediction value for the chrominance component of the current block, the adjacent sample values ​​of the current block are determined, and the MIP input sample value of the current block is determined based on the adjacent sample values ​​of the current block. Based on the MIP input sample value, MIP weighting matrix, shift offset parameter, and shift number parameter, the MIP prediction value of the chroma component of the current block is determined; wherein, the MIP prediction value is the prediction value of a portion of the sample points of the chroma component of the current block, the shift offset parameter is a first fixed value, and the shift number parameter is a second fixed value; The MIP prediction values ​​are filtered to determine the intra-frame prediction values ​​of the chroma components of the current block. When the LFNST index number indicates that LFNST is performed on the current block, the reconstruction transform coefficients of the current block are determined, and LFNST is performed on at least a portion of the reconstruction transform coefficients to obtain the second transform coefficients; The second transformation coefficients are transformed for the first time to obtain the reconstructed residual values ​​of the chromaticity components of the current block; The reconstructed value of the chroma component of the current block is determined based on the intra-frame prediction value of the chroma component of the current block and the reconstruction residual value.

9. The method according to claim 8, wherein, The step of determining the MIP prediction value of the chromaticity component of the current block based on the MIP input sample value, the MIP weighting matrix, the shift offset parameter, and the shift amount parameter includes: Determine the product of the sum of the MIP input sample values ​​and the value of the shift offset parameter; Set the value of the first offset to the difference between the first constant value and the product; Based on the prediction parameters, determine the MIP weighting matrix of the current block; Calculate the first weighted sum of the MIP weighting matrix and the MIP input sample values; Calculate the first sum of the first weighted sum and the first offset; The first sum is right-shifted by binary bits to obtain a first right-shifted value, wherein the number of bits shifted is equal to the value of the shift quantity parameter; The MIP prediction value of the chromaticity component of the current block is set to be equal to the sum of the first right shift value and the value corresponding to index number 0 in the first temporary reference value; wherein, the first temporary reference value is obtained by downsampling and filtering the adjacent sampled values ​​of the current block.

10. The method according to claim 8, characterized in that, The prediction parameters further include: the size parameter of the current block, and the step of determining the MIP input sample value of the current block based on the adjacent sample values ​​of the current block includes: Determine the block size index value of the current block based on the current block's size parameters; The adjacent sampled values ​​of the current block are downsampled and filtered to obtain a first temporary reference value; When the block size index value of the current block is within a preset range, the value corresponding to index number 0 in the MIP input sample value is set to be equal to the difference between the second constant value and the value corresponding to index number 0 in the first temporary reference value; the value corresponding to index number i in the MIP input sample value is set to be equal to the difference between the value corresponding to index number i in the first temporary reference value and the value corresponding to index number 0 in the first temporary reference value, where i is an integer greater than 0; When the block size index value of the current block is outside the preset range, the value corresponding to the index number j in the MIP input sample value is set to the difference between the value corresponding to the index number j+1 in the first temporary reference value and the value corresponding to the index number 0 in the first temporary reference value, where j is an integer greater than or equal to 0.

11. The method according to claim 10, wherein, The second constant value is equal to 1 << (BitDepth-1), where BitDepth represents the bit depth of the chroma component of the current block.

12. The method according to claim 9, wherein, The value of the shift quantity parameter is equal to 6; The first constant value is equal to 32; and The value of the shift offset parameter is 32.

13. The method according to claim 8, wherein, The parsing of the bitstream, obtaining the LFNST index number, includes: If the value of the LFNST index number is greater than zero, then it is determined that LFNST will be performed on the current block; If the value of the LFNST index number is equal to zero, then it is determined that LFNST will not be performed on the current block.

14. The method according to claim 8, wherein, Determining the reconstruction transformation coefficients of the current block includes: Parse the bitstream to obtain the quantization coefficients of the current block; The quantization coefficients are scaled to obtain the reconstruction transformation coefficients of the current block.

15. A decoder comprising at least one processor and a memory, the memory being coupled to the at least one processor and storing at least one computer-executable instruction, which, when executed by the at least one processor, causes the at least one processor to perform the method according to any one of claims 8-14.

16. An encoder comprising at least one processor and a memory, the memory being coupled to the at least one processor and storing at least one computer-executable instruction, which, when executed by the at least one processor, causes the at least one processor to perform the method according to any one of claims 1-7.

17. A computer-readable storage medium, characterized in that, The device stores a computer program and a bitstream, which, when executed by one or more processors, causes the one or more processors to implement the method as described in any one of claims 1 to 7 to generate the bitstream.

Images