Decoding method, encoding method, decoder, encoder and code stream

Abstract

The embodiment of the invention discloses a decoding method, an encoding method, a decoder, an encoder and a code stream, and the method comprises the steps: determining a prediction parameter of a current block, and the prediction parameter comprises a prediction mode parameter; when the prediction mode parameter indicates that an intra-frame prediction value is determined by using a matrix-based intra-frame prediction MIP mode for a chrominance component of the current block, obtaining an adjacent sampling value of the current block, and determining an MIP input sampling value of the current block according to the adjacent sampling value of the current block; determining an MIP predicted value of the chrominance component of the current block according to the MIP input sampling value, the MIP weighting matrix and the shift parameter; performing filtering processing on the MIP prediction value, and determining an intra-frame prediction value of a chrominance component of the current block; determining a prediction residual value of the chrominance component of the current block according to the intra-frame prediction value of the chrominance component of the current block; and performing low-frequency inseparable secondary transformation (LFNST) on the prediction residual value, and determining an LFNST parameter.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to the prior U.S. provisional patent application No. 62 / 958,582, filed on January 8, 2020 in the name of Junyan Huo, Shuai Wan, and Yanzhuo Ma, entitled “VIDEO ENCODING AND DECODING METHOD, APPARATUS AND COMMUNICATION SYSTEM,” the entire contents of which are incorporated herein by reference. Technical Field

[0003] The embodiments of the present application relate to the field of video coding and decoding technology, and in particular, to a decoding method, an encoding method, a decoder, an encoder, and a bit stream. Background Art

[0004] As people's requirements for video display quality increase, new video applications such as high-definition and ultra-high-definition video have emerged. H.265 / High Efficiency Video Coding (HEVC) can no longer meet the needs of the rapid development of video applications. The Joint Video Exploration Team (JVET) has proposed a new generation of video coding standard H.266 / Versatile Video Coding (VVC).

[0005] In H.266 / VVC, Matrix-based Intra Prediction (MIP) is an intra prediction mode used to obtain the intra prediction block of the current block; then the Low-Frequency Non-Separable Transform (LFNST) technology is used to determine the prediction residual of the current block. However, when LFNST technology is applied to MIP mode prediction, it has a negative impact on the encoding and decoding efficiency of VVC. Summary of the invention

[0006] The embodiments of the present application provide an encoding method, a decoding method, an encoder, a decoder and a storage medium, which can improve the encoding and decoding efficiency.

[0007] The technical solution of the embodiment of the present application can be implemented as follows:

[0008] In a first aspect, an embodiment of the present application provides an encoding method, which is applied to an encoder, and the method includes:

[0009] Determining prediction parameters of the current block, wherein the prediction parameters include prediction mode parameters;

[0010] When the prediction mode parameter indicates that a matrix-based intra prediction MIP mode is used to determine an intra prediction value for a chrominance component of the current block, obtaining adjacent sample values ​​of the current block, and determining a MIP input sample value of the current block according to the adjacent sample values ​​of the current block;

[0011] Determine the MIP prediction value of the chrominance component of the current block according to the MIP input sample value, the MIP weighting matrix and the shift parameter; wherein the MIP prediction value is the prediction value of some sampling points in the chrominance component of the current block;

[0012] Performing filtering processing on the MIP prediction value to determine an intra-frame prediction value of the chrominance component of the current block;

[0013] Determining a prediction residual value of a chrominance component of the current block according to an intra-frame prediction value of a chrominance component of the current block;

[0014] Performing a low frequency non-separable secondary transform LFNST on the prediction residual value to determine LFNST parameters;

[0015] The LFNST parameters are encoded and written into the bitstream.

[0016] In a second aspect, an embodiment of the present application provides a decoding method, which is applied to a decoder, and the method includes:

[0017] Parse the bitstream to obtain prediction parameters and LFNST parameters of the current block, wherein the prediction parameters include prediction mode parameters;

[0018] When the prediction mode parameter indicates that a matrix-based intra prediction MIP mode is used to determine an intra prediction value for a chrominance component of the current block, obtaining adjacent sample values ​​of the current block, and determining a MIP input sample value of the current block according to the adjacent sample values ​​of the current block;

[0019] Determine the MIP prediction value of the chrominance component of the current block according to the MIP input sample value, the MIP weighting matrix and the shift parameter; wherein the MIP prediction value is the prediction value of some sampling points in the chrominance component of the current block;

[0020] Performing filtering processing on the MIP prediction value to determine an intra-frame prediction value of the chrominance component of the current block;

[0021] When the LFNST parameter indicates that LFNST is performed on the current block, determining a reconstructed transform coefficient of the current block, and performing LFNST on at least part of the reconstructed transform coefficients to obtain a second transform coefficient;

[0022] Performing a first transform on the second transform coefficient to obtain a reconstructed residual value of the chrominance component of the current block;

[0023] A reconstructed value of the chrominance component of the current block is determined according to the intra-frame prediction value of the chrominance component of the current block and the reconstructed residual value.

[0024] In a third aspect, an embodiment of the present application provides an encoder, the encoder comprising a first memory and a first processor; wherein:

[0025] The first memory is used to store a computer program that can be run on the first processor;

[0026] The first processor is configured to execute the method according to the first aspect when running the computer program.

[0027] In a fourth aspect, an embodiment of the present application provides a decoder, the decoder comprising a second memory and a second processor; wherein:

[0028] The second memory is used to store a computer program that can be run on the second processor;

[0029] The second processor is used to execute the method as described in the second aspect when running the computer program.

[0030] In a fifth aspect, an embodiment of the present application provides a computer storage medium, which stores a computer program. When the computer program is executed by a first processor, it implements the method as described in the first aspect, or when it is executed by a second processor, it implements the method as described in the second aspect.

[0031] Embodiments of the present application provide an encoding method, a decoding method, an encoder, a decoder and a storage medium. On the encoder side, after determining the prediction parameters of the current block, when the prediction mode parameter indicates that the matrix-based intra-frame prediction MIP mode is used to determine the intra-frame prediction value for the chrominance component of the current block, the adjacent sampling values ​​of the current block are obtained, and the MIP input sampling values ​​of the current block are determined according to the adjacent sampling values ​​of the current block; the MIP prediction value of the chrominance component of the current block is determined according to the MIP input sampling value, the MIP weighting matrix and the shift parameter; the MIP prediction value is filtered to determine the intra-frame prediction value of the chrominance component of the current block; the prediction residual value of the chrominance component of the current block is determined according to the intra-frame prediction value of the chrominance component of the current block; a low-frequency non-separable secondary transform LFNST is performed on the prediction residual value to determine the LFNST parameters; the LFNST parameters are encoded and written into a bitstream. On the decoder side, after parsing the bitstream and obtaining the prediction parameters and LFNST parameters of the current block, when the prediction mode parameter indicates that the MIP mode is used to determine the intra-frame prediction value for the chrominance component of the current block, the adjacent sampling values ​​of the current block are obtained, and the MIP input sampling values ​​of the current block are determined according to the adjacent sampling values ​​of the current block; the MIP prediction value of the chrominance component of the current block is determined according to the MIP input sampling value, the MIP weighting matrix and the shift parameter; the MIP prediction value is filtered to determine the intra-frame prediction value of the chrominance component of the current block; when the LFNST parameter indicates that LFNST is performed on the current block, the reconstructed transform coefficient of the current block is determined, and LFNST is performed on at least part of the reconstructed transform coefficients to obtain a second transform coefficient; the second transform coefficient is transformed for the first time to obtain a reconstructed residual value of the chrominance component of the current block; and the reconstructed value of the chrominance component of the current block is determined according to the intra-frame prediction value of the chrominance component of the current block and the reconstructed residual value. In this way, for the MIP mode, the complexity can be reduced while ensuring the encoding and decoding performance, while reducing the storage space required in the encoding and decoding process, effectively improving the encoding and decoding efficiency; in addition, when LFNST technology is applied to MIP mode prediction, the introduction of MIP parameters makes the LFNST transformation more flexible, further improving the encoding and decoding efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 A schematic diagram of the structure of an encoder provided in an embodiment of the present application;

[0033] Figure 2 A schematic diagram of the structure of a decoder provided in an embodiment of the present application;

[0034] Figure 3 A schematic diagram of a flow chart of an encoding method provided in an embodiment of the present application;

[0035] Figure 4A flowchart of another encoding method provided in an embodiment of the present application;

[0036] Figure 5 A schematic diagram of a flow chart of an MPI prediction process provided in an embodiment of the present application;

[0037] Figure 6 This is a schematic diagram of a process of obtaining an intra-frame prediction block using the MIP mode in an embodiment of the present application;

[0038] Figure 7 A schematic diagram of a process framework for executing LFNST provided in an embodiment of the present application;

[0039] Figure 8 A schematic diagram of a process for encoding LFNST parameters provided in an embodiment of the present application;

[0040] Fig. 9 A flowchart of a decoding method provided in an embodiment of the present application;

[0041] Fig.10 A schematic diagram of a process for parsing LFNST parameters provided in an embodiment of the present application;

[0042] Fig.11 A schematic diagram of another process framework for executing LFNST provided in an embodiment of the present application;

[0043] Fig.12 A schematic diagram of the composition structure of another encoder provided in an embodiment of the present application;

[0044] Fig.13 A schematic diagram of a specific hardware structure of an encoder provided in an embodiment of the present application;

[0045] Fig.14 A schematic diagram of the composition structure of another decoder provided in an embodiment of the present application;

[0046] Fig.15 A schematic diagram of a specific hardware structure of a decoder provided in an embodiment of the present application;

[0047] Fig.16 A schematic diagram of the structure of a sending device provided in an embodiment of the present application;

[0048] Fig.17 A schematic diagram of the structure of a target device provided in an embodiment of the present application;

[0049] Fig.18 A schematic diagram of the structure of a communication system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0050] In order to enable a more detailed understanding of the features and technical contents of the embodiments of the present application, the implementation of the embodiments of the present application is described in detail below in conjunction with the accompanying drawings. The attached drawings are for reference only and are not used to limit the embodiments of the present application.

[0051] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0052] In the following description, "some embodiments" are referred to, which describe a subset of all possible embodiments, but it is understood that "some embodiments" can be the same subset or different subsets of all possible embodiments, and can be combined with each other without conflict. It should also be pointed out that the terms "first\second\third" involved in the embodiments of the present application are only used to distinguish similar objects and do not represent a specific order for the objects. It is understandable that "first\second\third" can be interchanged in a specific order or sequence where permitted, so that the embodiments of the present application described here can be implemented in an order other than that illustrated or described here.

[0053] With the development of video compression technology, the International Telecommunication Union-Telecommunication (ITU-T) and the International Organization for Standardization (ISO) / International Electrotechnical Commission (IEC) launched a standardization project called VVC to develop a new generation of video coding standards, the purpose of which is to improve the performance of VVC by about 50% compared with the H.265 / HEVC standard when encoding high-quality videos with one or more features of high resolution, high frame rate, high bit depth, high dynamic range, wide color gamut and omnidirectional viewing angle. The Joint Video Exploration Team (JVET) of ITU-T SG 16WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11 is responsible for the standardization project. Among them, various intra-frame prediction modes and inter-frame prediction modes have been verified to achieve high compression efficiency when encoding high-quality videos, and therefore have been adopted in the Working Draft (WD) of VVC.

[0054] Here, the matrix-based intra prediction (MIP) mode is an intra prediction mode. In VVC, the encoder or decoder can call the MIP mode to obtain the intra prediction block of the current block. Then, the low frequency non-separable transform (LFNST) as a secondary transform (or second transform) can be applied to the prediction residual of the current block.

[0055] In the related art, although LFNST can be applied to MIP mode prediction with a fixed configuration of LFNST transform kernel candidate set and LFNST transposition indication parameters, this has a negative impact on the encoding and decoding efficiency of VVC.

[0056] Based on this, an embodiment of the present application provides a coding method, the basic idea of ​​which is: determining prediction parameters of a current block, wherein the prediction parameters include prediction mode parameters; when the prediction mode parameters indicate that a matrix-based intra-frame prediction MIP mode is used to determine an intra-frame prediction value for the chrominance component of the current block, obtaining adjacent sampling values ​​of the current block, and determining the MIP input sampling value of the current block according to the adjacent sampling values ​​of the current block; determining the MIP prediction value of the chrominance component of the current block according to the MIP input sampling value, the MIP weighting matrix and the shift parameter; filtering the MIP prediction value to determine the intra-frame prediction value of the chrominance component of the current block; determining the prediction residual value of the chrominance component of the current block according to the intra-frame prediction value of the chrominance component of the current block; performing a low-frequency non-separable secondary transform LFNST on the prediction residual value to determine the LFNST parameters; encoding the LFNST parameters and writing them into the bitstream. Another embodiment of the present application provides a decoding method, the basic idea of ​​which is: parsing a bitstream to obtain prediction parameters and LFNST parameters of a current block, wherein the prediction parameters include prediction mode parameters; when the prediction mode parameters indicate that a matrix-based intra-frame prediction MIP mode is used to determine an intra-frame prediction value for a chrominance component of the current block, obtaining adjacent sampling values ​​of the current block, and determining a MIP input sampling value of the current block according to the adjacent sampling values ​​of the current block; determining a chrominance component of the current block according to the MIP input sampling value, a MIP weighting matrix, and a shift parameter MIP prediction value; filtering the MIP prediction value to determine the intra-frame prediction value of the chrominance component of the current block; when the LFNST parameter indicates to perform LFNST on the current block, determine the reconstructed transform coefficient block of the current block, perform LFNST on at least part of the reconstructed transform coefficients in the reconstructed transform coefficient block to obtain a second transform coefficient block; perform a first transform on the second transform coefficient block to obtain a reconstructed residual block of the chrominance component of the current block; determine the reconstructed block of the chrominance component of the current block according to the intra-frame prediction value of the chrominance component of the current block and the reconstructed residual block. In this way, for the MIP mode, the complexity can be reduced on the basis of ensuring the encoding and decoding performance, and the storage space required in the encoding and decoding process can be reduced, thereby effectively improving the encoding and decoding efficiency; in addition, when the LFNST technology is applied to the MIP mode prediction, the MIP parameters are introduced, so that the LFNST transform is more flexible, further improving the encoding and decoding efficiency.

[0057] The embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0058] See also Figure 1 , which shows an example of a system composition block diagram of an encoder provided in an embodiment of the present application. Figure 1As shown, the encoder 100 may include: a segmentation unit 101, a prediction unit 102, a first adder 107, a transform unit 108, a quantization unit 109, a dequantization unit 110, an inverse transform unit 111, a second adder 112, a filtering unit 113, a decoded picture buffer (DPB) unit 114 and an entropy coding unit 115. Here, the input of the encoder 100 may be a video consisting of a series of pictures or a static picture, and the output of the encoder 100 may be a bit stream (also referred to as a "code stream") for representing a compressed version of the input video.

[0059] Among them, the segmentation unit 101 divides the picture in the input video into one or more coding tree units (CTUs). The segmentation unit 101 divides the picture into multiple tiles, and can further divide a tile into one or more bricks, where a tile or a brick may include one or more complete and / or partial CTUs. In addition, the segmentation unit 101 can form one or more slices, where a slice can include one or more tiles arranged in a grid order in the picture, or one or more tiles covering a rectangular area in the picture. The segmentation unit 101 can also form one or more sub-pictures, where a sub-picture can include one or more slices, tiles or bricks.

[0060] During the encoding process of the encoder 100, the segmentation unit 101 transmits the CTU to the prediction unit 102. Generally, the prediction unit 102 can be composed of a block segmentation unit 103, a motion estimation (ME) unit 104, a motion compensation (MC) unit 105, and an intra-frame prediction unit 106. Specifically, the block segmentation unit 103 iteratively uses quadtree segmentation, binary tree segmentation, and ternary tree segmentation to further divide the input CTU into smaller coding units (Coding Units, CUs). The prediction unit 102 can use the ME unit 104 and the MC unit 105 to obtain the inter-frame prediction block of the CU. The intra-frame prediction unit 106 can use various intra-frame prediction modes including the MIP mode to obtain the intra-frame prediction block of the CU. In the example, the rate-distortion optimized motion estimation method can be called by the ME unit 104 and the MC unit 105 to obtain the inter-frame prediction block, and the rate-distortion optimized mode determination method can be called by the intra-frame prediction unit 106 to obtain the intra-frame prediction block.

[0061] The prediction unit 102 outputs the prediction block of the CU, and the first adder 107 calculates the difference between the CU in the output of the segmentation unit 101 and the prediction block of the CU, that is, the residual CU. The transform unit 108 reads the residual CU and performs one or more transform operations on the residual CU to obtain coefficients. The quantization unit 109 quantizes the coefficients and outputs the quantized coefficients (i.e., levels). The dequantization unit 110 performs a scaling operation on the quantized coefficients to output the reconstructed coefficients. The inverse transform unit 111 performs one or more inverse transforms corresponding to the transform in the transform unit 108 and outputs the reconstructed residual. The second adder 112 calculates the reconstructed CU by adding the reconstructed residual and the prediction block of the CU from the prediction unit 102. The second adder 112 also sends its output to the prediction unit 102 for use as an intra-frame prediction reference. After all CUs in the picture or sub-picture are reconstructed, the filtering unit 113 performs loop filtering on the reconstructed picture or sub-picture. Here, the filtering unit 113 includes one or more filters, such as a deblocking filter, a sample adaptive offset (Sample Adaptive Offset, SAO) filter, an adaptive loop filter (AdaptiveLoop Filter, ALF), a luminance mapping and chroma scaling (Luma Mapping with Chroma Scaling, LMCS) filter, and a neural network-based filter, etc. Alternatively, when the filtering unit 113 determines that the CU is not used as a reference for encoding other CUs, the filtering unit 113 performs loop filtering on one or more target pixels in the CU.

[0062] The output of the filtering unit 113 is a decoded picture or sub-picture, which is cached to the DPB unit 114. The DPB unit 114 outputs the decoded picture or sub-picture according to the timing and control information. Here, the picture stored in the DPB unit 114 can also be used as a reference for the prediction unit 102 to perform inter-frame prediction or intra-frame prediction. Finally, the entropy coding unit 115 converts the parameters (such as control parameters and supplementary information, etc.) necessary for obtaining the decoded picture from the encoder 100 into a binary form, and writes such a binary form into the bitstream according to the grammatical structure of each data unit, that is, the encoder 100 finally outputs the bitstream.

[0063] Furthermore, the encoder 100 may have a first processor and a first memory for recording a computer program. When the first processor reads and runs the computer program, the encoder 100 reads the input video and generates a corresponding bitstream. In addition, the encoder 100 may also be a computing device having one or more chips. These units implemented as integrated circuits on the chip have the same Figure 1 Similar connection and data exchange functions to corresponding units in the.

[0064] See also Figure 2, which shows an example of a system composition block diagram of a decoder provided in an embodiment of the present application. Figure 2 As shown, the decoder 200 may include: a parsing unit 201, a prediction unit 202, a scaling unit 205, a transform unit 206, an adder 207, a filtering unit 208, and a decoded picture buffer unit 209. Here, the input of the decoder 200 is a bit stream for representing a compressed version of a video or a static picture, and the output of the decoder 200 may be a decoded video consisting of a series of pictures or a decoded static picture.

[0065] The input code stream of the decoder 200 may be a code stream generated by the encoder 100. The parsing unit 201 parses the input code stream and obtains the value of the syntax element from the input code stream. The parsing unit 201 converts the binary representation of the syntax element into a digital value and sends the digital value to the unit in the decoder 200 to obtain one or more decoded pictures. The parsing unit 201 can also parse one or more syntax elements from the input code stream to display the decoded picture.

[0066] During the decoding process of the decoder 200 , the parsing unit 201 sends the values ​​of the syntax elements and one or more variables set or determined according to the values ​​of the syntax elements and used to obtain one or more decoded pictures to the units in the decoder 200 .

[0067] The prediction unit 202 determines a prediction block of a current decoding block (e.g., CU). Here, the prediction unit 202 may include a motion compensation unit 203 and an intra prediction unit 204. Specifically, when the inter-frame decoding mode is indicated to be used for decoding the current decoding block, the prediction unit 202 passes the relevant parameters from the parsing unit 201 to the motion compensation unit 203 to obtain the inter-frame prediction block; when the intra-frame prediction mode (including the MIP mode indicated based on the MIP mode index value) is indicated to be used for decoding the current decoding block, the prediction unit 202 transmits the relevant parameters from the parsing unit 201 to the intra-frame prediction unit 204 to obtain the intra-frame prediction block.

[0068] The scaling unit 205 has the same function as the inverse quantization unit 110 in the encoder 100. The scaling unit 205 performs a scaling operation on the quantization coefficients (ie, levels) from the parsing unit 201 to obtain reconstructed coefficients.

[0069] The transform unit 206 has the same function as the inverse transform unit 111 in the encoder 100. The transform unit 206 performs one or more transform operations (ie, inverse operations of one or more transform operations performed by the inverse transform unit 111 in the encoder 100) to obtain a reconstructed residual.

[0070] The adder 207 performs an addition operation on its input (the prediction block from the prediction unit 202 and the reconstructed residual from the transform unit 206) to obtain a reconstructed block of the current decoded block. The reconstructed block is also sent to the prediction unit 202 to be used as a reference for other blocks encoded in the intra prediction mode.

[0071] After all CUs in the picture or sub-picture are reconstructed, the filtering unit 208 performs loop filtering on the reconstructed picture or sub-picture. The filtering unit 208 includes one or more filters, such as a deblocking filter, a sampling adaptive compensation filter, an adaptive loop filter, a luminance mapping and chrominance scaling filter, and a filter based on a neural network. Alternatively, when the filtering unit 208 determines that the reconstructed block is not used as a reference for decoding other blocks, the filtering unit 208 performs loop filtering on one or more target pixels in the reconstructed block. Here, the output of the filtering unit 208 is a decoded picture or sub-picture, and the decoded picture or sub-picture is cached to the DPB unit 209. The DPB unit 209 outputs the decoded picture or sub-picture according to the timing and control information. The picture stored in the DPB unit 209 can also be used as a reference for performing inter-frame prediction or intra-frame prediction by the prediction unit 202.

[0072] Furthermore, the decoder 200 may have a second processor and a second memory for recording the computer program. When the first processor reads and runs the computer program, the decoder 200 reads the input code stream and generates a corresponding decoded video. In addition, the decoder 200 may also be a computing device having one or more chips. These units implemented as integrated circuits on the chip have the same Figure 2 Similar connection and data exchange functions to corresponding units in the.

[0073] It should be noted that the encoding method provided in the embodiment of the present application mainly acts on the intra-frame prediction unit 106 and the transform unit 108 of the encoder 100, and the decoding method provided in the embodiment of the present application mainly acts on the intra-frame prediction unit 204 and the transform unit 206 of the decoder 200. In other words, the embodiment of the present application can be applied to both the encoder and the decoder, and can even be applied to both the encoder and the decoder at the same time, but no limitation is made here.

[0074] It should also be noted that when the embodiment of the present application is applied to the encoder 100, the "current block" specifically refers to the current image block to be encoded in the intra-frame prediction (also referred to as the "encoding block"); when the embodiment of the present application is applied to the decoder 200, the "current block" specifically refers to the current image block to be decoded in the intra-frame prediction (also referred to as the "decoding block").

[0075] In addition, if the encoder 100 can obtain a better prediction effect through the encoding method provided in the embodiment of the present application to improve the encoding performance; then, correspondingly, the decoder 200 can also improve the video decoding recovery quality through the decoding method provided in the embodiment of the present application, thereby improving the decoding performance.

[0076] In one embodiment of the present application, see Figure 3 , which shows a schematic diagram of a flow chart of an encoding method provided in an embodiment of the present application. Figure 3 As shown, the method may include:

[0077] S301: Determine prediction parameters of a current block, wherein the prediction parameters include prediction mode parameters.

[0078] It should be noted that the image to be encoded can be divided into multiple image blocks, and each image block to be encoded can be called a coding block. Here, each coding block may include a first image component, a second image component, and a third image component. The current block is a coding block in the video image that is currently to be predicted by the first image component, the second image component, or the third image component.

[0079] Here, assuming that the first image component prediction is performed on the current block, the first image component is the brightness component, that is, the image component to be predicted is the brightness component, then the current block can also be called a brightness block to obtain the predicted value of the brightness component of the current block. Alternatively, assuming that the second image component prediction is performed on the current block, the second image component is the chrominance component, that is, the image component to be predicted is the chrominance component, then the current block can also be called a chrominance block to obtain the predicted value of the chrominance component of the current block.

[0080] It should also be noted that the prediction parameters may include prediction mode parameters, wherein the prediction mode parameters are used to indicate the prediction mode adopted by the current block, and different prediction modes correspond to different prediction mode parameters. Here, for determining the prediction mode parameters, a simple decision strategy may be adopted, such as determining according to the size of the distortion value; or a complex decision strategy may be adopted, such as determining according to the result of rate distortion optimization (RDO), which is not limited in any way in the embodiments of the present application. Generally speaking, the RDO method may be used to determine the prediction mode parameters of the current block, that is, to determine the prediction parameters of the current block.

[0081] Specifically, in some embodiments, determining the prediction parameter of the current block may include:

[0082] Pre-encoding the current block using multiple prediction modes to obtain rate-distortion cost values ​​corresponding to the multiple prediction modes;

[0083] An optimal rate-distortion cost value is selected from the obtained multiple rate-distortion cost values, and prediction parameters in the prediction mode corresponding to the optimal rate-distortion cost value are determined as prediction parameters of the current block.

[0084] That is to say, on the encoder side, multiple prediction modes can be used for the current block to perform pre-coding processing on the current block respectively. Here, the multiple prediction modes generally include inter-frame prediction mode, traditional intra-frame prediction mode and non-traditional intra-frame prediction mode. Among them, the traditional intra-frame prediction mode may include direct current (DC) mode, plane (PLANAR) mode and angle intra-frame prediction mode. Non-traditional intra-frame prediction modes may include MIP mode, cross-component linear model prediction (CCLM) mode, intra-frame block copy (IBC) mode and PLT (Palette) mode. The inter-frame prediction mode may include traditional inter-frame prediction mode and inter-frame geometric partitioning prediction (GEO) mode.

[0085] In this way, after pre-coding the current block using multiple prediction modes respectively, the rate-distortion cost values ​​corresponding to the multiple prediction modes can be obtained. Then, the optimal rate-distortion cost value is selected from the multiple rate-distortion cost values ​​obtained (usually, the minimum rate-distortion cost value is the optimal rate-distortion cost value), and the prediction parameters under the prediction mode corresponding to the optimal rate-distortion cost value are determined as the prediction parameters of the current block.

[0086] In addition, multiple prediction modes can be used to pre-encode the current block respectively to obtain distortion values ​​corresponding to the multiple prediction modes. Then, the optimal distortion value is selected from the obtained multiple distortion values, and the prediction parameters under the prediction mode corresponding to the optimal distortion value are determined as the prediction parameters of the current block. In this way, the encoder finally uses the prediction mode indicated by the prediction mode parameter in the determined prediction parameters to predict the current block, which can make the prediction residual smaller and improve the coding efficiency.

[0087] S302: When the prediction mode parameter indicates that a matrix-based intra-prediction MIP mode is used to determine an intra-prediction value for the chrominance component of the current block, adjacent sampling values ​​of the current block are obtained, and the MIP input sampling value of the current block is determined according to the adjacent sampling values ​​of the current block.

[0088] It should be noted that, for the current block, the embodiment of the present application uses the MIP mode to perform intra-frame prediction on the chrominance component of the current block. In this process, it is first necessary to obtain the adjacent sampling values ​​of the current block; then, based on the adjacent sampling values ​​of the current block, the MIP input sampling values ​​of the current block are determined.

[0089] It should be understood that for the MIP mode, the MIP core parameters need to be configured first. Here, the MIP core parameters may include the type of the current block (represented by mipSizeId), the number of reference samples for each edge (represented by boundySize), the number of MIP input samples (represented by inSize), and the MIP prediction block size output by matrix multiplication (arranged as predSize×predSize). Among them, the MIP mode can divide the current block into three categories according to the width and height of the current block, and mipSizeId can be equal to 0, 1 or 2. Here, mipSizeId represents the type of the current block, that is, the "block size index value of the current block" shown in the embodiment of the present application. For different mipSizeId, the number of reference samples (boundySize reference sampling points are required for each edge), the number of MIP input samples (inSize), and the MIP prediction block size output by matrix multiplication (arranged as predSize×predSize) are also different.

[0090] It should also be noted that, in addition to the prediction mode parameters, the prediction parameters may also include: the size parameters of the current block. The size parameters of the current block may include: the width (expressed as nTbW) and the height (expressed as nTbH) of the current block. And according to the size parameters of the current block, the block size index value (i.e., mipSizeId) of the current block may be determined.

[0091] In a possible implementation manner, determining the block size index value of the current block according to the size parameter of the current block may include:

[0092] If the width and height of the current block are both equal to 4, then the block size index value of the current block can be set to 0;

[0093] On the contrary, if the width and height of the current block are both equal to 8, or one of the width and height of the current block is equal to 4, then the block size index value of the current block may be set to 1;

[0094] On the contrary, if the current block is a block of another size, the block size index value of the current block may be set to 2.

[0095] In another possible implementation, determining the block size index value of the current block according to the size parameter of the current block may include:

[0096] If the width and height of the current block are both equal to 4, then the block size index value of the current block can be set to 0;

[0097] Conversely, if one of the width and height of the current block is equal to 4, the block size index value of the current block may be set to 1;

[0098] On the contrary, if the current block is a block of another size, the block size index value of the current block may be set to 2.

[0099] Thus, according to the block size index value of the current block, the number of adjacent boundary reference samples (variable is boundarySize) and the size of the MIP prediction block (variable is predSize, the size of the MIP prediction block is predSize×predSize) can be determined according to the look-up table (LUT) shown in Table 1, and the number of MIP input sample values ​​(expressed by inSize) used for the MIP matrix multiplication operation process can be calculated. The calculation formula is as follows:

[0100] inSize=(2×boundarySize)-(mipSizeId==2)? 1:0 (1)

[0101] The operation rules of the operators in formula (1) are the same as those of the operators defined in the ITU-T H.265 standard. For example, “==” is a logical “equal to” operator.

[0102] Table 1

[0103] mipSizeId boundarySize predSize 0 2 4 1 4 4 2 4 8

[0104] Thus, according to Table 1, when the value of mipSizeId is 0, the value of boundarySize can be 2, and the value of predSize can be 4. That is, at this time, the reference pixel selects two pixels for each edge, and the matrix multiplication output is a 4×4 MIP prediction block.

[0105] When the value of mipSizeId is 1, the value of boundarySize can be 4, and the value of predSize can be 4. That is to say, at this time, the reference pixel selects four pixels for each edge, and the matrix multiplication output is a 4×4 MIP prediction block.

[0106] When the value of mipSizeId is 2, the value of boundarySize can be 4, and the value of predSize can be 8; that is, at this time, the reference pixel selects four pixels for each edge, and the matrix multiplication output is an 8×8 MIP prediction block.

[0107] In addition, the values ​​of boundarySize, inSize and predSize can be determined according to the block size index value of the current block and the lookup table shown in Table 2.

[0108] Table 2

[0109] mipSizeId boundarySize inSize predSize 0 2 4 4 1 4 8 4 2 4 7 8

[0110] Thus, according to Table 2, when the value of mipSizeId is 0, the value of boundarySize can be 2, the value of inSize can be 4, and the value of predSize can be 4; that is, at this time, the reference pixel selects two pixel points for each edge, the number of matrix multiplication input sampling points is four, and the matrix multiplication output is a 4×4 MIP prediction block.

[0111] When the value of mipSizeId is 1, the value of boundarySize can be 4, the value of inSize can be 8, and the value of predSize can be 4; that is, at this time, the reference pixel selects four pixels for each edge, the number of matrix multiplication input sampling points is eight, and the matrix multiplication output is a 4×4 MIP prediction block.

[0112] When the value of mipSizeId is 2, the value of boundarySize can be 4, the value of inSize can be 7, and the value of predSize can be 8; that is, at this time, the reference pixel selects four pixels for each edge, the number of matrix multiplication input sampling points is seven, and the matrix multiplication output is an 8×8 MIP prediction block.

[0113] Furthermore, after configuring the MIP core parameters, it is also necessary to obtain reference pixels; then, based on the reference pixels and the MIP core parameters, the MIP input sample values ​​are constructed. The reference pixels here are the adjacent sample values ​​of the current block, which may include: the left adjacent sample values ​​of the current block and the upper adjacent sample values ​​of the current block. In other words, the MIP input sample values ​​of the current block may be determined based on the left adjacent sample values ​​and the upper adjacent sample values ​​of the current block.

[0114] In some embodiments, determining the MIP input sample value of the current block according to the adjacent sample values ​​of the current block may include:

[0115] Determining a block size index value of the current block according to a size parameter of the current block;

[0116] Performing down-sampling filtering on adjacent sample values ​​of the current block to obtain a first temporary reference value;

[0117] When the block size index value of the current block is within a preset range, determine a second constant value according to the bit depth of the adjacent sample values ​​of the current block; set the value corresponding to the index number 0 in the MIP input sample value to be equal to the difference between the second constant value and the value corresponding to the index number 0 in the first temporary reference value; set the value corresponding to the index number i in the MIP input sample value to be equal to the difference between the value corresponding to the index number i in the first temporary reference value and the value corresponding to the index number equal to 0 in the first temporary reference value, where i is an integer greater than 0;

[0118] When the block size index value of the current block is outside a preset range, the value corresponding to the index number j in the MIP input sampling value is set to be equal to the difference between the value corresponding to the index number j+1 in the first temporary reference value and the value corresponding to the index number 0 in the first temporary reference value, where j is an integer greater than or equal to 0.

[0119] Further, when the block size index value of the current block is within a preset range, the method may further include:

[0120] Determine a second constant value according to the bit depth of adjacent sampling values ​​of the current block; set the value corresponding to index number 0 in the MIP input sampling value to be equal to the difference between the value corresponding to index number 0 in the first temporary reference value and the second constant value; set the value corresponding to index number i in the MIP input sampling value to be equal to the difference between the value corresponding to index number i in the first temporary reference value and the value corresponding to index number 0 in the first temporary reference value, where i is an integer greater than 0.

[0121] It should be noted that after the adjacent sample values ​​of the current block are downsampled and filtered, the first temporary reference value can be obtained. Here, for the first temporary reference value, specifically, after the adjacent sample values ​​of the current block are downsampled and filtered, the filtered adjacent sample values ​​can be cached in a buffer (represented by pTemp). Among them, the value corresponding to the index number 0 in the first temporary reference value refers to pTemp[0], and the value corresponding to the index number i in the first temporary reference value refers to pTemp[i].

[0122] It should also be noted that, according to whether the block size index value (represented by mipSizeId) of the current block is within the preset range, it can be determined whether the size parameter value of the current block is within the preset range. Specifically, when mipSizeId=0 or 1, it indicates that the block size index value of the current block is within the preset range, that is, the size parameter value of the current block is within the preset range; when mipSizeId=2, it indicates that the block size index value of the current block is outside the preset range, that is, the size parameter value of the current block is not within the preset range.

[0123] That is to say, the MIP input sampling value is determined by the buffer (represented by pTemp), the block size index value of the current block (represented by MipSizeId), and the bit depth of the adjacent sampling value of the current block (represented by BitDepth), and the number of input samples contained in the MIP input sampling value is only related to the block size index value of the current block. Finally, the value corresponding to the index number x in the MIP input sampling value can be obtained (represented by p[x]).

[0124] Further, in some embodiments, determining the second constant value according to the bit depth of the adjacent sample values ​​of the current block may include:

[0125] The second constant value is set to be equal to an integer exponent power of 2, wherein the exponent of the power is equal to the bit depth of the neighboring sample values ​​of the current block minus 1.

[0126] Alternatively, in some embodiments, determining the second constant value according to the bit depth of the adjacent sample values ​​of the current block may include:

[0127] Perform a binary bit left shift on "1" to obtain the second constant value, wherein the number of bits of the bit left shift is equal to the bit depth of the adjacent sample values ​​of the current block minus 1.

[0128] That is, after obtaining the bit depth (expressed as BitDepth) of the adjacent sample values ​​of the current block, the second constant value can be expressed as 1<<(BitDepth-1) or 2^(BitDepth-1). In this way, when the size parameter value of the current block is within the preset range, the second constant value can be combined to determine the MIP input sample value of the current block.

[0129] It should also be noted that the MIP input sample is a matrix vector used for matrix multiplication. The current related technical solution is determined by the buffer (represented by pTemp), the type of the current block (i.e., the block size index value of the current block, represented by mipSizeId), the bit depth of the adjacent sample values ​​of the current block (represented by BitDepth) and the number of MIP input samples, and finally obtains the value corresponding to the index number x in the MIP input sample value (represented by p[x]).

[0130] Specifically, the construction process of the x-th input sample value p[x] is as follows:

[0131] In a possible implementation manner, when the block size index value of the current block is within a preset range, the construction process may include:

[0132] Calculate the difference between the second constant value and the value corresponding to the index number 0 in the first temporary reference value to obtain the value corresponding to the index number 0 in the MIP input sample value;

[0133] Subtract the value corresponding to index number i in the first temporary reference value from the value corresponding to index number 0 in the first temporary reference value to obtain the value corresponding to index number i in the MIP input sampling value; wherein i is a positive integer greater than 0 and less than N, and N represents the number of elements contained in the input sampling matrix.

[0134] In an embodiment of the present application, the minuend in the calculation of the difference is set equal to the second constant value, and the subtrahend in the calculation of the difference is set equal to the value corresponding to the index number 0 in the first temporary reference value.

[0135] That is to say, when mipSizeId = 0 or 1, p[0] can be obtained by subtracting pTemp[0] from 1<<(BitDepth-1); and when x is not equal to 0, p[x] can be obtained by subtracting pTemp[0] from pTemp[x]. The details are as follows:

[0136]

[0137] In another possible implementation, when the block size index value of the current block is within a preset range, the construction process may include:

[0138] Calculate the difference between the value corresponding to the index number 0 in the first temporary reference value and the second constant value to obtain the value corresponding to the index number 0 in the MIP input sample value;

[0139] Subtract the value corresponding to index number i in the first temporary reference value from the value corresponding to index number 0 in the first temporary reference value to obtain the value corresponding to index number i in the MIP input sampling value; wherein i is a positive integer greater than 0 and less than N, and N represents the number of elements contained in the input sampling matrix.

[0140] In an embodiment of the present application, the minuend in the calculation of the difference is set equal to the value corresponding to the index number 0 in the first temporary reference value, and the subtrahend in the calculation of the difference is set equal to the second constant value.

[0141] That is, when mipSizeId=0 or 1, when x is equal to 0, the value corresponding to the index number 0 in the MIP input sample value (represented by p[0]) can be obtained by subtracting the second constant value (i.e., 1<<(BitDepth-1)) from the value corresponding to the index number 0 in the first temporary reference value (i.e., pTemp[0]). When x is not equal to 0, the value corresponding to the index number x in the MIP input sample value (represented by p[x]) needs to be obtained by subtracting the value corresponding to the index number 0 in the first temporary reference value (i.e., pTemp[0]) from the value corresponding to the index number x in the first temporary reference value (i.e., pTemp[x]).

[0142] The details are as follows:

[0143]

[0144] In yet another possible implementation, when the block size index value of the current block is outside a preset range, the construction process may include:

[0145] Subtract the value corresponding to index number i+1 in the first temporary reference value from the value corresponding to index number 0 in the first temporary reference value to obtain the value corresponding to index number i in the MIP input sampling value; wherein i is a positive integer greater than or equal to 0 and less than N, and N represents the number of elements contained in the input sampling matrix.

[0146] That is to say, when mipSizeId=2, the value corresponding to index number 0 in the first temporary reference value, i.e., pTemp[0], can be ignored, and then the value corresponding to index number x in the MIP input sample value (represented by p[x]) can be obtained by subtracting the value corresponding to index number 0 in the first temporary reference value (i.e., pTemp[0]) from the value corresponding to index number x+1 in the first temporary reference value (i.e., pTemp[x+1]). Here, x is a positive integer greater than or equal to 0. Specifically, as shown below,

[0147] p[x]=pTemp[x+1]-pTemp[0] for x=0,...,inSize-1 (4)

[0148] Thus, still taking the 4×4 current block as an example, four values ​​are stored in the buffer pTemp, namely, the first temporary reference value includes: the value corresponding to the index number 0 (ie, pTemp[0]), the value corresponding to the index number 1 (ie, pTemp[1]), the value corresponding to the index number 2 (ie, pTemp[2]), and the value corresponding to the index number 3 (ie, pTemp[3]); at this time, according to formula (2) or formula (3) or formula (4), four MIP input sample values ​​can be determined, represented by p[x], x = 0, 1, 2, 3. Here, for these four MIP input sample values, they can also be combined into a 1×4 MIP input sample matrix.

[0149] In addition, the embodiment of the present application can also use a unified calculation method to obtain the value of p[x] without determining the value of mipSizeId. In some embodiments, determining the MIP input sample value of the current block according to the adjacent sample values ​​of the current block may include:

[0150] Performing down-sampling filtering on adjacent sample values ​​of the current block to obtain a first temporary reference value;

[0151] Determine a second constant value according to the bit depth of the adjacent sample values ​​of the current block, and cache the second constant value in a data unit after the first temporary reference value to obtain a second temporary reference value;

[0152] The value corresponding to the index number j in the MIP input sampling value is set to be equal to the difference between the value corresponding to the index number j+1 in the second temporary reference value and the value corresponding to the index number 0 in the second temporary reference value, where j is an integer greater than or equal to 0.

[0153] That is to say, the embodiment of the present application can add a second constant value (i.e., (1<<(BitDepth-1))) as an additional element at the end of the buffer pTemp. At this time, the size parameter of the current block is no longer considered (i.e., there is no need to consider the value of mipSizeId), and p[x] can be directly set to be equal to pTemp[x+1]-pTemp[0], x=0,…,inSize-1.

[0154] In this way, after the MIP input sample value of the current block is determined according to the adjacent sample values ​​of the current block, the MIP prediction value of the current block can be further determined.

[0155] S303: Determine the MIP prediction value of the chrominance component of the current block according to the MIP input sample value, the MIP weighting matrix and the shift parameter.

[0156] The MIP prediction value is the prediction value of some sampling points in the chrominance component of the current block.

[0157] It should be noted that the shift parameter may include a shift offset parameter and a shift quantity parameter. Figure 4 As shown, for S303, the following steps may be included:

[0158] S303-1: Determine the product of the sum of the MIP input sample values ​​and the shift offset parameter according to the value of the shift offset parameter, wherein the value of the shift offset parameter is a fixed constant.

[0159] It should be noted that the shift offset parameter may also be referred to as a shift compensation parameter or an offset factor, which may be represented by f0. In the embodiment of the present application, the value of the shift offset parameter may be set to a fixed constant, such as 32, 46, 56, 66, etc. The value of the shift offset parameter may also be set to be related to a shift offset parameter table, and the value of the shift offset parameter may be determined by looking up the table, which is not limited in any way.

[0160] S303-2: Determine a first constant value according to the value of the shift quantity parameter.

[0161] S303-3: Set the value of the first offset to the difference between the first constant value and the product.

[0162] It should be noted that the shift quantity parameter may also be referred to as a shift factor, a number of shift bits, or a weight shift value, etc., and may be represented by sW, shift, or weight shift. In the embodiment of the present application, the shift quantity parameter is represented by sW. In addition, the first offset may be represented by oW, and the first offset is related to both the shift quantity parameter and the shift offset parameter.

[0163] In the embodiment of the present application, the value of the shift quantity parameter can be set to a fixed constant, such as 5, 6, or 7. The value of the shift quantity parameter can also be set to be related to the shift quantity parameter table, and the value of the shift quantity parameter is determined by looking up the table, which is not limited here.

[0164] S303-4: Determine the MIP weighting matrix of the current block according to the prediction parameters.

[0165] It should be noted that the prediction parameters may include prediction mode parameters and may also include size parameters of the current block. In the prediction parameters, when the prediction mode parameters indicate that the chrominance component of the current block uses the MIP mode to determine the intra-frame prediction value, a weight matrix table may be pre-established at this time, and the weight matrix table may be stored in a memory or a storage unit, where the memory or storage unit may be integrated in the encoder, or may be set separately. In this way, according to the block size index value (mipSizeId) and the MIP mode index value (modeId) of the current block, the MIP weight matrix (or MIP weight matrix, or MIP matrix for short) required for the current block may be determined by looking up a table, and is represented by mWeight[x][y]. Among them, the block size index value (mipSizeId) of the current block is determined by the size parameter of the current block, and the size of the MIP weight matrix mWeight[x][y] is only related to the block size index value of the current block, as shown in Table 3.

[0166] In the MIP weighting matrix shown in Table 3, the number of columns is the number inSize of matrix multiplication input samples, and the number of rows is the number predSize×predSize of matrix multiplication output samples, so that the MIP weighting matrix of the current block can be determined.

[0167] Table 3

[0168] mipSizeId Number of columns Number of lines 0 4 16 1 8 16 2 7 64

[0169] It can be understood that in the encoder, a shift quantity parameter table can also be pre-established, and the shift quantity parameter table is also stored in a memory or a storage unit, where the memory or storage unit can be integrated in the encoder, or can also be set separately. In the embodiment of the present application, the determination of the shift quantity parameter (sW) can include the following methods:

[0170] In one possible implementation, for different block size index values ​​and different MIP mode index values, the shift quantity parameter may be different. In some embodiments, the method may further include:

[0171] When the chrominance component of the current block uses the MIP mode to determine the intra-frame prediction value, determine the MIP mode index value of the current block;

[0172] According to the MIP mode index value, querying a value corresponding to the MIP mode index value from a first preset lookup table, wherein the first preset lookup table is used to record a correspondence between the MIP mode index value and the value of the shift quantity parameter;

[0173] The queried value is determined as the value of the shift quantity parameter.

[0174] It should be noted that the value of the shift quantity parameter can be queried according to the block size index value (represented by mipSizeId) and the MIP mode index value (modeId) of the current block. As shown in the first preset lookup table in Table 4, for different mipSizeId and modeId, the shift quantity parameter required for matrix multiplication can be determined by a lookup table.

[0175] Table 4

[0176]

[0177] However, on the encoder side, Table 4 needs to be stored in a memory or storage unit in the form of a lookup table; however, storage requires a cost, and the lookup process also requires a cost; since the value of the shift quantity parameter in Table 4 is related to the block size and MIP mode index value of the current block, the memory usage is increased, and the computational complexity is also increased.

[0178] In order to reduce memory usage and computational complexity, the embodiment of the present application may simplify the method for determining the shift quantity parameter.

[0179] In another possible implementation, the value of the shift quantity parameter can be set to a fixed constant, and is a fixed constant that is independent of the block size index value and the MIP mode index value. For example, for different block size index values ​​and different MIP mode index values, the value of the shift quantity parameter can be set to 5; or, for different block size index values ​​and different MIP mode index values, the value of the shift quantity parameter can be set to 6; or, for different block size index values ​​and different MIP mode index values, the value of the shift quantity parameter can be set to 7. In the embodiment of the present application, preferably, the value of the shift quantity parameter is set to 6, but there is no limitation on this.

[0180] In another possible implementation, for the value of the shift quantity parameter, the method may further include:

[0181] Determining a block size index value of the current block according to a size parameter of the current block;

[0182] The value of the shift quantity parameter is determined according to the block size index value of the current block.

[0183] It should be noted that, according to the size parameter of the current block, the block size index value of the current block may be determined. In some embodiments, according to the size parameter of the current block, determining the block size index value of the current block may include:

[0184] When the width and height of the current block are both equal to 4, setting the block size index value of the current block to 0;

[0185] When the width and height of the current block are both equal to 8, or one of the width and height of the current block is equal to 4, setting the block size index value of the current block to 1;

[0186] When the width and height of the current block do not meet the aforementioned conditions, the block size index value of the current block is set to 2.

[0187] In this way, after the block size index value of the current block is determined, the value of the shift quantity parameter can be further determined according to the block size index value of the current block.

[0188] Optionally, in some embodiments, determining the value of the shift quantity parameter according to the block size index value of the current block may include:

[0189] When the block size index values ​​are equal to 0, 1, and 2 respectively, it is determined that the values ​​of the shift quantity parameter corresponding to the block size index value of the current block are equal to 5, 6, and 5 respectively.

[0190] Optionally, in some embodiments, determining the value of the shift quantity parameter according to the block size index value of the current block may include:

[0191] The value of the shift amount parameter is set to be equal to the ratio between the width or height of the current block and a first preset value of the block size index value corresponding to the current block.

[0192] Here, the first preset value represents the number of MIP input sample values ​​obtained from the boundary of the current block. In this case, the method may also include:

[0193] When the block size index values ​​of the current block are equal to 0, 1, and 2 respectively, it is determined that the first preset values ​​corresponding to the block size index values ​​of the current block are equal to 2, 4, and 4 respectively.

[0194] That is to say, when the first preset value represents the number of MIP input sampling values ​​obtained from the boundary of the current block, if the block size index value of the current block is equal to 0, then the corresponding first preset value is equal to 2; if the block size index value of the current block is equal to 1, then the corresponding first preset value is equal to 4; if the block size index value of the current block is equal to 2, then the corresponding first preset value is equal to 4, so that the value of the shift quantity parameter can be determined according to the ratio between the width or height of the current block and the corresponding first preset value.

[0195] Optionally, in some embodiments, determining the value of the shift quantity parameter according to the block size index value of the current block may include:

[0196] The value of the shift amount parameter is set to be equal to the ratio between the width or height of the current block and a second preset value of the block size index value corresponding to the current block.

[0197] Here, the second preset value represents the size of the MIP prediction block of the current block obtained by directly using the MIP weighting matrix. In this case, the method may further include:

[0198] When the block size index values ​​of the current block are equal to 0, 1, and 2 respectively, it is determined that the second preset values ​​corresponding to the block size index values ​​of the current block are equal to 4, 4, and 8 respectively.

[0199] That is to say, when the second preset value represents the size of the MIP prediction block of the current block obtained by directly using the MIP weighting matrix to calculate, at this time, if the block size index value of the current block is equal to 0, then the corresponding second preset value is equal to 4; if the block size index value of the current block is equal to 1, then the corresponding second preset value is equal to 4; if the block size index value of the current block is equal to 2, then the corresponding second preset value is equal to 8, so that the value of the shift quantity parameter can be determined according to the ratio between the width or height of the current block and the corresponding second preset value.

[0200] In another possible implementation, the shift quantity parameter table can be minimized, and the shift quantity parameter value can still be determined by using a lookup table. Optionally, in some embodiments, determining the shift quantity parameter value according to the block size index value of the current block can include:

[0201] According to the block size index value, querying a value corresponding to the block size index value from a second preset lookup table, wherein the second preset lookup table is used to record a correspondence between the block size index value and the value of the shift quantity parameter;

[0202] The queried value is determined as the value of the shift quantity parameter.

[0203] It should be noted that the value of the shift quantity parameter can be queried only according to the block size index value (represented by mipSizeId) of the current block. As shown in the second preset lookup table in Table 5, for each block size index value, a fixed value may correspond, that is, the size of each block or the size set of each block may have a fixed shift quantity parameter value as shown in Table 5.

[0204] Table 5

[0205] mipSizeId w 0 5 1 6 2 5

[0206] According to Table 5, when the block size index values ​​of the current block are equal to 0, 1, and 2 respectively, it can be determined that the values ​​of the shift quantity parameter corresponding to the block size index values ​​are equal to 5, 6, and 5 respectively.

[0207] In the above-mentioned implementation, by simplifying the method for determining the shift quantity parameter, especially minimizing the shift quantity parameter table or fixing the value of the shift quantity parameter, the storage of the lookup table can be minimized, thereby reducing the memory occupied by the shift quantity parameter table storage in the MIP mode without increasing the computational complexity.

[0208] In the encoder, a shift offset parameter table may also be pre-established, and the shift offset parameter table is also stored in a memory or a storage unit, where the memory or storage unit may be integrated in the encoder, or may be separately set. In the embodiment of the present application, the determination of the shift offset parameter (fO) may include the following methods:

[0209] In a possible implementation, for different block sizes and different MIP mode index values, the shift offset parameter may also be different. In some embodiments, the method may further include:

[0210] When the chrominance component of the current block uses the MIP mode to determine the intra-frame prediction value, determine the MIP mode index value of the current block;

[0211] According to the MIP mode index value, querying a value corresponding to the MIP mode index value from a third preset lookup table, wherein the third preset lookup table is used to record the correspondence between the MIP mode index value and the value of the shift offset parameter;

[0212] The queried value is determined as the value of the shift offset parameter.

[0213] It should be noted that the value of the shift offset parameter can be queried according to the block size index value (represented by MipSizeId) and the MIP mode index value (modeId) of the current block. As shown in the third preset lookup table in Table 6, for different mipSizeId and modeId, the shift offset parameter required for matrix multiplication can be determined by a lookup table.

[0214] Table 6

[0215]

[0216] However, on the encoder side, Table 6 also needs to be stored in a memory or storage unit in the form of a lookup table; however, storage requires a cost, and the lookup process also requires a cost; since the value of the shift offset parameter in Table 6 is related to the block size and MIP mode index value of the current block, the memory usage is increased, and the computational complexity is also increased.

[0217] In order to reduce memory usage and computational complexity, the embodiment of the present application may also simplify the method for determining the shift offset parameter.

[0218] In another possible implementation, the value of the shift offset parameter can be set to a fixed constant, and it is a fixed constant that is independent of the block size index value and the MIP mode index value. Typically, the fixed constant has a value range of 0 to 100. For example, for different block size index values ​​and different MIP mode index values, the value of the shift offset parameter can be set to 32; or, for different block size index values ​​and different MIP mode index values, the value of the shift offset parameter can be set to 46; or, for different block size index values ​​and different MIP mode index values, the value of the shift offset parameter can be set to 56; or, for different block size index values ​​and different MIP mode index values, the value of the shift offset parameter can be set to 66. In the embodiment of the present application, preferably, the value of the shift offset parameter is set to be equal to 32, but there is no limitation on this.

[0219] In another possible implementation, for the value of the shift offset parameter, the method may further include:

[0220] Determining a block size index value of the current block according to a size parameter of the current block;

[0221] The value of the shift offset parameter is determined according to the block size index value of the current block.

[0222] It should be noted that, according to the size parameter of the current block, the block size index value of the current block can be determined; and then according to the block size index value of the current block, the value of the shift offset parameter can be further determined.

[0223] Optionally, in some embodiments, determining the value of the shift offset parameter according to the block size index value of the current block may include:

[0224] When the block size index values ​​are equal to 0, 1, and 2 respectively, it is determined that the values ​​of the shift offset parameter corresponding to the block size index value of the current block are equal to 34, 23, and 46 respectively.

[0225] Optionally, in some embodiments, the shift offset parameter table may be minimized, and the offset factor may still be determined by using a lookup table. Optionally, in some embodiments, determining the value of the shift offset parameter according to the block size index value of the current block may include:

[0226] According to the block size index value, querying a value corresponding to the block size index value from a fourth preset lookup table, wherein the fourth preset lookup table is used to record a correspondence between the block size index value and the value of the shift offset parameter;

[0227] The queried value is determined as the value of the shift offset parameter.

[0228] It should be noted that the value of the shift offset parameter can be queried only according to the block size index value (represented by mipSizeId) of the current block. As shown in the fourth preset lookup table in Table 7, for each block size index value, a fixed value may correspond, that is, the size of each block or the size set of each block may have a fixed shift offset parameter value as shown in Table 7.

[0229] Table 7

[0230] mipSizeId fO 0 34 1 23 2 46

[0231] According to Table 7, when the block size index values ​​of the current block are respectively equal to 0, 1 and 2, it can be determined that the values ​​of the shift offset parameter corresponding to the block size index values ​​are respectively equal to 34, 23 and 46.

[0232] In the above-mentioned implementation, by simplifying the method of determining the shift offset parameter, especially minimizing the shift offset parameter table or fixing the value of the shift offset parameter, the storage of the lookup table can be minimized, thereby reducing the memory occupied by the shift offset parameter table storage in the MIP mode without increasing the computational complexity.

[0233] Thus, after determining the shift offset parameter (fO) and the shift quantity parameter (sW), for the first constant value, optionally, in some embodiments, determining the first constant value according to the value of the shift quantity parameter may include:

[0234] The first constant value is set to be equal to an integer exponent power of 2, wherein the exponent of the power is equal to the value of the shift amount parameter minus 1.

[0235] Optionally, in some embodiments, determining the first constant value according to the value of the shift quantity parameter may include:

[0236] Perform a binary bit left shift on "1" to obtain the first constant value, wherein the number of bits of the bit left shift is equal to the value of the shift quantity parameter minus 1.

[0237] That is, after obtaining the shift quantity parameter (sW), the first constant value can be expressed as 1<<(sW-1) or 2^(sW-1). At this time, when the shift quantity parameter is set to 6, the first constant value can be obtained to be equal to 32.

[0238] Here, if the first offset is represented by oW, the product of the sum of the MIP input sampling values ​​and the shift offset parameter (fO) can be calculated according to the shift offset parameter, which is Then the value of oW can be set to

[0239] In this way, in the MIP mode, the MIP weighting matrix, the MIP input sampling value, the shift quantity parameter and the first offset can be obtained, so as to subsequently determine the MIP prediction value of the current block.

[0240] S303-5: Determine a MIP prediction value of the chrominance component of the current block according to the MIP weighting matrix, the MIP input sample value, the shift quantity parameter and the first offset.

[0241] Here, the MIP prediction value represents the prediction value of some sampling points in the chrominance component of the current block.

[0242] It should be noted that determining the MIP prediction value of the current block according to the MIP weighting matrix, the MIP input sample value, the shift quantity parameter and the first offset may include:

[0243] Calculating a first weighted sum of the MIP weighting matrix and the MIP input sample value;

[0244] Calculating a first sum value of the first weighted sum and the first offset;

[0245] Performing a binary bit right shift on the first sum value to obtain a first right-shifted value, wherein the number of bits of the right shift is equal to the value of the shift quantity parameter;

[0246] The MIP prediction value of the current block is set to be equal to the sum of the first right shift value and the value corresponding to the index number 0 in the first temporary reference value; wherein the first temporary reference value is obtained based on downsampling filtering of adjacent sample values ​​of the current block.

[0247] Specifically, in MIP mode, the MIP weight matrix (represented by mWeight), the shift quantity parameter (represented by sW) and the shift offset parameter (represented by fO) can be determined according to the block size index value of the current block (represented by mipSizeId) and the MIP mode index value (represented by modeId); then the MIP input sampling value (represented by p[x]), mWeight, sW and fO are input into the matrix multiplication process to obtain the MIP prediction value output by the matrix multiplication (represented by predMip[x][y]), and the sampling points in predMip[x][y] are arranged into a matrix / array form according to predSize×predSize. The calculation formula is as follows,

[0248]

[0249] Wherein, [x][y] represents the position coordinates of the pixel point, x represents the horizontal direction, and y represents the vertical direction; inSize represents the number of input samples, p[i] represents the value corresponding to index number i in the MIP input sample value; pTemp[0] represents the value corresponding to index number 0 in the first temporary reference value; ">>" represents the binary right shift operator, and "<<" represents the binary left shift operator; x=0,…,predSize-1,y=0,…,predSize-1. Thus, according to the above formula (5), predMip[x][y] can be calculated to obtain the MIP prediction block.

[0250] In some embodiments, if sW is fixed at 6 and fO is fixed at 32, the calculation formula is as follows:

[0251]

[0252] Furthermore, it is also necessary to perform embedding processing on the predicted values ​​of the sampling points in the MIP prediction block to obtain the MIP prediction block of the current block; then determine whether to perform transposition processing on the MIP prediction block; if the judgment result is yes, then it is also necessary to perform transposition processing on the predicted sampling values ​​in the MIP prediction block, and determine the transposed MIP prediction block as the MIP prediction block of the current block; if the judgment result is no, then it is not necessary to perform transposition processing on the predicted sampling values ​​in the MIP prediction block, and the MIP prediction block can be directly determined as the MIP prediction block of the current block to obtain the MIP prediction value of the current block, and the MIP prediction value is the prediction value of some sampling points in the current block.

[0253] S304: Filter the MIP prediction value to determine an intra-frame prediction value of the chrominance component of the current block.

[0254] It should be noted that the filtering process on the MIP prediction value to determine the intra prediction value of the current block may include:

[0255] Determine whether the size parameter of the MIP prediction block is the same as the size parameter of the current block;

[0256] When the size parameter of the MIP prediction block is the same as the size parameter of the current block, setting the intra prediction block of the current block to be equal to the MIP prediction block; wherein the MIP prediction block contains prediction sample values ​​of all pixel positions in the current block;

[0257] When the size parameter of the MIP prediction block is different from the size parameter of the current block, filtering is performed on the MIP prediction block to obtain a filtered prediction block, and the filtered prediction block is set as an intra-frame prediction block of the current block.

[0258] Here, the filtering process may include an up-sampling filtering process or a low-pass filtering process.

[0259] It should be noted that the MIP prediction block is composed of MIP prediction values. After obtaining the MIP prediction block, since the size parameters of the MIP prediction block only include two types: 4×4 MIP prediction block and 8×8 MIP prediction block; thus, the size parameters of the current block may be the same as or different from the size parameters of the MIP prediction block. In other words, the sampling values ​​corresponding to the MIP prediction block may not be able to fill the current block, so that the generation of the final prediction value may require upsampling of the MIP prediction block, that is, by judging whether the size parameters of the MIP prediction block are the same as the size parameters of the current block, to determine whether to upsample the MIP prediction block.

[0260] It should also be noted that when the size parameters of the MIP prediction block are the same as the size parameters of the current block, that is, the width and height of the MIP prediction block are the same as those of the current block, it indicates that there is no need to upsample the MIP prediction block. At this time, the MIP prediction block can be directly filled into the current block, that is, there are no empty pixels in the filled current block. At this time, the intra-frame prediction value of each pixel in the current block can be directly set to the prediction value of each pixel in the MIP prediction block, as shown below.

[0261] predSamples[x][y]=predMip[x][y] (7)

[0262] Where [x][y] represents the position coordinates of the pixel point, x represents the horizontal direction, and y represents the vertical direction; predSamples[x][y] represents the intra-frame prediction value corresponding to the pixel point at the position coordinates [x][y] in the current block, and predMip[x][y] represents the prediction value corresponding to the pixel point at the position coordinates [x][y] in the MIP prediction block. Thus, according to formula (7), the MIP prediction block predMip[x][y] can be directly used as the intra-frame prediction block predSamples[x][y] of the current block, that is, the intra-frame prediction value of at least one pixel point in the current block.

[0263] It should also be noted that when the size parameter of the MIP prediction block is different from the size parameter of the current block, that is, at least one of the width and height of the MIP prediction block is different from the current block, the MIP prediction block cannot fill the current block, that is, there are spare pixels in the filled current block, indicating that the MIP prediction block needs to be filtered. In other words, if upsampling is required in both the horizontal and vertical directions, the MIP prediction block can be first sampled in the horizontal direction and then in the vertical direction to obtain a first upsampling block, which can be represented by predSamples[x][y]; then the MIP prediction block is sampled in the vertical direction and then in the horizontal direction to obtain a second upsampling block, which can be represented by predSamplesTemp[x][y]; finally, predSamples[x][y] and predSamplesTemp[x][y] are weighted averaged to finally obtain the intra-frame prediction value of the chrominance component of the current block, that is, the intra-frame prediction block of the current block.

[0264] Exemplarily, if the side length nTbS of the current block (here, S can be replaced by W and H respectively) is equal to the side length predSize of predMip (here, predSize is only related to blocksizeIdx of the current block), then the MIP prediction block can be directly set as the intra-frame prediction block of the current block; otherwise, the MIP prediction block needs to be filtered to obtain the intra-frame prediction block of the current block. Here, taking the 4×4 current block as an example, if the size parameters of the current block and the MIP prediction block are the same, it is not necessary to filter the MIP prediction block at this time, then the MIP prediction block can be directly set as the intra-frame prediction block of the current block, so that the intra-frame prediction value of at least one pixel in the current block can be obtained.

[0265] In short, for the prediction process of the MIP mode, the input data of the MIP mode may include: the position of the current block (xTbCmp, yTbCmp), the MIP mode index value applied to the current block (which can be represented by modeId), the height of the current block (represented by nTbH), the width of the current block (represented by nTbW), and the transposition indication flag indicating whether transposition processing is required (i.e., the MIP transposition indication parameter, which can be represented by isTransposed), etc. The output data of the MIP mode may include: the intra-frame prediction block of the current block. The prediction value corresponding to the pixel coordinate [x][y] in the intra-frame prediction block is predSamples[x][y]; wherein x=0,1,…,nTbW-1; y=0,1,…,nTbH-1.

[0266] like Figure 5 As shown in Figure 1, the MIP prediction process specifically includes the following steps:

[0267] S501: Configure core parameters.

[0268] For S501, the MIP core parameters may include the type of the current block (represented by mipSizeId), the number of reference samples for each edge (represented by boundySize), the number of MIP input samples (represented by inSize), and the MIP prediction block size output by matrix multiplication (arranged as predSize×predSize). According to the size of the current block, the current block can be divided into three categories, and the type of the current block is recorded by mipSizeId. Here, mipSizeId can be equal to 0, 1, or 2. Moreover, for different types of current blocks, the number of reference sampling points and the MIP prediction block size output by matrix multiplication are different.

[0269] S502: Acquire reference pixels.

[0270] For S502, when predicting the current block, the upper adjacent block and the left adjacent block of the current block are both coded blocks, and the reference pixels of the MIP mode are the reconstructed values ​​of the upper row of pixels and the left column of pixels adjacent to the current block. In other words, the process of obtaining the reference pixels adjacent to the upper side of the current block (represented by refT) and the reference pixels adjacent to the left side (represented by refL) is the process of obtaining the reference pixels.

[0271] S503: Construct input samples.

[0272] For S503, this step is used for the input of matrix multiplication, and may specifically include:

[0273] S5031: Obtain reference sampling;

[0274] S5032: construct a reference sampling buffer;

[0275] S5033: Derive matrix multiplication input samples.

[0276] In S5031, the process of obtaining the reference sample is the process of downsampling the reference pixel. In S5032, in the construction of the reference sample buffer, there are two filling methods, which may include:

[0277] S5032-1: How to fill the buffer when transposition is not required;

[0278] S5032-2: How to fill the buffer when transposition is required.

[0279] S504: Generate intra-frame prediction values.

[0280] For S504, this step is used to obtain the MIP prediction value of the current block, which may specifically include:

[0281] S5041: constructing a MIP prediction block of matrix multiplication output samples;

[0282] S5042: Performing embedding processing on the MIP prediction block of the matrix multiplication output sample;

[0283] S5043: performing transposition processing on the MIP prediction block of the matrix multiplication output sample;

[0284] S5044: Generate MIP prediction value.

[0285] Among them, for S5041, in the process of constructing the MIP prediction block, it may specifically include:

[0286] S5041-1: Obtain weight matrix;

[0287] S5041-2: Get shift parameters;

[0288] S5041-3: Matrix multiplication operation.

[0289] That is to say, in the process of constructing the MIP prediction block, it is necessary to obtain not only the weight matrix but also the shift parameters (including: shift offset parameters and shift quantity parameters); then perform matrix multiplication to obtain the MIP prediction block of the matrix multiplication output sample. For S5044, generating the final MIP prediction value includes two cases, which may include:

[0290] S5044-1: Generate a prediction value that does not require upsampling and filtering;

[0291] S5044-2: Generate prediction values ​​that require upsampling and filtering processing.

[0292] In this way, after the above four steps S501 to S504, the embodiment of the present application can obtain the intra-frame prediction value of at least one pixel in the chrominance component of the current block.

[0293] S305: Determine a prediction residual value of the chrominance component of the current block according to the intra-frame prediction value of the chrominance component of the current block.

[0294] It should be noted that after obtaining the intra-frame prediction value of the chrominance component of the current block, the difference calculation can be performed between the true value of the chrominance component of the current block and the intra-frame prediction value of the chrominance component of the current block, so as to obtain the predicted residual value of the chrominance component of the current block, that is, to obtain the residual block of the current block.

[0295] S306: Perform low frequency non-separable secondary transform LFNST on the prediction residual value to determine LFNST parameters.

[0296] It should be noted that for Figure 1 For the transform unit 108 shown in , the transform unit 108 may perform a first transform on the residual block of the current block, such as an integer transform initially designed based on a discrete cosine transform (DCT); and may also perform a second transform on the residual block of the current block, such as a low-frequency non-separable secondary transform (LFNST). Specifically, in some embodiments, performing a low-frequency non-separable secondary transform LFNST on the prediction residual value and determining the LFNST parameters may include:

[0297] Determine a residual block of the current block according to the prediction residual value, and perform a first transform on the residual block to obtain a first transform coefficient block;

[0298] determining whether to perform LFNST on at least part of the transform coefficients in the first transform coefficient block;

[0299] In a case where the determination result is yes, LFNST is performed on at least part of the transform coefficients in the first transform coefficient block to determine the LFNST parameters.

[0300] In an embodiment of the present application, the encoder may perform a first transform (also referred to as "Core Transform" or "primary transform" or "main transform") on the predicted residual value in the residual block to obtain a first transform coefficient block after the first transform; and then perform an LFNST transform (also referred to as "Secondary Transform" or "secondary transform") on part or all of the transform coefficients in the first transform coefficient block to determine the LFNST parameters. It should be noted that the first transform here is a transform different from LFNST.

[0301] It should also be noted that not all current blocks can be subjected to LFNST. After the first transformation, some conditions (such as the minimum value in the size parameter of the current block, the block size index value of the current block, etc.) need to be checked to determine whether to perform LFNST on at least some of the transform coefficients in the first transform coefficient block. Optionally, in some embodiments, the determining whether to perform LFNST on at least some of the transform coefficients in the first transform coefficient block may include:

[0302] When the chrominance component of the current block uses the MIP mode to determine the intra-frame prediction value, determine the minimum value of the size parameters of the current block;

[0303] According to the minimum value, it is determined whether to perform LFNST on at least part of the transform coefficients in the first transform coefficient block.

[0304] Further, the determining, according to the minimum value, whether to perform LFNST on at least part of the transform coefficients in the first transform coefficient block may include:

[0305] If the minimum value is greater than or equal to a first preset threshold, it is determined to perform LFNST on at least part of the transform coefficients in the first transform coefficient block.

[0306] Here, the first preset threshold value may be used to represent a preset threshold value for measuring whether to perform LFNST. In the embodiment of the present application, the first preset threshold value may be set to 8, but is not specifically limited thereto.

[0307] That is to say, when the MIP mode is used to perform intra-frame prediction on the chrominance component of the current block, the minimum value of the size parameters of the current block can be determined at this time; for example, when the minimum value of the width and height is greater than or equal to 8, it can be determined that LFNST is performed on at least part of the transform coefficients in the first transform coefficient block.

[0308] Optionally, in some embodiments, the determining whether to perform LFNST on at least part of the transform coefficients in the first transform coefficient block may include:

[0309] When the chrominance component of the current block uses the MIP mode to determine the intra-frame prediction value, determining the block size index value of the current block according to the size parameter of the current block;

[0310] According to the block size index value, it is determined whether to perform LFNST on at least part of the transform coefficients in the first transform coefficient block.

[0311] Further, the determining, according to the block size index value, whether to perform LFNST on at least part of the transform coefficients in the first transform coefficient block may include:

[0312] If the block size index value is equal to a second preset threshold, it is determined to perform LFNST on at least part of the transform coefficients in the first transform coefficient block.

[0313] Here, the second preset threshold value can also be used to represent a preset threshold value for measuring whether to perform LFNST. In the embodiment of the present application, the second preset threshold value can be set to a preset value (such as 2), or one of multiple preset values ​​(such as 1, 2), but is not specifically limited.

[0314] That is to say, when the MIP mode is used to perform intra-frame prediction on the chrominance component of the current block, the block size index value (mipSizeId) of the current block can be determined based on the size parameters of the current block; for example, when the value of mipSizeId is equal to 2, it can be determined that LFNST is performed on at least part of the transform coefficients in the first transform coefficient block.

[0315] Optionally, in some embodiments, the determining whether to perform LFNST on at least part of the transform coefficients in the first transform coefficient block may include:

[0316] calculating a first cost result without performing LFNST on at least part of the transform coefficients in the first transform coefficient block;

[0317] calculating a second cost result when LFNST is performed on at least part of the transform coefficients in the first transform coefficient block;

[0318] According to the first cost result and the second cost result, it is determined whether to perform LFNST on at least part of the transform coefficients in the first transform coefficient block.

[0319] Further, the determining whether to perform LFNST on at least part of the transform coefficients in the first transform coefficient block according to the first cost result and the second cost result may include:

[0320] If the first cost result is greater than the second cost result, it is determined to perform LFNST on at least part of the transform coefficients in the first transform coefficient block.

[0321] It should be noted that the embodiments of the present application can adopt the traditional rate-distortion optimization method. First, a first cost result is calculated without performing LFNST on at least part of the transform coefficients in the first transform coefficient block; then, a second cost result is calculated while performing LFNST on at least part of the transform coefficients in the first transform coefficient block; the first cost result is compared with the second cost result. When the first cost result is greater than the second cost result, it can be determined that LFNST is performed on at least part of the transform coefficients in the first transform coefficient block.

[0322] In this way, when the determination result is yes, that is, it is determined that LFNST needs to be performed on at least part of the transform coefficients in the first transform coefficient block, LFNST can be performed on at least part of the transform coefficients in the first transform coefficient block and LFNST parameters can be determined.

[0323] S307: Encode the LFNST parameters and write them into the bitstream.

[0324] It should be noted that the LFNST parameter includes a LFNST index number, which can be represented by lfnst_index. In some embodiments, determining the LFNST parameter may include:

[0325] If it is determined that LFNST is performed on at least part of the transform coefficients in the first transform coefficient block, setting the value of the LFNST index sequence number to be greater than zero;

[0326] If it is determined not to perform LFNST on at least part of the transform coefficients in the first transform coefficient block, the value of the LFNST index number is set to zero.

[0327] Further, for S307, encoding the LFNST parameter and writing it into the bitstream may include:

[0328] The LFNST index number is encoded and written into the bitstream.

[0329] That is, after determining the LFNST parameter, i.e., lfnst_index, if lfnst_index is equal to 0, it means that LFNST is not performed; if lfnst_index is greater than 0, it means that LFNST needs to be performed. In the encoding process, lfnst_index can be encoded using context-based adaptive binary arithmetic coding (CABAC) with a descriptor of "ae(v)".

[0330] For example, see Table 8, which shows an example of a syntax structure description of a LFNST parameter provided by the related art. At this time, the judgment condition for determining whether to perform LFNST on at least part of the transform coefficients in the first transform coefficient block may include at least one of the following: treeType==DUAL_TREE_CHROMA, ! IntraMipFlag[x0][y0], Min(lfnstWidth, lfnstHeight)>=16.

[0331] Table 8

[0332]

[0333] Refer to Table 9A and Table 9B, which respectively show an example of a syntax structure description of a LFNST parameter provided in an embodiment of the present application. At this time, the judgment condition for determining whether to perform LFNST on at least part of the transform coefficients in the first transform coefficient block may include at least one of the following: intra_mip_flag[x0][y0]==1&&Min(lfnstWidth,lfnstHeight)>=8, intra_mip_flag[x0][y0]==1&&mipSizeId==2.

[0334] Specifically, as shown in the syntax structure of Table 9A, when the minimum value in the size parameter of the current block is greater than or equal to 8 (that is, the width and height are both greater than or equal to 8), the MIP mode (that is, when intra_mip_flag[x0][y0]==1) can be used to encode the lfnst_index of the current block. Alternatively, the judgment condition can also be set to: the width and height are both greater than or equal to the first value (for example, 8), and the sum of the width and height is greater than or equal to the second value (for example, 16). As shown in the syntax structure of Table 9B, when the block size index value (mipSizeId) of the current block is equal to 2, the MIP mode (that is, when intra_mip_flag[x0][y0]==1) can be used to encode the lfnst_index of the current block. Alternatively, the judgment condition can also be set to: mipSizeId is equal to one of multiple preset values ​​(for example, 1, 2).

[0335] Table 9A

[0336] coding_unit(x0,y0,cbWidth,cbHeight,cqtDepth,treeType,modeType){ Descriptor …… if(…&&(intra_mip_flag[x0][y0]===1&&Min(lfnstWidth,lfnstHeight)>=8)&&…){ …… lfnst_idx ae(v) } …… }

[0337] Table 9B

[0338] coding_unit(x0,y0,cbWidth,cbHeight,cqtDepth,treeType,modeType){ Descriptor …… if(…&&(intra_mip_flag[x0][y0]==1&&mipSizeId==2)&&…){ …… lfnst_idx ae(v) } …… }

[0339] It should also be noted that when it is determined that the current block can perform LFNST, it is also necessary to determine the LFNST transformation kernel (represented by kernel) used by the current block. In some embodiments, performing LFNST on the prediction residual value and determining LFNST parameters may include: determining the LFNST transformation kernel used by the current block; and performing LFNST on the prediction residual value using the LFNST transformation kernel to determine LFNST parameters.

[0340] In the embodiment of the present application, there are 4 transform core candidate sets in LFNST, and these four transform core candidate sets may include set0, set1, set2 and set3. Here, the embodiment of the present application may use MIP parameters to determine the LFNST transform core candidate set, and then select the LFNST transform core used by the current block from the LFNST transform core candidate set. Therefore, in some embodiments, the method may also include:

[0341] When the chrominance component of the current block uses the MIP mode to determine the intra-frame prediction value, determine the MIP parameter from the prediction parameter of the current block;

[0342] Determine a LFNST transform kernel candidate set according to the MIP parameters;

[0343] In the LFNST transform core candidate set, the transform core indicated by the LFNST index number is set as the LFNST transform core used by the current block;

[0344] The value of the LFNST index number is set to indicate that LFNST is performed on the current block and the index number of the LFNST transform core in the LFNST transform core candidate set;

[0345] The LFNST transform kernel candidate set includes two or more preset LFNST transform kernels.

[0346] It should be noted that the MIP parameter may include at least one of the following: a MIP mode index value (expressed by modeId) and a MIP transposition indication parameter (expressed by isTransposed). Among them, for the LFNST transform kernel used by the current block, when the value of the MIP transposition indication parameter indicates that the sampling point input vector used by the MIP mode is transposed, it is also necessary to perform matrix transposition processing on the selected transform kernel to obtain the LFNST transform kernel used by the current block.

[0347] Further, in some embodiments, determining the LFNST transformation kernel candidate set according to the MIP parameters may include:

[0348] Determine the value of the LFNST intra prediction mode index number according to the MIP mode index value;

[0349] According to the value of the LFNST intra prediction mode index sequence number, determining the index value of the LFNST transform kernel candidate set through a fifth preset lookup table;

[0350] The LFNST transform core candidate set is selected from a plurality of candidate LFNST transform core candidate sets according to the index value of the LFNST transform core candidate set.

[0351] Here, the LFNST intra prediction mode index number may be represented by predModeIntra, and the index value of the LFNST transform kernel candidate set may be represented by lfnstTrSetIdx.

[0352] It should be noted that the fifth preset lookup table is shown in Table 10. According to the MIP mode index value (i.e., the value of modeId), the value of predModeIntra can be determined; then, according to the value of predModeIntra, the value of lfnstTrSetIdx can be directly determined in combination with Table 10, that is, the LFNST transform core candidate set selected for the current block is determined. Here, the value of lfnstTrSetIdx indicates the transform core candidate set used during LFNST; since the value of modeId can include 0, 1, 2, 3, 4, 5, the value of predModeIntra is also 0, 1, 2, 3, 4, 5; its corresponding relationship with lfnstTrSetIdx is as follows,

[0353] Table 10

[0354] predModeIntra lfnstTrSetIdx 0<=predModeIntra<=2 0 predModeIntra==3 2 predModeIntra==4 1 predModeIntra==5 3

[0355] In a specific embodiment, determining the value of the LFNST intra prediction mode index number according to the MIP mode index value may include: setting the value of the LFNST intra prediction mode index number to be equal to the MIP mode index value.

[0356] That is to say, the embodiment of the present application can set the value of predModeIntra to be equal to the value of modeId, and then directly determine the value of lfnstTrSetIdx based on the value of predModeIntra in combination with Table 10, that is, determine the LFNST transform core candidate set selected for the current block. It should also be noted that in another specific embodiment, the embodiment of the present application can also directly map the value of modeId to the value of the PLANAR mode, and then use the value of predModeIntra corresponding to the PLANAR mode to determine the value of lfnstTrSetIdx, that is, determine the LFNST transform core candidate set selected for the current block.

[0357] In some embodiments, the LFNST parameters may further include LFNST coefficients. Determining the LFNST parameters may include:

[0358] Performing LFNST on at least part of the transform coefficients in the first transform coefficient block to obtain the LFNST coefficients;

[0359] Accordingly, encoding the LFNST parameters and writing them into a bitstream includes:

[0360] quantizing the LFNST coefficients to obtain quantized coefficients;

[0361] The quantized coefficients are encoded and written into a bitstream.

[0362] It should be noted that when lfnst_index is not equal to 0, that is, it is determined to perform LFNST on the current block, the LFNST coefficients can be obtained by performing LFNST on at least part of the transform coefficients in the first transform coefficient block; then the LFNST coefficients are quantized and encoded and written into the bitstream.

[0363] It should be noted that, assuming that d[x][y] is the first transform coefficient block, where x=0,...,nTbW-1, y=0,...,nTbH-1, where nTbW and nTbH represent the width and height of the current block respectively. Assuming that v[x] represents a block including LFNST coefficients, where x=0,...,nLfnstOutSize-1, where nLfnstOutSize=(nTbW>=8&&nTbH>=8)? 48:16.

[0364] In a possible implementation, LFNST may be performed directly on at least part of the transform coefficients in the first transform coefficient block in a horizontal scanning order to obtain LFNST coefficients. Specifically,

[0365]

[0366] Among them, log2LfnstSize is determined as follows:

[0367] Log2LfnstSize=(nTbW>=8&&nTbH>=8)? 3:2 (9)

[0368] In another possible implementation, the scanning order may include a horizontal scanning order and a vertical scanning order. Here, the determination of the scanning order is related to the value of the MIP transposition indication parameter (expressed by isTransposed).

[0369] Optionally, in some embodiments, performing LFNST on at least part of the transform coefficients in the first transform coefficient block to obtain the LFNST coefficients may include:

[0370] When the value of the MIP transposition indication parameter indicates that a transposition process is performed on a sampling point input vector used in the MIP mode, performing LFNST on at least part of the transform coefficients in the first transform coefficient block according to a vertical scanning order to obtain the LFNST coefficients;

[0371] When the value of the MIP transposition indication parameter indicates that the sampling point input vector used by the MIP mode is not to be transposed, LFNST is performed on at least part of the transform coefficients in the first transform coefficient block according to a horizontal scanning order to obtain the LFNST coefficients.

[0372] Specifically, when the value of isTransposed is equal to 0, the sampling point input vector used by the MIP mode is not transposed, then LFNST can be performed on at least part of the transform coefficients in the first transform coefficient block according to the horizontal scanning order, as follows:

[0373]

[0374] When the value of isTransposed is equal to 1, the sampling point input vector used in the MIP mode is transposed, and then LFNST can be performed on at least part of the transform coefficients in the first transform coefficient block according to the vertical scanning order, as follows:

[0375]

[0376] Among them, the determination of log2LfnstSize is shown in the above formula (9).

[0377] Optionally, in some embodiments, performing LFNST on at least part of the transform coefficients in the first transform coefficient block to obtain the LFNST coefficients may include:

[0378] When the value of the MIP transposition indication parameter indicates that a transposition process is performed on a sampling point input vector used in the MIP mode, performing LFNST on at least part of the transform coefficients in the first transform coefficient block according to a horizontal scanning order to obtain the LFNST coefficients;

[0379] When the value of the MIP transposition indication parameter indicates that the sampling point input vector used by the MIP mode is not to be transposed, LFNST is performed on at least part of the transform coefficients in the first transform coefficient block according to a vertical scanning order to obtain the LFNST coefficients.

[0380] Specifically, when the value of isTransposed is equal to 1, the sampling point input vector used in the MIP mode is transposed, and LFNST can be performed on at least part of the transform coefficients in the first transform coefficient block according to the horizontal scanning order, as shown in the above formula (10).

[0381] When the value of isTransposed is equal to 0, the sampling point input vector used by the MIP mode is not transposed. Then LFNST can be performed on at least part of the transform coefficients in the first transform coefficient block according to the vertical scanning order, as shown in the above formula (11).

[0382] Among them, the determination of log2LfnstSize is shown in the above formula (9).

[0383] In addition, the LFNST in the embodiment of the present application can be applied to the traditional intra prediction mode (eg, angle intra prediction mode, DC mode, and PLANAR mode, etc.) in addition to the MIP mode. The specific implementation method is similar to that of the MIP mode.

[0384] In some embodiments, determining the LFNST transform kernel candidate set according to the MIP parameters may include:

[0385] According to the index value of the traditional intra prediction mode, determine the value of the LFNST intra prediction mode index number;

[0386] According to the value of the LFNST intra prediction mode index sequence number, determining the index value of the LFNST transform kernel candidate set through a sixth preset lookup table;

[0387] The LFNST transform core candidate set is selected from a plurality of candidate LFNST transform core candidate sets according to the index value of the LFNST transform core candidate set.

[0388] It should be noted that the sixth preset lookup table is shown in Table 11. According to the index value of the traditional intra prediction mode, the value of predModeIntra can be determined; then according to the value of predModeIntra, the value of lfnstTrSetIdx can be directly determined in combination with Table 11, that is, the LFNST transform core candidate set selected by the current block is determined. Here, the index value of the traditional intra prediction mode can be assigned to predModeIntra to determine the value of lfnstTrSetIdx.

[0389] Table 11

[0390] predModeIntra lfnstTrSetIdx predModeIntra<0 1 0<=predModeIntra<=1 0 2<=predModeIntra<=12 1 13<=predModeIntra<=23 2 24<=predModeIntra<=44 3 45<=predModeIntra<=55 2 56<=predModeIntra<=80 1

[0391] The following will be combined Figure 1 The encoder 100 shown specifically illustrates the encoding method of the embodiment of the present application.

[0392] It should be understood that the current block (or referred to as "coding block") in the embodiment of the present application can be a CU or a partition of a CU (e.g., a transform block). Specifically, when the intra-prediction unit 106 determines that the MIP mode is used to encode the current block, or when the intra-prediction unit 106 evaluates the MIP mode used to encode the current block using a rate-distortion optimization method, the intra-prediction unit 106 will obtain an intra-prediction block of the current block; wherein the intra-prediction block of the current block includes an intra-prediction value of at least one pixel (samples) in the current block.

[0393] Specifically, for the intra prediction unit 106, it uses the MIP mode to obtain the intra prediction block, the steps are as follows:

[0394] First, the intra-frame prediction unit 106 obtains one or more reference pixels from the adjacent pixels of the current block, for example, by downsampling the adjacent pixels, or directly extracting from the adjacent pixels.

[0395] Then, the intra prediction unit 106 uses the obtained reference pixel points, the MIP matrix and the shift parameter to determine one or more partial prediction pixel points corresponding to the pixel point position in the current block. Here, the pixel point position can be a preset pixel point position in the current block. For example, the pixel point position has uniform horizontal and vertical coordinate values. The shift parameter includes a shift amount parameter and a shift offset parameter, and the shift parameter can be used for an offset operation in the process of obtaining the intra prediction value of the partial pixel point.

[0396] Finally, if the predicted pixel points corresponding to the partial pixel points of the current block are obtained, the intra-frame prediction unit 106 also needs to obtain the predicted pixel points corresponding to the remaining pixel points in the current block except for the partial pixel points. For example, the intra-frame prediction unit 106 can use an interpolation filter to obtain the predicted pixel points corresponding to the remaining pixel points, where the input of the interpolation filter can be the partial pixel points and the adjacent pixel points.

[0397] like Figure 6 As shown, it shows an example of a flow chart of using the MIP mode to obtain an intra-frame prediction block in an embodiment of the present application. The process can be implemented on the encoder 100, where the "current block" refers to the "encoded block".

[0398] Step 601, the intra-frame prediction unit 106 obtains adjacent pixels of the current block, for example, adjacent pixels are marked as Figure 6 The gray filled squares adjacent to the current block in step 601 are shown. The intra-frame prediction unit 106 obtains one or more reference pixels from the adjacent pixels. Figure 6 In the example of step 601, the intra-frame prediction unit 106 may optionally calculate an average value of two adjacent pixels and use the average value as a reference pixel. Alternatively, the intra-frame prediction unit 106 selects an adjacent pixel as a reference pixel for every other adjacent pixel. Figure 6 In the example of step 601, the intra prediction unit 106 selects 4 reference pixels from the 8 upper adjacent pixels of the current block, and selects another 4 reference pixels from the 8 left adjacent pixels of the current block.

[0399] The specific process of step 601 is as follows:

[0400] The intra prediction unit 106 obtains the width and height of the current block from the block segmentation unit 103, and the width and height are represented by variables cbWidth and cbHeight, respectively. As an example, the intra prediction unit 106 calls a rate-distortion optimized mode determination method to determine an intra prediction mode, in which the current block is divided into one or more transform blocks. Let variables nTbW and nTbH be the width and height of the transform block, respectively. When the MIP mode is used as the intra prediction mode to obtain the intra prediction value of the current block, the intra prediction unit 106 determines the block size index value of the MIP, that is, the variable represented as mipSizeId.

[0401] Optionally, the intra prediction unit 106 determines the value of mipSizeId as follows:

[0402] - If nTbW and nTbH are both equal to 4, mipSizeId is set equal to 0;

[0403] - Otherwise, if cbWidth or cbHeight is equal to 4, mipSizeId is set equal to 1;

[0404] Otherwise, mipSizeId is set equal to 2.

[0405] Specifically, when the size parameter of the current block is 8×8 (ie, both cbWidth and cbHeight are equal to 8), mipSizeId is set to be equal to 2.

[0406] Optionally, the intra prediction unit 106 determines the value of mipSizeId as follows:

[0407] - If nTbW and nTbH are both equal to 4, mipSizeId is set equal to 0;

[0408] - Otherwise, if cbWidth or cbHeight is equal to 4, or if both cbWidth and cbHeight are equal to 8, then mipSizeId is set equal to 1;

[0409] Otherwise, mipSizeId is set equal to 2.

[0410] Specifically, when the size parameter of the current block is 8×8, mipSizeId is set to be equal to 1.

[0411] Further, the intra prediction unit 106 obtains the values ​​of boundarySize and predSize according to mipSizeId, as follows:

[0412] - If mipSizeId is equal to 0, boundarySize is set equal to 2 and predSize is set equal to 4;

[0413] - Otherwise, if mipSizeId is equal to 1, boundarySize is set equal to 4 and predSize is set equal to 4;

[0414] - Otherwise (if mipSizeId is equal to 2), boundarySize is set equal to 4 and predSize is set equal to 8;

[0415] Wherein, boundarySize represents the number of reference pixels obtained from each side of the reference pixels adjacent to the upper side and the reference pixels adjacent to the left side of the current block.

[0416] The intra-frame prediction unit 106 can also obtain the variable isTransposed to indicate the order of reference pixels stored in the buffer pTemp. For example, isTransposed equal to 0 means that the intra-frame prediction unit 106 first presents the reference pixels obtained from the reference pixels adjacent to the upper side of the current block, and then presents the reference pixels obtained from the reference pixels adjacent to the left side; otherwise, isTransposed equal to 1 means that the intra-frame prediction unit 106 first presents the reference pixels obtained from the reference pixels adjacent to the left side of the current block, and then presents the reference pixels obtained from the reference pixels adjacent to the upper side. Here, the value of isTransposed will be sent to the entropy coding unit 115 as one of the parameters of the MIP mode that needs to be encoded and written into the bitstream.

[0417] The intra prediction unit 106 obtains the value of inSize to indicate the number of reference pixels used when using the MIP mode, as shown in the above equation (1).

[0418] The intra prediction unit 106 calls the following process to obtain a set of reference pixels (stored as an array p[x], where x is 0, . . . , inSize-1) using reference pixels adjacent to the current block.

[0419] The intra prediction unit 106 obtains nTbW reference pixels from the reference pixels adjacent to the upper side of the current block (e.g., stored in the array refT), and obtains nTbH reference pixels from the reference pixels adjacent to the left side of the current block (e.g., stored in the array refL). The intra prediction unit 106 calls a downsampling process on refT to obtain a boundarySize reference pixel, and stores the boundarySize reference pixel in refT. The intra prediction unit 106 calls a downsampling process on refL to obtain a boundarySize reference pixel, and stores the boundarySize reference pixel in refL.

[0420] The intra-frame prediction unit 106 can also obtain a variable isTransposed to indicate the order of reference pixels stored in the buffer pTemp. For example, isTransposed equal to 0 (or FALSE) indicates that the intra-frame prediction unit 106 first presents the reference pixels obtained from the reference pixels adjacent to the upper side of the current block, and then presents the reference pixels obtained from the reference pixels adjacent to the left side; otherwise, isTransposed equal to 1 (or TRUE) indicates that the intra-frame prediction unit 106 first presents the reference pixels obtained from the reference pixels adjacent to the left side of the current block, and then presents the reference pixels obtained from the reference pixels adjacent to the upper side. The intra-frame prediction unit 106 can determine the value of isTransposed using a rate-distortion optimization method or based on a correlation comparison between adjacent reference pixels and the current block. The value of isTransposed will be sent to the entropy coding unit 115 as one of the parameters of the MIP mode that needs to be encoded and written into the bitstream.

[0421] The intra prediction unit 106 places the elements in refT and refL into the buffer pTemp according to the order indicated by isTransposed.

[0422] Optionally, the intra prediction unit 106 obtains p[x], where x=0, ..., inSize-1, as follows:

[0423] - If mipSizeId is equal to 2, then p[x] = pTemp[x+1] - pTemp[0];

[0424] - Otherwise (mipSizeId is less than 2), p[0] = pTemp[0] - (1 << (BitDepth - 1)), p[x] = pTemp[x] - pTemp[0];

[0425] Wherein, BitDepth is the bit depth of the color component of the pixel in the current block. Here, the color component can be one of the RGB components, or one of the YUV components, or one of the YCbCr components, for example, the Y component.

[0426] Optionally, the intra prediction unit 106 may obtain p[x], where x=0, ..., inSize-1, as follows:

[0427] - If mipSizeId is equal to 2, then p[x] = pTemp[x+1] - pTemp[0];

[0428] -Otherwise (mipSizeId is less than 2), p[0] = (1 << (BitDepth - 1)) - pTemp[0], p[x] = pTemp[x] - pTemp[0].

[0429] Alternatively, the intra-frame prediction unit 106 may use a unified calculation method to obtain the value of p[x] without determining the value of mipSizeId. For example, (1<<(BitDepth-1)) is added as an additional element to the buffer pTemp. In this case, the intra-frame prediction unit 106 calculates p[x] as pTemp[x+1]-pTemp[0].

[0430] In step 602, the intra prediction unit 106 obtains a MIP prediction value of the current block using a set of reference pixels and a MIP weighting matrix, wherein the MIP weighting matrix is ​​selected from a set of predefined MIP weighting matrices according to a MIP mode index value (expressed as ModeId) corresponding to the MIP mode and a block size index value (expressed as mipSizeId) of the MIP.

[0431] The intra-frame prediction unit 106 obtains the MIP prediction value (expressed as predMip[x][y]) of some predicted pixel points corresponding to one or more pixel point positions in the current block. Figure 6 In the example of step 602, the partially predicted pixels are the pixels marked as gray filled squares in the current block. The input of the prediction module 601 is the reference pixel p[x] obtained in step 601, and the prediction module 601 calculates the partially predicted pixels using the MIP weighting matrix and the shift parameter; here, the shift parameter includes a shift amount parameter and a shift offset parameter.

[0432] In a possible implementation, the prediction module 601 may set its coordinates to (x, y) and obtain the predicted pixel point represented by predMip[x][y]. Then the calculation formula of predMip[x][y] is as shown in Formula (5) or Formula (6).

[0433] In equation (5) or (6), mWeight[i][j] is a MIP weight matrix; here, the matrix elements may be predetermined constant values; or, they may be adaptively updated using, for example, a training method, the input of which is one or more coded pictures or blocks, or pictures in other bitstreams provided to the encoder 100 by an external device; fO is a shift offset parameter for determining oW; sW is a shift quantity parameter; p[i] is a MIP input sample value calculated using a reference pixel; pTemp[0] represents a value corresponding to an index number of 0 in the first temporary reference value; ">>" is a binary right shift operator as defined in VVC. When the MIP mode is finally determined to perform intra-frame prediction on the current block, the intra-frame prediction unit 106 may also send mWeight[i][j] to the entropy coding unit 115. The entropy coding unit 115 may write mWeight[i][j] into one or more data units in the bitstream.

[0434] The prediction module 601 may determine the values ​​of sW and fO according to the size of the current block and the MIP mode used for the current block. In one example, the prediction module 601 obtains the values ​​of sW and fO using a lookup table.

[0435] Optionally, the prediction module 601 may use the above-mentioned Table 4 to determine sW according to the size parameters of the current block and the MIP mode.

[0436] Optionally, the prediction module 601 may also use the above Table 5 to determine sW according to the size parameters of the current block.

[0437] Optionally, the prediction module 601 may also directly set sW to a constant value. For example, for blocks of various size parameters and different MIP modes, the prediction module 601 may set sW to 5; or, for blocks of various size parameters and different MIP modes, the prediction module 601 may set sW to 6; or, for blocks of various size parameters and different MIP modes, the prediction module 601 may set sW to 7.

[0438] Optionally, the prediction module 601 may use the above-mentioned Table 6 to determine fO according to the size parameters of the current block and the MIP mode.

[0439] Optionally, the prediction module 601 may also use the above-mentioned Table 7 to determine f0 according to the size parameters of the current block.

[0440] Optionally, the prediction module 601 may also directly set fO to a constant value (such as 0 to 100). For example, for blocks of various size parameters and different MIP modes, the prediction module 601 sets fO to 32; or, the prediction module 601 sets fO to 46; or, the prediction module 601 sets fO to 56; or, the prediction module 601 sets fO to 66.

[0441] The intra prediction unit 106 may perform an embedding operation on the MIP prediction value in predMip. When isTransposed is equal to 1 (or TRUE), the predSize×predSize array predMip[x][y] (where x=0, ..., predSize-1, y=0, ..., predSize-1) is converted to predTemp[y][x]=predMip[x][y], and then predMip=predTemp.

[0442] Exemplarily, optionally, when the size parameter of the current block is 8×8 (ie, both cbWidth and cbHeight are equal to 8), the intra prediction unit 106 determines that mipSizeId is equal to 2 and obtains 8×8 predMip.

[0443] Optionally, when the size parameter of the current block is 8×8 (ie, both cbWidth and cbHeight are equal to 8), the intra prediction unit 106 determines that mipSizeId is equal to 1 and obtains 4×4 predMip.

[0444] Step 603, the intra prediction unit 106 obtains the intra prediction value of the current block (stored in an array as predSamples[x][y], where x=0, ..., nTbW-1, y=0, ..., nTbH-1), as follows:

[0445] - If the intra prediction unit 106 determines that nTbW is greater than predSize or nTbH is greater than predSize, the intra prediction unit 106 calls the upsampling process to obtain predSamples using predMip. The intra prediction unit 106 obtains the predicted pixel points corresponding to the remaining pixel points in the current block except for some pixel points. Figure 6 In the step 601, the intra prediction unit 106 may use the filtering module 602 to obtain the predicted pixel points corresponding to the remaining pixel points in the current block except for some pixel points. The input of the filtering module 602 is the pixel points marked as gray filled squares in step 602. The filtering module 602 may use one or more interpolation filters to obtain the predicted pixel points corresponding to the remaining pixel points in the current block except for some pixel points using the input. For example, the input may include the reference pixel point and the partial predicted pixel points of one or more pixel point positions in the current block; or, the input may include the adjacent pixel point and the partial predicted pixel points of one or more pixel point positions in the current block; or, the input may include the reference pixel point, the adjacent pixel point and the partial predicted pixel points of one or more pixel point positions in the current block. For example, optionally, when the size of the current block is 8×8 (i.e., cbWidth and cbHeight are both equal to 8), the intra prediction unit 106 determines that mipSizeId is equal to 1, and the intra prediction unit 106 applies the upsampling process to 4×4predMip to obtain an 8×8 intra prediction block of the current block.

[0446] -Otherwise, the intra prediction unit 106 sets the intra prediction block of the current block to be equal to the MIP prediction block of the current block, that is, sets predSamples[x][y] (where x=0, ..., nTbW-1, y=0, ..., nTbH-1) to be equal to predMip[x][y]. For example, specifically, when the size parameter of the current block is 8×8 (that is, cbWidth and cbHeight are both equal to 8), the intra prediction unit 106 determines that mipSizeId is equal to 2, and the intra prediction unit 106 obtains predSamples of the current block, whose size is equal to 8×8 (that is, cbWidth and cbHeight are both equal to 8).

[0447] So, according to Figure 6 As shown in the flowchart, after step 603, the intra prediction unit 106 may obtain the intra prediction block of the current block (ie, CU), that is, determine the intra prediction value of at least one pixel in the current block.

[0448] The prediction unit 102 outputs the intra-frame prediction block of the current block. The first adder 107 calculates the difference between the current block in the output of the segmentation unit 101 and the intra-frame prediction block of the current block, that is, the residual block (i.e., residual CU). The transform unit 108 reads the residual block and performs one or more transform operations on the residual block to obtain coefficients. The quantization unit 109 quantizes the coefficients and outputs the quantized coefficients (i.e., levels).

[0449] The transform unit 108 performs a first transform on the residual block, for example, an integer transform originally designed based on DCT. The transform unit 108 determines whether to allow a secondary transform to be applied to the block. When it is determined that the application is allowed, the transform unit 108 further determines whether to apply the secondary transform to the coefficients obtained after performing the first transform. LFNST is an example of a secondary transform.

[0450] See also Figure 7 , which shows a schematic diagram of a process framework for executing LFNST provided in an embodiment of the present application. Figure 7 , block 701 represents a residual block.

[0451] In step 701, the transform unit 108 performs a first transform (ie, a core transform) on block 701, for example, an integer transform initially designed based on DCT, to obtain block 702 containing transform coefficients corresponding to the first transform (called "first transform coefficients").

[0452] In step 702 , the transform unit 108 determines whether to allow a secondary transform, such as LFNST, to be performed on all or part of the first transform coefficients in the block 702 .

[0453] Exemplarily, optionally, if the intra prediction mode of the transform block including the LFNST is the MIP mode, and the minimum value of the width and height of the transform block is greater than or equal to 8, the transform unit 108 is allowed to perform the LFNST on the transform block.

[0454] Optionally, if the intra prediction mode of the transform block including LFNST is MIP mode and the value of mipSizeId is equal to a preset value (eg, 2) or one of multiple preset values ​​(eg, 1, 2), the transform unit 108 is allowed to perform LFNST on the transform block.

[0455] When the transform unit 108 is allowed to perform LFNST on the transform block, the transform unit 108 determines LFNST parameters, such as lfnst_index. When lfnst_index is equal to 0, it indicates that the transform unit 108 does not perform LFNST on the transform block. When lfnst_index is greater than 0, it indicates that LFNST is performed on the transform block using a transform core in the LFNST transform core candidate set.

[0456] The transform unit 108 may adopt a conventional rate-distortion optimization method to determine the value of lfnst_index. First, the transform unit 108 obtains a first cost result without performing LFNST on the transform block.

[0457] The transform unit 108 selects a LFNST transform kernel candidate set according to the MIP parameters. The MIP parameters may include at least one of: a MIP mode index value (ie, modeId) and a MIP transposed indication parameter (ie, isTransposed).

[0458] The transform unit 108 sets the value of the variable predModeIntra to be equal to the value of modeId. The transform unit 108 selects a portion of the first transform coefficients in block 702, for example, the coefficients in sub-blocks 7001, 7002, and 7003, and determines a transform kernel for performing LFNST on the coefficients on the sub-block. The size of the sub-block may be a preset value, for example, 8×8. The transform unit 108 determines the index value (i.e., lfnstTrSetIdx) of the LFNST transform kernel candidate set according to the value of predModeIntra. For block coding in MIP mode, the transform unit 108 may use a lookup table as shown in Table 10 to obtain the value of lfnstTrSetIdx.

[0459] The transform unit 108 may use a conventional rate-distortion optimization method to determine the value of lfnst_index (i.e., the value is greater than 0, indicating that the transform core used in the LFNST transform core candidate set is indicated by lfnstTrSetIdx). The transform unit 108 sets the value of lfnst_index to the index number of the transform core in the LFNST transform core candidate set, so that the cost function has a minimum value and a second cost result is obtained.

[0460] If the first cost result is greater than the second cost result, the transform unit 108 determines to perform LFNST on the transform block, and sends lfnst_index indicating the index number of the transform core to the entropy coding unit 115. Otherwise, if the transform unit 108 determines not to perform LFNST on the transform block, lfnst_value is set to 0, and lfnst_index indicating that LFNST is not performed is sent to the entropy coding unit 115.

[0461] When lfnst_index is not equal to 0, the transform unit 108 performs LFNST on the transform block using coefficients from sub-blocks 7001, 7002, and 7003, obtains LFNST coefficients, and places the LFNST coefficients into sub-blocks 7011, 7012, and 7013 of block 703. Assume that d[x][y] is a block of first transform coefficients, where x=0, ..., nTbW-1, y=0, ..., nTbH-1, where nTbW and nTbH are the width and height of block 703, respectively. Assume that v[x] is a block of LFNST coefficients, where x=0, ..., nLfnstOutSize-1, and nLfnstOutSize=(nTbW>=8&&nTbH>=8)? 48:16.

[0462] Optionally, the transform unit 108 obtains the block 703 of transform coefficients using the following instructions, and transmits the block 703 of LFNST coefficients in d[x][y] to the quantization unit 109, as specifically shown in the above equation (8). Wherein, log2LfnstSize is shown in the above equation (9).

[0463] Optionally, when isTransposed is equal to 0, the transform unit 108 obtains the block 703 of LFNST coefficients using the following instructions, and transmits the block 703 of LFNST coefficients in d[x][y] to the quantization unit 109, as shown in the above formula (10);

[0464] When isTransposed is equal to 1, the transform unit 108 obtains the block 703 of LFNST coefficients using the following instructions, and transmits the block 703 of LFNST coefficients in d[x][y] to the quantization unit 109, as shown in the above formula (11).

[0465] Optionally, when isTransposed is equal to 1, the transform unit 108 obtains the block 703 of LFNST coefficients using the following instructions, and transmits the block 703 of LFNST coefficients in d[x][y] to the quantization unit 109, as shown in the above formula (10);

[0466] When isTransposed is equal to 0, the transform unit 108 obtains the block 703 of LFNST coefficients using the following instructions, and transmits the block 703 of LFNST coefficients in d[x][y] to the quantization unit 109, as shown in the above formula (11).

[0467] Among them, the determination of log2LfnstSize is shown in the above formula (9).

[0468] In addition, the transform unit 108 may also determine the value of lfnst_index for a transform block encoded in a conventional intra prediction mode (i.e., an angular intra prediction mode, a DC mode, and a PLANAR mode, etc.) Similar to the rate-distortion optimization method for determining the value of lfnst_index for a transform block in a MIP mode, the first cost result is determined without performing LFNST on the transform block.

[0469] The transform unit 108 uses a lookup table to determine the value of lfnstTrSetIdx according to the mode index value of the traditional intra prediction mode (assigned to predModeIntra), that is, to determine the LFNST transform kernel candidate set selected for the current block, as shown in Table 11 above.

[0470] Then, the transform unit 108 may determine the value of lfnst_index using a conventional rate-distortion optimization method (i.e., the value is greater than 0, indicating that the transform core used in the LFNST transform core candidate set is indicated by lfnstTrSetIdx). The transform unit 108 sets the value of lfnst_index to the index number of the transform core in the LFNST transform core candidate set, so that the cost function has a minimum value and a second cost result is obtained.

[0471] If the first cost result is greater than the second cost result, the transform unit 108 determines to perform LFNST on the transform block, and sends lfnst_index indicating the index number of the transform core to the entropy coding unit 115. Otherwise, if the transform unit 108 determines not to perform LFNST on the transform block, lfnst_value is set to 0, and lfnst_index indicating that LFNST is not performed is sent to the entropy coding unit 115.

[0472] When lfnst_index is not equal to 0, the transform unit 108 performs LFNST on the transform block using coefficients from sub-blocks 7001, 7002, and 7003, obtains LFNST coefficients, and places the LFNST coefficients into sub-blocks 7011, 7012, and 7013 of block 703. Assume that d[x][y] is a block of first transform coefficients, where x=0, ..., nTbW-1, y=0, ..., nTbH-1, where nTbW and nTbH are the width and height of block 703, respectively. Assume that v[x] is a block of LFNST coefficients, where x=0, ..., nLfnstOutSize-1, and nLfnstOutSize=(nTbW>=8&&nTbH>=8)? 48:16.

[0473] Optionally, when predModeIntra is less than or equal to 34, the transform unit 108 obtains the block 703 of LFNST coefficients using the following instructions, and transmits the block 703 of LFNST coefficients in d[x][y] to the quantization unit 109, as shown in the above formula (10);

[0474] When predModeIntra is greater than 34, the transform unit 108 obtains the block 703 of LFNST coefficients using the following instructions, and transmits the block 703 of LFNST coefficients in d[x][y] to the quantization unit 109, as specifically shown in the above formula (11).

[0475] Optionally, when predModeIntra is greater than 34, the transform unit 108 obtains the block 703 of LFNST coefficients using the following instructions, and transmits the block 703 of LFNST coefficients in d[x][y] to the quantization unit 109, as shown in the above formula (10);

[0476] When predModeIntra is less than or equal to 34, the transform unit 108 obtains the block 703 of LFNST coefficients using the following instructions, and transmits the block 703 of LFNST coefficients in d[x][y] to the quantization unit 109, as shown in the above formula (11).

[0477] Among them, the determination of log2LfnstSize is shown in the above formula (9).

[0478] Further, the inverse quantization unit 110 performs a scaling operation on the quantized coefficients to output a reconstructed coefficient. The inverse transform unit 111 performs one or more inverse transforms corresponding to the transform in the transform unit 108 and outputs a reconstructed residual. The second adder 112 calculates a reconstructed CU by adding the reconstructed residual and the intra-frame prediction block of the current block from the prediction unit 102. The second adder 112 also sends its output to the prediction unit 102 for use as an intra-frame prediction reference.

[0479] The output of the filter unit 113 is a decoded picture or sub-picture, which is sent to the DPB unit 114. The DPB unit 114 outputs the decoded picture according to the timing and control information. Among them, the picture stored in the DPB unit 114 can also be used as a reference for the prediction unit 102 to perform inter-frame prediction or intra-frame prediction.

[0480] The entropy coding unit 115 converts the parameters of the units in the encoder 100 (necessary to obtain the decoded picture) as well as the control parameters and supplementary information into binary representations, and writes such binary representations into the generated code stream (or "video bit stream") according to the grammatical structure of each data unit, as shown in Tables 9A and 9B.

[0481] Using the syntax structure in Table 9A, when the minimum value of the width and height of the transform block in the coding unit is greater than or equal to 8 (ie, both the width and the height are greater than or equal to 8), the entropy coding unit 115 encodes the lfnst_index of the coding unit using the MIP mode. Another optional condition may be set to: the width and the height are greater than or equal to a first value (e.g., 8), and the sum of the width and the height is greater than or equal to a second value (e.g., 16).

[0482] Using the syntax structure in Table 9B, the entropy encoding unit 115 encodes the lfnst_index of the coding unit using the MIP mode when the mipSizeId of the coding unit is equal to 2. Another optional condition may be set as: the mipSizeId of the coding unit is equal to one of multiple values ​​(eg, 1 and 2).

[0483] See also Figure 8 , which shows a schematic diagram of a process for encoding LFNST parameters provided by an embodiment of the present application. Figure 8 As shown, the process may include:

[0484] S801: The entropy encoding unit 115 checks the above conditions to determine whether to encode the LFNST parameters.

[0485] S802: The entropy coding unit 115 encodes the LFNST parameters using context-based adaptive binary arithmetic coding.

[0486] It should be noted that in step S801, the entropy coding unit 115 checks and determines the LFNST parameters, such as the value of lfnst_index; then in step S802, the value of lfnst_index can be encoded using CABAC with a descriptor of "ae(v)" as shown in Tables 9A and 9B.

[0487] In the embodiment of the present application, the encoder 100 may be a computing device having a processor and a storage medium recording an encoding program. When the processor reads and runs the encoding program, the encoder 100 reads the input video and generates a corresponding bitstream. In addition, the encoder 100 may also be a computing device having one or more chips. The units implemented as integrated circuits on the chip are connected to the processor. Figure 1 The corresponding units in the same embodiment have similar connections and data exchange and have similar functions.

[0488] The present embodiment provides a coding method, by determining prediction parameters of a current block, wherein the prediction parameters include prediction mode parameters; when the prediction mode parameters indicate that a matrix-based intra-frame prediction MIP mode is used to determine an intra-frame prediction value for a chroma component of the current block, adjacent sampling values ​​of the current block are obtained, and the MIP input sampling value of the current block is determined according to the adjacent sampling values ​​of the current block; the MIP prediction value of the chroma component of the current block is determined according to the MIP input sampling value, the MIP weighting matrix and the shift parameter; the MIP prediction value is filtered to determine the intra-frame prediction value of the chroma component of the current block; the prediction residual value of the chroma component of the current block is determined according to the intra-frame prediction value of the chroma component of the current block; a low-frequency non-separable secondary transform LFNST is performed on the prediction residual value to determine the LFNST parameter; the LFNST parameter is encoded and written into a bitstream. In this way, for the MIP mode, the complexity can be reduced while ensuring the coding performance, while reducing the storage space required in the coding process, effectively improving the coding efficiency; in addition, when LFNST technology is applied to MIP mode prediction, the introduction of MIP parameters makes the LFNST transformation more flexible, further improving the coding efficiency.

[0489] In another embodiment of the present application, see Fig. 9 , which shows a schematic flow chart of a decoding method provided by an embodiment of the present application. Fig. 9 As shown, the method may include:

[0490] S901: Parse the bitstream to obtain prediction parameters and LFNST parameters of the current block, wherein the prediction parameters include prediction mode parameters.

[0491] It should be noted that the image to be decoded can be divided into multiple image blocks, and each image block to be decoded can be called a decoding block. Here, each decoding block may include a first image component, a second image component, and a third image component; and the current block is a decoding block in the video image that is currently to be predicted for the first image component, the second image component, or the third image component.

[0492] It should also be noted that the prediction parameters may include prediction mode parameters, wherein the prediction mode parameters are used to indicate the prediction mode adopted by the current block, and different prediction modes correspond to different prediction mode parameters. Among them, the prediction modes generally include inter-frame prediction mode, traditional intra-frame prediction mode and non-traditional intra-frame prediction mode, etc., and the traditional intra-frame prediction mode may include DC mode, PLANAR mode and angle intra-frame prediction mode, etc., and the non-traditional intra-frame prediction mode may include MIP mode, CCLM mode, IBC mode and PLT mode, etc. In other words, the encoder will select the optimal prediction mode to encode the current block. In this process, the prediction mode of the current block can be determined, thereby obtaining the corresponding prediction mode parameters, and then the prediction parameters including the prediction mode parameters are written into the bitstream, which is transmitted from the encoder to the decoder.

[0493] In this way, in the decoder, the prediction parameters of the current block can be obtained by parsing the bitstream, and the prediction mode parameters included in the prediction parameters obtained through parsing can be used to determine whether the current block uses the MIP mode.

[0494] S902: When the prediction mode parameter indicates that the intra prediction value of the chrominance component of the current block is determined using a matrix-based intra prediction MIP mode, adjacent sampling values ​​of the current block are obtained, and the MIP input sampling value of the current block is determined according to the adjacent sampling values ​​of the current block.

[0495] It should be noted that, for the current block, the embodiment of the present application uses the MIP mode to perform intra-frame prediction on the chrominance component of the current block. In this process, it is first necessary to obtain the adjacent sampling values ​​of the current block; then, based on the adjacent sampling values ​​of the current block, the MIP input sampling values ​​of the current block are determined.

[0496] The adjacent sampling values ​​of the current block may include the left adjacent sampling values ​​of the current block and the upper adjacent sampling values ​​of the current block. That is, the MIP input sampling values ​​of the current block may be determined according to the left adjacent sampling values ​​and the upper adjacent sampling values ​​of the current block.

[0497] In some embodiments, determining the MIP input sample value of the current block according to the adjacent sample values ​​of the current block may include:

[0498] Determining a block size index value of the current block according to a size parameter of the current block;

[0499] Performing down-sampling filtering on adjacent sample values ​​of the current block to obtain a first temporary reference value;

[0500] When the block size index value of the current block is within a preset range, determine a second constant value according to the bit depth of the adjacent sample values ​​of the current block; set the value corresponding to the index number 0 in the MIP input sample value to be equal to the difference between the second constant value and the value corresponding to the index number 0 in the first temporary reference value; set the value corresponding to the index number i in the MIP input sample value to be equal to the difference between the value corresponding to the index number i in the first temporary reference value and the value corresponding to the index number equal to 0 in the first temporary reference value, where i is an integer greater than 0;

[0501] When the block size index value of the current block is outside a preset range, the value corresponding to the index number j in the MIP input sampling value is set to be equal to the difference between the value corresponding to the index number j+1 in the first temporary reference value and the value corresponding to the index number 0 in the first temporary reference value, where j is an integer greater than or equal to 0.

[0502] Further, when the block size index value of the current block is within a preset range, the method may further include:

[0503] Determine a second constant value according to the bit depth of adjacent sampling values ​​of the current block; set the value corresponding to index number 0 in the MIP input sampling value to be equal to the difference between the value corresponding to index number 0 in the first temporary reference value and the second constant value; set the value corresponding to index number i in the MIP input sampling value to be equal to the difference between the value corresponding to index number i in the first temporary reference value and the value corresponding to index number 0 in the first temporary reference value, where i is an integer greater than 0.

[0504] It should be noted that after downsampling and filtering the adjacent sample values ​​of the current block, the first temporary reference value can be obtained. Here, for the first temporary reference value, specifically, the adjacent sample values ​​after filtering can be cached in a buffer (represented by pTemp). Among them, the value corresponding to the index number 0 in the first temporary reference value refers to pTemp[0], and the value corresponding to the index number i in the first temporary reference value refers to pTemp[i].

[0505] In the embodiment of the present application, the prediction parameters may include, in addition to the prediction mode parameters, the size parameters of the current block. The size parameters of the current block may include the width (expressed as nTbW) and height (expressed as nTbH) of the current block. Furthermore, according to the size parameters of the current block, the block size index value (i.e., mipSizeId) of the current block may be determined.

[0506] It should also be noted that, according to whether the block size index value (represented by mipSizeId) of the current block is within the preset range, it can be determined whether the size parameter value of the current block is within the preset range. Specifically, when mipSizeId=0 or 1, it indicates that the block size index value of the current block is within the preset range, that is, the size parameter value of the current block is within the preset range; when mipSizeId=2, it indicates that the block size index value of the current block is outside the preset range, that is, the size parameter value of the current block is not within the preset range.

[0507] That is to say, the MIP input sampling value is determined by the buffer (represented by pTemp), the block size index value of the current block (represented by MipSizeId), and the bit depth of the adjacent sampling value of the current block (represented by BitDepth), and the number of input samples contained in the MIP input sampling value is only related to the block size index value of the current block. Finally, the value corresponding to the index number x in the MIP input sampling value can be obtained (represented by p[x]).

[0508] Further, in some embodiments, determining the second constant value according to the bit depth of the adjacent sample values ​​of the current block may include:

[0509] The second constant value is set to be equal to an integer exponent power of 2, wherein the exponent of the power is equal to the bit depth of the neighboring sample values ​​of the current block minus 1.

[0510] Alternatively, in some embodiments, determining the second constant value according to the bit depth of the adjacent sample values ​​of the current block may include:

[0511] Perform a binary bit left shift on "1" to obtain the second constant value, wherein the number of bits of the bit left shift is equal to the bit depth of the adjacent sample values ​​of the current block minus 1.

[0512] That is, after obtaining the bit depth (expressed as BitDepth) of the adjacent sample values ​​of the current block, the second constant value can be expressed as 1<<(BitDepth-1) or 2^(BitDepth-1). In this way, when the size parameter value of the current block is within the preset range, the second constant value can be combined to determine the MIP input sample value of the current block.

[0513] It should also be noted that the MIP input sample is a matrix vector used for matrix multiplication. The current related technical solution is determined by the buffer (represented by pTemp), the type of the current block (i.e., the block size index value of the current block, represented by mipSizeId), the bit depth of the adjacent sample values ​​of the current block (represented by BitDepth) and the number of MIP input samples, and finally obtains the value corresponding to the index number x in the MIP input sample value (represented by p[x]).

[0514] In a possible implementation, when mipSizeId=0 or 1, p[0] may be obtained by subtracting pTemp[0] from 1<<(BitDepth-1); and when x is not equal to 0, p[x] may be obtained by subtracting pTemp[0] from pTemp[x]. Specifically, as shown in the above formula (2).

[0515] In another possible implementation, when mipSizeId=0 or 1, p[0] can be obtained by subtracting 1<<(BitDepth-1) from pTemp[0]; and when x is not equal to 0, p[x] can be obtained by subtracting pTemp[0] from pTemp[x]. Specifically, as shown in the above formula (3).

[0516] In another possible implementation, when mipSizeId=2, p[x] is obtained by subtracting pTemp[0] from pTemp[x+1], as shown in the above formula (4).

[0517] In addition, the embodiment of the present application can also use a unified calculation method to obtain the value of p[x] without determining the value of mipSizeId. In this case, the embodiment of the present application can directly set p[x] to be equal to pTemp[x+1]-pTemp[0], x=0,…,inSize-1, by appending 1<<(BitDepth-1) as an additional element at the end of the buffer pTemp.

[0518] In this way, after the MIP input sample value of the current block is determined according to the adjacent sample values ​​of the current block, the MIP prediction value of the current block can be further determined.

[0519] S903: Determine the MIP prediction value of the chrominance component of the current block according to the MIP input sample value, the MIP weighting matrix and the shift parameter.

[0520] The MIP prediction value is the prediction value of some sampling points in the chrominance component of the current block.

[0521] It should be noted that the shift parameter may include a shift offset parameter and a shift quantity parameter. At this time, for S903, determining the MIP prediction value of the chrominance component of the current block according to the MIP input sample value, the MIP weighting matrix and the shift parameter may include:

[0522] Determine, according to the value of the shift offset parameter, the product of the sum of the MIP input sampling values ​​and the shift offset parameter, wherein the value of the shift offset parameter is a fixed constant;

[0523] Determine a first constant value according to the value of the shift quantity parameter, wherein the value of the shift quantity parameter is a fixed constant;

[0524] Setting the value of the first offset to the difference between the first constant value and the product;

[0525] Determining a MIP weighting matrix of the current block according to the prediction parameters;

[0526] A MIP prediction value of the chrominance component of the current block is determined according to the MIP weighting matrix, the MIP input sample value, the shift amount parameter and the first offset.

[0527] It should be noted that the shift offset parameter may also be referred to as a shift compensation parameter or an offset factor, which may be represented by f0. In the embodiment of the present application, the value of the shift offset parameter may be set to a fixed constant, such as 32, 46, 56, 66, etc.; the value of the shift offset parameter may also be set to be related to a shift offset parameter table, and the value of the shift offset parameter may be determined by looking up the table, and no limitation is made here.

[0528] It should also be noted that the shift quantity parameter may also be referred to as a shift factor, a number of shift bits, or a weight shift value, etc., and may be represented by sW, shift, or weight shift. In the embodiment of the present application, the shift quantity parameter is generally represented by sW. In the embodiment of the present application, the value of the shift quantity parameter may be set to a fixed constant, such as 5, or 6, or 7, etc. The value of the shift quantity parameter may also be set to be related to a shift quantity parameter table, and the value of the shift quantity parameter may be determined by looking up the table, and no limitation is made here.

[0529] In addition, the first offset can be represented by oW, and the first offset is related to both the shift quantity parameter and the shift offset parameter.

[0530] It should also be understood that the prediction parameters may include prediction mode parameters and size parameters of the current block. When the prediction mode parameters indicate that the chrominance component of the current block uses the MIP mode to determine the intra-frame prediction value, a weight matrix table may be pre-established at this time, and the weight matrix table may be stored in a memory or a storage unit, where the memory or storage unit may be integrated in the decoder, or may be set separately. In this way, according to the block size index value (mipSizeId) and the MIP mode index value (modeId) of the current block, the MIP weight matrix (or MIP weight matrix, or simply MIP matrix) required for the current block may be determined by table lookup, represented by mWeight[x][y].

[0531] It should also be noted that in the decoder, a shift quantity parameter table may be pre-established, and the shift quantity parameter table is also stored in a memory or a storage unit, where the memory or storage unit may be integrated in the decoder, or may be separately set. In the embodiment of the present application, the determination of the shift quantity parameter (sW) may include the following methods:

[0532] In one possible implementation, for different block sizes and different MIP mode index values, the shift quantity parameter may be different. In some embodiments, the method may further include:

[0533] When the chrominance component of the current block uses the MIP mode to determine the intra-frame prediction value, parsing the bitstream to obtain the MIP mode index value of the current block;

[0534] According to the MIP mode index value, querying a value corresponding to the MIP mode index value from a first preset lookup table, wherein the first preset lookup table is used to record a correspondence between the MIP mode index value and the value of the shift quantity parameter;

[0535] The queried value is determined as the value of the shift quantity parameter.

[0536] It should be noted that the value of the shift quantity parameter can be queried according to the block size index value (represented by mipSizeId) and the MIP mode index value (modeId) of the current block. As shown in the first preset lookup table in Table 4 above, for different mipSizeId and modeId, the shift quantity parameter required for matrix multiplication can be determined by a lookup table.

[0537] However, on the decoder side, Table 4 needs to be stored in a memory or storage unit in the form of a lookup table; however, storage requires a cost, and the lookup process also requires a cost; since the shift factor in Table 4 is related to the block size and MIP mode index value of the current block, the memory usage is increased, and the computational complexity is also increased.

[0538] In order to reduce memory usage and computational complexity, the embodiment of the present application simplifies the method for determining the shift factor.

[0539] In another possible implementation, the value of the shift quantity parameter can be set to a fixed constant, and is a fixed constant that is independent of the block size index value and the MIP mode index value. For example, for different block size index values ​​and different MIP mode index values, the value of the shift quantity parameter can be set to 5; or, for different block size index values ​​and different MIP mode index values, the value of the shift quantity parameter can be set to 6; or, for different block size index values ​​and different MIP mode index values, the value of the shift quantity parameter can be set to 7. In the embodiment of the present application, preferably, the value of the shift quantity parameter is equal to 6, but there is no limitation on this.

[0540] In another possible implementation, for the value of the shift quantity parameter, the method may further include:

[0541] Determining a block size index value of the current block according to a size parameter of the current block;

[0542] The value of the shift quantity parameter is determined according to the block size index value of the current block.

[0543] It should be noted that, according to the size parameter of the current block, the block size index value of the current block may be determined. In a possible implementation manner, according to the size parameter of the current block, determining the block size index value of the current block may include:

[0544] When the width and height of the current block are both equal to 4, setting the block size index value of the current block to 0;

[0545] When the width and height of the current block are both equal to 8, or one of the width and height of the current block is equal to 4, setting the block size index value of the current block to 1;

[0546] When the width and height of the current block do not meet the aforementioned conditions, the block size index value of the current block is set to 2.

[0547] In this way, after the block size index value of the current block is determined, the value of the shift quantity parameter can be further determined according to the block size index value of the current block.

[0548] Optionally, in some embodiments, determining the value of the shift quantity parameter according to the block size index value of the current block may include:

[0549] When the block size index values ​​are equal to 0, 1, and 2 respectively, it is determined that the values ​​of the shift quantity parameter corresponding to the block size index value of the current block are equal to 5, 6, and 5 respectively.

[0550] Optionally, in some embodiments, determining the value of the shift quantity parameter according to the block size index value of the current block may include:

[0551] The value of the shift amount parameter is set to be equal to the ratio between the width or height of the current block and a first preset value of the block size index value corresponding to the current block.

[0552] Here, the first preset value represents the number of MIP input sample values ​​obtained from the boundary of the current block. In this case, the method may also include:

[0553] When the block size index values ​​of the current block are equal to 0, 1, and 2 respectively, it is determined that the first preset values ​​corresponding to the block size index values ​​of the current block are equal to 2, 4, and 4 respectively.

[0554] That is to say, when the first preset value represents the number of MIP input sampling values ​​obtained from the boundary of the current block, if the block size index value of the current block is equal to 0, then the corresponding first preset value is equal to 2; if the block size index value of the current block is equal to 1, then the corresponding first preset value is equal to 4; if the block size index value of the current block is equal to 2, then the corresponding first preset value is equal to 4, so that the value of the shift quantity parameter can be determined according to the ratio between the width or height of the current block and the corresponding first preset value.

[0555] Optionally, in some embodiments, determining the value of the shift quantity parameter according to the block size index value of the current block may include:

[0556] The value of the shift amount parameter is set to be equal to the ratio between the width or height of the current block and a second preset value of the block size index value corresponding to the current block.

[0557] Here, the second preset value represents the size of the MIP prediction block of the current block obtained by directly using the MIP weighting matrix. In this case, the method may further include:

[0558] When the block size index values ​​of the current block are equal to 0, 1, and 2 respectively, it is determined that the second preset values ​​corresponding to the block size index values ​​of the current block are equal to 4, 4, and 8 respectively.

[0559] That is to say, when the second preset value represents the size of the MIP prediction block of the current block obtained by directly using the MIP weighting matrix to calculate, at this time, if the block size index value of the current block is equal to 0, then the corresponding second preset value is equal to 4; if the block size index value of the current block is equal to 1, then the corresponding second preset value is equal to 4; if the block size index value of the current block is equal to 2, then the corresponding second preset value is equal to 8, so that the value of the shift quantity parameter can be determined according to the ratio between the width or height of the current block and the corresponding second preset value.

[0560] In another possible implementation, the shift quantity parameter table may be minimized, and the shift quantity parameter value may still be determined by using a lookup table. Optionally, in some embodiments, determining the shift quantity parameter value according to the block size index value of the current block may include:

[0561] According to the block size index value, querying a value corresponding to the block size index value from a second preset lookup table, wherein the second preset lookup table is used to record a correspondence between the block size index value and the value of the shift quantity parameter;

[0562] The queried value is determined as the value of the shift quantity parameter.

[0563] It should be noted that the value of the shift quantity parameter can be queried only according to the block size index value (represented by mipSizeId) of the current block. As shown in the second preset lookup table in Table 5 above, for each block size index value, a fixed value may correspond, that is, the size of each block or the size set of each block may have a fixed shift quantity parameter value as shown in Table 5.

[0564] According to the above Table 5, when the block size index values ​​of the current block are respectively equal to 0, 1 and 2, it can be determined that the values ​​of the shift quantity parameter corresponding to the block size index values ​​are respectively equal to 5, 6 and 5.

[0565] In this way, by simplifying the method of determining the shift quantity parameter, especially minimizing the shift quantity parameter table or fixing the value of the shift quantity parameter, the storage of the lookup table can be minimized, and the memory occupied by the shift factor table storage in the MIP mode can be reduced without increasing the calculation complexity.

[0566] Furthermore, in the decoder, a shift offset parameter table may be pre-established, and the shift offset parameter table is also stored in a memory or a storage unit, where the memory or storage unit may be integrated in the decoder, or may be separately set. In the embodiment of the present application, the determination of the shift offset parameter (fO) may include the following methods:

[0567] In a possible implementation, for different block sizes and different MIP mode index values, the shift offset parameter may also be different. In some embodiments, the method may further include:

[0568] When the chrominance component of the current block uses the MIP mode to determine the intra-frame prediction value, parsing the bitstream to determine the MIP mode index value of the current block;

[0569] According to the MIP mode index value, querying a value corresponding to the MIP mode index value from a third preset lookup table, wherein the third preset lookup table is used to record the correspondence between the MIP mode index value and the value of the shift offset parameter;

[0570] The queried value is determined as the value of the shift offset parameter.

[0571] It should be noted that the value of the shift offset parameter can be queried according to the block size index value (represented by mipSizeId) and the MIP mode index value (modeId) of the current block. As shown in the third preset lookup table in Table 6 above, for different mipSizeId and modeId, the shift offset parameter required for matrix multiplication can be determined by a lookup table.

[0572] However, on the decoder side, Table 6 also needs to be stored in a memory or storage unit in the form of a lookup table; however, storage requires a cost, and the lookup process also requires a cost; since the value of the shift offset parameter in Table 6 is related to the block size and MIP mode index value of the current block, the memory usage is increased, and the computational complexity is also increased.

[0573] In order to reduce memory usage and computational complexity, the method for determining the shift offset parameter is also simplified in the embodiment of the present application.

[0574] In another possible implementation, the value of the shift offset parameter can be set to a fixed constant, and it is a fixed constant that is independent of the block size index value and the MIP mode index value. Typically, the fixed constant has a value range of 0 to 100. For example, for different block size index values ​​and different MIP mode index values, the value of the shift offset parameter can be set to 32; or, for different block size index values ​​and different MIP mode index values, the value of the shift offset parameter can be set to 46; or, for different block size index values ​​and different MIP mode index values, the value of the shift offset parameter can be set to 56; or, for different block size index values ​​and different MIP mode index values, the value of the shift offset parameter can be set to 66. In the embodiment of the present application, preferably, the value of the shift offset parameter is equal to 32, but there is no limitation on this.

[0575] In another possible implementation, for the value of the shift offset parameter, the method may further include:

[0576] Determining a block size index value of the current block according to a size parameter of the current block;

[0577] The value of the shift offset parameter is determined according to the block size index value of the current block.

[0578] It should be noted that, according to the size parameter of the current block, the block size index value of the current block can be determined; and then according to the block size index value of the current block, the value of the shift offset parameter can be further determined.

[0579] Optionally, in some embodiments, determining the value of the shift offset parameter according to the block size index value of the current block may include:

[0580] When the block size index values ​​are equal to 0, 1, and 2 respectively, it is determined that the values ​​of the shift offset parameter corresponding to the block size index value of the current block are equal to 34, 23, and 46 respectively.

[0581] Optionally, in some embodiments, the shift offset parameter table may be minimized, and the offset factor may still be determined by using a lookup table. Optionally, in some embodiments, determining the value of the shift offset parameter according to the block size index value of the current block may include:

[0582] According to the block size index value, querying a value corresponding to the block size index value from a fourth preset lookup table, wherein the fourth preset lookup table is used to record a correspondence between the block size index value and the value of the shift offset parameter;

[0583] The queried value is determined as the value of the shift offset parameter.

[0584] It should be noted that the value of the shift offset parameter can be queried only according to the block size index value (represented by mipSizeId) of the current block. As shown in the fourth preset lookup table in Table 7 above, for each block size index value, a fixed value may correspond, that is, the size of each block or the size set of each block may have a fixed shift offset parameter value as shown in Table 7.

[0585] According to the above Table 7, when the block size index values ​​of the current block are respectively equal to 0, 1 and 2, it can be determined that the values ​​of the shift offset parameters corresponding to the block size index values ​​are respectively equal to 34, 23 and 46.

[0586] In this way, by simplifying the method of determining the shift offset parameter, especially minimizing the shift offset parameter table or fixing the value of the shift offset parameter, the storage of the lookup table can be minimized, thereby reducing the memory occupied by the shift offset parameter table storage in the MIP mode without increasing the computational complexity.

[0587] Thus, after determining the shift offset parameter (fO) and the shift quantity parameter (sW), for the first constant value, optionally, in some embodiments, determining the first constant value according to the value of the shift quantity parameter may include:

[0588] The first constant value is set to an integer exponent power of 2, wherein the exponent of the power is equal to the value of the shift quantity parameter minus 1.

[0589] Optionally, in some embodiments, determining the first constant value according to the value of the shift quantity parameter may include:

[0590] Perform a binary bit left shift on "1" to obtain the first constant value, wherein the number of bits of the bit left shift is equal to the value of the shift quantity parameter minus 1.

[0591] That is, after obtaining the shift quantity parameter (sW), the first constant value can be expressed as 1<<(sW-1) or 2^(sW-1). At this time, when the shift quantity parameter is set to 6, the first constant value can be obtained to be equal to 32.

[0592] Here, if the first offset is represented by oW, the product of the sum of the MIP input sampling values ​​and the shift offset parameter (fO) can be calculated according to the shift offset parameter, which is Then the value of oW can be set to

[0593] In this way, in the MIP mode, the MIP weighting matrix, the MIP input sampling value, the shift quantity parameter and the first offset can be obtained, so as to subsequently determine the MIP prediction value of the current block. Specifically, in some embodiments, determining the MIP prediction value of the current block according to the MIP weighting matrix, the MIP input sampling value, the shift quantity parameter and the first offset may include:

[0594] Calculating a first weighted sum of the MIP weighting matrix and the MIP input sample value;

[0595] Calculating a first sum value of the first weighted sum and the first offset;

[0596] Performing a binary bit right shift on the first sum value to obtain a first right-shifted value, wherein the number of bits of the right shift is equal to the value of the shift quantity parameter;

[0597] The MIP prediction value of the current block is set to be equal to the sum of the first right shift value and the value corresponding to the index number 0 in the first temporary reference value; wherein the first temporary reference value is obtained based on downsampling filtering of adjacent sample values ​​of the current block.

[0598] That is, in MIP mode, the MIP weight matrix (represented by mWeight), the shift quantity parameter (represented by sW) and the shift offset parameter (represented by fO) can be determined according to the block size index value of the current block (represented by mipSizeId) and the MIP mode index value (represented by modeId); then the MIP input sample value (represented by p[x]), mWeight, sW and fO are input into the matrix multiplication process to obtain the MIP prediction value output by the matrix multiplication (represented by predMip[x][y]), and the sample points in predMip[x][y] are arranged into a matrix / array form according to predSize×predSize. The calculation formula is shown in the above formula (5) or formula (6).

[0599] S904: Filter the MIP prediction value to determine an intra-frame prediction value of the chrominance component of the current block.

[0600] It should be noted that the MIP prediction block is composed of MIP prediction values. After obtaining the MIP prediction block, it is possible to determine whether the size parameter of the MIP prediction block is the same as the size parameter of the current block in order to further determine the intra-frame prediction value of the current block. Specifically, according to the judgment result, when the size parameter of the MIP prediction block is the same as the size parameter of the current block, the intra-frame prediction block of the current block is set to be equal to the MIP prediction block; at this time, the MIP prediction block contains the prediction values ​​of all pixel positions in the current block; when the size parameter of the MIP prediction block is different from the size parameter of the current block, the MIP prediction block is filtered to obtain a filtered prediction block, and the filtered prediction block is set as the intra-frame prediction block of the current block. Here, the filtering process may include upsampling filtering or low-pass filtering.

[0601] S905: When the LFNST parameter indicates to perform LFNST on the current block, determine a reconstructed transform coefficient block of the current block, and perform LFNST on at least part of the reconstructed transform coefficients in the reconstructed transform coefficient block to obtain a second transform coefficient block.

[0602] It should be noted that not all current blocks can perform LFNST, which means that the LFNST parameters may not exist in the bitstream; at this time, some conditions need to be checked (such as the minimum value of the size parameter of the current block, the block size index value of the current block, etc.) to determine whether the LFNST parameters exist in the bitstream. Therefore, in some embodiments, the parsing of the bitstream to obtain the LFNST parameters may include:

[0603] Determining whether there is a LFNST parameter in the bitstream;

[0604] When it is determined that the code stream contains the LFNST parameter, the code stream is parsed to obtain the LFNST parameter.

[0605] That is, it is first necessary to determine whether the LFNST parameter exists in the bitstream. Only when the determination result is yes, that is, the LFNST parameter exists in the bitstream, the step of parsing the bitstream and obtaining the LFNST parameter can be performed.

[0606] Optionally, in some embodiments, the determining whether there is a LFNST parameter in the bitstream may include:

[0607] When the chrominance component of the current block uses the MIP mode to determine the intra-frame prediction value, determine the minimum value of the size parameters of the current block;

[0608] According to the minimum value, it is determined whether a LFNST parameter exists in the code stream.

[0609] Further, determining whether a LFNST parameter exists in the bitstream according to the minimum value may include:

[0610] If the minimum value is greater than or equal to the first preset threshold, it is determined that the LFNST parameter exists in the bitstream.

[0611] Here, the first preset threshold value may be used to represent a preset threshold value for measuring whether to perform LFNST. In the embodiment of the present application, the first preset threshold value may be set to 8, but is not specifically limited thereto.

[0612] That is to say, when the MIP mode is used to perform intra-frame prediction on the chrominance component of the current block, the minimum value of the size parameters of the current block can be determined at this time; for example, when the minimum value of the width and height is greater than or equal to 8, it is determined that the LFNST parameters exist in the bitstream, and the LFNST parameters, such as the value of lfnst_index, can be obtained by parsing the bitstream.

[0613] Optionally, in some embodiments, the determining whether there is a LFNST parameter in the bitstream may include:

[0614] When the chrominance component of the current block uses the MIP mode to determine the intra-frame prediction value, determining the block size index value of the current block according to the size parameter of the current block;

[0615] It is determined whether a LFNST parameter exists in the bitstream according to the block size index value.

[0616] Further, the determining whether there is a LFNST parameter in the bitstream according to the block size index value may include:

[0617] If the block size index value is equal to the second preset threshold, it is determined that the LFNST parameter exists in the bitstream.

[0618] Here, the second preset threshold value may also be used to represent a preset threshold value for measuring whether to perform LFNST. In the embodiment of the present application, the second preset threshold value may be set to a preset value (eg, 2), or one of multiple preset values ​​(eg, 1, 2), but is not specifically limited.

[0619] That is to say, when the MIP mode is used to perform intra-frame prediction on the chrominance component of the current block, the block size index value (mipSizeId) of the current block can be determined according to the size parameter of the current block; for example, when the value of mipSizeId is equal to 2, it is determined that the LFNST parameter exists in the bitstream, and the LFNST parameter, such as the value of lfnst_index, can be obtained by parsing the bitstream.

[0620] In some embodiments, parsing the code stream and obtaining the LFNST parameter may include:

[0621] Parse the code stream to obtain the value of the LFNST index number.

[0622] It should be noted that the LFNST index number can be represented by lfnst_index, and the value of the LFNST index number is used to determine whether to perform LFNST on the current block. Specifically, in some embodiments, the method may further include:

[0623] If the value of the LFNST index number is greater than zero, determining to perform LFNST on the current block;

[0624] If the value of the LFNST index sequence number is equal to zero, it is determined not to perform LFNST on the current block.

[0625] That is, after obtaining the value of lfnst_index, if the value of lfnst_index is greater than zero, it is determined that LFNST is performed on the current block; if the value of lfnst_index is equal to zero, it is determined that LFNST is not performed on the current block.

[0626] It should also be noted that after obtaining the LFNST parameter, such as the value of lfnst_index, if lfnst_index is greater than zero, it indicates that the LFNST parameter indicates that LFNST is performed on the current block. At this time, in some embodiments, when the LFNST parameter indicates that LFNST is performed on the current block, determining the reconstructed transform coefficient block of the current block may include:

[0627] Parsing the bitstream to obtain the quantization coefficient of the current block;

[0628] The quantized coefficients are inversely quantized to obtain a reconstructed transform coefficient block of the current block.

[0629] Specifically, the inverse quantization process is a scaling process. In a specific example, the inverse quantization of the quantized coefficients to obtain the reconstructed transform coefficient block of the current block may include: performing a scaling operation on the quantized coefficients to obtain the reconstructed transform coefficient block of the current block.

[0630] In this way, when the LFNST parameter indicates that LFNST is to be performed on the current block, LFNST may be performed on all or part of the reconstructed transform coefficients in the reconstructed transform coefficient block to obtain a second transform coefficient block.

[0631] It should also be noted that when it is determined that the current block can perform LFNST, it is also necessary to determine the LFNST transform kernel (represented by kernel) used by the current block. In some embodiments, performing LFNST on at least part of the reconstructed transform coefficients in the reconstructed transform coefficient block to obtain a second transform coefficient block may include: performing LFNST on at least part of the reconstructed transform coefficients in the reconstructed transform coefficient block using the LFNST transform kernel to obtain the second transform coefficient block.

[0632] In the embodiment of the present application, there are 4 transform core candidate sets in LFNST, and these four transform core candidate sets may include set0, set1, set2 and set3. Here, the embodiment of the present application may use MIP parameters to determine the LFNST transform core candidate set, and then select the LFNST transform core used by the current block from the LFNST transform core candidate set. Therefore, in some embodiments, the method may also include:

[0633] When the chrominance component of the current block uses the MIP mode to determine the intra-frame prediction value, determine the MIP parameter from the prediction mode parameter of the current block;

[0634] Determine a LFNST transform kernel candidate set according to the MIP parameters;

[0635] In the LFNST transform core candidate set, a transform core indicated by the value of the LFNST index number is determined as the LFNST transform core used by the current block; wherein the LFNST transform core candidate set includes two or more preset LFNST transform cores.

[0636] It should be noted that the MIP parameter may include at least one of the following: a MIP mode index value (expressed by modeId) and a MIP transposition indication parameter (expressed by isTransposed). Among them, for the LFNST transform kernel used by the current block, when the value of the MIP transposition indication parameter indicates that the sampling point input vector used by the MIP mode is transposed, it is also necessary to perform matrix transposition processing on the selected transform kernel to obtain the LFNST transform kernel used by the current block.

[0637] Further, in some embodiments, determining the LFNST transformation kernel candidate set according to the MIP parameters may include:

[0638] Determine the value of the LFNST intra prediction mode index number according to the MIP mode index value;

[0639] According to the value of the LFNST intra prediction mode index sequence number, determining the index value of the LFNST transform kernel candidate set through a fifth preset lookup table;

[0640] The LFNST transform core candidate set is selected from a plurality of candidate LFNST transform core candidate sets according to the index value of the LFNST transform core candidate set.

[0641] Here, the LFNST intra prediction mode index number may be represented by predModeIntra, and the index value of the LFNST transform kernel candidate set may be represented by lfnstTrSetIdx.

[0642] It should be noted that the fifth preset lookup table is shown in Table 10 above. According to the MIP mode index value (ie, modeId), the value of predModeIntra can be determined; then, according to the value of predModeIntra, the value of lfnstTrSetIdx can be directly determined in combination with Table 10, that is, the LFNST transform core candidate set selected by the current block is determined. Here, the value of lfnstTrSetIdx indicates the transform core candidate set used during LFNST; since the value of modeId can include 0, 1, 2, 3, 4, 5, the value of predModeIntra is also 0, 1, 2, 3, 4, 5; its corresponding relationship with lfnstTrSetIdx is specifically shown in Table 10.

[0643] In a specific embodiment, determining the value of the LFNST intra prediction mode index number according to the MIP mode index value may include: setting the value of the LFNST intra prediction mode index number to be equal to the MIP mode index value.

[0644] That is to say, the embodiment of the present application can set the value of predModeIntra to be equal to the value of modeId, and then directly determine the value of lfnstTrSetIdx based on the value of predModeIntra in combination with Table 10, that is, determine the LFNST transform core candidate set selected for the current block. It should also be noted that in another specific embodiment, the embodiment of the present application can also directly map the MIP mode index value to the value of the PLANAR mode, and then use the value of predModeIntra corresponding to the PLANAR mode to determine the value of lfnstTrSetIdx, that is, determine the LFNST transform core candidate set selected for the current block.

[0645] Furthermore, during the execution of LFNST, the scanning order may include a horizontal scanning order and a vertical scanning order. Here, the determination of the scanning order is related to the value of the MIP transposition indication parameter (expressed by isTransposed).

[0646] Optionally, in some embodiments, performing LFNST on at least part of the reconstructed transform coefficients in the reconstructed transform coefficient block to obtain a second transform coefficient block may include:

[0647] When the value of the MIP transposition indication parameter indicates that a transposition process is performed on a sampling point input vector used in the MIP mode, performing LFNST on at least part of the reconstructed transform coefficients in the reconstructed transform coefficient block according to a vertical scanning order to obtain the second transform coefficient block;

[0648] When the value of the MIP transposition indication parameter indicates that the sampling point input vector used by the MIP mode is not to be transposed, LFNST is performed on at least part of the reconstructed transform coefficients in the reconstructed transform coefficient block according to a horizontal scanning order to obtain the second transform coefficient block.

[0649] Specifically, when the value of isTransposed is equal to 0, the sampling point input vector used by the MIP mode is not transposed. Then LFNST can be performed on at least part of the transform coefficients in the first transform coefficient block according to the horizontal scanning order, as shown in the above formula (10).

[0650] When the value of isTransposed is equal to 1, the sampling point input vector used in the MIP mode is transposed. Then, LFNST can be performed on at least part of the transform coefficients in the first transform coefficient block according to the vertical scanning order, as shown in the above formula (11).

[0651] Among them, the determination of log2LfnstSize is shown in the above formula (9).

[0652] Optionally, in some embodiments, performing LFNST on at least part of the reconstructed transform coefficients in the reconstructed transform coefficient block to obtain a second transform coefficient block may include:

[0653] When the value of the MIP transposition indication parameter indicates that a transposition process is performed on a sampling point input vector used in the MIP mode, performing LFNST on at least part of the reconstructed transform coefficients in the reconstructed transform coefficient block according to a horizontal scanning order to obtain the second transform coefficient block;

[0654] When the value of the MIP transposition indication parameter indicates that the sampling point input vector used by the MIP mode is not to be transposed, LFNST is performed on at least part of the reconstructed transform coefficients in the reconstructed transform coefficient block according to a vertical scanning order to obtain the second transform coefficient block.

[0655] Specifically, when the value of isTransposed is equal to 1, the sampling point input vector used in the MIP mode is transposed, and LFNST can be performed on at least part of the transform coefficients in the first transform coefficient block according to the horizontal scanning order, as shown in the above formula (10).

[0656] When the value of isTransposed is equal to 0, the sampling point input vector used by the MIP mode is not transposed. Then LFNST can be performed on at least part of the transform coefficients in the first transform coefficient block according to the vertical scanning order, as shown in the above formula (11).

[0657] Among them, the determination of log2LfnstSize is shown in the above formula (9).

[0658] S906: Perform a first transform on the second transform coefficient block to obtain a reconstructed residual block of the chrominance component of the current block.

[0659] S907: Determine a reconstructed block of the chrominance component of the current block according to the intra-frame prediction value of the chrominance component of the current block and the reconstructed residual block.

[0660] It should be noted that in the decoder, by parsing the bitstream, the quantized value of the LFNST coefficient can be obtained, and the quantized value is dequantized by the dequantization unit (which can be called "Scaling") to obtain a reconstructed transform coefficient block, and at least part of the reconstructed transform coefficients in the reconstructed transform coefficient block are subjected to an inverse LFNST transform to obtain a second transform coefficient block; and then through the inverse first transform, a reconstructed residual block that restores the chrominance component of the current block can be obtained. Here, the inverse first transform is the inverse transform corresponding to the "first transform (Core Transform)" in the encoder. It should be noted that the standard only defines the "inverse transform" operation in the decoder, so the "inverse LFNST transform" in the standard can also be called the "LFNST transform".

[0661] It should also be noted that after restoring the reconstructed residual block of the chrominance component of the current block, the intra-frame prediction value of the chrominance component of the current block and the reconstructed residual block may be added to determine the reconstructed block of the chrominance component of the current block.

[0662] The following will be combined Figure 2 The decoder 200 shown specifically illustrates the decoding method of the embodiment of the present application.

[0663] The input code stream of the decoder 200 may be a code stream generated by the encoder 100. The parsing unit 201 parses the input code stream and obtains the value of the syntax element from the input code stream. The parsing unit 201 converts the binary representation of the syntax element into a digital value and sends the digital value to the unit in the decoder 200 to obtain one or more decoded pictures. The parsing unit 201 can also parse one or more syntax elements from the input code stream to render the decoded picture.

[0664] See also Fig.10 , which shows a schematic diagram of a process for parsing LFNST parameters provided by an embodiment of the present application. Fig.10 As shown, the process may include:

[0665] S1001: The parsing unit 201 checks to determine whether there is a LFNST parameter in the input code stream.

[0666] It should be noted that the LFNST parameter may refer to the value of lfnst_index. Here, the example of the syntax structure for representing the LFNST parameter may be the same as the example of the syntax structure shown in Table 8 above, or may be the same as the example of the syntax structure shown in Table 9A and Table 9B above.

[0667] Specifically, when the intra prediction mode of the current block is the MIP mode, the parsing unit 201 obtains the coded bits of lfnst_index from the input bitstream in the following manner. Optionally, using the syntax structure in Table 9A, when the minimum value of the width and height of the transform block in the current block is greater than or equal to 8 (that is, the width and height are both greater than or equal to 8), the parsing unit 201 uses the MIP mode to read the lfnst_index bit. Another optional condition can be set to: the width and height are greater than or equal to a first value (e.g., 8), and the sum of the width and height is greater than or equal to a second value (e.g., 16).

[0668] Optionally, using the syntax structure in Table 9B, when the mipSizeId of the current block is equal to 2, the parsing unit 201 uses the MIP mode to read the lfnst_index bit. Another optional condition may be set to: the mipSizeId of the current block is equal to one of multiple values ​​(eg, 1 and 2).

[0669] S1002: The parsing unit 201 performs adaptive binary arithmetic decoding on the LFNST parameters to obtain the LFNST parameters.

[0670] Here, since the entropy coding unit 115 on the encoder 100 side uses CABAC with a descriptor of "ae(v)" as shown in Tables 9A and 9B to encode the LFNST parameters (for example, the value of lfnst_index), then on the decoder 200 side, adaptive binary arithmetic decoding is performed by the parsing unit 201, and the value of lfnst_index can be obtained from the lfnst_index bits.

[0671] The parsing unit 201 sends the value of the syntax element and one or more variables set or determined according to the value of the syntax element for obtaining one or more decoded pictures to the units in the decoder 200. The prediction unit 202 determines the intra-frame prediction block of the current block (e.g., CU). When the inter-frame decoding mode is indicated for decoding the current decoding block, the prediction unit 202 transmits the relative parameters from the parsing unit 201 to the MC unit 203 to obtain the inter-frame prediction block. When the intra-frame prediction mode (including the MIP mode indicated based on the MIP mode index value) is indicated for decoding the current block, the prediction unit 202 transmits the relative parameters from the parsing unit 201 to the intra-frame prediction unit 204 to obtain the intra-frame prediction block.

[0672] It should be understood that the current block (or referred to as a "decoded block") in the embodiment of the present application may be a CU or a partition of a CU (e.g., a transform block). Specifically, when the MIP mode is indicated for decoding the current block, the intra prediction unit 204 obtains an intra prediction block of the current block, wherein the intra prediction block of the current block includes an intra prediction value of at least one pixel (samples) in the current block.

[0673] Specifically, for the intra prediction unit 204, it uses the MIP mode to obtain the intra prediction block, the steps are as follows:

[0674] First, the intra-frame prediction unit 204 obtains one or more reference pixels from the adjacent pixels of the current block, for example, by downsampling the adjacent pixels, or directly extracting from the adjacent pixels.

[0675] Then, the intra prediction unit 204 uses the obtained reference pixel points, the MIP matrix and the shift parameter to determine one or more partial prediction pixel points corresponding to the pixel point position in the current block. Here, the pixel point position can be a preset pixel point position in the current block. For example, the pixel point position has uniform horizontal and vertical coordinate values. The shift parameter includes a shift amount parameter and a shift offset parameter, and the shift parameter can be used for an offset operation in the process of obtaining the intra prediction value of the partial pixel point.

[0676] Finally, if the predicted pixel points corresponding to some of the pixels of the current block are obtained, the intra-frame prediction unit 204 also needs to obtain the predicted pixel points corresponding to the remaining pixels in the current block except for the partial pixels. For example, the intra-frame prediction unit 204 can use an interpolation filter to obtain the predicted pixel points corresponding to the remaining pixels, where the input of the interpolation filter can be the partial pixels and the adjacent pixels.

[0677] like Figure 6 As shown, the flowchart example of using the MIP mode to obtain the intra-frame prediction block can also be implemented on the decoder 200, wherein the "current block" here refers to the "decoded block".

[0678] Step 601, the intra-frame prediction unit 204 obtains adjacent pixels of the current block, for example, adjacent pixels are marked as Figure 6 The gray filled squares adjacent to the current block in step 601 are shown. The intra-frame prediction unit 204 obtains one or more reference pixels from the adjacent pixels. Figure 6 In the example of step 601, the intra-frame prediction unit 204 may optionally calculate an average value of two adjacent pixels and use the average value as a reference pixel. Alternatively, the intra-frame prediction unit 204 selects an adjacent pixel as a reference pixel for every other adjacent pixel. Figure 6 In the example shown in step 601, the intra prediction unit 204 selects 4 reference pixels from the 8 upper adjacent pixels of the current block, and selects another 4 reference pixels from the 8 left adjacent pixels of the current block.

[0679] The specific process of step 601 is as follows:

[0680] The intra prediction unit 204 obtains the width and height of the current block from the parsing unit 201, the width and height are represented by variables cbWidth and cbHeight, respectively. According to the parameters from the parsing unit 201, the current block is divided into one or more transform blocks. Let the variables nTbW and nTbH be the width and height of the transform block, respectively. When the MIP mode is used as the intra prediction mode to obtain the prediction of the current block, the intra prediction unit 204 determines the block size index value of the MIP, that is, the variable represented as mipSizeId.

[0681] Optionally, the intra prediction unit 204 determines the value of mipSizeId as follows:

[0682] - If nTbW and nTbH are both equal to 4, mipSizeId is set equal to 0;

[0683] - Otherwise, if cbWidth or cbHeight is equal to 4, mipSizeId is set equal to 1;

[0684] Otherwise, mipSizeId is set equal to 2.

[0685] Specifically, when the size parameter of the current block is 8×8 (ie, both cbWidth and cbHeight are equal to 8), mipSizeId is set to be equal to 2.

[0686] Optionally, the intra prediction unit 204 determines the value of mipSizeId as follows:

[0687] - If nTbW and nTbH are both equal to 4, mipSizeId is set equal to 0;

[0688] - Otherwise, if cbWidth or cbHeight is equal to 4, or if both cbWidth and cbHeight are equal to 8, then mipSizeId is set equal to 1;

[0689] Otherwise, mipSizeId is set equal to 2.

[0690] Specifically, when the size parameter of the current block is 8×8, mipSizeId is set to be equal to 1.

[0691] Further, the intra prediction unit 204 obtains the values ​​of boundarySize and predSize according to mipSizeId, as follows:

[0692] - If mipSizeId is equal to 0, boundarySize is set equal to 2 and predSize is set equal to 4;

[0693] - Otherwise, if mipSizeId is equal to 1, boundarySize is set equal to 4 and predSize is set equal to 4;

[0694] - Otherwise (if mipSizeId is equal to 2), boundarySize is set equal to 4 and predSize is set equal to 8;

[0695] Wherein, boundarySize represents the number of reference pixels obtained from each side of the reference pixels adjacent to the upper side and the reference pixels adjacent to the left side of the current block.

[0696] The intra prediction unit 204 may also obtain a parameter from the parsing unit 701 indicating the order of reference pixels stored in the buffer pTemp to obtain reference pixels for calculating the MIP prediction value. The intra prediction unit 204 sets the variable isTransposed to be equal to the parameter from the parsing unit 701 indicating the order of reference pixels stored in the buffer pTemp. For example, isTransposed equal to 0 (or FALSE) indicates that the intra prediction unit 204 first presents the reference pixels obtained from the reference pixels adjacent to the upper side of the current block, and then presents the reference pixels obtained from the reference pixels adjacent to the left side; otherwise, isTransposed equal to 1 (or TRUE) indicates that the intra prediction unit 204 first presents the reference pixels obtained from the reference pixels adjacent to the left side of the current block, and then presents the reference pixels obtained from the reference pixels adjacent to the upper side.

[0697] The intra prediction unit 204 obtains the value of the variable inSize to indicate the number of reference pixels used when using the MIP mode, as shown in the above equation (1).

[0698] The intra prediction unit 204 calls the following process to obtain a set of reference pixels (stored as an array p[x], where x is 0, . . . , inSize-1) using reference pixels adjacent to the current block.

[0699] The intra prediction unit 204 obtains nTbW reference pixels from the reference pixels adjacent to the upper side of the current block (e.g., stored in the array refT), and obtains nTbH reference pixels from the reference pixels adjacent to the left side of the current block (e.g., stored in the array refL). The intra prediction unit 204 calls a downsampling process on refT to obtain a boundarySize reference pixel, and stores the boundarySize reference pixel in refT. The intra prediction unit 204 calls a downsampling process on refL to obtain a boundarySize reference pixel, and stores the boundarySize reference pixel in refL.

[0700] The intra prediction unit 204 may also obtain the order indicated by isTransposed to place the elements in refT and refL into the buffer pTemp.

[0701] Optionally, the intra prediction unit 204 obtains p[x], where x=0, ..., inSize-1, as follows:

[0702] - If mipSizeId is equal to 2, then p[x] = pTemp[x+1] - pTemp[0];

[0703] - Otherwise (mipSizeId is less than 2), p[0] = pTemp[0] - (1 << (BitDepth - 1)), p[x] = pTemp[x] - pTemp[0];

[0704] Wherein, BitDepth is the bit depth of the color component of the pixel in the current block. Here, the color component can be one of the RGB components, or one of the YUV components, or one of the YCbCr components, for example, the Y component.

[0705] Optionally, the intra prediction unit 204 may obtain p[x], where x=0, ..., inSize-1, as follows:

[0706] - If mipSizeId is equal to 2, then p[x] = pTemp[x+1] - pTemp[0];

[0707] -Otherwise (mipSizeId is less than 2), p[0] = (1 << (BitDepth - 1)) - pTemp[0], p[x] = pTemp[x] - pTemp[0].

[0708] Alternatively, the intra-frame prediction unit 204 may use a unified calculation method to obtain the value of p[x] without determining the value of mipSizeId. For example, (1<<(BitDepth-1)) is added as an additional element to the buffer pTemp. In this case, the intra-frame prediction unit 204 calculates p[x] as pTemp[x+1]-pTemp[0].

[0709] Step 602, the intra prediction unit 204 uses a set of reference pixels and a MIP weighting matrix to obtain a MIP prediction value of the current block. The MIP weighting matrix is ​​selected from a set of predefined MIP weighting matrices according to a MIP mode index value (expressed as ModeId) corresponding to the MIP mode and a MIP block size index value (expressed as mipSizeId). The intra prediction unit 204 obtains the value of ModeId from the parsing unit 201.

[0710] The intra-frame prediction unit 204 obtains the MIP prediction values ​​(expressed as predMip[x][y]) of some predicted pixel points corresponding to one or more pixel point positions in the current block. Figure 6 In the example of step 602, the partially predicted pixels are the pixels marked as gray filled squares in the current block. The input of the prediction module 601 is the reference pixel p[x] obtained in step 601. The prediction module 601 calculates the partially predicted pixels using the MIP weighting matrix and the shift parameters; here, the shift parameters include a shift amount parameter and a shift offset parameter.

[0711] In a possible implementation, the prediction module 601 may represent a pixel point with coordinates (x, y), and the predicted pixel point is represented by predMip[x][y]. Then, the calculation formula of predMip[x][y] is as shown in Formula (5) or Formula (6).

[0712] In equation (5) or (6), mWeight[i][j] is a MIP weighting matrix; here, the matrix elements may be predetermined constant values; or, they may be adaptively updated using a training method, the input of which is one or more coded pictures or blocks, or pictures from other bitstreams provided to the decoder 200 by an external device, or pictures obtained by parsing a special data unit in an input bitstream containing a MIP weighting matrix from the parsing unit 201; mWeight[i][j] may be determined according to the MIP mode indicated by one or more corresponding parameters from the parsing unit 201; fO is a shift offset parameter for determining oW; sW is a shift quantity parameter; p[i] is calculated using a reference pixel, i.e., the MIP input sample value in the embodiment of the present application; pTemp[0] represents the first pixel in the reference pixel, i.e., the value corresponding to the index number 0 in the first temporary reference value in the embodiment of the present application; ">>" is a binary right shift operator as defined in VVC.

[0713] The prediction module 601 may determine the values ​​of sW and fO according to the size of the current block and the MIP mode used for the current block. In one example, the prediction module 601 obtains the values ​​of sW and fO using a lookup table.

[0714] Optionally, the prediction module 601 may use the above-mentioned Table 4 to determine sW according to the size parameters of the current block and the MIP mode.

[0715] Optionally, the prediction module 601 may also use the above Table 5 to determine sW according to the size parameters of the current block.

[0716] Optionally, the prediction module 601 may also directly set sW to a constant value. For example, for blocks of various size parameters and different MIP modes, the prediction module 601 may set sW to 5; or, for blocks of various size parameters and different MIP modes, the prediction module 601 may set sW to 6; or, for blocks of various size parameters and different MIP modes, the prediction module 601 may set sW to 7.

[0717] Optionally, the prediction module 601 may use the above-mentioned Table 6 to determine fO according to the size parameters of the current block and the MIP mode.

[0718] Optionally, the prediction module 601 may also use the above-mentioned Table 7 to determine f0 according to the size parameters of the current block.

[0719] Optionally, the prediction module 601 may also directly set fO to a constant value (such as 0 to 100). For example, for blocks of various size parameters and different MIP modes, the prediction module 601 sets fO to 32; or, the prediction module 601 sets fO to 46; or, the prediction module 601 sets fO to 56; or, the prediction module 601 sets fO to 66.

[0720] The intra prediction unit 204 may perform an embedding operation on the MIP prediction value in predMip. When isTransposed is equal to 1 (or TRUE), the predSize×predSize array predMip[x][y] (where x=0, ..., predSize-1, y=0, ..., predSize-1) is converted to predTemp[y][x]=predMip[x][y], and then predMip=predTemp.

[0721] Exemplarily, optionally, when the size parameter of the current block is 8×8 (ie, both cbWidth and cbHeight are equal to 8), the intra prediction unit 204 determines that mipSizeId is equal to 2 and obtains 8×8 predMip.

[0722] Optionally, when the size parameter of the current block is 8×8 (ie, both cbWidth and cbHeight are equal to 8), the intra prediction unit 204 determines that mipSizeId is equal to 1 and obtains 4×4 predMip.

[0723] Step 603, the intra prediction unit 204 obtains the intra prediction value of the current block (stored in an array as predSamples[x][y], where x=0, ..., nTbW-1, y=0, ..., nTbH-1), as follows:

[0724] - If the intra prediction unit 204 determines that nTbW is greater than predSize or nTbH is greater than predSize, the intra prediction unit 204 calls the upsampling process to obtain predSamples using predMip. The intra prediction unit 204 obtains the predicted pixel points corresponding to the remaining pixel points in the current block except for some pixel points. Figure 6In the step 602, the intra prediction unit 204 may use the filtering module 602 to obtain the predicted pixel points corresponding to the remaining pixel points except some pixel points in the current block. The input of the filtering module 602 is the pixel points marked as gray filled squares in step 602. The filtering module 602 may use one or more interpolation filters to obtain the predicted pixel points corresponding to the remaining pixel points except some pixel points in the current block using the input. For example, the input may include the reference pixel point and the partial predicted pixel points of one or more pixel point positions in the current block; or, the input may include the adjacent pixel point and the partial predicted pixel points of one or more pixel point positions in the current block; or, the input may include the reference pixel point, the adjacent pixel point and the partial predicted pixel points of one or more pixel point positions in the current block. For example, optionally, when the size of the current block is 8×8 (i.e., cbWidth and cbHeight are both equal to 8), the intra prediction unit 204 determines that mipSizeId is equal to 1, and the intra prediction unit 204 applies the upsampling process to 4×4predMip to obtain an 8×8 intra prediction block of the current block.

[0725] - Otherwise, the intra prediction unit 204 sets the intra prediction block of the current block to be equal to the MIP prediction block of the current block, that is, sets predSamples[x][y] (where x=0, ..., nTbW-1, y=0, ..., nTbH-1) to be equal to predMip[x][y]. For example, when the size parameter of the current block is 8×8 (that is, cbWidth and cbHeight are both equal to 8), the intra prediction unit 204 obtains predSamples of the current block, whose size is equal to 8×8 (that is, cbWidth and cbHeight are both equal to 8).

[0726] So, according to Figure 6 As shown in the flowchart, after step 603, the intra prediction unit 204 may obtain the intra prediction block of the current block (ie, CU), that is, determine the intra prediction value of at least one pixel in the current block.

[0727] The scaling unit 205 has the same function as that of the inverse quantization unit 110 in the encoder 100. The scaling unit 205 performs a scaling operation on the quantization coefficients (ie, levels) from the parsing unit 201 to obtain reconstructed coefficients.

[0728] The transform unit 206 has the same function as the inverse transform unit 111 in the encoder 100. The transform unit 206 performs one or more transform operations (ie, the inverse operation of the one or more transform operations performed by the inverse transform unit 111 in the encoder 100) to obtain a reconstructed residual.

[0729] When lfnst_index is not equal to 0, the transform unit 206 performs a secondary transform on the reconstructed coefficients to obtain a secondary transform coefficient block, and then performs a first transform on the secondary transform coefficient block, for example, an integer transform originally designed based on DCT, to obtain a reconstructed residual block. LFNST is an example of a secondary transform.

[0730] See also Fig.11 , which shows another schematic diagram of the process framework for executing LFNST provided in an embodiment of the present application. Fig.11 , block 1101 is a reconstructed transform coefficient block containing reconstructed transform coefficients.

[0731] In step 1101 , the transform unit 206 performs a secondary transform LFNST on all or part of the reconstructed transform coefficients in block 1101 .

[0732] When the intra prediction mode of the current block is the MIP mode, the transform unit 206 selects the LFNST transform kernel candidate set according to the MIP parameters. The MIP parameters may include at least one of the following: a MIP mode index value (ie, modeId) and a MIP transposed indication parameter (ie, isTransposed).

[0733] The transform unit 206 sets the value of the variable predModeIntra to be equal to the value of modeId. The transform unit 206 determines the transform kernel for performing LFNST on the coefficients on the sub-blocks 11001, 11002 and 11003. The size of the sub-block may be a preset value, such as 8×8. The transform unit 206 determines the index value (i.e., lfnstTrSetIdx) of the LFNST transform kernel candidate set according to the value of predModeIntra. For block decoding in MIP mode, the transform unit 206 uses a lookup table as shown in Table 10 to obtain the value of lfnstTrSetIdx. The transform unit 206 determines the transform kernel for performing LFNST as the transform kernel indicated by lfnst_index in the LFNST transform kernel candidate set indicated by lfnstTrSetIdx.

[0734] When lfnst_index is not equal to 0, the transform unit 206 performs LFNST on the transform block using coefficients from sub-blocks 11001, 11002, and 11003, obtains LFNST coefficients, and places the LFNST coefficients into sub-blocks 11011, 11012, and 11013 of block 1102. Assume that d[x][y] is a block of first transform coefficients, where x=0, ..., nTbW-1, y=0, ..., nTbH-1, where nTbW and nTbH are the width and height of block 1102, respectively. Assume that v[x] is a block of LFNST coefficients, where x=0, ..., nLfnstOutSize-1, and nLfnstOutSize=(nTbW>=8&&nTbH>=8)? 48:16.

[0735] Optionally, the transform unit 206 obtains the transform coefficient block 1102 using the following instruction, as specifically shown in the above equation (8). Wherein, log2LfnstSize is as shown in the above equation (9).

[0736] Optionally, when isTransposed is equal to 0, the transform unit 206 obtains the block 1102 of transform coefficients using the following instructions, as specifically shown in the above formula (10);

[0737] When isTransposed is equal to 1, the transform unit 206 uses the following instructions to obtain the block 1102 of transform coefficients, as specifically shown in the above formula (11).

[0738] Optionally, when isTransposed is equal to 1, the transform unit 206 obtains the block 1102 of transform coefficients using the following instructions, as specifically shown in the above formula (10);

[0739] When isTransposed is equal to 0, the transform unit 206 uses the following instructions to obtain the block 1102 of transform coefficients, as specifically shown in the above formula (11).

[0740] Among them, the determination of log2LfnstSize is shown in the above formula (9).

[0741] In addition, when the intra prediction mode of the current block is a conventional intra prediction mode, namely, an angular intra prediction mode, a DC mode, and a PLANAR mode, the transform unit 206 uses a lookup table as shown in Table 11 to determine the value of lfnstTrSetIdx according to the mode index value of the conventional intra prediction mode (assigned to predModeIntra), that is, to determine the LFNST transform core candidate set selected for the current block. The transform unit 206 determines the transform core for performing LFNST as the transform core indicated by lfnst_index in the LFNST transform core candidate set indicated by lfnstTrSetIdx.

[0742] When lfnst_index is not equal to 0, the transform unit 206 performs LFNST on the transform block using coefficients from sub-blocks 11001, 11002, and 11003, obtains LFNST coefficients, and places the LFNST coefficients into sub-blocks 11011, 11012, and 11013 of block 1102. Assume that d[x][y] is a block of first transform coefficients, where x=0, ..., nTbW-1, y=0, ..., nTbH-1, where nTbW and nTbH are the width and height of block 1102, respectively. Assume that v[x] is a block of LFNST coefficients, where x=0, ..., nLfnstOutSize-1, and nLfnstOutSize=(nTbW>=8&&nTbH>=8)? 48:16.

[0743] Optionally, when predModeIntra is less than or equal to 34, the transform unit 206 obtains the block 1102 of transform coefficients using the following instructions, as specifically shown in the above formula (10);

[0744] When predModeIntra is greater than 34, the transform unit 206 uses the following instructions to obtain a block 1102 of transform coefficients, as specifically shown in the above formula (11).

[0745] Optionally, when predModeIntra is greater than 34, the transform unit 206 obtains the block 1102 of transform coefficients using the following instructions, as specifically shown in the above formula (10);

[0746] When predModeIntra is less than or equal to 34, the transform unit 206 uses the following instructions to obtain a block 1102 of transform coefficients, as specifically shown in the above formula (11).

[0747] Among them, the determination of log2LfnstSize is shown in the above formula (9).

[0748] In step 1102 , the transform unit 206 performs a first transform (ie, a core transform) on the transform coefficients in the block 1102 , for example, an integer transform initially designed based on DCT, to obtain a reconstructed residual block 1103 of the current block.

[0749] Further, the adder 207 performs an addition operation on its input (the intra-frame prediction block from the prediction unit 202 and the reconstructed residual block from the transform unit 206) to obtain a reconstructed block of the current block. The reconstructed block is also sent to the prediction unit 202 to be used as a reference for other blocks encoded in the intra-frame prediction mode.

[0750] After all CUs in the picture or sub-picture are reconstructed, the filtering unit 208 performs loop filtering on the reconstructed picture or sub-picture. The filtering unit 208 includes one or more filters, such as a deblocking filter, a sample adaptive offset filter, an adaptive loop filter, a luminance mapping and chrominance scaling filter, and a filter based on a neural network. Alternatively, when the filtering unit 208 determines that the reconstructed block is not used as a reference for decoding other blocks, the filtering unit 208 performs loop filtering on one or more target pixels in the reconstructed block.

[0751] The output of the filtering unit 208 is a decoded picture or sub-picture, which is sent to the DPB unit 209. The DPB unit 209 outputs the decoded picture according to the timing and control information. The picture stored in the DPB unit 209 can also be used as a reference for performing inter-frame prediction or intra-frame prediction by the prediction unit 202.

[0752] In the embodiment of the present application, the decoder 200 may be a computing device having a processor and a storage medium recording a decoding program. When the processor reads and runs the decoding program, the decoder 200 reads the input bitstream and generates a corresponding decoded video. In addition, the decoder 200 may be a computing device having one or more chips. The units implemented as integrated circuits on the chip are connected to the processor. Figure 2 The corresponding units in the same embodiment have similar connections and data exchange and have similar functions.

[0753] The present embodiment provides a decoding method, which obtains prediction parameters and LFNST parameters of a current block by parsing a bitstream, wherein the prediction parameters include prediction mode parameters; when the prediction mode parameters indicate that a matrix-based intra-frame prediction MIP mode is used to determine an intra-frame prediction value for a chrominance component of the current block, obtains adjacent sample values ​​of the current block, and determines a MIP input sample value of the current block according to the adjacent sample values ​​of the current block; determines a MIP prediction value of the chrominance component of the current block according to the MIP input sample value, a MIP weighting matrix, and a shift parameter; The MIP prediction value is filtered to determine the intra-frame prediction value of the chrominance component of the current block; when the LFNST parameter indicates that LFNST is to be performed on the current block, the reconstructed transform coefficient block of the current block is determined, and LFNST is performed on at least part of the reconstructed transform coefficients in the reconstructed transform coefficient block to obtain a second transform coefficient block; the second transform coefficient block is transformed for the first time to obtain a reconstructed residual block of the chrominance component of the current block; according to the intra-frame prediction value of the chrominance component of the current block and the reconstructed residual block, the reconstructed block of the chrominance component of the current block is determined. In this way, for the MIP mode, the complexity can be reduced while ensuring the decoding performance, and the storage space required in the decoding process can be reduced, thereby effectively improving the decoding efficiency; in addition, when the LFNST technology is applied to the MIP mode prediction, the MIP parameters are introduced, so that the LFNST transform is more flexible, further improving the decoding efficiency.

[0754] In another embodiment of the present application, based on the same inventive concept as the above embodiment, see Fig.12 , which shows a schematic diagram of the structure of an encoder 120 provided in an embodiment of the present application. Fig.12 As shown, the encoder 120 may include: a first determination unit 1201, a first prediction unit 1202, a first transformation unit 1203 and an encoding unit 1204; wherein,

[0755] A first determining unit 1201 is configured to determine prediction parameters of a current block, wherein the prediction parameters include prediction mode parameters;

[0756] The first determining unit 1201 is further configured to, when the prediction mode parameter indicates that a matrix-based intra-prediction MIP mode is used to determine an intra-prediction value for a chrominance component of the current block, obtain adjacent sample values ​​of the current block, and determine a MIP input sample value of the current block according to the adjacent sample values ​​of the current block;

[0757] The first prediction unit 1202 is configured to determine the MIP prediction value of the chrominance component of the current block according to the MIP input sample value, the MIP weighting matrix and the shift parameter; wherein the MIP prediction value is the prediction value of some sampling points in the chrominance component of the current block; and filter the MIP prediction value to determine the intra-frame prediction value of the chrominance component of the current block;

[0758] The first transform unit 1203 is configured to determine a prediction residual value of the chrominance component of the current block according to the intra-frame prediction value of the chrominance component of the current block; and perform a low-frequency non-separable secondary transform LFNST on the prediction residual value to determine a LFNST parameter;

[0759] The encoding unit 1204 is configured to encode the LFNST parameters and write them into a bitstream.

[0760] In some embodiments, the adjacent sampling values ​​of the current block include left adjacent sampling values ​​and upper adjacent sampling values ​​of the current block.

[0761] In some embodiments, the shift parameters include a shift offset parameter and a shift amount parameter;

[0762] The first determination unit 1201 is specifically configured to determine the product of the sum of the MIP input sampling values ​​and the shift offset parameter according to the value of the shift offset parameter, wherein the value of the shift offset parameter is a fixed constant; and determine the first constant value according to the value of the shift quantity parameter, wherein the value of the shift quantity parameter is a fixed constant; and set the value of the first offset to the difference between the first constant value and the product; and determine the MIP weighting matrix of the current block according to the prediction parameters; and determine the MIP prediction value of the chrominance component of the current block according to the MIP weighting matrix, the MIP input sampling value, the shift quantity parameter and the first offset.

[0763] In some embodiments, the prediction parameters further include: a size parameter of the current block.

[0764] In some embodiments, the first determination unit 1201 is further configured to determine the block size index value of the current block according to the size parameter of the current block; and perform downsampling filtering on the adjacent sampling values ​​of the current block to obtain a first temporary reference value; and when the block size index value of the current block is within a preset range, determine a second constant value according to the bit depth of the adjacent sampling values ​​of the current block; set the value corresponding to the index number 0 in the MIP input sampling value to be equal to the difference between the second constant value and the value corresponding to the index number 0 in the first temporary reference value; The value corresponding to the index number i in the MIP input sampling value is set to be equal to the difference between the value corresponding to the index number i in the first temporary reference value and the value corresponding to the index number equal to 0 in the first temporary reference value, wherein i is an integer greater than 0; when the block size index value of the current block is outside a preset range, the value corresponding to the index number j in the MIP input sampling value is set to be equal to the difference between the value corresponding to the index number j+1 in the first temporary reference value and the value corresponding to the index number equal to 0 in the first temporary reference value, wherein j is an integer greater than or equal to 0.

[0765] In some embodiments, the first determination unit 1201 is further configured to determine a second constant value according to the bit depth of adjacent sampling values ​​of the current block when the block size index value of the current block is within a preset range; set the value corresponding to the index number 0 in the MIP input sampling value to be equal to the difference between the value corresponding to the index number 0 in the first temporary reference value and the second constant value; set the value corresponding to the index number i in the MIP input sampling value to be equal to the difference between the value corresponding to the index number i in the first temporary reference value and the value corresponding to the index number equal to 0 in the first temporary reference value, where i is an integer greater than 0.

[0766] In some embodiments, the first determination unit 1201 is further configured to, when the width and height of the current block are both equal to 4, set the block size index value of the current block to 0; when the width and height of the current block are both equal to 8, or one of the width and height of the current block is equal to 4, set the block size index value of the current block to 1; when the width and height of the current block do not meet the aforementioned conditions, set the block size index value of the current block to 2.

[0767] In some embodiments, the first determination unit 1201 is further configured to perform downsampling filtering on the adjacent sampling values ​​of the current block to obtain a first temporary reference value; and determine a second constant value according to the bit depth of the adjacent sampling values ​​of the current block, cache the second constant value in a data unit after the first temporary reference value to obtain a second temporary reference value; and set the value corresponding to the index number equal to j in the MIP input sampling value to be equal to the difference between the value corresponding to the index number equal to j+1 in the second temporary reference value and the value corresponding to the index number equal to 0 in the second temporary reference value, where j is an integer greater than or equal to 0.

[0768] In some embodiments, the first determination unit 1201 is further configured to set the second constant value to be equal to an integer exponent power of 2, wherein the exponent of the power is equal to the bit depth of the adjacent sample values ​​of the current block minus 1.

[0769] In some embodiments, the first determination unit 1201 is further configured to perform a binary bit left shift on "1" to obtain the second constant value, wherein the number of bits of the bit left shift is equal to the bit depth of the adjacent sampling values ​​of the current block minus 1.

[0770] In some embodiments, the value of the shift amount parameter is a fixed constant that is independent of the block size index value and the MIP mode index value.

[0771] In some embodiments, the value of the shift amount parameter is set to be equal to 6.

[0772] In some embodiments, the value of the shift offset parameter is a fixed constant that is independent of the block size index value and the MIP mode index value.

[0773] In some embodiments, the value of the shift offset parameter is set to be equal to 32.

[0774] In some embodiments, the first determining unit 1201 is further configured to set the first constant value to be equal to an integer exponent power of 2, wherein the exponent of the power is equal to the value of the shift quantity parameter minus 1.

[0775] In some embodiments, the first determination unit 1201 is further configured to perform a binary bit left shift on "1" to obtain the first constant value, wherein the number of bits of the bit left shift is equal to the value of the shift quantity parameter minus 1.

[0776] In some embodiments, see Fig.12, the encoder 120 may also include a first calculation unit 1205, configured to calculate a first weighted sum of the MIP weighting matrix and the MIP input sample value; and calculate a first sum value of the first weighted sum and the first offset; and perform a binary bit right shift on the first sum value to obtain a first right shift value, wherein the number of bits of the right shift is equal to the value of the shift quantity parameter;

[0777] The first determination unit 1201 is also configured to set the MIP prediction value of the chrominance component of the current block to be equal to the sum of the first right shift value and the value corresponding to the index number 0 in the first temporary reference value; wherein the first temporary reference value is obtained based on downsampling filtering processing of adjacent sampling values ​​of the current block.

[0778] In some embodiments, the first transform unit 1203 is configured to determine a residual block of the current block according to the prediction residual value, and perform a first transform on the residual block to obtain a first transform coefficient block;

[0779] The first determining unit 1201 is further configured to determine whether to perform LFNST on at least part of the transform coefficients in the first transform coefficient block;

[0780] The first transform unit 1203 is further configured to, when the determination result is yes, perform LFNST on at least part of the transform coefficients in the first transform coefficient block to determine the LFNST parameters.

[0781] In some embodiments, the first determination unit 1201 is further configured to determine the minimum value among the size parameters of the current block when the chrominance component of the current block uses the MIP mode to determine the intra-frame prediction value; and if the minimum value is greater than or equal to a first preset threshold, determine to perform LFNST on at least part of the transform coefficients in the first transform coefficient block.

[0782] In some embodiments, the LFNST parameter includes a LFNST index number, and the first determining unit 1201 is further configured to set the value of the LFNST index number to be greater than zero if it is determined that LFNST is performed on at least part of the transform coefficients in the first transform coefficient block; if it is determined that LFNST is not performed on at least part of the transform coefficients in the first transform coefficient block, set the value of the LFNST index number to be equal to zero;

[0783] The encoding unit 1204 is further configured to encode the LFNST index number and write it into the bitstream.

[0784] In some embodiments, the LFNST parameters also include LFNST coefficients, see Fig.12 , the encoder 120 may further include a quantization unit 1206; wherein,

[0785] A first transform unit 1203 is configured to perform LFNST on at least part of the transform coefficients in the first transform coefficient block to obtain the LFNST coefficients;

[0786] A quantization unit 1206, configured to quantize the LFNST coefficients to obtain quantized coefficients;

[0787] The encoding unit 1204 is further configured to encode the quantization coefficients and write them into a bit stream.

[0788] It is understandable that in the embodiments of the present application, a "unit" may be a part of a circuit, a part of a processor, a part of a program or software, etc., and of course, it may be a module, or it may be non-modular. Moreover, the components in the present embodiment may be integrated into a processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of a software functional module.

[0789] If the integrated unit is implemented in the form of a software function module and is not sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this embodiment is essentially or the part that contributes to the prior art or the whole or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) or a processor to perform all or part of the steps of the method described in this embodiment. The aforementioned storage medium includes: U disk, mobile hard disk, read only memory (ROM), random access memory (RAM), disk or optical disk, etc., various media that can store program codes.

[0790] Therefore, an embodiment of the present application provides a computer storage medium, which is applied to the encoder 120, and the computer storage medium stores a computer program, and when the computer program is executed by the first processor, the method described in any one of the above embodiments is implemented.

[0791] Based on the composition of the encoder 120 and the computer storage medium, see Fig.13 , which shows a schematic diagram of the specific hardware structure of the encoder 120 provided in the embodiment of the present application. Fig.13As shown, it may include: a first communication interface 1301, a first memory 1302 and a first processor 1303; each component is coupled together through a first bus system 1304. It is understandable that the first bus system 1304 is used to realize the connection and communication between these components. In addition to the data bus, the first bus system 1304 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, Fig.13 In the figure, various buses are labeled as the first bus system 1304. Among them,

[0792] The first communication interface 1301 is used for receiving and sending signals during the process of sending and receiving information with other external network elements;

[0793] A first memory 1302, used to store a computer program that can be run on the first processor 1303;

[0794] The first processor 1303 is configured to, when running the computer program, execute:

[0795] Determining prediction parameters of the current block, wherein the prediction parameters include prediction mode parameters;

[0796] When the prediction mode parameter indicates that a matrix-based intra prediction MIP mode is used to determine an intra prediction value for a chrominance component of the current block, obtaining adjacent sample values ​​of the current block, and determining a MIP input sample value of the current block according to the adjacent sample values ​​of the current block;

[0797] Determine the MIP prediction value of the chrominance component of the current block according to the MIP input sample value, the MIP weighting matrix and the shift parameter; wherein the MIP prediction value is the prediction value of some sampling points in the chrominance component of the current block;

[0798] Performing filtering processing on the MIP prediction value to determine an intra-frame prediction value of the chrominance component of the current block;

[0799] Determining a prediction residual value of a chrominance component of the current block according to an intra-frame prediction value of a chrominance component of the current block;

[0800] Performing a low frequency non-separable secondary transform LFNST on the prediction residual value to determine LFNST parameters;

[0801] The LFNST parameters are encoded and written into the bitstream.

[0802] It can be understood that the first memory 1302 in the embodiment of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM) and direct memory bus random access memory (DRRAM). The first memory 1302 of the system and method described in the present application is intended to include, but is not limited to, these and any other suitable types of memory.

[0803] The first processor 1303 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the first processor 1303. The above-mentioned first processor 1303 can be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application can be directly embodied as a hardware decoding processor to execute, or the hardware and software modules in the decoding processor are combined and executed. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the first memory 1302, and the first processor 1303 reads the information in the first memory 1302 and completes the steps of the above method in combination with its hardware.

[0804] It is to be understood that these embodiments described in the present application can be implemented with hardware, software, firmware, middleware, microcode or a combination thereof. For hardware implementation, the processing unit can be implemented in one or more application specific integrated circuits (Application Specific Integrated Circuits, ASIC), digital signal processors (Digital Signal Processing, DSP), digital signal processing equipment (DSP Device, DSPD), programmable logic devices (Programmable Logic Device, PLD), field programmable gate arrays (Field-Programmable Gate Array, FPGA), general-purpose processors, controllers, microcontrollers, microprocessors, other electronic units or combinations thereof for performing functions described in the present application. For software implementation, the technology described in the present application can be implemented by a module (such as a process, function, etc.) that performs functions described in the present application. The software code can be stored in a memory and executed by a processor. The memory can be implemented in a processor or outside a processor.

[0805] Optionally, as another embodiment, the first processor 1303 is further configured to execute any one of the methods described in the foregoing embodiments when running the computer program.

[0806] This embodiment provides an encoder, which may include a first determination unit, a first prediction unit, a first transformation unit, and an encoding unit. In this way, for the MIP mode, the complexity can be reduced on the basis of ensuring the encoding performance, and the storage space required in the encoding process can be reduced, thereby effectively improving the encoding efficiency; in addition, when the LFNST technology is applied to the MIP mode prediction, the MIP parameters are introduced, so that the LFNST transformation is more flexible, further improving the encoding efficiency.

[0807] In another embodiment of the present application, based on the same inventive concept as the above embodiment, see Fig.14 , which shows a schematic diagram of the structure of a decoder 140 provided in an embodiment of the present application. Fig.14 As shown, the decoder 140 may include: a parsing unit 1401, a second determining unit 1402, a second prediction unit 1403, and a second transforming unit 1404; wherein,

[0808] The parsing unit 1401 is configured to parse the bitstream to obtain prediction parameters and LFNST parameters of the current block, wherein the prediction parameters include prediction mode parameters;

[0809] The second determining unit 1402 is configured to, when the prediction mode parameter indicates that the intra prediction value is determined using the matrix-based intra prediction MIP mode for the chrominance component of the current block, obtain the adjacent sample values ​​of the current block, and determine the MIP input sample value of the current block according to the adjacent sample values ​​of the current block;

[0810] The second prediction unit 1403 is configured to determine the MIP prediction value of the chrominance component of the current block according to the MIP input sample value, the MIP weighting matrix and the shift parameter; wherein the MIP prediction value is the prediction value of some sampling points in the chrominance component of the current block; and to determine the intra-frame prediction value of the chrominance component of the current block by filtering the MIP prediction value;

[0811] The second transform unit 1404 is configured to, when the LFNST parameter indicates that LFNST is performed on the current block, determine a reconstructed transform coefficient block of the current block, perform LFNST on at least part of the reconstructed transform coefficients in the reconstructed transform coefficient block to obtain a second transform coefficient block; and perform a first transform on the second transform coefficient block to obtain a reconstructed residual block of the chrominance component of the current block;

[0812] The second determining unit 1402 is further configured to determine a reconstructed block of the chrominance component of the current block according to the intra-frame prediction value of the chrominance component of the current block and the reconstructed residual block.

[0813] In some embodiments, the adjacent sampling values ​​of the current block include left adjacent sampling values ​​and upper adjacent sampling values ​​of the current block.

[0814] In some embodiments, the shift parameters include a shift offset parameter and a shift amount parameter;

[0815] The second determination unit 1402 is specifically configured to determine the product of the sum of the MIP input sampling values ​​and the shift offset parameter according to the value of the shift offset parameter, wherein the value of the shift offset parameter is a fixed constant; and determine the first constant value according to the value of the shift quantity parameter, wherein the value of the shift quantity parameter is a fixed constant; and set the value of the first offset to the difference between the first constant value and the product; and determine the MIP weighting matrix of the current block according to the prediction parameters; and determine the MIP prediction value of the chrominance component of the current block according to the MIP weighting matrix, the MIP input sampling value, the shift quantity parameter and the first offset.

[0816] In some embodiments, the prediction parameters further include: a size parameter of the current block.

[0817] In some embodiments, the second determination unit 1402 is further configured to determine the block size index value of the current block according to the size parameter of the current block; and perform downsampling filtering on the adjacent sampling values ​​of the current block to obtain a first temporary reference value; and when the block size index value of the current block is within a preset range, determine a second constant value according to the bit depth of the adjacent sampling values ​​of the current block; set the value corresponding to the index number 0 in the MIP input sampling value to be equal to the difference between the second constant value and the value corresponding to the index number 0 in the first temporary reference value; The value corresponding to the index number i in the MIP input sampling value is set to be equal to the difference between the value corresponding to the index number i in the first temporary reference value and the value corresponding to the index number equal to 0 in the first temporary reference value, wherein i is an integer greater than 0; when the block size index value of the current block is outside a preset range, the value corresponding to the index number j in the MIP input sampling value is set to be equal to the difference between the value corresponding to the index number j+1 in the first temporary reference value and the value corresponding to the index number equal to 0 in the first temporary reference value, wherein j is an integer greater than or equal to 0.

[0818] In some embodiments, the second determination unit 1402 is further configured to determine a second constant value according to the bit depth of the adjacent sampling values ​​of the current block when the block size index value of the current block is within a preset range; set the value corresponding to the index number 0 in the MIP input sampling value to be equal to the difference between the value corresponding to the index number 0 in the first temporary reference value and the second constant value; set the value corresponding to the index number i in the MIP input sampling value to be equal to the difference between the value corresponding to the index number i in the first temporary reference value and the value corresponding to the index number equal to 0 in the first temporary reference value, where i is an integer greater than 0.

[0819] In some embodiments, the second determination unit 1402 is further configured to, when the width and height of the current block are both equal to 4, set the block size index value of the current block to 0; when the width and height of the current block are both equal to 8, or one of the width and height of the current block is equal to 4, set the block size index value of the current block to 1; when the width and height of the current block do not meet the aforementioned conditions, set the block size index value of the current block to 2.

[0820] In some embodiments, the second determination unit 1402 is further configured to perform downsampling filtering on the adjacent sampling values ​​of the current block to obtain a first temporary reference value; and determine a second constant value according to the bit depth of the adjacent sampling values ​​of the current block, cache the second constant value in a data unit after the first temporary reference value to obtain a second temporary reference value; and set the value corresponding to the index number equal to j in the MIP input sampling value to be equal to the difference between the value corresponding to the index number equal to j+1 in the second temporary reference value and the value corresponding to the index number equal to 0 in the second temporary reference value, where j is an integer greater than or equal to 0.

[0821] In some embodiments, the second determining unit 1402 is further configured to set the second constant value to be equal to an integer exponent power of 2, wherein the exponent of the power is equal to the bit depth of the adjacent sample values ​​of the current block minus 1.

[0822] In some embodiments, the second determination unit 1402 is further configured to perform a binary bit left shift on "1" to obtain the second constant value, wherein the number of bits of the bit left shift is equal to the bit depth of the adjacent sampling values ​​of the current block minus 1.

[0823] In some embodiments, the value of the shift amount parameter is a fixed constant that is independent of the block size index value and the MIP mode index value.

[0824] In some embodiments, the value of the shift amount parameter is set to be equal to 6.

[0825] In some embodiments, the value of the shift offset parameter is a fixed constant that is independent of the block size index value and the MIP mode index value.

[0826] In some embodiments, the value of the shift offset parameter is set to be equal to 32.

[0827] In some embodiments, the second determining unit 1402 is further configured to set the first constant value to be equal to an integer exponent power of 2, wherein the exponent of the power is equal to the value of the shift quantity parameter minus 1.

[0828] In some embodiments, the second determination unit 1402 is further configured to perform a binary bit left shift on "1" to obtain the first constant value, wherein the number of bits of the bit left shift is equal to the value of the shift quantity parameter minus 1.

[0829] In some embodiments, see Fig.14 The decoder 140 may further include a second calculation unit 1405 configured to calculate a first weighted sum of the MIP weighting matrix and the MIP input sample value; and calculate a first sum value of the first weighted sum and the first offset; and perform a binary bit right shift on the first sum value to obtain a first right shift value, wherein the number of bits of the right shift is equal to the value of the shift quantity parameter;

[0830] The second determination unit 1402 is also configured to set the MIP prediction value of the chrominance component of the current block to be equal to the sum of the first right shift value and the value corresponding to the index number 0 in the first temporary reference value; wherein the first temporary reference value is obtained based on downsampling filtering of adjacent sampling values ​​of the current block.

[0831] In some embodiments, the second determining unit 1402 is further configured to determine whether there is a LFNST parameter in the bitstream;

[0832] The parsing unit 1401 is further configured to parse the code stream to obtain the LFNST parameter when it is determined that the code stream contains the LFNST parameter.

[0833] In some embodiments, the second determination unit 1402 is further configured to determine the minimum value among the size parameters of the current block when the chrominance component of the current block uses the MIP mode to determine the intra-frame prediction value; and if the minimum value is greater than or equal to the first preset threshold, determine that the LFNST parameter exists in the code stream.

[0834] In some embodiments, see Fig.14 , the decoder 140 may further include an inverse quantization unit 1406;

[0835] The parsing unit 1401 is further configured to parse the bitstream to obtain a quantization coefficient of the current block when the LFNST parameter indicates that LFNST is performed on the current block;

[0836] The dequantization unit 1406 is configured to dequantize the quantized coefficients to obtain a reconstructed transform coefficient block of the current block.

[0837] In some embodiments, the inverse quantization unit 1406 is specifically configured to perform a scaling operation on the quantization coefficients to obtain a reconstructed transform coefficient block of the current block.

[0838] In some embodiments, the parsing unit 1401 is specifically configured to parse the code stream to obtain the value of the LFNST index number; and if the value of the LFNST index number is greater than zero, determine to perform LFNST on the current block; if the value of the LFNST index number is equal to zero, determine not to perform LFNST on the current block.

[0839] It can be understood that in this embodiment, a "unit" can be a part of a circuit, a part of a processor, a part of a program or software, etc., and of course it can also be a module, or it can be non-modular. Moreover, the components in this embodiment can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional module.

[0840] If the integrated unit is implemented in the form of a software function module and is not sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, this embodiment provides a computer storage medium, which is applied to the decoder 140, and the computer storage medium stores a computer program. When the computer program is executed by the second processor, the method described in any one of the above embodiments is implemented.

[0841] Based on the above-mentioned components of the decoder 140 and the computer storage medium, see Fig.15 , which shows a schematic diagram of the specific hardware structure of the decoder 140 provided in the embodiment of the present application. Fig.15 As shown, it may include: a second communication interface 1501, a second memory 1502, and a second processor 1503; each component is coupled together via a second bus system 1504. It is understandable that the second bus system 1504 is used to realize the connection and communication between these components. In addition to the data bus, the second bus system 1504 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, Fig.15 In FIG. 1 , various buses are labeled as the second bus system 1504. Among them,

[0842] The second communication interface 1501 is used for receiving and sending signals during the process of sending and receiving information with other external network elements;

[0843] The second memory 1502 is used to store a computer program that can be run on the second processor 1503;

[0844] The second processor 1503 is configured to execute, when running the computer program:

[0845] Parse the bitstream to obtain prediction parameters and LFNST parameters of the current block, wherein the prediction parameters include prediction mode parameters;

[0846] When the prediction mode parameter indicates that a matrix-based intra prediction MIP mode is used to determine an intra prediction value for a chrominance component of the current block, obtaining adjacent sample values ​​of the current block, and determining a MIP input sample value of the current block according to the adjacent sample values ​​of the current block;

[0847] Determine the MIP prediction value of the chrominance component of the current block according to the MIP input sample value, the MIP weighting matrix and the shift parameter; wherein the MIP prediction value is the prediction value of some sampling points in the chrominance component of the current block;

[0848] Performing filtering processing on the MIP prediction value to determine an intra-frame prediction value of the chrominance component of the current block;

[0849] When the LFNST parameter indicates that LFNST is performed on the current block, determining a reconstructed transform coefficient block of the current block, and performing LFNST on at least part of the reconstructed transform coefficients in the reconstructed transform coefficient block to obtain a second transform coefficient block;

[0850] Performing a first transform on the second transform coefficient block to obtain a reconstructed residual block of the chrominance component of the current block;

[0851] A reconstructed block of the chrominance component of the current block is determined according to the intra-frame prediction value of the chrominance component of the current block and the reconstructed residual block.

[0852] Optionally, as another embodiment, the second processor 1503 is further configured to execute any one of the methods described in the foregoing embodiments when running the computer program.

[0853] It can be understood that the hardware functions of the second memory 1502 and the first memory 1302 are similar, and the hardware functions of the second processor 1503 and the first processor 1303 are similar; they will not be described in detail here.

[0854] This embodiment provides a decoder, which may include a parsing unit, a second determination unit, a second prediction unit, and a second transformation unit. In this way, for the MIP mode, the complexity can be reduced on the basis of ensuring the decoding performance, and the storage space required in the decoding process can be reduced, thereby effectively improving the decoding efficiency; in addition, when the LFNST technology is applied to the MIP mode prediction, the MIP parameters are introduced, so that the LFNST transformation is more flexible, further improving the decoding efficiency.

[0855] In another embodiment of the present application, see Fig.16 , which shows a schematic diagram of the composition structure of a sending device provided in an embodiment of the present application. Fig.16 As shown, an exemplary sending device 1600 is shown. Among them, the acquisition unit 1601 acquires a video signal and sends the video signal to the encoder 1602. The acquisition unit 1601 may be a device including one or more cameras (including a depth camera). The acquisition unit 1601 may be a device that partially or completely decodes the code stream to obtain a video. The acquisition unit 1601 may also include one or more elements to capture an audio signal. A specific implementation of the encoder 1602 is the encoder 100 or the encoder 120 described in the aforementioned embodiment, which encodes the video signal from the acquisition unit 1601 as its input video and generates a video code stream. The encoder 1602 may also include one or more audio encoders to encode the audio signal to generate an audio code stream. The storage / transmission unit 1603 receives the video code stream from the encoder 1602. The storage / transmission unit 1603 may also receive the audio code stream from the encoder 1602 and compress the video code stream and the audio code stream together to form a media file (e.g., an ISO-based media file format) or a transport stream.

[0856] Optionally, the storage / transmission unit 1603 writes the media file or the transport stream to a storage unit, such as a hard disk, a DVD disk, a cloud, or a portable storage device.

[0857] Optionally, the storage / transmission unit 1603 transmits the bit stream to a transmission network, such as the Internet, a wired network, a cellular network, a wireless local area network, etc.

[0858] In another embodiment of the present application, see Fig.17 , which shows a schematic diagram of the composition structure of a target device provided in an embodiment of the present application. Fig.17As shown, an exemplary target device 1700 is shown. Among them, the receiving unit 1701 receives a media file or a transport stream from a network, or reads a media file or a transport stream from a storage device. The receiving unit 1701 separates the video code stream and the audio code stream from the media file or the transport stream. The receiving unit 1701 can also generate a new video code stream by extracting the video code stream. The receiving unit 1701 can also generate a new audio code stream by extracting the audio code stream. The decoder 1702 includes one or more video decoders, for example, a specific implementation is the decoder 200 or the decoder 140 described in the above embodiment.

[0859] The decoder 1702 may also include one or more audio decoders. The decoder 1702 decodes the video bitstream and the audio bitstream from the receiving unit 1701 to obtain a decoded video and one or more decoded audios corresponding to one or more channels. The rendering unit 1703 performs operations on the reconstructed video to make it suitable for display. Such operations may include one or more of the following operations to improve the perceived quality: noise reduction, synthesis, conversion of color space, upsampling, downsampling, etc. The rendering unit 1703 may also perform operations on the decoded audio to improve the perceived quality of the displayed audio signal.

[0860] In another embodiment of the present application, see Fig.18 , which shows a schematic diagram of the structure of a communication system provided by an embodiment of the present application. Fig.18 As shown, an exemplary communication system 1800 is shown. Among them, the source device 1801 can be Fig.16 The output of the storage / transmission unit 1603 is processed by the storage medium / transmission network 1802 to store or transmit the code stream. The target device 1803 can be Fig.17 The target device 1700 is shown. Here, the receiving unit 1701 can obtain a code stream from a storage medium / transmission network 1802. The receiving unit 1701 can extract a new video code stream from a media file or a transmission stream. The receiving unit 1701 can also extract a new audio code stream from a media file or a transmission stream.

[0861] It should be noted that, in this application, the terms "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0862] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0863] The methods disclosed in several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0864] The features disclosed in several product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0865] The features disclosed in several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments or device embodiments.

[0866] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0867] Industrial Applicability

[0868] In an embodiment of the present application, on the encoder side, after determining the prediction parameters of the current block, when the prediction mode parameters indicate that the matrix-based intra-frame prediction MIP mode is used to determine the intra-frame prediction value for the chrominance component of the current block, the adjacent sampling values ​​of the current block are obtained, and the MIP input sampling values ​​of the current block are determined according to the adjacent sampling values ​​of the current block; the MIP prediction value of the chrominance component of the current block is determined according to the MIP input sampling value, the MIP weighting matrix and the shift parameter; the MIP prediction value is filtered to determine the intra-frame prediction value of the chrominance component of the current block; the prediction residual value of the chrominance component of the current block is determined according to the intra-frame prediction value of the chrominance component of the current block; a low-frequency non-separable secondary transform LFNST is performed on the prediction residual value to determine the LFNST parameters; the LFNST parameters are encoded and written into the bitstream. At the decoder side, after parsing the bitstream and obtaining the prediction parameters and LFNST parameters of the current block, when the prediction mode parameter indicates that the matrix-based intra-frame prediction MIP mode is used to determine the intra-frame prediction value for the chrominance component of the current block, the adjacent sampling values ​​of the current block are obtained, and the MIP input sampling values ​​of the current block are determined according to the adjacent sampling values ​​of the current block; the MIP prediction value of the chrominance component of the current block is determined according to the MIP input sampling value, the MIP weighting matrix and the shift parameter; the MIP prediction value is filtered to determine the intra-frame prediction value of the chrominance component of the current block; when the LFNST parameter indicates that LFNST is performed on the current block, a reconstructed transform coefficient block of the current block is determined, and LFNST is performed on at least part of the reconstructed transform coefficients in the reconstructed transform coefficient block to obtain a second transform coefficient block; the second transform coefficient block is transformed for the first time to obtain a reconstructed residual block of the chrominance component of the current block; and the reconstructed block of the chrominance component of the current block is determined according to the intra-frame prediction value of the chrominance component of the current block and the reconstructed residual block. In this way, for the MIP mode, the complexity can be reduced while ensuring the encoding and decoding performance, while reducing the storage space required in the encoding and decoding process, effectively improving the encoding and decoding efficiency; in addition, when LFNST technology is applied to MIP mode prediction, the introduction of MIP parameters makes the LFNST transformation more flexible, further improving the encoding and decoding efficiency.

Claims

1. A coding method, applied to an encoder, characterized in that: The method comprises: Determining prediction parameters of the current block, wherein the prediction parameters include prediction mode parameters; When the prediction mode parameter indicates that a matrix-based intra prediction MIP mode is used to determine an intra prediction value for a chrominance component of the current block, obtaining adjacent sample values ​​of the current block, and determining a MIP input sample value of the current block according to the adjacent sample values ​​of the current block; Determine the product of the sum of the MIP input sample values ​​and a shift offset parameter; Setting the value of the first offset to the difference between the first constant value and the product; Determining a MIP weighting matrix of the current block according to the prediction parameters; Calculating a first weighted sum of the MIP weighting matrix and the MIP input sample value; Calculating a first sum value of the first weighted sum and the first offset; Performing a binary bit right shift on the first sum value to obtain a first right-shifted value, wherein the number of bits of the right shift is equal to the value of the shift quantity parameter; Setting the MIP prediction value of the chrominance component of the current block to be equal to the sum of the first right shift value and the value corresponding to the index number 0 in the first temporary reference value; wherein the first temporary reference value is obtained by performing downsampling filtering on adjacent sample values ​​of the current block; Performing filtering processing on the MIP prediction value to determine an intra-frame prediction value of the chrominance component of the current block; Determining a prediction residual value of a chrominance component of the current block according to an intra-frame prediction value of a chrominance component of the current block; Performing a low frequency non-separable secondary transform LFNST on the prediction residual value to determine LFNST parameters; The LFNST parameters are encoded and written into the bitstream.

2. The method according to claim 1, characterized in that The value of the shift quantity parameter is equal to 6, the first constant value is equal to 32, and the value of the shift offset parameter is equal to 32.

3. The method according to claim 1, characterized in that The prediction parameters also include: a size parameter of the current block, wherein determining the MIP input sample value of the current block according to the adjacent sample values ​​of the current block includes: Determining a block size index value of the current block according to a size parameter of the current block; Performing down-sampling filtering on adjacent sample values ​​of the current block to obtain a first temporary reference value; When the block size index value of the current block is within a preset range, determine a second constant value according to the bit depth of the adjacent sample values ​​of the current block; set the value corresponding to the index number 0 in the MIP input sample value to be equal to the difference between the second constant value and the value corresponding to the index number 0 in the first temporary reference value; set the value corresponding to the index number i in the MIP input sample value to be equal to the difference between the value corresponding to the index number i in the first temporary reference value and the value corresponding to the index number equal to 0 in the first temporary reference value, where i is an integer greater than 0; When the block size index value of the current block is outside a preset range, the value corresponding to the index number j in the MIP input sampling value is set to be equal to the difference between the value corresponding to the index number j+1 in the first temporary reference value and the value corresponding to the index number 0 in the first temporary reference value, where j is an integer greater than or equal to 0.

4. The method according to claim 3, characterized in that The second constant value is expressed as 1<<(BitDepth-1), where BitDepth is the bit depth of the color component of the current block.

5. The method according to claim 3, characterized in that: The determining, according to the size parameter of the current block, a block size index value of the current block comprises: When the width and height of the current block are both equal to 4, setting the block size index value of the current block to 0; When the width and height of the current block are both equal to 8, or one of the width and height of the current block is equal to 4, setting the block size index value of the current block to 1; When the width and height of the current block do not meet the aforementioned conditions, the block size index value of the current block is set to 2.

6. The method according to claim 1, characterized in that The performing a low frequency non-separable secondary transform LFNST on the prediction residual value to determine a LFNST parameter comprises: Performing a first transform on the prediction residual value of the current block to obtain a first transform coefficient block; determining whether to perform LFNST on at least part of the transform coefficient blocks in the first transform coefficient block; If the determination result is yes, performing LFNST on at least part of the transform coefficients in the first transform coefficient block to determine the LFNST parameters; quantizing the LFNST coefficients to obtain quantized coefficients; The quantized coefficients are encoded and written into a bitstream.

7. The method according to claim 6, characterized in that The LFNST parameter includes a LFNST index number, and the determining of the LFNST parameter further includes: If it is determined that LFNST is performed on at least part of the transform coefficients in the first transform coefficient block, setting the value of the LFNST index sequence number to be greater than zero; If it is determined that LFNST is not to be performed on at least part of the transform coefficients in the first transform coefficient block, setting the value of the LFNST index sequence number to zero; The step of encoding the LFNST parameters and writing them into a bitstream includes: The LFNST index number is encoded, and the encoded LFNST index number is written into the bitstream.

8. A decoding method, applied to a decoder, characterized in that: The method comprises: Parse the bitstream to obtain prediction parameters and LFNST parameters of the current block, wherein the prediction parameters include prediction mode parameters; When the prediction mode parameter indicates that a matrix-based intra prediction MIP mode is used to determine an intra prediction value for a chrominance component of the current block, obtaining adjacent sample values ​​of the current block, and determining a MIP input sample value of the current block according to the adjacent sample values ​​of the current block; Determine the product of the sum of the MIP input sample values ​​and a shift offset parameter; Setting the value of the first offset to the difference between the first constant value and the product; Determining a MIP weighting matrix of the current block according to the prediction parameters; Calculating a first weighted sum of the MIP weighting matrix and the MIP input sample value; Calculating a first sum value of the first weighted sum and the first offset; Performing a binary bit right shift on the first sum value to obtain a first right-shifted value, wherein the number of bits of the right shift is equal to the value of the shift quantity parameter; Setting the MIP prediction value of the chrominance component of the current block to be equal to the sum of the first right shift value and the value corresponding to the index number 0 in the first temporary reference value; wherein the first temporary reference value is obtained by performing downsampling filtering on adjacent sample values ​​of the current block; Performing filtering processing on the MIP prediction value to determine an intra-frame prediction value of the chrominance component of the current block; When the LFNST parameter indicates that LFNST is performed on the current block, determining a reconstructed transform coefficient block of the current block, and performing LFNST on at least part of the reconstructed transform coefficient blocks in the reconstructed transform coefficient block to obtain a second transform coefficient block; Performing a first transform on the second transform coefficient block to obtain a reconstructed residual value of the chrominance component of the current block; A reconstructed value of the chrominance component of the current block is determined according to the intra-frame prediction value of the chrominance component of the current block and the reconstructed residual value.

9. The method according to claim 8, characterized in that The value of the shift quantity parameter is equal to 6, the first constant value is equal to 32, and the value of the shift offset parameter is equal to 32.

10. The method according to claim 8, characterized in that The prediction parameters also include: a size parameter of the current block, wherein determining the MIP input sample value of the current block according to the adjacent sample values ​​of the current block includes: Determining a block size index value of the current block according to a size parameter of the current block; Performing down-sampling filtering on adjacent sample values ​​of the current block to obtain a first temporary reference value; When the block size index value of the current block is within a preset range, determine a second constant value according to the bit depth of the adjacent sample values ​​of the current block; set the value corresponding to the index number 0 in the MIP input sample value to be equal to the difference between the second constant value and the value corresponding to the index number 0 in the first temporary reference value; set the value corresponding to the index number i in the MIP input sample value to be equal to the difference between the value corresponding to the index number i in the first temporary reference value and the value corresponding to the index number equal to 0 in the first temporary reference value, where i is an integer greater than 0; When the block size index value of the current block is outside a preset range, the value corresponding to the index number j in the MIP input sampling value is set to be equal to the difference between the value corresponding to the index number j+1 in the first temporary reference value and the value corresponding to the index number 0 in the first temporary reference value, where j is an integer greater than or equal to 0.

11. The method according to claim 10, characterized in that The second constant value is expressed as 1<<(BitDepth-1), where BitDepth is the bit depth of the color component of the current block.

12. The method according to claim 10, characterized in that The determining, according to the size parameter of the current block, a block size index value of the current block comprises: When the width and height of the current block are both equal to 4, setting the block size index value of the current block to 0; When the width and height of the current block are both equal to 8, or one of the width and height of the current block is equal to 4, setting the block size index value of the current block to 1; When the width and height of the current block do not meet the aforementioned conditions, the block size index value of the current block is set to 2.

13. The method according to claim 8, characterized in that The parsing of the code stream to obtain the LFNST parameters of the current block includes: Parse the code stream to obtain the value of the LFNST index number; If the value of the LFNST index number is greater than zero, determining to perform LFNST on the current block; If the value of the LFNST index sequence number is equal to zero, it is determined not to perform LFNST on the current block.

14. The method according to claim 8, characterized in that The determining of the reconstructed transform coefficient block of the current block comprises: Parsing the bitstream to obtain the quantization coefficient of the current block; The quantized coefficients are scaled to obtain a reconstructed transform coefficient block of the current block.

15. An encoder, characterized in that: The encoder includes a first memory and a first processor; wherein, The first memory is used to store a computer program that can be run on the first processor; The first processor is configured to execute the method according to any one of claims 1 to 7 when running the computer program.

16. A decoder, characterized in that: The decoder comprises a second memory and a second processor; wherein, The second memory is used to store a computer program that can be run on the second processor; The second processor is configured to execute the method according to any one of claims 8 to 14 when running the computer program.

17. A code stream, characterized in that: The code stream is obtained by the encoder by executing the method according to any one of claims 1-7.

18. A code stream, characterized in that: The code stream is used to input into a decoder, so that the decoder executes the method according to any one of claims 8 to 14.

Images