Encoding method, decoding method, encoder, decoder, and storage medium

Abstract

To provide an encoding method, a decoding method, an encoder, a decoder and a storage medium that improve the encoding and decoding efficiency.SOLUTION: A method comprises: obtaining neighboring sample values of a current block and determining matrix-based intra prediction (MIP) input sample values based on the neighboring sample values, when a prediction mode parameter indicates that an MIP mode is used to determine an intra prediction value of a chroma component of the current block; determining an MIP prediction value of the chroma component of the current block based on the MIP input sample values, an MIP weighting matrix and a shifting parameter; determining the intra prediction value based on the MIP prediction value; determining a prediction residual value of the chroma component of the current block based on the intra prediction value of the chroma component of the current block; determining a low-frequency non-separable transform (LFNST) parameter by performing an LFNST on the prediction residual value; and encoding the parameter and writing the parameter in a bitstream.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] [Reference to Related Applications] This application claims priority to U.S. Provisional Application No. 62 / 958,582, entitled “VIDEO ENCODING AND DECODING METHOD, APPARATUS AND COMMUNICATION SYSTEM,” filed on January 8, 2020 in the names of Junyan Huo, Shuai Wan, and Yanzhuo Ma, the entire contents of which are incorporated herein by reference.

[0002] FIELD OF THEINVENTION Embodiments of the present invention relate to the technical field of video encoding, and in particular to an encoding method, a decoding method, an encoder, a decoder and a storage medium. [Background technology]

[0003] As people's demands for video display quality increase, new video application forms such as high definition and ultra-high definition video are emerging. H.265 / High Efficiency Video Coding (HEVC) is already unable to meet the needs of the rapidly developing video applications. Therefore, JVET (Joint Video Exploration Team) has proposed the next-generation video coding standard, H.266 / Versatile Video Coding (VVC).

[0004] In H.266 / VVC, matrix-based intra prediction (MIP) is an intra prediction mode that is used to obtain an intra prediction block of a current block. Then, a low-frequency non-separable transform (LFNST) technique is used to determine the prediction residual of the current block. However, there is a problem that the LFNST technique has a negative impact on the coding efficiency of VVC when applied to MIP mode prediction. Summary of the Invention

[0005] In accordance with an embodiment of the present invention, there are provided an encoding method, a decoding method, an encoder, a decoder, and a storage medium capable of improving encoding / decoding efficiency.

[0006] The technical solutions of the embodiments of the present invention can be realized as follows.

[0007] In a first aspect, an embodiment of the present invention provides an encoding method applied to an encoder. The method may include:

[0008] Determining prediction parameters for the current block, the prediction parameters including a prediction mode parameter. When the prediction mode parameter indicates that a matrix-based intra prediction (MIP) mode is used to determine an intra prediction value of a chroma component of the current block, neighboring sample values ​​of the current block are obtained, and a MIP input sample value of the current block is determined based on the neighboring sample values ​​of the current block. Based on the MIP input sample values, the MIP weighting matrix and the shift parameters, a MIP prediction value of the chroma component of the current block is determined, where the MIP prediction value is a prediction value of some samples in the chroma component of the current block. An intra prediction value of the chroma component of the current block is determined by filtering the MIP prediction value. A prediction residual value of the chroma component of the current block is determined based on the intra prediction value of the chroma component of the current block. LFNST parameters are determined by performing a low-frequency non-separable transform (LFNST) on the prediction residual values. Encode and write the LFNST parameters into the bitstream.

[0009] In a second aspect, an embodiment of the present invention provides a decoding method applied in a decoder. The method may include:

[0010] The bitstream is parsed to obtain prediction parameters and low-frequency non-separable transform (LFNST) parameters of the current block, where the prediction parameters include a prediction mode parameter. When the prediction mode parameter indicates that a matrix-based intra prediction (MIP) mode is used to determine an intra prediction value of a chroma component of the current block, neighboring sample values ​​of the current block are obtained, and an MIP input sample value of the current block is determined based on the neighboring sample values ​​of the current block. Based on the MIP input sample values, the MIP weighting matrix and the shift parameters, a MIP prediction value of the chroma component of the current block is determined, where the MIP prediction value is a prediction value of some samples in the chroma component of the current block. An intra prediction value of the chroma component of the current block is determined by filtering the MIP prediction value. If the LFNST parameter indicates to perform LFNST on the current block, a reconstructed transform coefficient block of the current block is determined, and a second transform coefficient block is obtained by performing LFNST on at least some of the reconstructed transform coefficients in the reconstructed transform coefficient block. A reconstructed residual block of the chroma component of the current block is obtained by performing a first transform on the second transform coefficient block. A reconstructed block of the chroma component of the current block is determined based on the intra predicted value of the chroma component of the current block and the reconstructed residual block.

[0011] In a third aspect, an embodiment of the present invention provides an encoder, the encoder including a first decision unit, a first prediction unit, a first transform unit, and a coding unit.

[0012] The first determining unit is configured to determine prediction parameters for a current block, where the prediction parameters include a prediction mode parameter. The first determination unit is further configured to obtain neighboring sample values ​​of the current block and determine MIP input sample values ​​of the current block based on the neighboring sample values ​​of the current block when the prediction mode parameter indicates to determine the intra predicted value of the chroma component of the current block using a matrix-based intra prediction (MIP) mode. The first prediction unit is configured to determine a MIP predicted value of a chroma component of the current block based on a MIP input sample value, a MIP weighting matrix and a shift parameter, and to determine an intra predicted value of the chroma component of the current block by filtering the MIP predicted value, where the MIP predicted value is a predicted value of some samples in the chroma component of the current block. The first transform unit is configured to determine a prediction residual value of a chroma component of the current block based on an intra-prediction value of the chroma component of the current block, and determine LFNST parameters by performing a low-frequency non-separable transform (LFNST) on the prediction residual value. The encoding unit is configured to encode and write the LFNST parameters into a bitstream.

[0013] In a fourth aspect, an embodiment of the present invention provides an encoder, the encoder including a first memory and a first processor. The first memory is configured to store a computer program executable by the first processor. The first processor is configured to execute a computer program to perform the method according to the first aspect.

[0014] In a fifth aspect, an embodiment of the present invention provides a decoder, the decoder including a parsing unit, a second decision unit, a second prediction unit and a second transform unit.

[0015] The parsing unit is configured to parse the bitstream to obtain prediction parameters and low-frequency non-separable transform (LFNST) parameters of a current block, where the prediction parameters include a prediction mode parameter. The second determination unit is configured to obtain neighboring sample values ​​of the current block and determine MIP input sample values ​​of the current block based on the neighboring sample values ​​of the current block when the prediction mode parameter indicates to determine an intra-prediction value of the chroma component of the current block using a matrix-based intra-prediction (MIP) mode. The second prediction unit is configured to determine a MIP prediction value of the chroma component of the current block based on the MIP input sample value, the MIP weighting matrix and the shift parameter, and to determine an intra prediction value of the chroma component of the current block by filtering the MIP prediction value, where the MIP prediction value is a prediction value of some samples in the chroma component of the current block. The second transform unit is configured to, when the LFNST parameter indicates to perform LFNST on the current block, determine a reconstructed transform coefficient block of the current block, perform LFNST on at least some of the reconstructed transform coefficients in the reconstructed transform coefficient block to obtain a second transform coefficient block, and perform a first transform on the second transform coefficient block to obtain a reconstructed residual block of a chroma component of the current block. The second determining unit is further configured to determine, based on the intra predicted value of the chroma component of the current block and the reconstructed residual block, a reconstructed block of the chroma component of the current block.

[0016] In a sixth aspect, an embodiment of the present invention provides a decoder, the decoder including a second memory and a second processor. The second memory is configured to store a computer program executable by the second processor. The second processor is configured to execute the computer program to perform the method according to the second aspect.

[0017] In a seventh aspect, an embodiment of the present invention provides a computer storage medium storing a computer program which, when executed by a first processor, performs the method according to the first aspect, or, when executed by a second processor, performs the method according to the second aspect.

[0018] In the embodiment of the present invention, an encoding method, a decoding method, an encoder, a decoder and a storage medium are provided. At the encoder side, after determining a prediction parameter of a current block, when a prediction mode parameter indicates that a matrix-based intra prediction (MIP) mode is used to determine an intra prediction value of a chroma component of the current block, obtain neighboring sample values ​​of the current block, and determine a MIP input sample value of the current block according to the neighboring sample values ​​of the current block. According to the MIP input sample value, a MIP weighting matrix and a shift parameter, determine a MIP prediction value of a chroma component of the current block. By filtering the MIP prediction value, determine an intra prediction value of a chroma component of the current block. According to the intra prediction value of a chroma component of the current block, determine a prediction residual value of a chroma component of the current block. By performing a low-frequency non-separable transform (LFNST) on the prediction residual value, determine an LFNST parameter. Encode the LFNST parameter and write it into a bitstream. At the decoder side, after parsing the bitstream to obtain prediction parameters and LFNST parameters of the current block, if the prediction mode parameter indicates that the MIP mode is used to determine the intra predicted value of the chroma component of the current block, obtain neighboring sample values ​​of the current block, and determine MIP input sample values ​​of the current block according to the neighboring sample values ​​of the current block; determine MIP predicted values ​​of the chroma component of the current block according to the MIP input sample values, the MIP weighting matrix, and the shift parameter; determine intra predicted values ​​of the chroma component of the current block by filtering the MIP predicted values; determine a reconstructed transform coefficient block of the current block, if the LFNST parameter indicates that LFNST is performed on the current block, and obtain a second transform coefficient block by performing LFNST on at least some reconstructed transform coefficients in the reconstructed transform coefficient block; obtain a reconstructed residual block of the chroma component of the current block by performing a first transform on the second transform coefficient block.Based on the intra prediction value of the chroma component of the current block and the reconstructed residual block, the reconstructed block of the chroma component of the current block is determined. In this way, in the MIP mode, while ensuring the performance of the encoding and decoding, the complexity is reduced, and at the same time, the memory space required for the encoding and decoding process is reduced, and the encoding and decoding efficiency is effectively improved. In addition, when the LFNST technology is applied to the MIP mode prediction, the introduction of the MIP parameters makes the LFNST more flexible, and the encoding and decoding efficiency can be further improved. [Brief description of the drawings]

[0019] [Figure 1] FIG. 1 is a schematic diagram illustrating the structure of an encoder according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a schematic diagram illustrating the structure of a decoder according to an embodiment of the present invention. [Diagram 3] FIG. 3 is a flow chart illustrating an encoding method according to an embodiment of the present invention. [Figure 4] FIG. 4 is a flow chart illustrating another encoding method according to an embodiment of the present invention. [Diagram 5] FIG. 5 is a flow chart illustrating a matrix-based intra prediction (MIP) prediction process according to an embodiment of the present invention. [Figure 6] FIG. 6 is a flowchart illustrating obtaining an intra-prediction block using MIP mode according to an embodiment of the present invention. [Figure 7] FIG. 7 is a schematic diagram illustrating a flow of performing a low-frequency non-separable transform (LFNST) according to an embodiment of the present invention. [Figure 8] FIG. 8 is a flowchart illustrating encoding of LFNST parameters according to an embodiment of the present invention. [Figure 9] FIG. 9 is a flow chart illustrating a decoding method according to an embodiment of the present invention. [Figure 10] FIG. 10 is a flow chart illustrating analysis of LFNST parameters according to an embodiment of the present invention. [Figure 11] FIG. 11 is a schematic diagram showing another flow of performing LFNST according to an embodiment of the present invention. [Figure 12] FIG. 12 is a schematic diagram illustrating another encoder structure according to an embodiment of the present invention. [Figure 13] FIG. 13 is a schematic diagram showing a specific hardware structure of an encoder according to an embodiment of the present invention. [Figure 14] FIG. 14 is a schematic diagram illustrating another decoder structure according to an embodiment of the present invention. [Figure 15] FIG. 15 is a schematic diagram showing a specific hardware structure of a decoder according to an embodiment of the present invention. [Figure 16] FIG. 16 is a schematic diagram showing the structure of a transmitting device according to an embodiment of the present invention. [Figure 17] FIG. 17 is a schematic diagram illustrating a structure of a target device according to an embodiment of the present invention. [Figure 18] FIG. 18 is a schematic diagram showing the structure of a communication system according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0020] In order to more fully understand the features and technical contents of the embodiments of the present invention, the implementation of the embodiments of the present invention will be described in detail below with reference to the accompanying drawings, in which the accompanying drawings are used for explanation purposes only and are not intended to limit the embodiments of the present invention.

[0021] All technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art of the present invention unless otherwise defined. The terms used herein are for the purpose of describing the embodiments of the present invention and are not intended to limit the present invention.

[0022] In the following description, although "some embodiments" refers to a subset of all possible embodiments, it can be understood that "some embodiments" may be the same or different subsets of all possible embodiments, and they may be combined with each other unless there is a contradiction. It should also be noted that the terms "first / second / third" in the embodiments of the present invention merely distinguish similar objects and do not imply a specific order of objects. "First / second / third" may be interchanged with a specific order or priority when permitted, so that the embodiments of the present invention described herein may be implemented in an order other than the order illustrated or described herein.

[0023] With the development of video compression technology, the International Telecommunication Union-Telecommunication Sector (ITU-T) and the International Organization for Standardization (ISO) / International Electrotechnical Commission (IEC) have launched a standardization project entitled Versatile Video Coding (VVC) to develop a new generation of video coding standard, aiming to improve the performance of VVC by about 50% compared with the latest H.265 / HEVC standard when coding high-quality video with one or more of the following characteristics: high resolution, high frame rate, high bit depth, high dynamic range, wide color gamut, and omnidirectional viewing angle. ITU-T SG16WP3 and JVET (Joint Video Exploration Team) of ISO / IEC JTC1 / SC29 / WG11 are in charge of the standardization project. Various intra- and inter-prediction modes have been demonstrated to achieve high compression efficiency when encoding high-quality video, and have been adopted into the VVC Working Draft (WD).

[0024] Matrix-based intra-prediction (MIP) mode is one intra-prediction mode. In VVC, an encoder or decoder can invoke the MIP mode to obtain an intra-prediction block of a current block. Then, a low-frequency non-separable transform (LFNST) can be applied to the prediction residual of the current block as a secondary transform (or called the second transform).

[0025] In the related art, LFNST can be applied to MIP mode prediction with a fixed configuration of LFNST transform kernel candidate set and LFNST transposition indication parameter, which adversely affects the encoding and decoding efficiency of VVC.

[0026] Based on the above, in one embodiment of the present invention, an encoding method is provided. The basic idea of ​​the encoding method is as follows: Determine prediction parameters of a current block. The prediction parameters include a prediction mode parameter. If the prediction mode parameter indicates that a MIP mode is used to determine an intra-prediction value of a chroma component of the current block, obtain neighboring sample values ​​of the current block, and determine a MIP input sample value of the current block according to the neighboring sample values ​​of the current block. Determine a MIP predicted value of a chroma component of the current block according to the MIP input sample value, a MIP weighting matrix, and a shift parameter. Determine an intra-prediction value of a chroma component of the current block by filtering the MIP predicted value. Determine a prediction residual value of a chroma component of the current block according to the intra-prediction value of a chroma component of the current block. Determine an LFNST parameter by performing LFNST on the prediction residual value. Encode the LFNST parameter and write it into a bitstream. In another embodiment of the present invention, a decoding method is provided. The basic idea of ​​the decoding method is as follows: Analyze a bitstream to obtain prediction parameters and LFNST parameters of a current block. The prediction parameters include a prediction mode parameter. If the prediction mode parameter indicates to determine an intra predicted value of a chroma component of the current block using a MIP mode, obtain neighboring sample values ​​of the current block and determine MIP input sample values ​​of the current block based on the neighboring sample values ​​of the current block. Determine an MIP predicted value of a chroma component of the current block based on the MIP input sample values, a MIP weighting matrix and a shift parameter. Determine an intra predicted value of a chroma component of the current block by filtering the MIP predicted value. If the LFNST parameter indicates to perform LFNST on the current block, determine a reconstructed transform coefficient block of the current block and obtain a second transform coefficient block by performing LFNST on at least some of the reconstructed transform coefficients in the reconstructed transform coefficient block.A reconstructed residual block of the chroma component of the current block is obtained by performing a first transform on the second transform coefficient block. A reconstructed block of the chroma component of the current block is determined based on the intra-predicted value of the chroma component of the current block and the reconstructed residual block. In this way, in the MIP mode, while ensuring the performance of encoding and decoding, the complexity can be reduced, and at the same time, the storage space required for the encoding and decoding process can be reduced, and the encoding and decoding efficiency can be effectively improved. In addition, when the LFNST technology is applied to the MIP mode prediction, the introduction of the MIP parameters makes the LFNST more flexible, and the encoding and decoding efficiency can be further improved.

[0027] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

[0028] Referring to Fig. 1, Fig. 1 shows a block diagram of an example of the structure of an encoder system according to an embodiment of the present invention. As shown in Fig. 1, the encoder 100 includes a division unit 101, a prediction unit 102, a first adder 107, a transformation unit 108, a quantization unit 109, an inverse quantization unit 110, an inverse transformation unit 111, a second adder 112, a filtering unit 113, a decoded picture buffer (DPB) unit 114 and an entropy coding unit 115. Here, the input of the encoder 100 may be a video consisting of a sequence of images or a single still image, and the output of the encoder 100 may be a bitstream representing a compressed version of the input video.

[0029] The division unit 101 divides an image in the input video into one or more coding tree units (CTUs). The division unit 101 may divide the image into tiles, and further divide a tile into one or more bricks, where a tile or a brick may include one or more complete and / or partial CTUs. The division unit 101 may also form one or more slices, where a slice may include one or more tiles in an image in raster order, or one or more tiles in an image covering a rectangular area. The division unit 101 may further form one or more sub-images, where a sub-image may include one or more slices, tiles, or bricks.

[0030] In the encoding process of the encoder 100, the division unit 101 sends the CTU to the prediction unit 102. The prediction unit 102 may generally include a block division unit 103, a motion estimation (ME) unit 104, a motion compensation (MC) unit 105, and an intra prediction unit 106. Specifically, the block division unit 103 further divides the input CTU into smaller coding units (Coding Units, CUs) by iteratively using quad-tree division, binary tree division, and ternary tree division. The prediction unit 102 may obtain an inter prediction block of the CU by using the ME unit 104 and the MC unit 105. The intra prediction unit 106 may obtain an intra prediction block of the CU by using various intra prediction modes including MIP mode. As an example, a rate-distortion optimized motion estimation method can be invoked by ME unit 104 and MC unit 105 to obtain an inter-predicted block, and a rate-distortion optimized mode decision method can be invoked by intra prediction unit 106 to obtain an inter-predicted block.

[0031] The prediction unit 102 outputs a prediction block of a CU, and the first adder 107 calculates the difference between the CU at the output of the division unit 101 and the prediction block of the CU, that is, the residual CU. The transform unit 108 reads the residual CU and performs one or more transform operations on the residual CU to obtain coefficients. The quantization unit 109 quantizes the coefficients and outputs quantized coefficients (i.e., levels). The inverse quantization unit 110 outputs reconstructed coefficients by performing a scaling operation on the quantized coefficients. The inverse transform unit 111 performs one or more inverse transforms corresponding to the transform in the transform unit 108, and outputs a reconstructed residual. The second adder 112 calculates a reconstructed CU by adding the reconstructed residual and the prediction block of the CU from the prediction unit 102. The second adder 112 also sends its output to the prediction unit 102 to be used as a reference for intra prediction. After all CUs in an image or sub-image are reconstructed, the filtering unit 113 performs in-loop filtering on the reconstructed image or sub-image, where the filtering unit 113 includes one or more filters, such as a deblocking filter, a sample adaptive offset (SAO) filter, an adaptive loop filter (ALF), a luma mapping with chroma scaling (LMCS) filter, and a neural-network-based filter, etc. Or, the filtering unit 113 performs in-loop filtering on one or more target samples in a CU when it determines that the CU is not used as a reference for encoding other CUs.

[0032] The output of the filtering unit 113 is a decoded image or sub-image, which is buffered in the DPB unit 114. The DPB unit 114 outputs the decoded image or sub-image based on timing and control information. Here, the image stored in the DPB unit 114 can further be used as a reference for inter-prediction or intra-prediction by the prediction unit 102. Finally, the entropy coding unit 115 converts parameters required for decoding an image (such as control parameters and supplementary information) from the encoder 100 into a binary format, and writes the binary format into a bitstream based on the syntax structure of each data unit, i.e. the encoder 100 finally outputs a bitstream.

[0033] Furthermore, the encoder 100 may include a first processor and a first memory for recording a computer program. The first processor reads and executes the computer program, which causes the encoder 100 to read the input video and generate a corresponding bitstream. The encoder 100 may also be a computing device with one or more chips. The above units implemented as integrated circuits on the chips have similar connections and data exchange functions as the corresponding units in FIG. 1.

[0034] Referring to Fig. 2, Fig. 2 shows a block diagram of an example of the structure of a decoder system according to an embodiment of the present invention. As shown in Fig. 2, the decoder 200 may include an analysis unit 201, a prediction unit 202, a scaling unit 205, a transformation unit 206, an adder 207, a filtering unit 208 and a decoded image buffer unit 209. Here, the input of the decoder 200 may be a bitstream representing a compressed version of a video or a single still image, and the output of the decoder 200 may be a decoded video consisting of a sequence of images or a single decoded still image.

[0035] The input bitstream of the decoder 200 may be the bitstream generated by the encoder 100. The parsing unit 201 parses the input bitstream and obtains values ​​of syntax elements from the input bitstream. The parsing unit 201 converts the binary representations of the syntax elements into numerical values ​​to obtain one or more decoded images, and sends the numerical values ​​to units in the decoder 200. The parsing unit 201 may further parse one or more syntax elements from the input bitstream to display the decoded images.

[0036] In the decoding process of the decoder 200, the parsing unit 201 sends the values ​​of the syntax elements and one or more variables set or determined based on the values ​​of the syntax elements to units in the decoder 200 for obtaining one or more decoded images.

[0037] The prediction unit 202 determines a prediction block of a current decoded block (e.g., a CU). Here, the prediction unit 202 may include a motion compensation unit 203 and an intra prediction unit 204. Specifically, if it is indicated that an inter decoding mode is used to decode the current decoded block, the prediction unit 202 sends related parameters from the analysis unit 201 to the motion compensation unit 203 to obtain an inter prediction block. If it is indicated that an intra prediction mode (including a MIP mode indicated based on a MIP mode index value) is used to decode the current decoded block, the prediction unit 202 sends related parameters from the analysis unit 201 to the intra prediction unit 204 to obtain an intra prediction block.

[0038] The scaling unit 205 has the same function as the inverse quantization unit 110 in the encoder 100. The scaling unit 205 performs a scaling operation on the quantized coefficients (i.e., levels) from the analysis unit 201 to obtain reconstructed coefficients.

[0039] The transform unit 206 has the same function as the inverse transform unit 111 in the encoder 100. The transform unit 206 performs one or more transform operations (i.e., the inverse operations of one or more transform operations performed by the inverse transform unit 111 in the encoder 100) to obtain a reconstructed residual.

[0040] The adder 207 performs an addition operation on its inputs (the prediction block from the prediction unit 202 and the reconstructed residual from the transform unit 206) to obtain a reconstructed block of the current decoded block. The reconstructed block is also sent to the prediction unit 202 to be used as a reference for other blocks coded under intra prediction modes.

[0041] After all CUs in an image or sub-image are reconstructed, the filtering unit 208 performs in-loop filtering on the reconstructed image or sub-image. The filtering unit 208 includes one or more filters, such as a deblocking filter, a sample adaptive offset (SAO) filter, an adaptive loop filter (ALF), a luma mapping with chroma scaling (LMCS) filter, and a neural network-based filter. Or, when the filtering unit 208 determines that the reconstructed block is not used as a reference for decoding other blocks, it performs in-loop filtering on one or more target samples in the reconstructed block. The output of the filtering unit 208 is a decoded image or sub-image, and the decoded image or sub-image is buffered in the DPB unit 209. The DPB unit 209 outputs the decoded image or sub-image based on timing and control information. The image stored in the DPB unit 209 can further be used as a reference for inter prediction or intra prediction by the prediction unit 202.

[0042] In addition, the decoder 200 may include a second processor and a second memory for recording a computer program. When the second processor reads and executes the computer program, the decoder 200 reads the input bitstream and generates a corresponding decoded video. The decoder 200 may also be a computing device with one or more chips. The above units implemented as integrated circuits on the chips have similar connections and data exchange functions as the corresponding units in FIG. 2.

[0043] It should be noted that the encoding method according to the embodiment of the present invention is mainly applied to the intra prediction unit 106 and the transform unit 108 of the encoder 100, and the decoding method according to the embodiment of the present invention is mainly applied to the intra prediction unit 204 and the transform unit 206 of the decoder 200. That is, the embodiment of the present invention may be applied to the encoder, the decoder, or even simultaneously to both the encoder and the decoder, which is not limited in this specification.

[0044] Also, please note that when an embodiment of the present invention is applied to encoder 100, the "current block" specifically refers to the image block currently to be encoded in intra prediction (which may also be referred to as the "encoding block"), and when an embodiment of the present invention is applied to decoder 200, the "current block" specifically refers to the image block currently to be decoded in intra prediction (which may also be referred to as the "decoding block").

[0045] When the encoder 100 can improve the encoding performance by achieving a better prediction effect through the encoding method according to an embodiment of the present invention, the decoder 200 can accordingly improve the decoding performance by improving the video decoding quality through the decoding method according to an embodiment of the present invention.

[0046] In one embodiment of the present invention, refer to Fig. 3, which shows a flowchart of an encoding method according to an embodiment of the present invention. As shown in Fig. 3, the method may include:

[0047] S301: Determine prediction parameters of a current block, where the prediction parameters include a prediction mode parameter.

[0048] It should be noted that the image to be coded may be divided into a number of image blocks, and each image block to be coded may be called a coding block, where each coding block may include a first image component, a second image component, and a third image component. The current block is the coding block on which prediction of the first image component, the second image component, or the third image component in the image of the video is currently being performed.

[0049] A prediction of a first image component is performed on the current block, and assuming that the first image component is a luma component, i.e., the image component to be predicted is a luma component, the current block can also be called a luma block, and a predicted value of the luma component of the current block is obtained; or, a prediction of a second image component is performed on the current block, and assuming that the second image component is a chroma component, i.e., the image component to be predicted is a chroma component, the current block can also be called a chroma block, and a predicted value of the chroma component of the current block is obtained.

[0050] It should be noted that the prediction parameters may include prediction mode parameters, and the prediction mode parameters are used to indicate the prediction mode adopted by the current block, and different prediction modes correspond to different prediction mode parameters. Here, a simple decision strategy may be adopted for determining the prediction mode parameters, for example, based on the magnitude of a distortion value. A complex decision strategy may also be adopted, for example, based on the result of rate distortion optimization (RDO). The embodiment of the present invention is not limited thereto. In general, the RDO method is adopted to determine the prediction mode parameters of the current block, i.e., to determine the prediction parameters of the current block.

[0051] Specifically, in some embodiments, determining prediction parameters of the current block may include: By pre-encoding the current block using multiple prediction modes, rate-distortion cost values ​​corresponding to the multiple prediction modes are obtained. An optimal rate-distortion cost value is selected from the obtained multiple rate-distortion cost values, and the original prediction parameters of the prediction mode corresponding to the optimal rate-distortion cost value are determined as prediction parameters of the current block.

[0052] That is, the encoder may pre-encode the current block using a plurality of prediction modes for the current block. Here, the plurality of prediction modes generally include an inter prediction mode, a conventional intra prediction mode, and a non-conventional intra prediction mode. The conventional intra prediction mode may include a direct current (DC) mode, a planar (PLANAR) mode, an angular intra prediction mode, and the like. The non-conventional intra prediction mode may include an MIP mode, a cross-component linear model prediction (CCLM) mode, an intra block copy (IBC) mode, a palette (PLT) mode, and the like. The inter prediction mode may include a conventional inter prediction mode and a geometrical (GEO) partitioning for inter blocks mode, and the like.

[0053] In this way, after pre-encoding the current block using multiple prediction modes respectively, rate-distortion cost values ​​corresponding to the multiple prediction modes can be obtained. Then, an optimal rate-distortion cost value is selected from the obtained multiple rate-distortion cost values ​​(generally, the minimum rate-distortion cost value is the optimal rate-distortion cost value), and the original prediction parameters of the prediction mode corresponding to the optimal rate-distortion cost value are determined as the prediction parameters of the current block.

[0054] Furthermore, the current block is pre-encoded using a plurality of prediction modes, so that distortion values ​​corresponding to the plurality of prediction modes can be obtained. Next, an optimal distortion value is selected from the obtained plurality of distortion values, and the original prediction parameters of the prediction mode corresponding to the optimal distortion value are determined as the prediction parameters of the current block. In this way, the encoder finally performs predictive encoding on the current block using the prediction mode indicated by the prediction mode parameter in the determined prediction parameters. This allows a smaller prediction residual to be obtained, and improves the encoding efficiency.

[0055] S302: When the prediction mode parameter indicates that a matrix-based intra prediction (MIP) mode is used to determine an intra prediction value of a chroma component of the current block, obtain neighboring sample values ​​of the current block, and determine a MIP input sample value of the current block based on the neighboring sample values ​​of the current block.

[0056] However, for the current block, in the embodiment of the present invention, MIP mode is used to perform intra prediction on the chroma component of the current block, in this process, first obtain the adjacent sample values ​​of the current block, and then determine the MIP input sample values ​​of the current block according to the adjacent sample values ​​of the current block.

[0057] It should be understood that for MIP mode, MIP core parameters need to be configured first. Here, the MIP core parameters can include the type of current block (represented by mipSizeId), the number of reference samples for each edge (represented by boundySize), the number of MIP input samples (represented by inSize), and the size of the MIP prediction block output from the matrix multiplication (arranged as predSize×predSize). In MIP mode, the current block can be classified into three types based on the width and height of the current block, and mipSizeId may be equal to 0, 1, or 2. In this specification, mipSizeId indicates the type of the current block, i.e., the "block size index value of the current block" shown in the embodiment of the present invention. Different mipSizeIds result in different numbers of reference samples (the number of reference samples required for each edge is boundySize), the number of MIP input samples (inSize), and the size of the MIP prediction block output from the matrix multiplication (arranged as predSize×predSize).

[0058] Also, note that the prediction parameters can further include a size parameter of the current block in addition to the prediction mode parameter. The size parameter of the current block can include the width (represented by nTbW) and height (represented by nTbH) of the current block. Also, the block size index value (i.e., mipSizeId) of the current block can be determined based on the size parameter of the current block.

[0059] In one possible embodiment, determining a block size index value for the current block based on the size parameter of the current block comprises: setting the block size index value of the current block to 0 if both the width and height of the current block are equal to 4; Otherwise, if both the width and height of the current block are equal to 8 or one of the width and height of the current block is equal to 4, setting the block size index value of the current block to 1; Otherwise, if the current block is of another size, setting the block size index value of the current block to 2.

[0060] In another possible embodiment, determining a block size index value for the current block based on the size parameter of the current block comprises: setting the block size index value of the current block to 0 if both the width and height of the current block are equal to 4; Otherwise, if one of the width and height of the current block is equal to 4, then set the block size index value of the current block to 1; Otherwise, if the current block is of another size, setting the block size index value of the current block to 2.

[0061] In this way, based on the block size index value of the current block, the number of adjacent boundary reference samples (variable is boundarySize) and the size of the MIP prediction block (variable is predSize, and the size of the MIP prediction block is predSize×predSize) can be determined by the look-up table (LUT) shown in Table 1, and the number of MIP input sample values ​​(represented by inSize) used in the process of MIP matrix multiplication is calculated. The calculation formula is as follows:

number

[0062] The rules for operation of the operators in equation (1) are the same as those defined in the ITU-TH.265 standard; for example, "==" is the logical "equal to" operator.

[0063] [Table 1]

[0064] Thus, according to Table 1, when the value of mipSizeId is equal to 0, the value of boundarySize is equal to 2 and the value of predSize is equal to 4. That is, in this case, the reference samples are the two samples selected at each edge, and the output of the matrix multiplication is a 4x4 MIP prediction block.

[0065] When the value of mipSizeId is equal to 1, the value of boundarySize is equal to 4, and the value of predSize is equal to 4. That is, in this case, the reference samples are the four samples selected at each edge, and the output of the matrix multiplication is a 4×4 MIP prediction block.

[0066] When the value of mipSizeId is equal to 2, the value of boundarySize is equal to 4, and the value of predSize is equal to 8, i.e., in this case, the reference samples are the four samples selected at each edge, and the output of the matrix multiplication is an 8x8 MIP prediction block.

[0067] Furthermore, based on the block size index value of the current block, the values ​​of boundarySize, inSize and predSize can be determined simultaneously by the lookup table shown in Table 2.

[0068] [Table 2]

[0069] Thus, according to Table 2, when the value of mipSizeId is equal to 0, the value of boundarySize is equal to 2, the value of inSize is equal to 4, and the value of predSize is equal to 4. That is, in this case, the reference samples are the two samples selected at each edge, the number of input samples of the matrix multiplication is 4, and the output of the matrix multiplication is a 4×4 MIP prediction block.

[0070] When the value of mipSizeId is equal to 1, the value of boundarySize is equal to 4, the value of inSize is equal to 8, and the value of predSize is equal to 4. That is, in this case, the reference samples are the four samples selected at each edge, the number of input samples of the matrix multiplication is 8, and the output of the matrix multiplication is a 4×4 MIP prediction block.

[0071] When the value of mipSizeId is equal to 2, the value of boundarySize is equal to 4, the value of inSize is equal to 7, and the value of predSize is equal to 8. That is, in this case, the reference samples are the four samples selected at each edge, the number of input samples of the matrix multiplication is 7, and the output of the matrix multiplication is an 8×8 MIP prediction block.

[0072] In addition, after configuring the MIP core parameters, a reference sample needs to be obtained. Then, a MIP input sample value is constructed according to the reference sample and the MIP core parameters. Here, the reference sample refers to the adjacent sample value of the current block, and can include the left adjacent sample value of the current block and the upper adjacent sample value of the current block. That is, the MIP input sample value of the current block can be determined according to the left adjacent sample value and the upper adjacent sample value of the current block.

[0073] In some embodiments, determining MIP input sample values ​​of the current block based on neighboring sample values ​​of the current block may include:

[0074] A block size index value for the current block is determined based on the size parameter of the current block.

[0075] A first temporary reference value is obtained by downsampling the neighboring sample values ​​of the current block.

[0076] If the block size index value of the current block is within a preset range, determine a second constant value based on a bit depth of adjacent sample values ​​of the current block, set a value corresponding to index 0 in the MIP input sample value to a difference between the second constant value and a value corresponding to index 0 in the first temporary reference value, and set a value corresponding to index i in the MIP input sample value to a difference between a value corresponding to index i in the first temporary reference value and a value corresponding to index 0 in the first temporary reference value, where i is an integer greater than 0.

[0077] If the block size index value of the current block is outside a preset range, set a value corresponding to index j in the MIP input sample value to a difference between a value corresponding to index (j+1) in the first temporary reference value and a value corresponding to index 0 in the first temporary reference value, where j is an integer equal to or greater than 0.

[0078] Furthermore, if the block size index value of the current block is within a preset range, the method may further include: Determine a second constant value based on the bit depth of the adjacent sample values ​​of the current block, set a value corresponding to index 0 in the MIP input sample value to a difference between the value corresponding to index 0 in the first temporary reference value and the second constant value, and set a value corresponding to index i in the MIP input sample value to a difference between the value corresponding to index i in the first temporary reference value and the value corresponding to index 0 in the first temporary reference value, where i is an integer greater than 0.

[0079] It should be noted that after downsampling the adjacent sample values ​​of the current block, a first temporary reference value can be obtained. For the first temporary reference value, specifically, after downsampling the adjacent sample values ​​of the current block, the filtered adjacent sample values ​​can be buffered in a buffer (represented by pTemp). The value corresponding to index 0 in the first temporary reference value refers to pTemp[0], and the value corresponding to index i in the first temporary reference value refers to pTemp[i].

[0080] It should also be noted that whether the value of the size parameter of the current block is within a preset range can be determined based on whether the block size index value (represented by mipSizeId) of the current block is within a preset range. Specifically, when mipSizeId=0 or 1, it indicates that the block size index value of the current block is within the preset range, i.e., the value of the size parameter of the current block is within the preset range. When mipSizeId=2, it indicates that the block size index value of the current block is outside the preset range, i.e., the value of the size parameter of the current block is not within the preset range.

[0081] In other words, the MIP input sample value is determined by the buffer (represented by pTemp), the block size index value of the current block (represented by MipSizeId), and the bit depth (represented by BitDepth) of the neighboring sample values ​​of the current block, and the number of input samples corresponding to the MIP input sample value is related to the block size index value of the current block. Finally, a value (represented by p[x]) corresponding to index x in the MIP input sample value can be obtained.

[0082] Furthermore, in some embodiments, determining the second constant value based on a bit depth of adjacent sample values ​​of the current block may include: Set the second constant value to a power of 2. The exponent of the power is an integer equal to the bit depth of the adjacent sample values ​​of the current block minus 1.

[0083] Or, in some embodiments, determining the second constant value based on a bit depth of adjacent sample values ​​of the current block may include: The second constant value is obtained by binary left-shifting "1". The number of left-shifted bits is equal to the bit depth of the adjacent sample values ​​of the current block minus 1.

[0084] In other words, after the bit depth (represented by BitDepth) of the adjacent sample values ​​of the current block is obtained, the second constant value can be expressed as 1<<(BitDepth-1) or 2^(BitDepth-1). In this way, when the value of the size parameter of the current block is within a preset range, the MIP input sample value of the current block can be determined in combination with the second constant value.

[0085] Also, it should be noted that the MIP input sample is a matrix vector used in matrix multiplication operation. The current related technical solution is as follows: According to a buffer (represented by pTemp), a type of a current block (i.e., a block size index value of the current block represented by mipSizeId), a bit depth (represented by BitDepth) of adjacent sample values ​​of the current block, and the number of MIP input samples, a value (represented by p[x]) corresponding to an index x in the MIP input sample value is finally obtained.

[0086] Specifically, the process of constructing the x-th input sample value p[x] is as follows:

[0087] In one possible embodiment, if the block size index value of the current block is within a preset range, the construction process may include: The value corresponding to index 0 in the MIP input sample values ​​is obtained by calculating the difference between the second constant value and the value corresponding to index 0 in the first temporary reference value. A value corresponding to index i in the MIP input sample values ​​is obtained by performing a subtraction operation on a value corresponding to index i in the first temporary reference value and a value corresponding to index 0 in the first temporary reference value, where i is a positive integer greater than 0 and less than N, and N represents the number of elements included in the input sample matrix.

[0088] In an embodiment of the invention, the minuend in the difference calculation is set equal to a second constant value, and the subtrahend in the difference calculation is set equal to a value corresponding to index 0 in the first temporary reference value.

[0089] In other words, when mipSizeId=0 or 1, p[0] is obtained by subtracting pTemp[0] from 1<<(BitDepth-1), and when x is not 0, p[x] is obtained by subtracting pTemp[0] from pTemp[x]. The details are as follows.

number

[0090] In another possible embodiment, if the block size index value of the current block is within a preset range, the construction process may include: The value corresponding to index 0 in the MIP input sample values ​​is obtained by calculating the difference between the value corresponding to index 0 in the first temporary reference value and a second constant value. A value corresponding to index i in the MIP input sample values ​​is obtained by performing a subtraction operation on a value corresponding to index i in the first temporary reference value and a value corresponding to index 0 in the first temporary reference value, where i is a positive integer greater than 0 and less than N, and N represents the number of elements included in the input sample matrix.

[0091] In an embodiment of the present invention, the minuend in the difference calculation is set equal to a value corresponding to index 0 in the first temporary reference value, and the subtrahend in the difference calculation is set equal to a second constant value.

[0092] In other words, when mipSizeId=0 or 1, if x is equal to 0, the value corresponding to index 0 in the MIP input sample value (denoted by p[0]) is obtained by subtracting the second constant value (i.e., 1<<(BitDepth-1)) from the value corresponding to index 0 in the first temporary reference value (i.e., pTemp[0]). If x is not equal to 0, the value corresponding to index x in the MIP input sample value (denoted by p[x]) is obtained by subtracting the value corresponding to index 0 in the first temporary reference value (i.e., pTemp[0]) from the value corresponding to index x in the first temporary reference value (i.e., pTemp[x]).

[0093] The specific details are as follows:

number

[0094] In another possible embodiment, if the block size index value of the current block is outside a preset range, the construction process may include: A value corresponding to index i in the MIP input sample values ​​is obtained by performing a subtraction operation on a value corresponding to index (i+1) in the first temporary reference value and a value corresponding to index 0 in the first temporary reference value, where i is a positive integer greater than or equal to 0 and less than N, and N represents the number of elements in the input sample matrix.

[0095] In other words, when mipSizeId=2, the value corresponding to index 0 in the first temporary reference value, i.e., pTemp[0], may be ignored. Next, the value corresponding to index x in the MIP input sample value (denoted as p[x]) is obtained by subtracting the value corresponding to index 0 in the first temporary reference value (i.e., pTemp[0]) from the value corresponding to index (x+1) in the first temporary reference value (i.e., pTemp[x+1]), where x is a positive integer equal to or greater than 0. The specific details are as follows:

number

[0096] Here, still taking the 4×4 current block as an example, four values ​​are stored in the buffer pTemp. That is, the first temporary reference value includes a value corresponding to index 0 (i.e., pTemp[0]), a value corresponding to index 1 (i.e., pTemp[1]), a value corresponding to index 2 (i.e., pTemp[2]), and a value corresponding to index 3 (i.e., pTemp[3]). In this case, four MIP input sample values ​​represented by p[x] can be determined according to equation (2), equation (3), or equation (4), where x=0, 1, 2, 3. Here, the four MIP input sample values ​​can also form a 1×4 MIP input sample matrix.

[0097] In addition, in the embodiments of the present invention, it is also possible to obtain the value of p[x] by a unified calculation method without determining the value of mipSizeId. In some embodiments, determining the MIP input sample value of the current block based on the neighboring sample values ​​of the current block can include:

[0098] A first temporary reference value is obtained by downsampling the neighboring sample values ​​of the current block. A second constant value is determined based on a bit depth of adjacent sample values ​​of the current block, and the second temporary reference value is obtained by buffering the second constant value within one data unit after the first temporary reference value. Set the value corresponding to index j in the MIP input sample values ​​to the difference between the value corresponding to index (j+1) in the second temporary reference values ​​and the value corresponding to index 0 in the second temporary reference values, where j is an integer equal to or greater than 0.

[0099] In other words, in an embodiment of the present invention, a second constant value (i.e., (1<<(BitDepth-1))) is added as an additional element to the end of the buffer pTemp. In this case, there is no need to take into account the size parameter of the current block (i.e., there is no need to take into account the value of mipSizeId), and p[x] can be directly set as p[x]=pTemp[x+1]-pTemp[0], where x=0,...,inSize-1.

[0100] In this way, after determining the MIP input sample values ​​of the current block based on the neighboring sample values ​​of the current block, the MIP predicted value of the current block can be further determined.

[0101] S303: Determine a MIP predicted value of the chroma component of the current block based on the MIP input sample value, the MIP weighting matrix and the shift parameter.

[0102] The MIP predictor is a prediction of some samples in the chroma component of the current block.

[0103] In addition, the shifting parameter may include a shifting offset parameter and a shifting number parameter. In this case, as shown in FIG 4, S303 may include the following steps:

[0104] S303-1: Determine a product of the sum of the MIP input sample values ​​and the shift offset parameter based on the value of the shift offset parameter, where the value of the shift offset parameter is a fixed constant.

[0105] In addition, the shift offset parameter may be called a shifting compensation parameter or an offset factor, and may be expressed as fO. In an embodiment of the present invention, the value of the shift offset parameter may be set to a fixed constant, for example, 32, 46, 56, or 66. Furthermore, the value of the shift offset parameter may be set to be associated with a shift offset parameter table, and the value of the shift offset parameter may be determined by a look-up table. This specification is not limited thereto.

[0106] S303-2: A first constant value is determined based on the value of the shift number parameter.

[0107] S303-3: A first offset value is set to the difference between the first constant value and the above multiplication product.

[0108] In addition, the shift number parameter may be called a shifting factor, a number of shift bits, or a weight shifting value, and may be represented as sW, shift, or weight shift. In an embodiment of the present invention, the shift number parameter is represented as sW. In addition, the first offset may be represented as oW, and the first offset is related to both the shift number parameter and the shift offset parameter.

[0109] In an embodiment of the present invention, the value of the shift number parameter may be set to a fixed constant, such as 5, 6, or 7. Furthermore, the value of the shift number parameter may be set to be associated with a shift number parameter table, and the value of the shift number parameter may be determined by a look-up table. This specification is not limited thereto.

[0110] S303-4: Determine the MIP weighting matrix for the current block based on the prediction parameters.

[0111] It should be noted that the prediction parameters may include prediction mode parameters and may also include size parameters of the current block. In the prediction parameters, if the prediction mode parameter indicates that the MIP mode is used to determine the intra prediction value of the chroma component of the current block, a weighting matrix table may be established in advance, and the weighting matrix table may be stored in a memory or storage unit. The memory or storage unit may be integrated in the encoder or may be provided separately. In this way, based on the block size index value (mipSizeId) of the current block and the MIP mode index value (modelId), the MIP weighting matrix (also called MIP weight matrix or MIP matrix) that needs to be used for the current block may be determined by a look-up table, and is represented by mWeight[x][y]. The block size index value (mipSizeId) of the current block is determined based on the size parameter of the current block, and the size of the MIP weighting matrix mWeight[x][y] is related to the block size index value of the current block as shown in Table 3.

[0112] In the MIP weighting matrix shown in Table 3, the number of columns is the number of input samples of the matrix multiplication equal to inSize, and the number of rows is the number of output samples of the matrix multiplication equal to predSize × predSize, and the MIP weighting matrix of the current block can be determined.

[0113] [Table 3]

[0114] In the encoder, a shift number parameter table can be pre-established, and the shift number parameter table is also stored in a memory or storage unit. The memory or storage unit here can be integrated in the encoder or provided separately. In the embodiment of the present invention, the shift number parameter (sW) can be determined in several ways:

[0115] In one possible embodiment, the shift number parameter may be different for different block size index values ​​and different MIP mode index values. In some embodiments, the above method may further include:

[0116] When the MIP mode is used to determine the intra predicted value of the chroma component of the current block, the MIP mode index value of the current block is determined. According to the MIP mode index value, query a value corresponding to the MIP mode index value from a first preset lookup table, where the first preset lookup table is used to record the corresponding relationship between the MIP mode index value and the value of the shift number parameter. The queried value is determined as the value of the shift number parameter.

[0117] Note that the value of the shift number parameter can be queried based on the block size index value (represented by mipSizeId) and the MIP mode index value (modelId) of the current block. In the first pre-defined lookup table shown in Table 4, for different mipSizeId and modelId, the shift number parameter required in the matrix multiplication can be determined by the lookup table.

[0118] [Table 4]

[0119] However, on the encoder side, Table 4 needs to be stored in a memory or storage unit in the form of a look-up table, but the storage is costly and the look-up process is also costly. The value of the shift number parameter in Table 4 is related to both the block size and the MIP mode index value of the current block, which increases the memory occupation and the computational complexity.

[0120] In order to reduce memory occupation and calculation complexity, in the embodiment of the present invention, the method for determining the shift number parameter can be simplified.

[0121] In another possible embodiment, the value of the shift number parameter can be set to a fixed constant that is not related to the block size index value and the MIP mode index value. For example, the value of the shift number parameter can be set to 5 for different block size index values ​​and different MIP mode index values. Or, the value of the shift number parameter can be set to 6 for different block size index values ​​and different MIP mode index values. Or, the value of the shift number parameter can be set to 7 for different block size index values ​​and different MIP mode index values. In the embodiment of the present invention, it is preferable to set the value of the shift number parameter to 6, but is not limited thereto.

[0122] In another possible embodiment, for the value of the shift number parameter, the method further comprises: A block size index value for the current block is determined based on the size parameter of the current block. Based on the block size index value of the current block, a value of a shift number parameter is determined.

[0123] It should be noted that the block size index value of the current block may be determined based on the size parameter of the current block. In some embodiments, determining the block size index value of the current block based on the size parameter of the current block may include:

[0124] Set the block size index value of the current block to 0 if both the width and height of the current block are equal to 4. If the width and height of the current block are both equal to 8, or if either the width or height of the current block is equal to 4, set the block size index value of the current block to 1. If the width and height of the current block do not satisfy the above conditions, set the block size index value of the current block to 2.

[0125] In this manner, after determining the block size index value of the current block, the value of the shift number parameter can be further determined based on the block size index value of the current block.

[0126] Optionally, in some embodiments, determining a value of a shift number parameter based on a block size index value of the current block may include: It is determined that if the block size index value is equal to 0, 1 or 2, respectively, the value of the shift number parameter corresponding to the block size index value of the current block is equal to 5, 6 or 5, respectively.

[0127] Optionally, in some embodiments, determining a value of a shift number parameter based on a block size index value of the current block may include: Set the value of the shift number parameter equal to the ratio of the width or height of the current block to a first preset value that corresponds to the block size index value of the current block.

[0128] Here, the first preset value represents the number of MIP input sample values ​​obtained from the boundary of the current block. In this case, the method may also include: Determine that if the block size index value of the current block is equal to 0, 1 or 2, respectively, a first preset value corresponding to the block size index value of the current block is equal to 2, 4 or 4, respectively.

[0129] In other words, when the first preset value represents the number of MIP input sample values ​​taken from the boundary of the current block, when the block size index value of the current block is equal to 0, the corresponding first preset value is equal to 2, when the block size index value of the current block is equal to 1, the corresponding first preset value is equal to 4, and when the block size index value of the current block is equal to 2, the corresponding first preset value is equal to 4. In this way, the value of the shift number parameter can be determined based on the ratio of the width or height of the current block to the corresponding first preset value.

[0130] Optionally, in some embodiments, determining a value of a shift number parameter based on a block size index value of the current block may include: Set the value of the shift number parameter equal to the ratio of the width or height of the current block to a second preset value that corresponds to the block size index value of the current block.

[0131] Wherein, the second preset value represents the size of the MIP prediction block of the current block obtained by directly using the MIP weighting matrix. In this case, the method may also include: Determine that if the block size index value of the current block is equal to 0, 1 or 2, respectively, the second preset value corresponding to the block size index value of the current block is equal to 4, 4 or 8, respectively.

[0132] In other words, when the second preset value represents the size of the MIP prediction block of the current block obtained by directly using the MIP weighting matrix, when the block size index value of the current block is equal to 0, the corresponding second preset value is equal to 4, when the block size index value of the current block is equal to 1, the corresponding second preset value is equal to 4, and when the block size index value of the current block is equal to 2, the corresponding second preset value is equal to 8. In this way, the value of the shift number parameter can be determined based on the ratio of the width or height of the current block to the corresponding second preset value.

[0133] In another possible embodiment, the shift number parameter table can be minimized, and still determine the value of the shift number parameter by the method of a lookup table. Optionally, in some embodiments, determining the value of the shift number parameter based on the block size index value of the current block can include: According to the block size index value, query a value corresponding to the block size index value from a second preset lookup table, where the second preset lookup table is used to record the corresponding relationship between the block size index value and the value of the shift number parameter. The queried value is determined as the value of the shift number parameter.

[0134] Note that the value of the shift number parameter can be queried based only on the block size index value (represented by mipSizeId) of the current block. In the second pre-defined lookup table shown in Table 5, each block size index value can correspond to a fixed value. That is, each block size or each block size set can have a fixed shift number parameter value as shown in Table 5.

[0135] [Table 5]

[0136] According to Table 5, it can be determined that if the block size index value of the current block is equal to 0, 1 or 2, respectively, the value of the shift number parameter corresponding to the block size index value is equal to 5, 6 or 5, respectively.

[0137] In the above embodiment, by simplifying the method of determining the shift number parameter, particularly by minimizing the shift number parameter table or fixing the value of the shift number parameter, the storage amount of the look-up table can be minimized, so that the memory occupied by the storage of the original shift number parameter table of the MIP mode can be reduced without increasing the calculation complexity.

[0138] In the encoder, a shift offset parameter table can be pre-established, and the shift offset parameter table is also stored in a memory or storage unit. The memory or storage unit here can be integrated in the encoder or provided separately. In the embodiment of the present invention, the shift offset parameter (fO) can be determined in several ways:

[0139] In one possible embodiment, the shift offset parameter may be different for different block size index values ​​and different MIP mode index values. In some embodiments, the above method may further include: When the MIP mode is used to determine the intra predicted value of the chroma component of the current block, the MIP mode index value of the current block is determined. According to the MIP mode index value, a value corresponding to the MIP mode index value is queried from a third preset lookup table, where the third preset lookup table is used for recording the corresponding relationship between the MIP mode index value and the value of the shift offset parameter. The queried value is determined as the value of the shift offset parameter.

[0140] Note that the value of the shift offset parameter can be queried based on the block size index value (represented by mipSizeId) and the MIP mode index value (modelId) of the current block. In the third pre-defined lookup table shown in Table 6, for different mipSizeId and modelId, the shift offset parameter required in the matrix multiplication can be determined by the lookup table.

[0141] [Table 6]

[0142] However, at the encoder side, Table 6 needs to be stored in a memory or storage unit in the form of a look-up table, but the storage is costly and the look-up process is also costly. The value of the shift offset parameter in Table 6 is related to both the block size and the MIP mode index value of the current block, which increases the memory occupation and the computational complexity.

[0143] In order to reduce memory occupancy and computational complexity, in the embodiment of the present invention, the method for determining the shift offset parameter can be simplified.

[0144] In another possible embodiment, the value of the shift offset parameter can be set to a fixed constant that is not related to the block size index value and the MIP mode index value. Typically, the fixed constant is in the range of 0 to 100. For example, the value of the shift offset parameter can be set to 32 for different block size index values ​​and different MIP mode index values. Or, the value of the shift offset parameter can be set to 46 for different block size index values ​​and different MIP mode index values. Or, the value of the shift offset parameter can be set to 56 for different block size index values ​​and different MIP mode index values. Or, the value of the shift offset parameter can be set to 66 for different block size index values ​​and different MIP mode index values. In the embodiment of the present invention, it is preferable to set the value of the shift offset parameter to 32, but the present invention is not limited thereto.

[0145] In another possible embodiment, for the value of the shift offset parameter, the method may further include: A block size index value for the current block is determined based on the size parameter of the current block. A value of a shift offset parameter is determined based on the block size index value of the current block.

[0146] It should be noted that the block size index value of the current block can be determined based on the size parameter of the current block. Then, the value of the shift offset parameter can be further determined based on the block size index value of the current block.

[0147] Optionally, in some embodiments, determining a value of a shift offset parameter based on a block size index value of the current block may include: Determine that if the block size index value is equal to 0, 1 or 2, respectively, the value of the shift offset parameter corresponding to the block size index value of the current block is equal to 34, 23 or 46, respectively.

[0148] Optionally, in some embodiments, the shift offset parameter table can be further minimized and still determine the shift factor by the lookup table method. Optionally, in some embodiments, determining a value of the shift offset parameter based on the block size index value of the current block can include: According to the block size index value, query a value corresponding to the block size index value from a fourth preset lookup table, the fourth preset lookup table being used for recording the correspondence relationship between the block size index value and the value of the shift offset parameter. The queried value is determined as the value of the shift offset parameter.

[0149] It should be noted that the value of the shift offset parameter can be queried based only on the block size index value (represented by mipSizeId) of the current block. In the fourth pre-defined lookup table shown in Table 7, each block size index value can correspond to a fixed value. That is, each block size or each block size set can have a fixed shift offset parameter value as shown in Table 7.

[0150] [Table 7]

[0151] According to Table 7, it can be determined that when the block size index value of the current block is equal to 0, 1 or 2, respectively, the value of the shift offset parameter corresponding to the block size index value is equal to 34, 23 or 46, respectively.

[0152] In the above embodiment, by simplifying the method of determining the shift offset parameter, particularly by minimizing the shift offset parameter table or fixing the value of the shift offset parameter, the storage amount of the look-up table can be minimized, so that the memory occupied by the storage of the original shift offset parameter table of the MIP mode can be reduced without increasing the calculation complexity.

[0153] Thus, after determining the shift offset parameter (fO) and the shift number parameter (sW), optionally, in some embodiments, determining the first constant value based on the value of the shift number parameter may include: Set the first constant value to a power of 2. The exponent of the power is an integer and is equal to the value of the shift number parameter minus 1.

[0154] Optionally, in some embodiments, determining a first constant value based on the value of the shift number parameter may include: The first constant value is obtained by binary left-shifting a "1". The number of bits shifted left is equal to the value of the shift number parameter minus 1.

[0155] In other words, after the shift number parameter (sW) is obtained, the first constant value may be expressed as 1<<(sW-1), or may be expressed as 2^(sW-1). In this case, when the value of the shift number parameter is set to 6, the first constant value equal to 32 can be obtained.

[0156] JPEG2025061393000013.jpg37170

[0157] In this way, under the MIP mode, the MIP weighting matrix, the MIP input sample value, the shift number parameter and the first offset can be obtained, which facilitates the subsequent determination of the MIP predicted value of the current block.

[0158] S303-5: Determine a MIP predicted value of the chroma component of the current block based on the MIP weighting matrix, the MIP input sample value, the shift number parameter and a first offset.

[0159] Here, the MIP prediction value is a prediction value of some samples in the chroma component of the current block.

[0160] Determining a MIP predicted value of the current block based on the MIP weighting matrix, the MIP input sample value, the shift number parameter, and the first offset may include:

[0161] A first weighted sum of the MIP weighting matrix and the MIP input sample values ​​is calculated. A first sum of the first weighted sum and the first offset is calculated. A first right shift value is obtained by binary right shifting the first sum, the number of bits shifted right equal to the value of the shift number parameter. Set the MIP predicted value of the current block to the sum of the first right shift value and the value corresponding to index 0 in the first temporary reference value, where the first temporary reference value is obtained by downsampling the neighboring sample values ​​of the current block.

[0162] Specifically, under MIP mode, a MIP weighting matrix (represented by mWeight), a shift number parameter (represented by sW) and a shift offset parameter (represented by fO) can be determined based on the block size index value (represented by mipSizeId) and the MIP mode index value (represented by modelId) of the current block. Then, the MIP input sample value (represented by p[x]), mWeight, sW and fO are input into the matrix multiplication to obtain the MIP predicted value (represented by predMip[x][y]) output from the matrix multiplication. The samples in predMip[x][y] are arranged in the form of a matrix / array based on predSize×predSize. The calculation formula is as follows:

number

[0163] [x][y] represents the position coordinate of the sample, x represents the horizontal direction, y represents the vertical direction, inSize represents the number of input samples, p[i] represents the value corresponding to index i in the MIP input sample value, pTemp[0] represents the value corresponding to index 0 in the first temporary reference value, ">>" represents the binary right shift operator, and "<<" represents the binary left shift operator, where x=0,...,predSize-1,y=0,...,predSize-1. In this way, the MIP prediction block can be obtained by calculating predMip[x][y] based on the above formula (5).

[0164] In some embodiments, when sW has a fixed value of 6 and fO has a fixed value of 32, the calculation is as follows:

number

[0165] In addition, in order to obtain the MIP predicted block of the current block, clipping is required for the predicted values ​​of the samples in the MIP predicted block. Next, it is determined whether the MIP predicted block is to be transposed. If the determination result is Yes, the predicted sample values ​​in the MIP predicted block need to be transposed, and the transposed MIP predicted block is determined as the MIP predicted block of the current block. If the determination result is No, it is not required to transpose the predicted sample values ​​in the MIP predicted block, and the MIP predicted block is directly determined as the MIP predicted block of the current block, so that the MIP predicted value of the current block can be obtained, and the MIP predicted value is the predicted value of some samples in the current block.

[0166] S304: Determine an intra prediction value of the chroma component of the current block by filtering the MIP prediction value.

[0167] Determining an intra predicted value of the current block by filtering the MIP predicted value may include:

[0168] It is determined whether the size parameter of the MIP prediction block and the size parameter of the current block are the same. If a size parameter of the MIP prediction block and a size parameter of the current block are the same, set the intra prediction block of the current block equal to the MIP prediction block, where the MIP prediction block contains predicted sample values ​​for all sample positions in the current block. When a size parameter of the MIP prediction block is different from a size parameter of the current block, a filtered prediction block is obtained by filtering the MIP prediction block, and the filtered prediction block is set as an intra prediction block of the current block.

[0169] Here, the filtering may include upsampling or low-pass filtering.

[0170] It should be noted that the MIP prediction block is composed of MIP prediction values. After obtaining the MIP prediction block, the size parameter of the MIP prediction block only includes two types, namely, a 4×4 MIP prediction block and an 8×8 MIP prediction block, so the size parameter of the current block and the size parameter of the MIP prediction block may or may not be the same. In other words, the sample value corresponding to the MIP prediction block does not necessarily completely fill the current block, so the MIP prediction block may need to be upsampled to generate the final prediction value. That is, whether the size parameter of the MIP prediction block and the size parameter of the current block are the same is determined to determine whether the MIP prediction block is upsampled.

[0171] It is also shown that if the size parameters of the MIP prediction block and the size parameters of the current block are the same, that is, if the width and height of the MIP prediction block are both the same as the current block, upsampling of the MIP prediction block is not required. In this case, the MIP prediction block can be directly embedded in the current block, that is, there is no vacant sample in the current block after embedding. In this case, the intra prediction value of each sample in the current block can be directly set to the prediction value of each sample in the MIP prediction block, as shown below.

number

[0172] [x][y] represents the position coordinate of a sample, where x represents the horizontal direction and y represents the vertical direction, predSamples[x][y] represents the intra prediction value corresponding to the sample whose position coordinate is [x][y] in the current block, and predMip[x][y] represents the prediction value corresponding to the sample whose position coordinate is [x][y] in the MIP prediction block. Thus, according to formula (7), the MIP prediction block predMip[x][y] can be directly used as the intra prediction block predSamples[x][y] of the current block, that is, as the intra prediction value of at least one sample in the current block.

[0173] It is noted that when the size parameter of the MIP prediction block is different from that of the current block, i.e., when at least one of the width and height of the MIP prediction block is different from that of the current block, the MIP prediction block cannot completely fill the current block, i.e., there are vacant samples in the current block after filling, and in this case, it is indicated that the MIP prediction block needs to be filtered. In other words, when upsampling is required in both the horizontal and vertical directions, the MIP prediction block is first upsampled horizontally and then upsampled vertically to obtain a first upsample block that can be represented by predSamples[x][y]. Then, the MIP prediction block is upsampled vertically and then upsampled horizontally to obtain a second upsample block that can be represented by predSamplesTemp[x][y]. Finally, the intra-prediction value of the chroma component of the current block is finally obtained by weighted averaging predSamples[x][y] and predSamplesTemp[x][y], i.e., the intra-prediction block of the current block is obtained.

[0174] Exemplarily, if the edge length nTbS (where S can be replaced by W and H respectively) of the current block is equal to the edge length predSize of predMip (where predSize is only related to the blocksizeIdx of the current block), the MIP prediction block can be directly set as the intra prediction block of the current block. Otherwise, the MIP prediction block needs to be filtered to obtain the intra prediction block of the current block. Here, taking the current block of 4×4 as an example, if the size parameter of the current block and the size parameter of the MIP prediction block are the same, there is no need to filter the MIP prediction block, and the MIP prediction block can be directly set as the intra prediction block of the current block, and the intra prediction value of at least one sample in the current block is obtained.

[0175] Briefly, for the prediction process under MIP mode, the input data of MIP mode may include the position (xTbCmp, yTbCmp) of the current block, the MIP mode index value applied to the current block (which may be represented by modelId), the height (nTbH) of the current block, the width (nTbW) of the current block, and a transposition indication flag indicating whether transposition is required (i.e., MIP transposition indication parameter (which may be represented by isTransposed)). The output data of MIP mode may include an intra-predicted block of the current block. The predicted value corresponding to the sample coordinate [x][y] in the intra-predicted block is predSamples[x][y], where x=0, 1, ... nTbW-1, y=0, 1, ... nTbH-1.

[0176] As shown in FIG. 5, the MIP prediction process specifically includes the following steps:

[0177] S501: Configure core parameters.

[0178] For S501, the MIP core parameters may include the type of the current block (represented by mipSizeId), the number of reference samples for each edge (represented by boundySize), the number of MIP input samples (represented by inSize), and the size of the MIP prediction block output from the matrix multiplication (arranged as predSize×predSize). Based on the size of the current block, the current block may be classified into three types, and the type of the current block is recorded by mipSizeId, where mipSizeId may be equal to 0, 1, or 2. Also, different types of the current block have different numbers of reference samples and sizes of the MIP prediction block output from the matrix multiplication.

[0179] S502: A reference sample is obtained.

[0180] Regarding S502, when predicting the current block, the upper adjacent block and the left adjacent block of the current block are both coded blocks, and the original reference sample of MIP mode is the reconstructed value of the sample in the upper row and the left column adjacent to the current block. In other words, the process of obtaining the upper adjacent reference sample (represented by refT) and the left adjacent reference sample (represented by refL) of the current block is the process of obtaining reference samples.

[0181] S503: Construct an input sample.

[0182] Regarding S503, this step is used for input of matrix multiplication, and specifically may include: S5031: Obtain a reference sample. S5032: Build a reference sample buffer. S5033: Derive input samples for matrix multiplication.

[0183] For S5031, the process of obtaining the reference sample is a process of downsampling the reference sample. For S5031, in constructing the reference sample buffer, there are two kinds of filling methods, which can include the following: S5032-1: A method for filling a buffer without requiring transposition. S5032-2: A method for filling a buffer that requires transposition.

[0184] S504: An intra prediction value is generated.

[0185] Regarding S504, this step is used to obtain a MIP predicted value of the current block, and specifically may include: S5041: Construct a MIP prediction block of output samples of the matrix multiplication. S5042: Perform clipping on the MIP predicted block of output samples of the matrix multiplication. S5043: Transpose the MIP prediction block of output samples of the matrix multiplication. S5044: Generate MIP predicted values.

[0186] For S5041, the process of constructing a MIP prediction block may specifically include: S5041-1: Obtain the weighting matrix. S5041-2: Get shift parameters. S5041-3: Perform a matrix multiplication operation.

[0187] In other words, in the process of constructing the MIP prediction block, it is necessary to obtain not only the weighting matrix but also the shift parameters (including the shift offset parameter and the shift number parameter). Then, perform a matrix multiplication operation to obtain the MIP prediction block of the output sample of the matrix multiplication. Regarding S5044, the generation of the final MIP prediction value includes two situations, which can specifically include the following: S5044-1: Generate a prediction that does not require upsampling. S5044-2: Generate a prediction that requires upsampling.

[0188] In this way, after the above four steps S501 to S504, in the embodiment of the present invention, an intra predicted value of at least one sample in the chroma component of the current block can be obtained.

[0189] S305: Determine a prediction residual value of the chroma component of the current block based on the intra prediction value of the chroma component of the current block.

[0190] In addition, after obtaining the intra-prediction value of the chroma component of the current block, the prediction residual value of the chroma component of the current block is obtained by calculating the difference between the actual value of the chroma component of the current block and the intra-prediction value of the chroma component of the current block, that is, the residual block of the current block is obtained.

[0191] S306: Determine LFNST parameters by performing a low frequency non-separable transform (LFNST) on the prediction residual values.

[0192] It should be noted that, for the transform unit 108 shown in FIG. 1, the transform unit 108 may perform a first transform on the residual block of the current block, for example, an integer transform that is initially designed based on a Discrete Cosine Transform (DCT), and may perform a second transform on the residual block of the current block, for example, a Low Frequency Non-Separable Transform (LFNST). Specifically, in some embodiments, determining LFNST parameters by performing LFNST on the prediction residual values ​​may include:

[0193] A residual block of the current block is determined based on the prediction residual value, and a first transform coefficient block is obtained by performing a first transform on the residual block. It is determined whether to perform LFNST on at least some of the transform coefficients in the first transform coefficient block. If the result of the determination is Yes, LFNST parameters are determined by performing LFNST on at least some of the transform coefficients in the first transform coefficient block.

[0194] In an embodiment of the present invention, the encoder performs a first transform (also called a "core transform", "primary transform" or "main transform") on prediction residual values ​​in a residual block to obtain a first transform coefficient block after the first transform. Then, the encoder determines LFNST parameters by performing LFNST (also called a "secondary transform" or "secondary transform") on some or all of the transform coefficients in the first transform coefficient block. Note that the first transform in this specification is a transform different from LFNST.

[0195] Also, note that LFNST is not performed on any current block. After the first transform, some conditions (e.g., the minimum value of size parameters of the current block, the block size index value of the current block, etc.) need to be checked to determine whether to perform LFNST on at least some transform coefficients in the first transform coefficient block. Optionally, in some embodiments, determining whether to perform LFNST on at least some transform coefficients in the first transform coefficient block may include: When determining an intra prediction value of the chroma component of the current block using the MIP mode, the minimum value of the size parameters of the current block is determined. Based on the minimum value, it is determined whether or not to perform LFNST on at least some of the transform coefficients in the first transform coefficient block.

[0196] Furthermore, determining whether to perform LFNST on at least some transform coefficients in the first transform coefficient block based on the minimum value may include: If the minimum value is equal to or greater than a first preset threshold, it is determined to perform LFNST on at least some of the transform coefficients in the first transform coefficient block.

[0197] Here, the first preset threshold may be used to represent a preset threshold for determining whether to perform LFNST. In the embodiment of the present invention, the first preset threshold may be set to, but is not limited to, 8.

[0198] In other words, when intra prediction is performed on the chroma component of the current block using the MIP mode, a minimum value of the size parameters of the current block may be determined. For example, when the minimum values ​​of the width and height are equal to or greater than 8, it may be determined to perform LFNST on at least some transform coefficients in the first transform coefficient block.

[0199] Optionally, in some embodiments, determining whether to perform LFNST on at least some transform coefficients in the first transform coefficient block may include: When the MIP mode is used to determine the intra prediction value of the chroma component of the current block, a block size index value of the current block is determined based on the size parameter of the current block. Based on the block size index value, it is determined whether to perform LFNST on at least some of the transform coefficients in the first transform coefficient block.

[0200] Furthermore, determining whether to perform LFNST on at least some of the transform coefficients in the first transform coefficient block based on the block size index value includes: determining to perform LFNST on at least some transform coefficients in the first transform coefficient block when the block size index value is equal to a second preset threshold.

[0201] Here, the second preset threshold may be used to represent a preset threshold for determining whether to perform LFNST. In an embodiment of the present invention, the second preset threshold may be set to, but is not limited to, one preset value (e.g., 2) or one of a plurality of preset values ​​(e.g., 1, 2).

[0202] In other words, when intra prediction is performed on the chroma component of the current block using the MIP mode, a block size index value (mipSizeId) of the current block may be determined based on a size parameter of the current block. For example, when the value of mipSizeId is equal to 2, it may be determined to perform LFNST on at least some transform coefficients in the first transform coefficient block.

[0203] Optionally, in some embodiments, determining whether to perform LFNST on at least some transform coefficients in the first transform coefficient block may include: A first cost result is calculated when LFNST is not performed on at least some transform coefficients in the first transform coefficient block. A second cost result is calculated when LFNST is performed on at least some of the transform coefficients in the first transform coefficient block. A determination is made as to whether to perform LFNST on at least some of the transform coefficients in the first transform coefficient block based on the first cost result and the second cost result.

[0204] Moreover, determining whether to perform LFNST on at least some of the transform coefficients in the first transform coefficient block based on the first cost result and the second cost result includes: determining to perform LFNST on at least some transform coefficients in the first transform coefficient block if the first cost result is greater than the second cost result.

[0205] It should be noted that in the embodiment of the present invention, a conventional rate-distortion optimization method may be adopted. First, a first cost result is calculated when LFNST is not performed on at least some transform coefficients in the first transform coefficient block. Then, a second cost result is calculated when LFNST is performed on at least some transform coefficients in the first transform coefficient block. The first cost result and the second cost result are compared, and if the first cost result is greater than the second cost result, it may be determined to perform LFNST on at least some transform coefficients in the first transform coefficient block.

[0206] In this way, if the result of the determination is Yes, i.e., if LFNST needs to be performed on at least some of the transform coefficients in the first transform coefficient block, the LFNST parameters can be determined by performing LFNST on at least some of the transform coefficients in the first transform coefficient block.

[0207] S307: Encode the LFNST parameters and write them into the bitstream.

[0208] Note that the LFNST parameter includes an LFNST index and may be represented by lfnst_index. In some embodiments, determining the LFNST parameter may include: When it is determined that LFNST is to be performed on at least some of the transform coefficients in the first transform coefficient block, the value of the LFNST index is set to be greater than zero. If it is determined that LFNST is not to be performed on at least some of the transform coefficients in the first transform coefficient block, the value of the LFNST index is set to 0.

[0209] Furthermore, regarding S307, encoding the LFNST parameters and writing them into the bitstream includes: and encoding and writing the LFNST index into the bitstream.

[0210] In other words, after determining the LFNST parameters, i.e., lfnst_index, if lfnst_index is equal to 0, it indicates that LFNST should not be performed. If lfnst_index is greater than 0, it indicates that LFNST needs to be performed. In the encoding process, lfnst_index can be encoded using Context-based Adaptive Binary Arithmetic Coding (CABAC) with the descriptor “ae(v)”.

[0211] For example, refer to Table 8, which shows an example of a description of a syntax structure of LFNST parameters according to the related art. In this case, the condition for determining whether to perform LFNST on at least some transform coefficient blocks in the first transform coefficient block may include at least one of treeType == DUAL_TREE_CHROMA, !IntraMipFlag[x0][y0], and Min(lfnstWidth, lfnstHeight) >= 16.

[0212] [Table 8]

[0213] Referring to Tables 9A and 9B, Tables 9A and 9B respectively show an example of a description of a syntax structure of LFNST parameters according to an embodiment of the present invention. In this case, the condition for determining whether to perform LFNST on at least some transform coefficient blocks in the first transform coefficient block may include at least one of intra_mip_flag[x0][y0]==1&&Min(lfnstWidth, lfnstHeight)>=8, intra_mip_flag[x0][y0]==1&&mipSizeId==2.

[0214] Specifically, as shown in the syntax structure of Table 9A, if the minimum value of the size parameter of the current block is 8 or more (i.e., both the width and the height are 8 or more), the lfnst_index of the current block may be encoded using the MIP mode (i.e., when intra_mip_flag[x0][y0]==1). Or, the determination condition may be set such that both the width and the height are equal to or more than a first value (e.g., 8) and the sum of the width and the height is equal to or more than a second value (e.g., 16). As shown in the syntax structure of Table 9B, if the block size index value (mipSizeId) of the current block is equal to 2, the lfnst_index of the current block may be encoded using the MIP mode (i.e., when intra_mip_flag[x0][y0]==1). Or, the determination condition may be set such that mipSizeId is equal to one of a plurality of set values ​​(e.g., 1, 2).

[0215] [Table 9A]

[0216] [Table 9B]

[0217] Also, if it is determined that LFNST can be performed on the current block, an LFNST transform kernel (denoted as "kernel") to be used for the current block needs to be determined. In some embodiments, determining LFNST parameters by performing LFNST on the prediction residual values ​​may include: determining an LFNST transform kernel to be used for the current block, and determining the LFNST parameters by performing LFNST on the prediction residual values ​​using the LFNST transform kernel.

[0218] In an embodiment of the present invention, there are four transform kernel candidate sets for LFNST. The four transform kernel candidate sets may include set 0, set 1, set 2 and set 3. Here, in an embodiment of the present invention, the MIP parameters may be used to determine the LFNST transform kernel candidate set, and then select an LFNST transform kernel to be used for the current block from the LFNST transform kernel candidate set. Thus, in some embodiments, the above method may further include:

[0219] When the MIP mode is used to determine the intra prediction value of the chroma component of the current block, the MIP parameters are determined from the prediction parameters of the current block. Based on the MIP parameters, a set of LFNST transformation kernel candidates is determined. In the LFNST transform kernel candidate set, the transform kernel indicated by the LFNST index is set as the LFNST transform kernel to be used for the current block. The value of the LFNST index is set to indicate to perform LFNST on the current block and the index of the LFNST transform kernel in the LFNST transform kernel candidate set. The LFNST transformation kernel candidate set includes two or more predefined LFNST transformation kernels.

[0220] Note that the MIP parameters may include at least one of a MIP mode index value (represented by modelId) and a MIP transposition instruction parameter (represented by isTransposed). For the LFNST transform kernel used for the current block, if the value of the MIP transposition instruction parameter indicates that a transposition is to be performed on the sample input vector used for the MIP mode, then a matrix transposition needs to be performed on the selected transform kernel to obtain the LFNST transform kernel to be used for the current block.

[0221] Furthermore, in some embodiments, determining the LFNST transformation kernel candidate set based on the MIP parameters may include: Based on the MIP mode index value, a value of the LFNST intra-prediction mode index is determined. According to the value of the LFNST intra-prediction mode index, a fifth preset lookup table is used to determine an index value of the LFNST transform kernel candidate set. The LFNST transformation kernel candidate set is selected from a plurality of candidate LFNST transformation kernel candidate sets based on index values ​​of the LFNST transformation kernel candidate set.

[0222] Here, the LFNST intra prediction mode index may be represented by predModeIntra, and the index value of the LFNST transform kernel candidate set may be represented by lfnstTrSetIdx.

[0223] In addition, the fifth preset lookup table is as shown in Table 10. The value of predModeIntra can be determined based on the MIP mode index value (i.e., the value of modelId). Then, the value of lfnstTrSetIdx can be directly determined based on the value of predModeIntra in conjunction with Table 10, that is, the LFNST transformation kernel candidate set selected for the current block can be determined. Here, the value of lfnstTrSetIdx indicates the transformation kernel candidate set used for LFNST. Since the value of modelId can include 0, 1, 2, 3, 4, and 5, the value of predModeIntra is also 0, 1, 2, 3, 4, or 5. The correspondence relationship between predModeIntra and lfnstTrSetIdx is as follows:

[0224] [Table 10]

[0225] In one specific embodiment, determining a value of the LFNST intra-prediction mode index based on the MIP mode index value includes: Setting the value of the LFNST intra-prediction mode index equal to the MIP mode index value.

[0226] In other words, in an embodiment of the present invention, the value of predModeIntra can be set equal to the value of modelId, and then the value of lfnstTrSetIdx is directly determined based on the value of predModeIntra in conjunction with Table 10, i.e., the LFNST transformation kernel candidate set selected for the current block is determined. In addition, in another specific embodiment of the present invention, the value of modelId is directly mapped to the value of PLANAR mode, and then the value of predModeIntra corresponding to PLANAR mode is used to determine the value of lfnstTrSetIdx, i.e., the LFNST transformation kernel candidate set selected for the current block is determined.

[0227] In some embodiments, the LFNST parameters further include an LFNST coefficient. obtaining LFNST coefficients by performing LFNST on at least some of the transform coefficients in the first transform coefficient block.

[0228] Correspondingly, encoding the LFNST parameters and writing them into the bitstream includes: quantizing the LFNST coefficients to obtain quantized coefficients; encoding the quantized coefficients and writing them into a bitstream.

[0229] Note that, when lfnst_index is not equal to 0, that is, when it is determined to perform LFNST on the current block, LFNST coefficients can be obtained by performing LFNST on at least some transform coefficients in the first transform coefficient block. Then, the LFNST coefficients are quantized, coded, and written into a bitstream.

[0230] Let d[x][y] denote the first block of transform coefficients, where x=0,...,nTbW-1, y=0,...,nTbH-1, nTbW and nTbH denote the width and height of the current block, respectively. Let v[x] denote a block containing LFNST coefficients, where x=0,...,nLfnstOutSize-1, nLfnstOutSize=(nTbW>=8 && nTbH>=8)? 48 : 16.

[0231] In one possible implementation, LFNST may be performed on at least some transform coefficients in the first transform coefficient block in horizontal scan order to obtain LFNST coefficients.

number

[0232] log2LfnstSize is determined as follows:

number

[0233] In another possible embodiment, the scan order can include a horizontal scan order and a vertical scan order, where the determination of the scan order is related to a MIP transposition indication parameter (denoted as isTransposed).

[0234] Optionally, in some embodiments, obtaining LFNST coefficients by performing LFNST on at least some transform coefficients in the first transform coefficient block may include:

[0235] If the value of the MIP transposition indication parameter indicates that a transposition is to be performed on the sample input vector used for the MIP mode, LFNST coefficients are obtained by performing LFNST on at least some of the transform coefficients in the first transform coefficient block in vertical scan order. If the value of the MIP transposition indication parameter indicates that no transposition is to be performed on the sample input vector used for the MIP mode, then LFNST coefficients are obtained by performing LFNST on at least some of the transform coefficients in the first transform coefficient block in horizontal scan order.

[0236] Specifically, when the value of isTransposed is equal to 0, no transposition is performed on the sample input vector used for the MIP mode. Then, LFNST can be performed on at least some transform coefficients in the first transform coefficient block in horizontal scan order. Specifically, as follows.

number

[0237] A transposition is performed on the sample input vector used for the MIP mode when the value of isTransposed is equal to 1. Then, LFNST may be performed on at least some transform coefficients in the first transform coefficient block in vertical scan order. Specifically, as follows.

number

[0238] log2LfnstSize is determined as shown in equation (9) above.

[0239] Optionally, in some embodiments, obtaining LFNST coefficients by performing LFNST on at least some transform coefficients in the first transform coefficient block may include:

[0240] If the value of the MIP transposition indication parameter indicates that a transposition is to be performed on the sample input vector used for the MIP mode, LFNST coefficients are obtained by performing LFNST on at least some of the transform coefficients in the first transform coefficient block in horizontal scan order. If the value of the MIP transposition indication parameter indicates that no transposition is to be performed on the sample input vector used for the MIP mode, then LFNST coefficients are obtained by performing LFNST on at least some of the transform coefficients in the first transform coefficient block in vertical scan order.

[0241] Specifically, when the value of isTransposed is equal to 1, a transposition is performed on the sample input vector used for the MIP mode. Then, LFNST can be performed on at least some transform coefficients in the first transform coefficient block in the horizontal scan order, as shown in the above formula (10).

[0242] When the value of isTransposed is equal to 0, no transposition is performed on the sample input vector used for MIP mode, and LFNST can be performed on at least some transform coefficients in the first transform coefficient block in vertical scan order, as shown in equation (11) above.

[0243] log2LfnstSize is determined as shown in equation (9) above.

[0244] In addition, the LFNST according to the embodiment of the present invention can be applied to conventional intra prediction modes (e.g., angular intra prediction mode, DC mode, PLANAR mode, etc.) as well as the MIP mode, and the specific implementation method thereof is similar to that of the MIP mode.

[0245] In some embodiments, determining the LFNST transformation kernel candidate set based on the MIP parameters may include: The value of the LFNST intra-prediction mode index is determined based on the index value of the conventional intra-prediction mode. According to the value of the LFNST intra-prediction mode index, a sixth pre-defined lookup table is used to determine an index value of the LFNST transform kernel candidate set. Select an LFNST transformation kernel candidate set from the multiple candidate LFNST transformation kernel candidate sets based on index values ​​of the LFNST transformation kernel candidate set.

[0246] In addition, the sixth preset lookup table is as shown in Table 11. The value of predModeIntra may be determined based on the index value of the conventional intra prediction mode. Then, the value of lfnstTrSetIdx may be directly determined based on the value of predModeIntra in conjunction with Table 11, that is, the LFNST transform kernel candidate set selected for the current block may be determined. Here, to determine the value of lfnstTrSetIdx, the index value of the conventional intra prediction mode may be assigned to predModeIntra.

[0247] [Table 11]

[0248] The encoding method according to the embodiment of the present invention will be specifically described below with reference to the encoder 100 shown in FIG.

[0249] It should be understood that the current block (or called "encoding block") in the embodiment of the present invention may be a CU or a partition of a CU (e.g., a transform block). Specifically, when the intra prediction unit 106 determines to use a MIP mode to encode the current block, or when the intra prediction unit 106 evaluates a MIP mode for encoding the current block in a rate-distortion optimization manner, the intra prediction unit 106 obtains an intra prediction block of the current block. The intra prediction block of the current block includes an intra prediction value of at least one sample in the current block.

[0250] Specifically, the steps of the intra prediction unit 106 obtaining an intra prediction block using MIP mode are as follows:

[0251] First, the intra prediction unit 106 obtains one or more reference samples from neighboring samples of the current block, for example, by downsampling the neighboring samples or by directly extracting from the neighboring samples.

[0252] Then, the intra prediction unit 106 uses the obtained reference sample, MIP matrix and shift parameter to determine one or more partial prediction samples corresponding to sample positions in the current block. Here, the sample positions can be preset sample positions in the current block, and for example, the sample positions have uniform horizontal and vertical coordinate values. In addition, the shift parameters include a shift number parameter and a shift offset parameter, and the shift parameter can be used for offset operation in the process of obtaining the intra prediction value of the partial sample.

[0253] Finally, when the prediction samples corresponding to the partial samples of the current block are obtained, the intra prediction unit 106 needs to further obtain prediction samples corresponding to the remaining samples other than the partial samples in the current block. For example, the intra prediction unit 106 can use an interpolation filter to obtain the prediction samples corresponding to the remaining samples, and the inputs of the interpolation filter can be the partial samples and the adjacent samples.

[0254] As shown in Fig. 6, Fig. 6 shows a flow chart of obtaining an intra prediction block using MIP mode according to an embodiment of the present invention. This flow can be implemented in an encoder 100, and the "current block" here refers to the "coding block".

[0255] In step 601, the intra prediction unit 106 obtains adjacent samples of the current block, and as an example, the adjacent samples are marked as gray squares adjacent to the current block in step 601 shown in FIG. 6. The intra prediction unit 106 obtains one or more reference samples from the adjacent samples. In the example of step 601 shown in FIG. 6, optionally, the intra prediction unit 106 can calculate an average value of two adjacent samples, and use the average value as a reference sample. Alternatively, the intra prediction unit 106 can select one adjacent sample every other adjacent sample as a reference sample. For example, in the example of step 601 shown in FIG. 6, the intra prediction unit 106 selects four reference samples from the eight upper adjacent samples of the current block, and selects another four reference samples from the eight left adjacent samples of the current block.

[0256] The specific process of step 601 is as follows:

[0257] The intra prediction unit 106 obtains the width and height of the current block from the block division unit 103, and the width and height are represented by variables cbWidth and cbHeight, respectively. For example, the intra prediction unit 106 determines the intra prediction mode by invoking a rate-distortion optimization mode decision method. The current block is divided into one or more transform blocks. The variables nTbW and nTbH are the width and height of the transform blocks, respectively. When using the MIP mode as the intra prediction mode to obtain the intra prediction value of the current block, the intra prediction unit 106 determines the block size index value of MIP, i.e., a variable represented by mipSizeId.

[0258] Optionally, the intra prediction unit 106 determines the value of mipSizeId as follows. - If nTbW and nTbH are both equal to 4, set mipSizeId equal to 0. - Else, if cbWidth or cbHeight is equal to 4, set mipSizeId equal to 1. - Otherwise, set mipSizeId equal to 2.

[0259] Specifically, if the size parameter of the current block is 8x8 (ie, cbWidth and cbHeight are both equal to 8), then set mipSizeId equal to 2.

[0260] Optionally, the intra prediction unit 106 determines the value of mipSizeId as follows. - If nTbW and nTbH are both equal to 4, set mipSizeId equal to 0. - Else, if cbWidth or cbHeight is equal to 4, or cbWidth and cbHeight are both equal to 8, then set mipSizeId equal to 1. - Otherwise, set mipSizeId equal to 2.

[0261] Specifically, if the size parameter of the current block is 8x8, then set mipSizeId equal to 1.

[0262] In addition, the intra prediction unit 106 obtains the values ​​of boundarySize and predSize according to mipSizeId. -If mipSizeId is equal to 0, set boundarySize equal to 2 and set predSize equal to 4. - Else, if mipSizeId is equal to 1, then set boundarySize equal to 4 and set predSize equal to 4. - Otherwise (if mipSizeId is equal to 2), set boundarySize equal to 4 and set predSize equal to 8. The boundarySize represents the number of reference samples obtained from each of the upper adjacent reference samples and the left adjacent reference samples of the current block.

[0263] The intra prediction unit 106 may further obtain a variable isTransposed to indicate the order of the reference samples stored in the buffer pTemp. For example, isTransposed is equal to 0 indicates that the intra prediction unit 106 first presents the reference sample obtained from the upper neighboring reference sample of the current block, and then presents the reference sample obtained from the left neighboring reference sample. Otherwise, isTransposed is equal to 1 indicates that the intra prediction unit 106 first presents the reference sample obtained from the left neighboring reference sample of the current block, and then presents the reference sample obtained from the upper neighboring reference sample. The value of isTransposed is sent to the entropy coding unit 115 as one of the parameters of the MIP mode that needs to be coded and written into the bitstream.

[0264] The intra prediction unit 106 takes the value of inSize to indicate the number of reference samples used for MIP mode, as shown in equation (1) above.

[0265] The intra-prediction unit 106 invokes the following process to obtain reference samples (stored as an array p[x], where x is 0, . . . , inSize-1) using neighboring reference samples of the current block.

[0266] The intra prediction unit 106 obtains nTbW reference samples from the current block's upper neighboring reference samples (e.g., stored in array refT) and obtains nTbH reference samples from the current block's left neighboring reference samples (e.g., stored in array refL). The intra prediction unit 106 invokes downsampling on refT to obtain boundarySize reference samples and stores the boundarySize reference samples in refT. The intra prediction unit 106 invokes downsampling on refL to obtain boundarySize reference samples and stores the boundarySize reference samples in refL.

[0267] The intra prediction unit 106 may further obtain a variable isTransposed to indicate the order of the reference samples stored in the buffer pTemp. For example, isTransposed is equal to 0 (or FALSE) indicates that the intra prediction unit 106 first presents the reference samples obtained from the upper neighboring reference samples of the current block, and then presents the reference samples obtained from the left neighboring reference samples. Otherwise, isTransposed is equal to 1 (or TRUE) indicates that the intra prediction unit 106 first presents the reference samples obtained from the left neighboring reference samples of the current block, and then presents the reference samples obtained from the upper neighboring reference samples. The intra prediction unit 106 may determine the value of isTransposed by using a rate distortion optimization method or based on a comparison of the correlation between the neighboring reference samples and the current block. The value of isTransposed is sent to the entropy coding unit 115 as one of the parameters of the MIP mode that needs to be coded and written into the bitstream.

[0268] The intra-prediction unit 106 places elements in refT and refL into a buffer pTemp based on the order indicated by isTransposed.

[0269] Optionally, the intra prediction unit 106 obtains p[x] as follows, where x=0, . . . , inSize-1. - When mipSizeId is equal to 2, p[x] = pTemp[x + 1] - pTemp[0]. - Otherwise (mipSizeId is less than 2), p[0] = pTemp[0] - (1 << (BitDepth-1)), p[x] = pTemp[x] - pTemp[0]. BitDepth is the bit depth of the color component of the sample in the current block, where the color component can be one of the RGB components, one of the YUV components, or one of the YCbCr components, such as the Y component.

[0270] Alternatively, the intra prediction unit 106 may obtain p[x] as follows, where x=0, . . . , inSize-1: - When mipSizeId is equal to 2, p[x] = pTemp[x + 1] - pTemp[0]. - else (mipSizeId is less than 2), then p[0] = (1 << (BitDepth-1)) - pTemp[0], p[x] = pTemp[x] - pTemp[0].

[0271] Alternatively, the intra prediction unit 106 may not determine the value of mipSizeId, and may obtain the value of p[x] in a unified calculation manner. For example, (1<<(BitDepth-1)) is added to the buffer pTemp as an additional factor. In this case, the intra prediction unit 106 calculates p[x] as pTemp[x+1]-pTemp[0].

[0272] In step 602, the intra prediction unit 106 obtains a MIP prediction value of the current block using the reference samples and a MIP weighting matrix, which is selected from a predefined MIP weighting matrix group based on a MIP mode index value (represented by ModeId) corresponding to a MIP mode and a MIP block size index value (represented by mipSizeId).

[0273] The intra prediction unit 106 obtains MIP prediction values ​​(denoted as predMip[x][y]) of partial prediction samples corresponding to one or more sample positions in the current block. In the example of step 602 shown in FIG. 6, the partial prediction samples are samples marked as gray squares in the current block. The input of the prediction module 601 is the reference sample p[x] obtained in step 601, and the prediction module 601 calculates the partial prediction samples using the MIP weighting matrix and the shift parameters. The shift parameters include a shift number parameter and a shift offset parameter.

[0274] In one possible embodiment, the prediction module 601 obtains a predicted sample represented as predMip[x][y] by taking its coordinate as (x, y). The calculation formula of predMip[x][y] is as shown in Equation (5) or Equation (6).

[0275] In equation (5) or equation (6), mWeight[i][j] is a MIP weighting matrix, and the matrix elements may be predetermined constants or may be adaptively updated using a training method or the like. The input of the training method is one or more encoded images or blocks, or images from other bitstreams provided to the encoder 100 by an external device. fO is a shift offset parameter for determining oW, sW is a shift number parameter, p[i] is a MIP input sample value calculated using a reference sample, pTemp[0] is a value corresponding to index 0 in the first temporary reference value, and ">>" is a binary right shift operator defined in VVC. When it is finally determined to perform intra prediction on the current block using the MIP mode, the intra prediction unit 106 may further send mWeight[i][j] to the entropy coding unit 115. The entropy coding unit 115 may write mWeight[i][j] into one or more data units in the bitstream.

[0276] The prediction module 601 can determine the values ​​of sW and fO based on the size of the current block and the MIP mode used for the current block. In one example, the prediction module 601 obtains the values ​​of sW and fO using a lookup table.

[0277] Optionally, the prediction module 601 can determine sW based on the size parameter and MIP mode of the current block by utilizing Table 4 above.

[0278] Optionally, the prediction module 601 can also determine sW based on the size parameter of the current block by utilizing Table 5 above.

[0279] Alternatively, the prediction module 601 may also directly set sW to a constant value. For example, for blocks with various size parameters and different MIP modes, the prediction module 601 may set sW to 5. Or, for blocks with various size parameters and different MIP modes, the prediction module 601 may set sW to 6. Or, for blocks with various size parameters and different MIP modes, the prediction module 601 may set sW to 7.

[0280] Optionally, the prediction module 601 can determine fO based on the size parameter and MIP mode of the current block by utilizing Table 6 above.

[0281] Optionally, the prediction module 601 can also determine fO based on the size parameter of the current block by utilizing Table 7 above.

[0282] Optionally, the prediction module 601 can also directly set fO to a constant value (e.g., 0 to 100). For example, for blocks with various size parameters and different MIP modes, the prediction module 601 can set fO to 32. Or, the prediction module 601 sets fO to 46. Or, the prediction module 601 sets fO to 56. Or, the prediction module 601 sets fO to 66.

[0283] The intra-prediction unit 106 may perform a clipping operation on the MIP predicted value in predMip. If isTransposed is equal to 1 (or TRUE), the predSize×predSize array predMip[x][y], where x = 0, ..., predSize-1, y = 0, ..., predSize-1, is transformed to predTemp[y][x] = predMip[x][y], and then predMip = predTemp.

[0284] As an example, optionally, if the size parameter of the current block is 8x8 (ie, cbWidth and cbHeight are both equal to 8), the intra prediction unit 106 sets mipSizeId equal to 2 and obtains a predMip of 8x8.

[0285] Optionally, if the size parameter of the current block is 8x8 (ie, cbWidth and cbHeight are both equal to 8), intra prediction unit 106 sets mipSizeId equal to 1 and obtains predMip of 4x4.

[0286] In step 603, the intra prediction unit 106 obtains intra prediction values ​​of the current block (stored in an array as predSamples[x][y], where x=0,...,nTbW-1, y=0,...,nTbH-1) as follows:

[0287] When the intra prediction unit 106 determines that nTbW is greater than predSize or nTbH is greater than predSize, it invokes an upsampling process to obtain predSamples using predMip. The intra prediction unit 106 obtains prediction samples corresponding to remaining samples other than partial samples in the current block. In FIG. 6, the intra prediction unit 106 can use a filtering module 602 to obtain prediction samples corresponding to remaining samples other than partial samples in the current block. The input of the filtering module 602 is the sample marked as a gray square in step 602. The filtering module 602 can use one or more interpolation filters to obtain prediction samples corresponding to remaining samples other than partial samples in the current block using the input. For example, the input can include a reference sample and partial prediction samples of one or more sample positions in the current block. Or, the input can include adjacent samples and partial prediction samples of one or more sample positions in the current block. Or, the input can include a reference sample, adjacent samples and partial prediction samples of one or more sample positions in the current block. For example, optionally, if the size of the current block is 8x8 (i.e., cbWidth and cbHeight are both equal to 8), the intra prediction unit 106 determines that mipSizeId is equal to 1, and the intra prediction unit 106 obtains an 8x8 intra prediction block of the current block by applying an upsampling process to the 4x4 predMip.

[0288] - Otherwise, the intra prediction unit 106 sets the intra prediction block of the current block equal to the MIP prediction block of the current block, i.e., sets predSamples[x][y] (x=0,...,nTbW-1,y=0,...,nTbH-1) equal to predMip[x][y]. For example, specifically, if the size parameter of the current block is 8x8 (i.e., cbWidth and cbHeight are both equal to 8), the intra prediction unit 106 determines that mipSizeId is equal to 2, and obtains the predSamples of the current block whose size is 8x8 (i.e., cbWidth and cbHeight are both equal to 8).

[0289] Thus, according to the flow block diagram shown in FIG. 6, after step 603, the intra prediction unit 106 can obtain an intra prediction block of the current block (i.e., the CU), i.e., determine an intra prediction value of at least one sample of the current block.

[0290] The prediction unit 102 outputs an intra-predicted block of the current block. The first adder 107 calculates the difference between the current block and the intra-predicted block of the current block at the output of the division unit 101, that is, a residual block (i.e., a residual CU). The transformation unit 108 reads the residual block and obtains coefficients by performing one or more transformation operations on the residual block. The quantization unit 109 quantizes the coefficients and outputs quantized coefficients (i.e., levels).

[0291] The transform unit 108 performs a first transform on the residual block, for example an integer transform that is initially designed based on DCT. The transform unit 108 determines whether a secondary transform is allowed to be applied to the block. If so, the transform unit 108 further determines whether a secondary transform is allowed to be applied to the coefficients obtained after performing the first transform. The LFNST is an example of a secondary transform.

[0292] Referring to Fig. 7, Fig. 7 shows a schematic diagram of a flow of performing LFNST according to an embodiment of the present invention. In Fig. 7, block 701 represents a residual block.

[0293] In step 701, the transform unit 108 performs a first transform (i.e., a core transform), for example an integer transform initially designed based on a DCT, on a block 701 to obtain a block 702 including transform coefficients (referred to as "first transform coefficients") corresponding to the first transform.

[0294] In step 702, transform unit 108 determines whether it is permitted to perform a secondary transform, such as LFNST, on all or a portion of the first transform coefficients in block 702.

[0295] As an example, selectively, transform unit 108 is allowed to perform LFNST on a transform block if the intra prediction mode of the transform block including the LFNST is MIP mode and the minimum values ​​of the width and height of the transform block are 8 or greater.

[0296] Optionally, if the intra prediction mode of the transform block including the LFNST is MIP mode and the value of mipSizeId is equal to a set value (e.g., 2) or one of multiple set values ​​(e.g., 1, 2), transform unit 108 is allowed to perform LFNST on the transform block.

[0297] If transform unit 108 is allowed to perform LFNST on the transform block, transform unit 108 determines an LFNST parameter, e.g., lfnst_index. lfnst_index equal to 0 indicates that transform unit 108 does not perform LFNST on the transform block. lfnst_index greater than 0 indicates that LFNST is performed on the transform block using a transform kernel in the LFNST transform kernel candidate set.

[0298] Transform unit 108 may determine the value of lfnst_index using a conventional rate-distortion optimization method. First, transform unit 108 obtains a first cost result when not performing LFNST on the transform block.

[0299] The transform unit 108 selects the LFNST transform kernel candidate set based on the MIP parameters, which may include one of a MIP mode index value (i.e., modelId) and a MIP transposition indication parameter (i.e., isTransposed).

[0300] The transform unit 108 sets the value of the variable predModeIntra equal to the value of modelId. The transform unit 108 selects a portion of the first transform coefficients in the block 702, for example, coefficients in the sub-blocks 7001, 7002, and 7003, and determines a transform kernel for performing LFNST on the coefficients in the sub-blocks. The size of the sub-blocks may be a set value, for example, 8×8. The transform unit 108 determines an index value of the LFNST transform kernel candidate set (i.e., lfnstTrSetIdx) based on the value of predModeIntra. For original block coding in MIP mode, the transform unit 108 may obtain the value of lfnstTrSetIdx using a lookup table such as that shown in Table 10.

[0301] The transform unit 108 may use a conventional rate-distortion optimization method to determine the value of lfnst_index (i.e., a value greater than 0 indicates that the transform kernel used in the LFNST transform kernel candidate set is indicated by lfnstTrSetIdx). The transform unit 108 sets the value of lfnst_index to the index of the transform kernel in the LFNST transform kernel candidate set. Then, the cost function has a minimum value, and a second cost result is obtained.

[0302] If the first cost result is greater than the second cost result, transform unit 108 determines to perform LFNST on the transform block, and sends lfnst_index indicating an index of the transform kernel to entropy coding unit 115. Otherwise, if transform unit 108 does not perform LFNST on the transform block, it sets lfnst_value equal to 0 and sends lfnst_index indicating not to perform LFNST to entropy coding unit 115.

[0303] If lfnst_index is not equal to 0, transform unit 108 performs LFNST on the transform block using coefficients from sub-blocks 7001, 7002, and 7003 to obtain LFNST coefficients, and puts the LFNST coefficients into sub-blocks 7011, 7012, and 7013 of block 703. Assume d[x][y] is a block of first transform coefficients, where x=0,...,nTbW-1, y=0,...,nTbH-1, nTbW and nTbH are the width and height, respectively, of block 703. Assume v[x] is a block of LFNST coefficients, where x=0,...,nLfnstOutSize-1, nLfnstOutSize=(nTbW>=8 && nTbH>=8)? 48:16.

[0304] Alternatively, the transform unit 108 may employ the following command to obtain the block of transform coefficients 703, and send the block of LFNST coefficients 703 in d[x][y] to the quantization unit 109, as shown in the above equation (8). log2LfnstSize is determined as the above equation (9).

[0305] Optionally, when isTransposed is equal to 0, the transform unit 108 employs the following instruction to obtain the block of LFNST coefficients 703, and sends the block of LFNST coefficients 703 in d[x][y] to the quantization unit 109, as shown in the above equation (10): When isTransposed is equal to 1, the transform unit 108 adopts the following instruction to obtain the block of LFNST coefficients 703, and sends the block of LFNST coefficients 703 in d[x][y] to the quantization unit 109, as shown in the above equation (11).

[0306] Optionally, when isTransposed is equal to 1, the transform unit 108 employs the following instruction to obtain the block of LFNST coefficients 703, and sends the block of LFNST coefficients 703 in d[x][y] to the quantization unit 109, as shown in the above equation (10): When isTransposed is equal to 0, the transform unit 108 adopts the following instruction to obtain the block of LFNST coefficients 703, and sends the block of LFNST coefficients 703 in d[x][y] to the quantization unit 109, as shown in the above formula (11).

[0307] log2LfnstSize is determined as shown in equation (9) above.

[0308] Additionally, transform unit 108 may determine a value of lfnst_index for a transform block coded under a conventional intra-prediction mode (i.e., angular intra-prediction mode, DC mode, PLANAR mode, etc.) Similar to the rate-distortion optimization method used to determine a value of lfnst_index for a transform block under MIP mode, a first cost result is determined without performing LFNST on the transform block.

[0309] The transform unit 108 determines the value of lfnstTrSetIdx according to the mode index value of the conventional intra-prediction mode (assigned to predModeIntra) by using a lookup table, i.e., determines the LFNST transform kernel candidate set selected for the current block, as shown in Table 11 above.

[0310] Next, transform unit 108 can determine the value of lfnst_index using a conventional rate-distortion optimization method (i.e., a value greater than 0 indicates that the transform kernel used for the LFNST transform kernel candidate set is indicated by lfnstTrSetIdx). Transform unit 108 sets the value of lfnst_index to the index of the transform kernel in the LFNST transform kernel candidate set. Then, the cost function has a minimum value, and a second cost result is obtained.

[0311] If the first cost result is greater than the second cost result, transform unit 108 determines to perform LFNST on the transform block, and sends lfnst_index indicating an index of the transform kernel to entropy coding unit 115. Otherwise, if transform unit 108 does not perform LFNST on the transform block, it sets lfnst_value equal to 0 and sends lfnst_index indicating not to perform LFNST to entropy coding unit 115.

[0312] If lfnst_index is not equal to 0, transform unit 108 performs LFNST on the transform block using coefficients from sub-blocks 7001, 7002, and 7003 to obtain LFNST coefficients, and puts the LFNST coefficients into sub-blocks 7011, 7012, and 7013 of block 703. Assume d[x][y] is a block of first transform coefficients, where x=0,...,nTbW-1, y=0,...,nTbH-1, nTbW and nTbH are the width and height, respectively, of block 703. Assume v[x] is a block of LFNST coefficients, where x=0,...,nLfnstOutSize-1, nLfnstOutSize=(nTbW>=8 && nTbH>=8)? 48:16.

[0313] Optionally, when predModeIntra is less than or equal to 34, the transform unit 108 adopts the following instruction to obtain the block of LFNST coefficients 703, and sends the block of LFNST coefficients 703 in d[x][y] to the quantization unit 109, as shown in the above equation (10): Optionally, when predModeIntra is greater than 34, the transform unit 108 adopts the following instruction to obtain the block of LFNST coefficients 703, and sends the block of LFNST coefficients 703 in d[x][y] to the quantization unit 109, as shown in the above formula (11):

[0314] Optionally, when predModeIntra is greater than 34, the transform unit 108 adopts the following instruction to obtain the block of LFNST coefficients 703, and sends the block of LFNST coefficients 703 in d[x][y] to the quantization unit 109, as shown in the above equation (10): Optionally, when predModeIntra is less than or equal to 34, the transform unit 108 adopts the following instruction to obtain the block of LFNST coefficients 703, and sends the block of LFNST coefficients 703 in d[x][y] to the quantization unit 109, as shown in the above equation (11):

[0315] log2LfnstSize is determined as shown in equation (9) above.

[0316] Furthermore, the inverse quantization unit 110 performs a scaling operation on the quantized coefficients to output reconstructed coefficients. The inverse transform unit 111 performs one or more inverse transforms corresponding to the transform in the transform unit 108, and outputs a reconstructed residual. The second adder 112 calculates a reconstructed CU by adding the reconstructed residual and the intra prediction block of the current block from the prediction unit 102. The second adder 112 further sends its output to the prediction unit 102 to be used as an intra prediction reference.

[0317] The output of the filtering unit 113 is a decoded image or sub-image, which is sent to the DPB unit 114. The DPB unit 114 outputs a decoded image based on timing and control information. The image stored in the DPB unit 114 can be used as a reference for the prediction unit 102 to perform inter prediction or intra prediction.

[0318] The entropy coding unit 115 converts the parameters (necessary for obtaining the decoded image), control parameters and supplementary information from the units in the encoder 100 into binary representations, and writes the binary representations into the generated bitstream (also called "video bitstream") according to the syntax structure of each data unit, as shown in Table 9A and Table 9B.

[0319] Using the syntax structure in Table 9A, if the minimum value of the width and height of the transform block in a coding unit is greater than or equal to 8 (i.e., both the width and the height are greater than or equal to 8), the entropy coding unit 115 uses the MIP mode to code the lfnst_index of the coding unit. Another selectable condition may be set as both the width and the height are greater than or equal to a first value (e.g., 8) and the sum of the width and the height is greater than or equal to a second value (e.g., 16).

[0320] Using the syntax structure in Table 9B, entropy coding unit 115 encodes the lfnst_index of a coding unit using MIP mode if the mipSizeId of the coding unit is equal to 2. Another selectable condition may be set when the mipSizeId of the coding unit is equal to one of multiple values ​​(e.g., 1, 2).

[0321] Referring to Figure 8, Figure 8 shows a flowchart of encoding LFNST parameters according to an embodiment of the present invention. As shown in Figure 8, the flow may include:

[0322] S801: The entropy encoding unit 115 checks the above conditions to determine whether to encode the LFNST parameters.

[0323] S802: The entropy encoding unit 115 encodes the LFNST parameters using context-adaptive binary arithmetic coding (CABAC).

[0324] Note that in step S801, the entropy encoding unit 115 checks to determine the value of an LFNST parameter, e.g., lfnst_index, and then in step S802, encodes the value of lfnst_index using CABAC, whose descriptor is “ae(v)”, as shown in Tables 9A and 9B.

[0325] In an embodiment of the present invention, the encoder 100 may be a computing device including a processor and a storage medium storing an encoding program. When the processor reads and executes the encoding program, the encoder 100 reads the input video and generates a corresponding bitstream. The encoder 100 may also be a computing device including one or more chips. The units implemented as integrated circuits in the chips have similar connections and data exchange functions as the corresponding units in FIG. 1.

[0326] In this embodiment, an encoding method is provided. A prediction parameter of a current block is determined. The prediction parameter includes a prediction mode parameter. When the prediction mode parameter indicates that the MIP mode is used to determine the intra-prediction value of the chroma component of the current block, the neighboring sample value of the current block is obtained, and the MIP input sample value of the current block is determined according to the neighboring sample value of the current block. The MIP predicted value of the chroma component of the current block is determined according to the MIP input sample value, the MIP weighting matrix, and the shift parameter. The MIP predicted value is determined by filtering the MIP predicted value. The prediction residual value of the chroma component of the current block is determined according to the intra-prediction value of the chroma component of the current block. The LFNST parameter is determined by performing LFNST on the prediction residual value. The LFNST parameter is coded and written into the bitstream. In this way, in the MIP mode, while ensuring coding performance, the complexity is reduced, and at the same time, the storage space required for the coding process is reduced, and coding efficiency can be effectively improved. In addition, when the LFNST technique is applied to MIP mode prediction, the introduction of MIP parameters makes LFNST more flexible and can further improve coding efficiency.

[0327] In another embodiment of the present invention, refer to Fig. 9, which shows a flowchart of a decoding method according to an embodiment of the present invention. As shown in Fig. 9, the method may include:

[0328] S901: Parse a bitstream to obtain prediction parameters and LFNST parameters of a current block. The prediction parameters include a prediction mode parameter.

[0329] It should be noted that the image to be decoded can be divided into a number of image blocks, and each image block to be decoded can be called a decoding block, where each decoding block can include a first image component, a second image component, and a third image component. A current block is a decoding block on which prediction of the first image component, the second image component, or the third image component in an image of a video is currently performed.

[0330] It should be noted that the prediction parameters may also include a prediction mode parameter, which is used to indicate the prediction mode adopted by the current block, and different prediction modes correspond to different prediction mode parameters. The prediction modes generally include an inter prediction mode, a conventional intra prediction mode, and a non-conventional intra prediction mode. The conventional intra prediction modes may include a DC mode, a PLANAR mode, an angular intra prediction mode, etc. The non-conventional intra prediction modes may include a MIP mode, a CCLM mode, an IBC mode, a PLT mode, etc. In other words, the encoder selects an optimal prediction mode to encode the current block. In this process, the prediction mode of the current block may be determined, and the prediction mode parameter may be obtained correspondingly. After that, the prediction parameters including the prediction mode parameter are written into a bitstream and sent from the encoder to the decoder.

[0331] In this way, at the decoder, prediction parameters for the current block can be obtained by parsing the bitstream, and the prediction mode parameter contained in the prediction parameters obtained by parsing can be used to determine whether or not MIP mode is used for the current block.

[0332] S902: When the prediction mode parameter indicates that the MIP mode is used to determine the intra prediction value of the chroma component of the current block, obtain neighboring sample values ​​of the current block, and determine the MIP input sample value of the current block based on the neighboring sample values ​​of the current block.

[0333] However, for the current block, in the embodiment of the present invention, MIP mode is used to perform intra prediction on the chroma component of the current block, in this process, first obtain the adjacent sample values ​​of the current block, and then determine the MIP input sample values ​​of the current block according to the adjacent sample values ​​of the current block.

[0334] The neighboring sample values ​​of the current block may include a left neighboring sample value of the current block and an upper neighboring sample value of the current block, that is, the MIP input sample value of the current block may be determined based on the left neighboring sample value and the upper neighboring sample value of the current block.

[0335] In some embodiments, determining MIP input sample values ​​of the current block based on neighboring sample values ​​of the current block may include:

[0336] A block size index value for the current block is determined based on the size parameter of the current block.

[0337] A first temporary reference value is obtained by downsampling the neighboring sample values ​​of the current block.

[0338] If the block size index value of the current block is within a preset range, determine a second constant value based on a bit depth of adjacent sample values ​​of the current block, set a value corresponding to index 0 in the MIP input sample value to a difference between the second constant value and a value corresponding to index 0 in the first temporary reference value, and set a value corresponding to index i in the MIP input sample value to a difference between a value corresponding to index i in the first temporary reference value and a value corresponding to index 0 in the first temporary reference value, where i is an integer greater than 0.

[0339] If the block size index value of the current block is outside a preset range, set a value corresponding to index j in the MIP input sample value to a difference between a value corresponding to index (j+1) in the first temporary reference value and a value corresponding to index 0 in the first temporary reference value, where j is an integer equal to or greater than 0.

[0340] Furthermore, if the block size index value of the current block is within a preset range, the method may further include: Determine a second constant value based on the bit depth of the adjacent sample values ​​of the current block, set a value corresponding to index 0 in the MIP input sample value to a difference between the value corresponding to index 0 in the first temporary reference value and the second constant value, and set a value corresponding to index i in the MIP input sample value to a difference between the value corresponding to index i in the first temporary reference value and the value corresponding to index 0 in the first temporary reference value, where i is an integer greater than 0.

[0341] It should be noted that after downsampling the adjacent sample values ​​of the current block, a first temporary reference value can be obtained. For the first temporary reference value, specifically, the filtered adjacent sample values ​​can be buffered in a buffer (represented by pTemp). The value corresponding to index 0 in the first temporary reference value refers to pTemp[0], and the value corresponding to index i in the first temporary reference value refers to pTemp[i].

[0342] In an embodiment of the present invention, in addition to the prediction mode parameter, the prediction parameters may further include a size parameter of the current block. The size parameter of the current block may include the width (represented by nTbW) and height (represented by nTbH) of the current block. In addition, the block size index value (i.e., mipSizeId) of the current block may be determined based on the size parameter of the current block.

[0343] It should also be noted that whether the value of the size parameter of the current block is within a preset range can be determined based on whether the block size index value (represented by mipSizeId) of the current block is within a preset range. Specifically, when mipSizeId=0 or 1, it indicates that the block size index value of the current block is within the preset range, i.e., the value of the size parameter of the current block is within the preset range. When mipSizeId=2, it indicates that the block size index value of the current block is outside the preset range, i.e., the value of the size parameter of the current block is not within the preset range.

[0344] In other words, the MIP input sample value is determined by the buffer (represented by pTemp), the block size index value of the current block (represented by MipSizeId), and the bit depth (represented by BitDepth) of the neighboring sample values ​​of the current block, and the number of input samples corresponding to the MIP input sample value is related to the block size index value of the current block. Finally, a value (represented by p[x]) corresponding to index x in the MIP input sample value can be obtained.

[0345] Furthermore, in some embodiments, determining the second constant value based on a bit depth of adjacent sample values ​​of the current block may include: Set the second constant value to a power of 2. The exponent of the power is an integer equal to the bit depth of the adjacent sample values ​​of the current block minus 1.

[0346] Or, in some embodiments, determining the second constant value based on a bit depth of adjacent sample values ​​of the current block may include: The second constant value is obtained by binary left-shifting "1". The number of left-shifted bits is equal to the bit depth of the adjacent sample values ​​of the current block minus 1.

[0347] In other words, after the bit depth (represented by BitDepth) of the adjacent sample values ​​of the current block is obtained, the second constant value can be expressed as 1<<(BitDepth-1) or 2^(BitDepth-1). In this way, when the value of the size parameter of the current block is within a preset range, the MIP input sample value of the current block can be determined in combination with the second constant value.

[0348] Also, it should be noted that the MIP input sample is a matrix vector used in matrix multiplication operation. The current related technical solution is as follows: According to a buffer (represented by pTemp), a type of a current block (i.e., a block size index value of the current block represented by mipSizeId), a bit depth (represented by BitDepth) of adjacent sample values ​​of the current block, and the number of MIP input samples, a value (represented by p[x]) corresponding to an index x in the MIP input sample value is finally obtained.

[0349] In one possible embodiment, when mipSizeId=0 or 1, p[0] is obtained by subtracting pTemp[0] from 1<<(BitDepth-1), and when x is not 0, p[x] is obtained by subtracting pTemp[0] from pTemp[x]. Specifically, this is as shown in the above formula (2).

[0350] In another possible embodiment, when mipSizeId=0 or 1, pTemp[0] minus 1<<(BitDepth-1) is set to p[0], and when x is not 0, pTemp[x] minus pTemp[0] is set to p[x]. Specifically, this is as shown in the above formula (3).

[0351] In another possible embodiment, when mipSizeId=2, pTemp[x+1] minus pTemp[0] is set to p[x], as shown in the above formula (4).

[0352] In addition, in the embodiment of the present invention, it is also possible to obtain the value of p[x] by a unified calculation method without determining the value of mipSizeId. In this case, in the embodiment of the present invention, (1<<(BitDepth-1)) is added as an additional element to the end of the buffer pTemp. p[x] can be directly set to p[x]=pTemp[x+1]-pTemp[0], where x=0,...,inSize-1.

[0353] In this way, after determining the MIP input sample values ​​of the current block based on the neighboring sample values ​​of the current block, the MIP predicted value of the current block can be further determined.

[0354] S903: Determine a MIP predicted value of the chroma component of the current block based on the MIP input sample value, the MIP weighting matrix and the shift parameter.

[0355] The MIP predictor is a prediction of some samples in the chroma component of the current block.

[0356] Wherein, the shift parameter may include a shift offset parameter and a shift number parameter. For S903, determining the MIP predicted value of the chroma component of the current block based on the MIP input sample value, the MIP weighting matrix, and the shift parameter may include:

[0357] A product of the sum of the MIP input sample values ​​and the shift offset parameter is determined based on the value of the shift offset parameter, the value of the shift offset parameter being a fixed constant. A first constant value is determined based on the value of the shift number parameter, the value of the shift number parameter being a fixed constant. A first offset value is set to the difference between a first constant value and the multiplication product. Based on the prediction parameters, a MIP weighting matrix for the current block is determined. A MIP prediction of the chroma component of the current block is determined based on the MIP weighting matrix, the MIP input sample value, the shift number parameter, and the first offset.

[0358] In addition, the shift offset parameter may be referred to as a shift compensation parameter or an offset factor, and may be represented by fO. In an embodiment of the present invention, the value of the shift offset parameter may be set to a fixed constant, for example, 32, 46, 56, 66, or 32. Furthermore, the value of the shift offset parameter may be set to be associated with a shift offset parameter table, and the value of the shift offset parameter may be determined by a look-up table. This specification is not limited thereto.

[0359] In addition, the shift number parameter may be called a shift factor, a number of shift bits, or a weight shift value, and may be represented as sW, shift, or weight shift. In an embodiment of the present invention, the shift number parameter is generally represented as sW. In an embodiment of the present invention, the value of the shift number parameter may be set to a fixed constant, for example, 5, 6, or 7. Furthermore, the value of the shift number parameter may be set to be associated with a shift number parameter table, and the value of the shift number parameter may be determined by a lookup table. This specification is not limited thereto.

[0360] Additionally, the first offset can be expressed as oW, where the first offset is associated with both the shift number parameter and the shift offset parameter.

[0361] In addition, the prediction parameters may include a prediction mode parameter and a size parameter of the current block. If the prediction mode parameter indicates that the MIP mode is used to determine the intra prediction value of the chroma component of the current block, a weighting matrix table may be established in advance, and the weighting matrix table may be stored in a memory or storage unit. The memory or storage unit may be integrated in the decoder or may be provided separately. In this way, based on the block size index value (mipSizeId) and the MIP mode index value (modelId) of the current block, the MIP weighting matrix (also called MIP weight matrix or MIP matrix) that needs to be used for the current block may be determined by a lookup table, and is represented by mWeight[x][y].

[0362] In addition, in the decoder, a shift number parameter table can be pre-established, and the shift number parameter table is also stored in a memory or storage unit. The memory or storage unit here can be integrated in the decoder or provided separately. In the embodiment of the present invention, the shift number parameter (sW) can be determined in the following several ways:

[0363] In one possible embodiment, the shift number parameter may be different for different block sizes and different MIP mode index values. In some embodiments, the method may further include: If the MIP mode is used to determine the intra prediction value of the chroma component of the current block, the bitstream is parsed to obtain the MIP mode index value of the current block. According to the MIP mode index value, query a value corresponding to the MIP mode index value from a first preset lookup table, where the first preset lookup table is used to record the corresponding relationship between the MIP mode index value and the value of the shift number parameter. The queried value is determined as the value of the shift number parameter.

[0364] Note that the value of the shift number parameter can be queried based on the block size index value (represented by mipSizeId) and the MIP mode index value (modelId) of the current block. In the first pre-defined lookup table shown in Table 4 above, for different mipSizeId and modelId, the shift number parameter required in the matrix multiplication can be determined by the lookup table.

[0365] However, at the decoder side, Table 4 needs to be stored in a memory or storage unit in the form of a look-up table, but the storage is costly and the look-up process is also costly. The shift factor in Table 4 is related to both the block size and the MIP mode index value of the current block, so the memory occupation and the calculation complexity are increased.

[0366] In order to reduce memory occupation and calculation complexity, in the embodiment of the present invention, the method for determining the shift factor can be simplified.

[0367] In another possible embodiment, the value of the shift number parameter can be set to a fixed constant that is not related to the block size index value and the MIP mode index value. For example, the value of the shift number parameter can be set to 5 for different block size index values ​​and different MIP mode index values. Or, the value of the shift number parameter can be set to 6 for different block size index values ​​and different MIP mode index values. Or, the value of the shift number parameter can be set to 7 for different block size index values ​​and different MIP mode index values. In the embodiment of the present invention, it is preferable to set the value of the shift number parameter to 6, but is not limited thereto.

[0368] In another possible embodiment, for the value of the shift number parameter, the method further comprises: A block size index value for the current block is determined based on the size parameter of the current block. Based on the block size index value of the current block, a value of a shift number parameter is determined.

[0369] It should be noted that the block size index value of the current block may be determined based on the size parameter of the current block. In some embodiments, determining the block size index value of the current block based on the size parameter of the current block may include: Set the block size index value of the current block to 0 if both the width and height of the current block are equal to 4. If the width and height of the current block are both equal to 8, or if either the width or height of the current block is equal to 4, set the block size index value of the current block to 1. If the width and height of the current block do not satisfy the above conditions, set the block size index value of the current block to 2.

[0370] In this manner, after determining the block size index value of the current block, the value of the shift number parameter can be further determined based on the block size index value of the current block.

[0371] Optionally, in some embodiments, determining a value of a shift number parameter based on a block size index value of the current block may include: It is determined that if the block size index value is equal to 0, 1 or 2, respectively, the value of the shift number parameter corresponding to the block size index value of the current block is equal to 5, 6 or 5, respectively.

[0372] Optionally, in some embodiments, determining a value of a shift number parameter based on a block size index value of the current block may include: Set the value of the shift number parameter equal to the ratio of the width or height of the current block to a first preset value that corresponds to the block size index value of the current block.

[0373] Here, the first preset value represents the number of MIP input sample values ​​obtained from the boundary of the current block. In this case, the method may also include: Determine that if the block size index value of the current block is equal to 0, 1 or 2, respectively, a first preset value corresponding to the block size index value of the current block is equal to 2, 4 or 4, respectively.

[0374] In other words, when the first preset value represents the number of MIP input sample values ​​taken from the boundary of the current block, when the block size index value of the current block is equal to 0, the corresponding first preset value is equal to 2, when the block size index value of the current block is equal to 1, the corresponding first preset value is equal to 4, and when the block size index value of the current block is equal to 2, the corresponding first preset value is equal to 4. In this way, the value of the shift number parameter can be determined based on the ratio of the width or height of the current block to the corresponding first preset value.

[0375] Optionally, in some embodiments, determining a value of a shift number parameter based on a block size index value of the current block may include: Set the value of the shift number parameter equal to the ratio of the width or height of the current block to a second preset value that corresponds to the block size index value of the current block.

[0376] Wherein, the second preset value represents a size of the MIP prediction block of the current block obtained by directly using the MIP weighting matrix. In this case, the method may also include: Determine that if the block size index value of the current block is equal to 0, 1 or 2, respectively, the second preset value corresponding to the block size index value of the current block is equal to 4, 4 or 8, respectively.

[0377] In other words, when the second preset value represents the size of the MIP prediction block of the current block obtained by directly using the MIP weighting matrix, when the block size index value of the current block is equal to 0, the corresponding second preset value is equal to 4, when the block size index value of the current block is equal to 1, the corresponding second preset value is equal to 4, and when the block size index value of the current block is equal to 2, the corresponding second preset value is equal to 8. In this way, the value of the shift number parameter can be determined based on the ratio of the width or height of the current block to the corresponding second preset value.

[0378] In another possible embodiment, the shift number parameter table can be minimized, and still determine the value of the shift number parameter by the method of a lookup table. Optionally, in some embodiments, determining the value of the shift number parameter based on the block size index value of the current block can include: According to the block size index value, query a value corresponding to the block size index value from a second preset lookup table, where the second preset lookup table is used to record the corresponding relationship between the block size index value and the value of the shift number parameter. The queried value is determined as the value of the shift number parameter.

[0379] Note that the value of the shift number parameter can be queried based only on the block size index value (represented by mipSizeId) of the current block. In the second pre-defined lookup table shown in Table 5 above, each block size index value can correspond to a fixed value. That is, each block size or each block size set can have a fixed shift number parameter value as shown in Table 5.

[0380] According to Table 5 above, it can be determined that if the block size index value of the current block is equal to 0, 1 or 2, respectively, the value of the shift number parameter corresponding to the block size index value is equal to 5, 6 or 5, respectively.

[0381] In this way, by simplifying the method of determining the shift number parameter, particularly by minimizing the shift number parameter table or fixing the value of the shift number parameter, the storage amount of the look-up table can be minimized, and the memory occupied by storing the original shift factor table of the MIP mode can be reduced without increasing the calculation complexity.

[0382] In addition, in the decoder, a shift offset parameter table can be pre-established, and the shift offset parameter table is also stored in a memory or storage unit. The memory or storage unit here can be integrated in the decoder or provided separately. In the embodiment of the present invention, the shift offset parameter (fO) can be determined in the following several ways:

[0383] In one possible embodiment, the shift offset parameter may be different for different block sizes and different MIP mode index values. In some embodiments, the above method may further include: If the MIP mode is used to determine the intra prediction value of the chroma component of the current block, the bitstream is parsed to obtain the MIP mode index value of the current block. According to the MIP mode index value, a value corresponding to the MIP mode index value is queried from a third preset lookup table, where the third preset lookup table is used for recording the corresponding relationship between the MIP mode index value and the value of the shift offset parameter. The queried value is determined as the value of the shift offset parameter.

[0384] Note that the value of the shift offset parameter can be queried based on the block size index value (represented by mipSizeId) and the MIP mode index value (modelId) of the current block. In the third pre-defined lookup table shown in Table 6 above, for different mipSizeId and modelId, the shift offset parameter required in the matrix multiplication can be determined by the lookup table.

[0385] However, at the decoder side, Table 6 needs to be stored in a memory or storage unit in the form of a look-up table, but the storage is costly and the look-up process is also costly. The value of the shift offset parameter in Table 6 is related to both the block size and the MIP mode index value of the current block, so the memory occupation and the calculation complexity are increased.

[0386] In order to reduce memory occupancy and computational complexity, in the embodiment of the present invention, the method for determining the shift offset parameter can be simplified.

[0387] In another possible embodiment, the value of the shift offset parameter can be set to a fixed constant that is not related to the block size index value and the MIP mode index value. Typically, the fixed constant is in the range of 0 to 100. For example, the value of the shift offset parameter can be set to 32 for different block size index values ​​and different MIP mode index values. Or, the value of the shift offset parameter can be set to 46 for different block size index values ​​and different MIP mode index values. Or, the value of the shift offset parameter can be set to 56 for different block size index values ​​and different MIP mode index values. Or, the value of the shift offset parameter can be set to 66 for different block size index values ​​and different MIP mode index values. In the embodiment of the present invention, it is preferable to set the value of the shift offset parameter to 32, but the present invention is not limited thereto.

[0388] In another possible embodiment, for the value of the shift offset parameter, the method may further include: A block size index value for the current block is determined based on the size parameter of the current block. A value of a shift offset parameter is determined based on the block size index value of the current block.

[0389] It should be noted that the block size index value of the current block can be determined based on the size parameter of the current block. Then, the value of the shift offset parameter can be further determined based on the block size index value of the current block.

[0390] Optionally, in some embodiments, determining a value of a shift offset parameter based on a block size index value of the current block may include: Determine that if the block size index value is equal to 0, 1 or 2, respectively, the value of the shift offset parameter corresponding to the block size index value of the current block is equal to 34, 23 or 46, respectively.

[0391] Optionally, in some embodiments, the shift offset parameter table can be further minimized and still determine the shift factor by the lookup table method. Optionally, in some embodiments, determining a value of the shift offset parameter based on the block size index value of the current block can include: According to the block size index value, query a value corresponding to the block size index value from a fourth preset lookup table, the fourth preset lookup table being used for recording the correspondence relationship between the block size index value and the value of the shift offset parameter. The queried value is determined as the value of the shift offset parameter.

[0392] It should be noted that the value of the shift offset parameter can be queried based only on the block size index value (represented by mipSizeId) of the current block. In the fourth pre-defined lookup table shown in Table 7 above, each block size index value can correspond to a fixed value. That is, each block size or each block size set can have a fixed shift offset parameter value as shown in Table 7.

[0393] According to Table 7 above, it can be determined that if the block size index value of the current block is equal to 0, 1 or 2, respectively, the value of the shift offset parameter corresponding to the block size index value is equal to 34, 23 or 46, respectively.

[0394] In this way, by simplifying the method of determining the shift offset parameter, particularly by minimizing the shift offset parameter table or fixing the value of the shift offset parameter, the storage amount of the look-up table can be minimized, and thus the memory occupied by the storage of the original shift offset parameter table in the MIP mode can be reduced without increasing the calculation complexity.

[0395] Thus, after determining the shift offset parameter (fO) and the shift number parameter (sW), optionally, in some embodiments, determining the first constant value based on the value of the shift number parameter may include: Set the first constant value to a power of 2. The exponent of the power is an integer and is equal to the value of the shift number parameter minus 1.

[0396] Optionally, in some embodiments, determining a first constant value based on the value of the shift number parameter may include: The first constant value is obtained by binary left-shifting a "1". The number of bits shifted left is equal to the value of the shift number parameter minus 1.

[0397] In other words, after the shift number parameter (sW) is obtained, the first constant value may be expressed as 1<<(sW-1), or may be expressed as 2^(sW-1). In this case, when the value of the shift number parameter is set to 6, the first constant value equal to 32 can be obtained.

[0398] JPEG2025061393000026.jpg36170

[0399] In this way, under the MIP mode, the MIP weighting matrix, the MIP input sample value, the shift number parameter and the first offset can be obtained, which facilitates the subsequent determination of the MIP predicted value of the current block. Specifically, in some embodiments, determining the MIP predicted value of the current block based on the above MIP weighting matrix, the MIP input sample value, the shift number parameter and the first offset can include:

[0400] A first weighted sum of the MIP weighting matrix and the MIP input sample values ​​is calculated. A first sum of the first weighted sum and the first offset is calculated. A first right shift value is obtained by binary right shifting the first sum, the number of bits shifted right equal to the value of the shift number parameter. Set the MIP predicted value of the current block to the sum of the first right shift value and the value corresponding to index 0 in the first temporary reference value, where the first temporary reference value is obtained by downsampling the neighboring sample values ​​of the current block.

[0401] In other words, under MIP mode, a MIP weighting matrix (represented by mWeight), a shift number parameter (represented by sW) and a shift offset parameter (represented by fO) can be determined based on the block size index value (represented by mipSizeId) and the MIP mode index value (represented by modelId) of the current block. Then, the MIP input sample value (represented by p[x]), mWeight, sW and fO are input into the matrix multiplication to obtain the MIP predicted value (represented by predMip[x][y]) output from the matrix multiplication. The samples in predMip[x][y] are arranged in the form of a matrix / array based on predSize×predSize. The calculation formula is as shown in the above formula (5) or formula (6).

[0402] S904: Determine an intra prediction value of the chroma component of the current block by filtering the MIP prediction value.

[0403] Here, the MIP prediction block is composed of MIP prediction values. After obtaining the MIP prediction block, the intra prediction value of the current block can be further determined by determining whether the size parameter of the MIP prediction block is the same as the size parameter of the current block. Specifically, based on the determination result, if the size parameter of the MIP prediction block is the same as the size parameter of the current block, the intra prediction block of the current block is set equal to the MIP prediction block. In this case, the MIP prediction block includes prediction values ​​of all sample positions in the current block. If the size parameter of the MIP prediction block is different from the size parameter of the current block, a filtered prediction block is obtained by filtering the MIP prediction block, and the filtered prediction block is set as the intra prediction block of the current block. Here, the filtering can include upsampling or low-pass filtering.

[0404] S905: If the LFNST parameter indicates to perform LFNST on the current block, determine a reconstructed transform coefficient block of the current block, and obtain a second transform coefficient block by performing LFNST on at least some of the reconstructed transform coefficients in the reconstructed transform coefficient block.

[0405] Also, note that LFNST is not performed for any current block, which means that there is not necessarily an LFNST parameter in the bitstream. In this case, some conditions (e.g., the minimum value of the size parameters of the current block, the block size index value of the current block, etc.) need to be checked to determine whether the bitstream has an LFNST parameter. Thus, in some embodiments, parsing the bitstream to obtain the LFNST parameter may include: Determines whether the bitstream has an LFNST parameter. Parse the bitstream to get the LFNST parameter, if any, in the bitstream.

[0406] In other words, it is first necessary to determine whether the bitstream has an LFNST parameter or not, and only if the result of the determination is Yes, i.e., only if the bitstream has an LFNST parameter, can the step of parsing the bitstream to obtain the LFNST parameter be performed.

[0407] Optionally, in some embodiments, determining whether there is an LFNST parameter in the bitstream may include: When determining an intra prediction value of the chroma component of the current block using the MIP mode, the minimum value of the size parameters of the current block is determined. Based on the minimum value, it is determined whether the bitstream has an LFNST parameter or not.

[0408] Furthermore, determining whether the bitstream has an LFNST parameter based on the minimum value includes: determining that the bitstream has an LFNST parameter if the minimum value is greater than or equal to a first preset threshold.

[0409] Here, the first preset threshold may be used to represent a preset threshold for determining whether to perform LFNST. In the embodiment of the present invention, the first preset threshold may be set to, but is not limited to, 8.

[0410] In other words, when intra prediction is performed on the chroma component of the current block using the MIP mode, the minimum value of the size parameters of the current block may be determined. For example, if the minimum values ​​of width and height are equal to or greater than 8, it may be determined that the bitstream has an LFNST parameter. In this case, the LFNST parameter, for example, the value of lfnst_index, may be obtained by analyzing the bitstream.

[0411] Optionally, in some embodiments, determining whether there is an LFNST parameter in the bitstream may include: When the MIP mode is used to determine the intra prediction value of the chroma component of the current block, a block size index value of the current block is determined based on the size parameter of the current block. Based on the block size index value, it is determined whether the LFNST parameter is present in the bitstream.

[0412] Further, determining whether the bitstream has an LFNST parameter based on the block size index value includes: determining that the bitstream has an LFNST parameter present if the block size index value is equal to a second preset threshold.

[0413] Here, the second preset threshold may be used to represent a preset threshold for determining whether to perform LFNST. In an embodiment of the present invention, the second preset threshold may be set to, but is not limited to, one preset value (e.g., 2) or one of a plurality of preset values ​​(e.g., 1, 2).

[0414] In other words, when intra prediction is performed on the chroma component of the current block using the MIP mode, a block size index value (mipSizeId) of the current block can be determined based on the size parameter of the current block. For example, when the value of mipSizeId is equal to 2, it is determined that there is an LFNST parameter in the bitstream. In this case, the value of the LFNST parameter, for example, lfnst_index, can be obtained by analyzing the bitstream.

[0415] In some embodiments, parsing the bitstream to obtain the LFNST parameters comprises: and parsing the bitstream to obtain the value of the LFNST index.

[0416] In addition, the LFNST index may be represented by lfnst_index, and the value of the LFNST index is used to determine whether to perform LFNST on the current block. Specifically, in some embodiments, the method may further include: If the value of the LFNST index is greater than 0, it is determined that LFNST is to be performed on the current block. If the value of the LFNST index is equal to 0, it is determined that LFNST is not to be performed on the current block.

[0417] That is, after obtaining the value of lfnst_index, if the value of lfnst_index is greater than 0, it is determined that LFNST is to be performed on the current block. If the value of lfnst_index is equal to 0, it is determined that LFNST is not to be performed on the current block.

[0418] Also, after obtaining the value of the LFNST parameter, e.g., lfnst_index, it is indicated that the LFNST parameter indicates to perform LFNST on the current block if the value of lfnst_index is greater than 0. In this case, in some embodiments, determining a reconstructed transform coefficient block for the current block when the LFNST parameter indicates to perform LFNST on the current block may include: Parse the bitstream to obtain the quantized coefficients of the current block. A reconstructed transform coefficient block of the current block is obtained by inverse quantizing the quantized coefficients.

[0419] Specifically, the inverse quantization is scaling. In one specific example, obtaining a reconstructed transform coefficient block of the current block by inverse quantizing the quantized coefficients includes: obtaining a reconstructed transform coefficient block of the current block by scaling the quantized coefficients.

[0420] In this manner, when the LFNST parameters indicate to perform LFNST on the current block, the second transform coefficient block may be obtained by performing LFNST on all or a portion of the reconstructed transform coefficients in the reconstructed transform coefficient block.

[0421] Also, when it is determined that LFNST can be performed on the current block, an LFNST transform kernel (denoted as kernel) to be used for the current block needs to be determined. In some embodiments, obtaining a second transform coefficient block by performing LFNST on at least some of the reconstructed transform coefficients in the reconstructed transform coefficient block includes: and obtaining a second transform coefficient block by performing LFNST on at least some of the reconstructed transform coefficients in the reconstructed transform coefficient block using an LFNST transform kernel.

[0422] In an embodiment of the present invention, there are four transform kernel candidate sets for LFNST. The four transform kernel candidate sets may include set 0, set 1, set 2 and set 3. Here, in an embodiment of the present invention, the MIP parameters may be used to determine the LFNST transform kernel candidate set, and then select an LFNST transform kernel to be used for the current block from the LFNST transform kernel candidate set. Thus, in some embodiments, the above method may further include:

[0423] When the MIP mode is used to determine the intra prediction value of the chroma component of the current block, the MIP parameters are determined from the prediction mode parameters of the current block. Based on the MIP parameters, a set of LFNST transformation kernel candidates is determined. In the LFNST transformation kernel candidate set, a transformation kernel indicated by the value of the LFNST index is determined as the LFNST transformation kernel to be used for the current block. The LFNST transformation kernel candidate set includes two or more predefined LFNST transformation kernels.

[0424] Note that the MIP parameters may include at least one of a MIP mode index value (represented by modelId) and a MIP transposition instruction parameter (represented by isTransposed). For the LFNST transform kernel used for the current block, if the value of the MIP transposition instruction parameter indicates that a transposition is to be performed on the sample input vector used for the MIP mode, then a matrix transposition needs to be performed on the selected transform kernel to obtain the LFNST transform kernel to be used for the current block.

[0425] Furthermore, in some embodiments, determining the LFNST transformation kernel candidate set based on the MIP parameters may include: Based on the MIP mode index value, a value of the LFNST intra-prediction mode index is determined. According to the value of the LFNST intra-prediction mode index, a fifth preset lookup table is used to determine an index value of the LFNST transform kernel candidate set. The LFNST transformation kernel candidate set is selected from a plurality of candidate LFNST transformation kernel candidate sets based on index values ​​of the LFNST transformation kernel candidate set.

[0426] Here, the LFNST intra prediction mode index may be represented by predModeIntra, and the index value of the LFNST transform kernel candidate set may be represented by lfnstTrSetIdx.

[0427] In addition, the fifth preset lookup table is as shown in Table 10 above. The value of predModeIntra can be determined based on the MIP mode index value (i.e., the value of modelId). Then, the value of lfnstTrSetIdx can be directly determined based on the value of predModeIntra in conjunction with Table 10, that is, the LFNST transformation kernel candidate set selected for the current block can be determined. Here, the value of lfnstTrSetIdx indicates the transformation kernel candidate set used for LFNST. Since the value of modelId can include 0, 1, 2, 3, 4, and 5, the value of predModeIntra is also 0, 1, 2, 3, 4, or 5. The correspondence relationship between predModeIntra and lfnstTrSetIdx is as shown in Table 10.

[0428] In one specific embodiment, determining a value of the LFNST intra-prediction mode index based on the MIP mode index value includes: Setting the value of the LFNST intra-prediction mode index equal to the MIP mode index value.

[0429] In other words, in an embodiment of the present invention, the value of predModeIntra can be set equal to the value of modelId, and then the value of lfnstTrSetIdx is directly determined based on the value of predModeIntra in conjunction with Table 10, i.e., the LFNST transformation kernel candidate set selected for the current block is determined. In addition, in another specific embodiment of the present invention, the MIP mode index value is directly mapped to the value of PLANAR mode, and then the value of predModeIntra corresponding to PLANAR mode is used to determine the value of lfnstTrSetIdx, i.e., the LFNST transformation kernel candidate set selected for the current block is determined.

[0430] Furthermore, in the process of performing LFNST, the scanning order can include a horizontal scanning order and a vertical scanning order, where the determination of the scanning order is related to the MIP transposition indication parameter (represented by isTransposed).

[0431] Optionally, in some embodiments, obtaining a second transform coefficient block by performing LFNST on at least some of the reconstructed transform coefficients in the reconstructed transform coefficient block may include:

[0432] If the value of the MIP transposition indication parameter indicates that a transposition is to be performed on the sample input vector used for the MIP mode, a second transform coefficient block is obtained by performing LFNST on at least some of the reconstructed transform coefficients in the reconstructed transform coefficient block in vertical scan order. If the value of the MIP transposition indication parameter indicates that no transposition is to be performed on the sample input vector used for the MIP mode, a second transform coefficient block is obtained by performing LFNST on at least some of the reconstructed transform coefficients in the reconstructed transform coefficient block in horizontal scan order.

[0433] Specifically, when the value of isTransposed is equal to 0, no transposition is performed on the sample input vector used for the MIP mode. Then, LFNST can be performed on at least some transform coefficients in the first transform coefficient block in horizontal scan order, as shown in the above formula (10).

[0434] If the value of isTransposed is equal to 1, a transposition is performed on the sample input vector used for MIP mode. Then, LFNST can be performed on at least some of the transform coefficients in the first transform coefficient block in vertical scan order, as shown in equation (11) above.

[0435] log2LfnstSize is determined as shown in equation (9) above.

[0436] Optionally, in some embodiments, obtaining a second transform coefficient block by performing LFNST on at least some of the reconstructed transform coefficients in the reconstructed transform coefficient block may include:

[0437] If the value of the MIP transposition indication parameter indicates that a transposition is to be performed on the sample input vector used for the MIP mode, a second transform coefficient block is obtained by performing LFNST on at least some of the reconstructed transform coefficients in the reconstructed transform coefficient block in horizontal scan order. If the value of the MIP transposition indication parameter indicates that no transposition is to be performed on the sample input vector used for the MIP mode, a second transform coefficient block is obtained by performing LFNST on at least some of the reconstructed transform coefficients in the reconstructed transform coefficient block in vertical scan order.

[0438] Specifically, when the value of isTransposed is equal to 1, a transposition is performed on the sample input vector used for the MIP mode. Then, LFNST can be performed on at least some of the transform coefficients in the first transform coefficient block in the horizontal scan order, as shown in the above formula (10).

[0439] When the value of isTransposed is equal to 0, no transposition is performed on the sample input vector used for MIP mode. Then, LFNST can be performed on at least some transform coefficients in the first transform coefficient block in vertical scan order, as shown in equation (11) above.

[0440] log2LfnstSize is determined as shown in equation (9) above.

[0441] S906: Obtain a reconstructed residual block of a chroma component of the current block by performing a first transform on the second transform coefficient block.

[0442] S907: Determine a reconstructed block of the chroma component of the current block based on the intra predicted value of the chroma component of the current block and the reconstructed residual block.

[0443] In addition, in the decoder, the quantized values ​​of the LFNST coefficients can be obtained by analyzing the bitstream. The inverse quantization unit inversely quantizes (may be called "scaling") the quantized values ​​to obtain a reconstructed transform coefficient block. A second transform coefficient block can be obtained by performing an inverse LFNST on at least some of the reconstructed transform coefficients in the reconstructed transform coefficient block. Then, a reconstructed residual block of the chroma component of the current block can be recovered by an inverse first transform, where the inverse first transform is an inverse transform corresponding to the "first transform (core transform)" in the encoder. In addition, since only the "inverse transform" in the decoder is defined in the standard, the "inverse LFNST" may be called "LFNST" in the standard.

[0444] In addition, after recovering the reconstructed residual block of the chroma component of the current block, the reconstructed block of the chroma component of the current block can be determined by adding the intra predicted value of the chroma component of the current block and the reconstructed residual block.

[0445] The decoding method according to the embodiment of the present invention will be specifically described below with reference to the decoder 200 shown in FIG.

[0446] The input bitstream of the decoder 200 may be the bitstream generated by the encoder 100. The parsing unit 201 parses the input bitstream and obtains values ​​of syntax elements from the input bitstream. The parsing unit 201 converts binary representations of the syntax elements into numerical values ​​and sends the numerical values ​​to units in the decoder 200 to obtain one or more decoded images. The parsing unit 201 may further parse one or more syntax elements from the input bitstream to render the decoded images.

[0447] 10, which shows a flow chart of analyzing LFNST parameters according to an embodiment of the present invention. As shown in FIG. 10, the flow may include:

[0448] S1001: Parsing unit 201 checks to determine whether there is an LFNST parameter in the input bitstream.

[0449] The LFNST parameter can indicate the value of lfnst_index. Here, an example of the syntax structure for expressing the LFNST parameter may be the same as the example of the syntax structure shown in Table 8 above, or may be the same as the example of the syntax structure shown in Tables 9A and 9B above.

[0450] Specifically, when the intra prediction mode of the current block is the MIP mode, the parsing unit 201 obtains the coding bit of lfnst_index from the input bitstream using the following method. Optionally, when the minimum value of the width and height of the transform block of the current block is equal to or greater than 8 (i.e., both the width and the height are equal to or greater than 8), the parsing unit 201 reads the lfnst_index bit using the MIP mode by using the syntax structure in Table 9A. Another selectable condition may be set as the width and the height are equal to a first value (e.g., 8) and the sum of the width and the height is equal to or greater than a second value (e.g., 16).

[0451] Optionally, using the syntax structure in Table 9B, parsing unit 201 reads the lfnst_index bits using MIP mode if the mipSizeId of the current block is equal to 2. Another selectable condition may be set as the mipSizeId of the current block is equal to one of multiple values ​​(e.g., 1, 2).

[0452] S1002: The parsing unit 201 obtains the LFNST parameters by performing adaptive binary arithmetic decoding on the LFNST parameters.

[0453] Here, in the encoder 100, the entropy coding unit 115 encodes the LFNST parameters (e.g., the value of lfnst_index) using CABAC whose descriptor is “ae(v)” as shown in Tables 9A and 9B. In the decoder 200, the analysis unit 201 obtains the value of lfnst_index from the lfnst_index bits by performing adaptive binary arithmetic decoding.

[0454] The parsing unit 201 sends the value of the syntax element and one or more variables set or determined based on the value of the syntax element to obtain one or more decoded images to a unit in the decoder 200. The prediction unit 202 determines an intra-prediction block of a current block (e.g., a CU). If it is indicated that an inter-decoding mode is used to decode the current decoded block, the prediction unit 202 sends related parameters from the parsing unit 201 to the MC unit 203 to obtain an inter-prediction block. If it is indicated that an intra-prediction mode (including a MIP mode indicated based on a MIP mode index value) is used to decode the current block, the prediction unit 202 sends related parameters from the parsing unit 201 to the intra-prediction unit 204 to obtain an intra-prediction block.

[0455] It should be understood that the current block (or called "decoded block") in the embodiment of the present invention may be a CU or a partition of a CU (for example, a transform block). Specifically, when it is indicated that the current block is decoded using MIP mode, the intra prediction unit 204 obtains the intra prediction block of the current block. The intra prediction block of the current block includes the intra prediction value of at least one sample in the current block.

[0456] Specifically, the steps of the intra prediction unit 204 obtaining an intra prediction block using MIP mode are as follows:

[0457] First, the intra prediction unit 204 obtains one or more reference samples from neighboring samples of the current block, for example, by downsampling the neighboring samples or by directly extracting from the neighboring samples.

[0458] Then, the intra prediction unit 204 uses the obtained reference sample, MIP matrix and shift parameter to determine one or more partial prediction samples corresponding to sample positions in the current block. Here, the sample positions can be preset sample positions in the current block, and for example, the sample positions have uniform horizontal and vertical coordinate values. In addition, the shift parameters include a shift number parameter and a shift offset parameter, and the shift parameter can be used for offset operation in the process of obtaining intra prediction values ​​of partial samples.

[0459] Finally, when the prediction samples corresponding to the partial samples of the current block are obtained, the intra prediction unit 204 needs to further obtain prediction samples corresponding to the remaining samples other than the partial samples in the current block. For example, the intra prediction unit 204 can use an interpolation filter to obtain the prediction samples corresponding to the remaining samples, and the inputs of the interpolation filter can be the partial samples and the adjacent samples.

[0460] As shown in Figure 6, Figure 6 shows a flow chart of obtaining an intra prediction block using MIP mode. This flow can be implemented in a decoder 200, where the "current block" refers to the "decoded block".

[0461] In step 601, the intra prediction unit 204 obtains adjacent samples of the current block, and for example, the adjacent samples are marked as gray squares adjacent to the current block in step 601 shown in FIG. 6. The intra prediction unit 204 obtains one or more reference samples from the adjacent samples. In the example of step 601 shown in FIG. 6, optionally, the intra prediction unit 204 can calculate an average value of two adjacent samples, and use the average value as a reference sample. Alternatively, the intra prediction unit 204 can select one adjacent sample every other adjacent sample as a reference sample. For example, in the example of step 601 shown in FIG. 6, the intra prediction unit 204 selects four reference samples from the eight upper adjacent samples of the current block, and selects another four reference samples from the eight left adjacent samples of the current block.

[0462] The specific process of step 601 is as follows:

[0463] The intra prediction unit 204 obtains the width and height of the current block from the parsing unit 201, where the width and height are represented by variables cbWidth and cbHeight, respectively. Based on the parameters from the parsing unit 201, the current block is divided into one or more transform blocks. Let the variables nTbW and nTbH be the width and height of the transform blocks, respectively. When using the MIP mode as the intra prediction mode to obtain the prediction of the current block, the intra prediction unit 204 determines the MIP block size index value, i.e., a variable represented by mipSizeId.

[0464] Optionally, the intra prediction unit 204 determines the value of mipSizeId as follows. - If nTbW and nTbH are both equal to 4, set mipSizeId equal to 0. - Else, if cbWidth or cbHeight is equal to 4, set mipSizeId equal to 1. - Otherwise, set mipSizeId equal to 2.

[0465] Specifically, if the size parameter of the current block is 8x8 (ie, cbWidth and cbHeight are both equal to 8), then set mipSizeId equal to 2.

[0466] Optionally, the intra prediction unit 204 determines the value of mipSizeId as follows. - If nTbW and nTbH are both equal to 4, set mipSizeId equal to 0. - Else, if cbWidth or cbHeight is equal to 4, or cbWidth and cbHeight are both equal to 8, then set mipSizeId equal to 1. - Otherwise, set mipSizeId equal to 2.

[0467] Specifically, if the size parameter of the current block is 8x8, then set mipSizeId equal to 1.

[0468] In addition, the intra prediction unit 204 obtains the values ​​of boundarySize and predSize according to mipSizeId. -If mipSizeId is equal to 0, set boundarySize equal to 2 and set predSize equal to 4. - Else, if mipSizeId is equal to 1, then set boundarySize equal to 4 and set predSize equal to 4. - Otherwise (if mipSizeId is equal to 2), set boundarySize equal to 4 and set predSize equal to 8. The boundarySize represents the number of reference samples obtained from each of the upper adjacent reference samples and the left adjacent reference samples of the current block.

[0469] The intra prediction unit 204 may further obtain a reference sample for calculating the MIP prediction value based on a parameter indicating the order of the reference samples stored in the buffer pTemp from the analysis unit 701. The intra prediction unit 204 sets a variable isTransposed equal to a parameter indicating the order of the reference samples stored in the buffer pTemp from the analysis unit 701. For example, isTransposed equal to 0 (or FALSE) indicates that the intra prediction unit 204 first presents a reference sample obtained from the upper neighboring reference sample of the current block, and then presents a reference sample obtained from the left neighboring reference sample. Otherwise, isTransposed equal to 1 (or TRUE) indicates that the intra prediction unit 204 first presents a reference sample obtained from the left neighboring reference sample of the current block, and then presents a reference sample obtained from the upper neighboring reference sample.

[0470] The intra prediction unit 204 takes the value of inSize to indicate the number of reference samples used for MIP mode, as shown in equation (1) above.

[0471] The intra prediction unit 204 invokes the following process to obtain reference samples (stored as an array p[x], where x is 0, . . . , inSize-1) using neighboring reference samples of the current block.

[0472] The intra prediction unit 204 obtains nTbW reference samples from the current block's above-neighboring reference samples (e.g., stored in array refT) and obtains nTbH reference samples from the current block's left-neighboring reference samples (e.g., stored in array refL). The intra prediction unit 204 invokes downsampling on refT to obtain boundarySize reference samples and stores the boundarySize reference samples in refT. The intra prediction unit 204 invokes downsampling on refL to obtain boundarySize reference samples and stores the boundarySize reference samples in refL.

[0473] The intra-prediction unit 204 may further obtain the order indicated by isTransposed to place elements in refT and refL into buffer pTemp.

[0474] Optionally, the intra prediction unit 204 obtains p[x] as follows, where x=0, . . . , inSize-1. - When mipSizeId is equal to 2, p[x] = pTemp[x + 1] - pTemp[0]. - Otherwise (mipSizeId is less than 2), p[0] = pTemp[0] - (1 << (BitDepth-1)), p[x] = pTemp[x] - pTemp[0]. BitDepth is the bit depth of the color component of the sample in the current block, where the color component can be one of the RGB components, one of the YUV components, or one of the YCbCr components, such as the Y component.

[0475] Alternatively, the intra prediction unit 204 may obtain p[x] as follows, where x=0, . . . , inSize-1. - When mipSizeId is equal to 2, p[x] = pTemp[x + 1] - pTemp[0]. - else (mipSizeId is less than 2), then p[0] = (1 << (BitDepth-1)) - pTemp[0], p[x] = pTemp[x] - pTemp[0].

[0476] Alternatively, the intra prediction unit 204 may not determine the value of mipSizeId, and may obtain the value of p[x] in a unified calculation manner. For example, (1<<(BitDepth-1)) is added to the buffer pTemp as an additional factor. In this case, the intra prediction unit 204 calculates p[x] as pTemp[x+1]-pTemp[0].

[0477] In step 602, the intra prediction unit 204 obtains a MIP prediction value of the current block using the reference samples and a MIP weighting matrix. The MIP weighting matrix is ​​selected from a predefined MIP weighting matrix group based on a MIP mode index value (denoted as ModeId) corresponding to a MIP mode and a MIP block size index value (denoted as mipSizeId). The intra prediction unit 204 obtains the value of ModeId from the analysis unit 201.

[0478] The intra prediction unit 204 obtains MIP prediction values ​​(denoted as predMip[x][y]) of partial prediction samples corresponding to one or more sample positions in the current block. In the example of step 602 shown in Figure 6, the partial prediction samples are the samples marked as grey squares in the current block. The input of the prediction module 601 is the reference sample p[x] obtained in step 601, and the prediction module 601 calculates a partial prediction sample using a MIP weighting matrix and shift parameters. The shift parameters include a shift number parameter and a shift offset parameter.

[0479] In one possible embodiment, the prediction module 601 can represent a sample with coordinates (x, y), and the predicted sample can be represented as predMip[x][y]. The calculation formula of predMip[x][y] is as shown in Equation (5) or Equation (6).

[0480] In formula (5) or formula (6), mWeight[i][j] is a MIP weighting matrix, and the matrix elements may be predetermined constants or may be adaptively updated using a training method or the like. The input of the training method is one or more coded images or blocks, or an image from another bitstream provided to the decoder 200 by an external device, or an image obtained from the analysis unit 201 by analyzing a special data unit in the input bitstream containing the MIP weighting matrix. mWeight[i][j] is determined based on the MIP mode indicated by the corresponding one or more parameters from the analysis unit 201. fO is a shift offset parameter for determining oW, sW is a shift number parameter, p[i] is calculated using the reference sample, i.e., the MIP input sample value in the embodiment of the present invention, pTemp[0] represents the first sample in the reference sample, i.e., the value corresponding to index 0 in the first temporary reference value in the embodiment of the present invention, and ">>" is a binary right shift operator defined in VVC.

[0481] The prediction module 601 can determine the values ​​of sW and fO based on the size of the current block and the MIP mode used for the current block. In one example, the prediction module 601 obtains the values ​​of sW and fO using a lookup table.

[0482] Optionally, the prediction module 601 can determine sW based on the size parameter and MIP mode of the current block by utilizing Table 4 above.

[0483] Optionally, the prediction module 601 can also determine sW based on the size parameter of the current block by utilizing Table 5 above.

[0484] Alternatively, the prediction module 601 may also directly set sW to a constant value. For example, for blocks with various size parameters and different MIP modes, the prediction module 601 may set sW to 5. Or, for blocks with various size parameters and different MIP modes, the prediction module 601 may set sW to 6. Or, for blocks with various size parameters and different MIP modes, the prediction module 601 may set sW to 7.

[0485] Optionally, the prediction module 601 can determine fO based on the size parameter and MIP mode of the current block by utilizing Table 6 above.

[0486] Optionally, the prediction module 601 can also determine fO based on the size parameter of the current block by utilizing Table 7 above.

[0487] Optionally, the prediction module 601 can also directly set fO to a constant value (e.g., 0 to 100). For example, for blocks with various size parameters and different MIP modes, the prediction module 601 can set fO to 32. Or, the prediction module 601 sets fO to 46. Or, the prediction module 601 sets fO to 56. Or, the prediction module 601 sets fO to 66.

[0488] The intra prediction unit 204 may perform a clipping operation on the MIP predicted value in predMip. If isTransposed is equal to 1 (or TRUE), the predSize×predSize array predMip[x][y], where x = 0, ..., predSize-1, y = 0, ..., predSize-1, is transformed to predTemp[y][x] = predMip[x][y], and then predMip = predTemp.

[0489] Optionally, by way of example, if the size parameter of the current block is 8x8 (ie, cbWidth and cbHeight are both equal to 8), the intra prediction unit 204 sets mipSizeId equal to 2 and obtains a predMip of 8x8.

[0490] Optionally, if the size parameter of the current block is 8x8 (ie, cbWidth and cbHeight are both equal to 8), the intra prediction unit 204 sets mipSizeId equal to 1 and obtains a predMip of 4x4.

[0491] In step 603, the intra prediction unit 204 obtains intra prediction values ​​of the current block (stored in an array as predSamples[x][y], where x=0,...,nTbW-1, y=0,...,nTbH-1) as follows:

[0492] When the intra prediction unit 204 determines that nTbW is greater than predSize or nTbH is greater than predSize, it invokes an upsampling process to obtain predSamples using predMip. The intra prediction unit 204 obtains prediction samples corresponding to remaining samples other than partial samples in the current block. In FIG. 6, the intra prediction unit 204 can use a filtering module 602 to obtain prediction samples corresponding to remaining samples other than partial samples in the current block. The input of the filtering module 602 is the sample marked as a gray square in step 602. The filtering module 602 can use one or more interpolation filters to obtain prediction samples corresponding to remaining samples other than partial samples in the current block using the input. For example, the input can include a reference sample and partial prediction samples of one or more sample positions in the current block. Or, the input can include adjacent samples and partial prediction samples of one or more sample positions in the current block. Or, the input can include a reference sample, adjacent samples and partial prediction samples of one or more sample positions in the current block. For example, optionally, if the size of the current block is 8x8 (i.e., cbWidth and cbHeight are both equal to 8), the intra prediction unit 204 determines that mipSizeId is equal to 1, and the intra prediction unit 204 obtains an 8x8 intra prediction block of the current block by applying an upsampling process to the 4x4 predMip.

[0493] - Otherwise, the intra prediction unit 204 sets the intra prediction block of the current block equal to the MIP prediction block of the current block, i.e., sets predSamples[x][y] (x=0,...,nTbW-1,y=0,...,nTbH-1) equal to predMip[x][y]. For example, if the size parameter of the current block is 8x8 (i.e., cbWidth and cbHeight are both equal to 8), the intra prediction unit 204 obtains predSamples of the current block whose size is 8x8 (i.e., cbWidth and cbHeight are both equal to 8).

[0494] Thus, according to the flow block diagram shown in FIG. 6, after step 603, the intra prediction unit 204 can obtain an intra prediction block of the current block (i.e., the CU), i.e., determine an intra prediction value of at least one sample of the current block.

[0495] The scaling unit 205 has the same function as that of the inverse quantization unit 110 in the encoder 100. The scaling unit 205 performs a scaling operation on the quantized coefficients (i.e. levels) from the analysis unit 201 to obtain reconstructed coefficients.

[0496] The transform unit 206 has the same function as that of the inverse transform unit 111 in the encoder 100. The transform unit 206 performs one or more transform operations (i.e., the inverse operations of one or more transform operations performed by the inverse transform unit 111 in the encoder 100) to obtain a reconstructed residual.

[0497] If lfnst_index is not equal to 0, the transform unit 206 performs a secondary transform on the reconstructed coefficients to obtain a secondary transform coefficient block. Then, the transform unit 206 performs a first transform, for example an integer transform initially designed based on a DCT, on the secondary transform coefficient block to obtain a reconstructed residual block. The LFNST is an example of a secondary transform.

[0498] 11, which shows a schematic diagram of another flow of performing LFNST according to an embodiment of the present invention. In FIG. 11, block 1101 is a reconstructed transform coefficient block including reconstructed transform coefficients.

[0499] In step 1101 , the transform unit 206 performs a secondary transform, such as LFNST, on all or a portion of the reconstructed transform coefficients in block 1101 .

[0500] When the intra-prediction mode of the current block is a MIP mode, the transform unit 206 selects the LFNST transform kernel candidate set based on the MIP parameters. The MIP parameters may include one of a MIP mode index value (i.e., modelId) and a MIP transposition indication parameter (i.e., isTransposed).

[0501] The transform unit 206 sets the value of the variable predModeIntra equal to the value of modelId. The transform unit 206 determines a transform kernel for performing LFNST on coefficients in the sub-blocks 11001, 11002, and 11003. The size of the sub-blocks may be a set value, for example, 8×8. The transform unit 206 determines an index value of an LFNST transform kernel candidate set (i.e., lfnstTrSetIdx) based on the value of predModeIntra. For original block decoding in MIP mode, the transform unit 206 obtains the value of lfnstTrSetIdx using a lookup table such as that shown in Table 10. The transform unit 206 determines the transform kernel for performing LFNST as the transform kernel indicated by lfnst_index in the LFNST transform kernel candidate set indicated by lfnstTrSetIdx.

[0502] If lfnst_index is not equal to 0, transform unit 206 performs LFNST on the transform block using coefficients from sub-blocks 11001, 11002, and 11003 to obtain LFNST coefficients, and puts the LFNST coefficients into sub-blocks 11001, 11002, and 11003 of block 1102. Assume d[x][y] is a block of first transform coefficients, where x=0,...,nTbW-1, y=0,...,nTbH-1, nTbW and nTbH are the width and height, respectively, of block 1102. Assume v[x] is a block of LFNST coefficients, where x=0,...,nLfnstOutSize-1, nLfnstOutSize=(nTbW>=8 && nTbH>=8)? 48:16.

[0503] Optionally, the transform unit 206 may employ the following instructions to obtain the block of transform coefficients 1102, as shown in the above equation (8): log2LfnstSize is determined as the above equation (9).

[0504] Optionally, when isTransposed is equal to 0, the transform unit 206 employs the following instructions to obtain the block of transform coefficients 1102, as shown in the above equation (10): If isTransposed is equal to 1, the transform unit 206 employs the following instructions to obtain the block of transform coefficients 1102, as shown in the above equation (11):

[0505] Optionally, if isTransposed is equal to 1, the transform unit 206 employs the following instructions to obtain the block of transform coefficients 1102, as shown in equation (10) above: If isTransposed is equal to 0, the transform unit 206 employs the following instructions to obtain the block of transform coefficients 1102, as shown in the above equation (11):

[0506] log2LfnstSize is determined as shown in equation (9) above.

[0507] Also, when the intra prediction mode of the current block is a conventional intra prediction mode (i.e., angular intra prediction mode, DC mode, PLANAR mode, etc.), transform unit 206 determines the value of lfnstTrSetIdx, i.e., the LFNST transform kernel candidate set selected for the current block, based on the mode index value (assigned to predModeIntra) of the conventional intra prediction mode by using the lookup table shown in Table 11. Transform unit 206 determines the transform kernel for performing LFNST as the transform kernel indicated by lfnst_index in the LFNST transform kernel candidate set indicated by lfnstTrSetIdx.

[0508] If lfnst_index is not equal to 0, transform unit 206 performs LFNST on the transform block using coefficients from sub-blocks 11001, 11002, and 11003 to obtain LFNST coefficients, and puts the LFNST coefficients into sub-blocks 11001, 11002, and 11003 of block 1102. Assume d[x][y] is a block of first transform coefficients, where x=0,...,nTbW-1, y=0,...,nTbH-1, nTbW and nTbH are the width and height, respectively, of block 1102. Assume v[x] is a block of LFNST coefficients, where x=0,...,nLfnstOutSize-1, nLfnstOutSize=(nTbW>=8 && nTbH>=8)? 48:16.

[0509] Optionally, when predModeIntra is less than or equal to 34, the transform unit 206 adopts the following instruction to obtain the block 1102 of transform coefficients, as shown in the above equation (10): Optionally, when predModeIntra is greater than 34, the transform unit 206 employs the following instructions to obtain the block 1102 of transform coefficients, as shown in the above equation (11):

[0510] Optionally, when predModeIntra is greater than 34, the transform unit 206 employs the following instructions to obtain the block 1102 of transform coefficients, as shown in the above equation (10): Optionally, when predModeIntra is less than or equal to 34, the transform unit 206 adopts the following instruction to obtain the block 1102 of transform coefficients, specifically as shown in the above equation (11):

[0511] log2LfnstSize is determined as shown in equation (9) above.

[0512] In step 1102, the transform unit 206 obtains a reconstructed residual block 1103 of the current block by performing a first transform (i.e., a core transform), for example an integer transform initially designed based on a DCT, on the transform coefficients in the block 1102.

[0513] Furthermore, adder 207 performs an addition operation on its inputs (the intra prediction block from prediction unit 202 and the reconstructed residual block from transform unit 206) to obtain a reconstructed block of the current block. The reconstructed block is also sent to prediction unit 202 to be used as a reference for other blocks coded under intra prediction modes.

[0514] After all CUs in an image or sub-image are reconstructed, the filtering unit 208 performs in-loop filtering on the reconstructed image or sub-image. The filtering unit 208 includes one or more filters, such as a deblocking filter, a sample adaptive offset (SAO) filter, an adaptive loop filter (ALF), a luma mapping with chroma scaling (LMCS) filter, and a neural network-based filter. Or, the filtering unit 208 performs in-loop filtering on one or more target samples in the reconstructed block when it determines that the reconstructed block is not used as a reference for decoding other blocks.

[0515] The output of the filtering unit 208 is a decoded image or sub-image, which is sent to the DPB unit 209. The DPB unit 209 outputs a decoded image based on timing and control information. The images stored in the DPB unit 209 can further be used as references for inter- or intra-prediction by the prediction unit 202.

[0516] In an embodiment of the present invention, the decoder 200 may be a computing device including a processor and a storage medium for recording a decoding program. When the processor reads and executes the decoding program, the decoder 200 reads the input bitstream and generates a corresponding decoded video. The decoder 200 may also be a computing device including one or more chips. The units implemented as integrated circuits in the chips have similar connections and data exchange functions as the corresponding units in FIG. 2.

[0517] In an embodiment of the present invention, a decoding method is provided. The bitstream is analyzed to obtain prediction parameters and LFNST parameters of a current block. The prediction parameters include a prediction mode parameter. When the prediction mode parameter indicates that a MIP mode is used to determine an intra prediction value of a chroma component of the current block, neighboring sample values ​​of the current block are obtained, and MIP input sample values ​​of the current block are determined based on the neighboring sample values ​​of the current block. The MIP prediction value of the chroma component of the current block is determined based on the MIP input sample values, a MIP weighting matrix, and a shift parameter. The intra prediction value of the chroma component of the current block is determined by filtering the MIP prediction value. When the LFNST parameter indicates that LFNST is performed on the current block, a reconstructed transform coefficient block of the current block is determined, and a second transform coefficient block is obtained by performing LFNST on at least some reconstructed transform coefficients in the reconstructed transform coefficient block. The reconstructed residual block of the chroma component of the current block is obtained by performing a first transform on the second transform coefficient block. The reconstructed block of the chroma component of the current block is determined based on the intra prediction value of the chroma component of the current block and the reconstructed residual block. In this way, in the MIP mode, while ensuring the decoding performance, the complexity can be reduced, and at the same time, the memory space required for the decoding process can be reduced, and the decoding efficiency can be effectively improved. In addition, when the LFNST technology is applied to the MIP mode prediction, the introduction of the MIP parameters makes the LFNST more flexible, and the decoding efficiency can be further improved.

[0518] In another embodiment of the present invention, based on the same inventive idea as the above embodiment, refer to Fig. 12, which shows a schematic diagram of the structure of an encoder 120 according to an embodiment of the present invention. As shown in Fig. 12, the encoder 120 can include a first decision unit 1201, a first prediction unit 1202, a first transform unit 1203 and an encoding unit 1204.

[0519] The first determining unit 1201 is configured to determine prediction parameters of a current block, where the prediction parameters include a prediction mode parameter. The first determination unit 1201 is further configured to obtain neighboring sample values ​​of the current block, and determine MIP input sample values ​​of the current block based on the neighboring sample values ​​of the current block when the prediction mode parameter indicates to use a MIP mode to determine the intra prediction value of the chroma component of the current block. The first prediction unit 1202 is configured to determine a MIP prediction value of a chroma component of the current block based on a MIP input sample value, a MIP weighting matrix and a shift parameter, and determine an intra prediction value of the chroma component of the current block by filtering the MIP prediction value, where the MIP prediction value is a prediction value of some samples in the chroma component of the current block. The first transform unit 1203 is configured to determine, based on the intra-predicted value of the chroma component of the current block, a prediction residual value of the chroma component of the current block, and determine LFNST parameters by performing LFNST on the prediction residual value. The encoding unit 1204 is configured to encode and write the LFNST parameters into a bitstream.

[0520] In some embodiments, the neighboring sample values ​​of the current block include a left neighboring sample value and an above neighboring sample value of the current block.

[0521] In some embodiments, the shift parameters include a shift offset parameter and a shift number parameter. The first determination unit 1201 is specifically configured to determine a product of a sum of MIP input sample values ​​and a shift offset parameter based on a value of a shift offset parameter which is a fixed constant, determine a first constant value based on a value of a shift number parameter which is a fixed constant, set a value of the first offset to a difference between the first constant value and the above multiplication product, determine a MIP weighting matrix of the current block based on the prediction parameters, and determine a MIP predicted value of a chroma component of the current block based on the MIP weighting matrix, the MIP input sample values, the shift number parameter and the first offset.

[0522] In some embodiments, the prediction parameters further include a size parameter of the current block.

[0523] In some embodiments, the first determining unit 1201 is configured to determine a block size index value of the current block based on a size parameter of the current block, and obtain the first temporary reference value by down-sampling adjacent sample values ​​of the current block. In addition, the first determination unit 1201 is configured to determine a second constant value based on the bit depth of the adjacent sample values ​​of the current block when the block size index value of the current block is within a predetermined range, set a value corresponding to index 0 in the MIP input sample value to a difference between the second constant value and a value corresponding to index 0 in the first temporary reference value, and set a value corresponding to index i in the MIP input sample value to a difference between a value corresponding to index i in the first temporary reference value and a value corresponding to index 0 in the first temporary reference value, where i is an integer greater than 0. In addition, the first determination unit 1201 is further configured to: set a value corresponding to index j in the MIP input sample value to a difference between a value corresponding to index (j+1) in the first temporary reference value and a value corresponding to index 0 in the first temporary reference value, when the block size index value of the current block is outside a preset range, where j is an integer greater than or equal to 0.

[0524] In some embodiments, the first determination unit 1201 is further configured to: determine a second constant value based on the bit depth of the adjacent sample values ​​of the current block when the block size index value of the current block is within a preset range; set a value corresponding to index 0 in the MIP input sample value to a difference between the value corresponding to index 0 in the first temporary reference value and the second constant value; and set a value corresponding to index i in the MIP input sample value to a difference between the value corresponding to index i in the first temporary reference value and the value corresponding to index 0 in the first temporary reference value, where i is an integer greater than 0.

[0525] In some embodiments, the first determining unit 1201 is further configured to: set a block size index value of the current block to 0 when the width and height of the current block are both equal to 4; set a block size index value of the current block to 1 when the width and height of the current block are both equal to 8 or one of the width and height of the current block is equal to 4; and set a block size index value of the current block to 2 when the width and height of the current block do not satisfy the above conditions.

[0526] In some embodiments, the first determination unit 1201 is further configured to obtain a first temporary reference value by downsampling neighboring sample values ​​of the current block, determine a second constant value based on a bit depth of the neighboring sample values ​​of the current block, obtain the second temporary reference value by buffering the second constant value within one data unit after the first temporary reference value, and set a value corresponding to index j in the MIP input sample value to a difference between a value corresponding to index (j+1) in the second temporary reference value and a value corresponding to index 0 in the second temporary reference value, where j is an integer greater than or equal to 0.

[0527] In some embodiments, the first determining unit 1201 is further configured to set the second constant value to a power of two, where the exponent of the power is an integer and is equal to a bit depth of the adjacent sample values ​​of the current block minus one.

[0528] In some embodiments, the first determining unit 1201 is further configured to obtain the second constant value by binary left-shifting “1”, where the number of left-shifted bits is equal to the bit depth of the adjacent sample values ​​of the current block minus 1.

[0529] In some embodiments, the value of the shift number parameter is a fixed constant that is unrelated to the block size index value and the MIP mode index value.

[0530] In some embodiments, the value of the shift number parameter is set to six.

[0531] In some embodiments, the value of the shift offset parameter is a fixed constant that is unrelated to the block size index value and the MIP mode index value.

[0532] In some embodiments, the value of the shift offset parameter is set to 32.

[0533] In some embodiments, the first determining unit 1201 is further configured to set the first constant value to a power of two, the exponent of the power being an integer and equal to the value of the shift number parameter minus one.

[0534] In some embodiments, the first determination unit 1201 is further configured to obtain the first constant value by binary left-shifting a “1”, where the number of left-shifted bits is equal to the value of the shift number parameter minus 1.

[0535] 12, in some embodiments, the encoder 120 may further include a first calculation unit 1205. The first calculation unit 1205 is configured to calculate a first weighted sum of a MIP weighting matrix and a MIP input sample value, calculate a first sum of the first weighted sum and a first offset, and obtain a first right shift value by binary right shifting the first sum, where the number of right shifted bits is equal to the value of a shift number parameter. The first determining unit 1201 is further configured to set the MIP predicted value of the chroma component of the current block to a sum of the first right-shifted value and a value corresponding to index 0 in the first temporary reference value, where the first temporary reference value is obtained by downsampling adjacent sample values ​​of the current block.

[0536] In some embodiments, the first transform unit 1203 is configured to determine a residual block of the current block based on the prediction residual value, and obtain a first transform coefficient block by performing a first transform on the residual block. The first determining unit 1201 is further configured to determine whether to perform LFNST on at least some transform coefficients in the first transform coefficient block. The first transform unit 1203 is further configured to determine LFNST parameters by performing LFNST on at least some of the transform coefficients in the first transform coefficient block when the result of the decision is to perform LFNST on at least some of the transform coefficients.

[0537] In some embodiments, the first determination unit 1201 is further configured to: determine a minimum value among size parameters of the current block when determining an intra-prediction value of a chroma component of the current block using the MIP mode; and, if the minimum value is greater than or equal to a first preset threshold, determine to perform LFNST on at least some transform coefficients in the first transform coefficient block.

[0538] In some embodiments, the LFNST parameters include an LFNST index, and the first determining unit 1201 is further configured to: set a value of the LFNST index to greater than 0 when it is determined to perform LFNST on at least some transform coefficients in the first transform coefficient block, and set a value of the LFNST index to 0 when it is determined to not perform LFNST on at least some transform coefficients in the first transform coefficient block. The encoding unit 1204 is configured to encode and write the LFNST index into a bitstream.

[0539] In some embodiments, the LFNST parameters further include LFNST coefficients. Referring to FIG. The first transform unit 1203 is configured to obtain LFNST coefficients by performing LFNST on at least some of the transform coefficients in the first transform coefficient block. The quantization unit 1206 is configured to quantize the LFNST coefficients to obtain quantized coefficients. The encoding unit 1204 is further configured to encode and write the quantized coefficients into a bitstream.

[0540] In the embodiment of the present invention, it can be understood that a "unit" can be a part of a circuit, a part of a processor, a part of a program, or a part of software. Of course, a "unit" can be a module or a non-module. In addition, each constituent unit according to the present embodiment can be integrated into one processing unit, each unit can exist physically alone, or two or more units can be integrated into one unit. The integrated unit can be realized in the form of a hardware or software functional module.

[0541] When the integrated unit is realized as a software functional module and sold or used as an independent product, it may be stored in a computer-readable recording medium. According to this understanding, the essential part of the technical solution of the present embodiment, or the part that can contribute to the prior art, or all or part of the technical solution may be expressed as a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) or a processor to execute all or part of the steps of the method described in the present embodiment. The storage medium includes various kinds of media capable of storing program code, such as a universal serial bus (USB) flash disk, a mobile hard disk, a read only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0542] Therefore, in an embodiment of the present invention, a computer storage medium is provided for application to the encoder 120. The computer storage medium stores a computer program which, when executed by a first processor, performs any of the methods in the above embodiments.

[0543] Based on the above configuration of the encoder 120 and the computer storage medium, refer to FIG. 13, which shows a schematic diagram of a specific hardware structure of the encoder 120 according to an embodiment of the present invention. As shown in FIG. 13, the encoder 120 may include a first communication interface 1301, a first memory 1302 and a first processor 1303. Each component is coupled through a first bus system 1304. The first bus system 1304 is used to realize the connection and communication between these components. In addition to a data bus, the first bus system 1304 further includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, each type of bus is referred to as the first bus system 1304 in FIG. 13.

[0544] The first communication interface 1301 is configured to receive and transmit signals in the process of receiving and transmitting information to and from other external network elements.

[0545] The first memory 1302 is configured to store a computer program executable by the first processor 1303 .

[0546] The first processor 1303, when executing a computer program, is configured to perform the following:

[0547] Determining prediction parameters for the current block, the prediction parameters including a prediction mode parameter. When the prediction mode parameter indicates that an MIP mode is used to determine an intra prediction value of a chroma component of the current block, neighboring sample values ​​of the current block are obtained, and an MIP input sample value of the current block is determined based on the neighboring sample values ​​of the current block. Based on the MIP input sample values, the MIP weighting matrix and the shift parameters, a MIP prediction value of the chroma component of the current block is determined, where the MIP prediction value is a prediction value of some samples in the chroma component of the current block. An intra prediction value of the chroma component of the current block is determined by filtering the MIP prediction value. A prediction residual value of the chroma component of the current block is determined based on the intra prediction value of the chroma component of the current block. LFNST parameters are determined by performing LFNST on the prediction residual values. Encode and write the LFNST parameters into the bitstream.

[0548] The first memory 1302 of the embodiment of the present invention can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. The non-volatile memory can be a Read-Only Memory (ROM), a Programmable Read-Only Memory (PROM), an Erasable Programmable Read-Only Memory (EPROM), an Electrically Erasable Programmable Read-Only Memory (EEPROM), or a Flash Memory. The volatile memory can be a Random Access Memory (RAM) that functions as an external high-speed cache. By way of illustrative, but non-limiting examples, various RAMs are available, such as static random access memory (Static RAM, SRAM), dynamic random access memory (DRAM, DRAM), synchronous dynamic random access memory (Synchronous DRAM, SDRAM), double data rate synchronous dynamic random access memory (Double Data Rate SDRAM, DDRSDRAM), enhanced synchronous dynamic random access memory (Enhanced SDRAM, ESDRAM), synchronous link dynamic random access memory (Synch-link DRAM, SLDRAM), and direct Rambus random access memory (Dirct Rambus RAM, DRRAM). The first memory 1302 of the systems and methods described herein may include, but is not limited to, these and any other suitable types of memory.

[0549] The first processor 1303 can be an integrated circuit chip having signal processing capabilities. In the implementation process, each step of the above method embodiment can be completed by the first processor 1303's integrated logic circuit in the form of hardware or instructions in the form of software. The first processor 1303 can be a general-purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component. The processor can realize or execute various methods, steps and logic block diagrams disclosed in the embodiments of the present invention. The general-purpose processor can be a microprocessor or any ordinary processor, etc. The steps of the method disclosed in the embodiments of the present invention can be directly performed and completed by a hardware decoding processor, or can be performed and completed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium well known in the art, such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the first memory 1302. The first processor 1303 reads the information in the first memory 1302 and completes the steps of the above method together with the hardware of the processor.

[0550] It can be understood that the embodiments described in the present invention can be realized by hardware, software, firmware, middleware, microcode, or a combination thereof. When realized by hardware, the processing unit can be realized by one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processing (DSPs), DSP Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), general-purpose processors, controllers, microcontrollers, microprocessors, other electronic units, or combinations thereof, used to perform the functions described in the present invention. When realized by software, the techniques described in the present invention can be realized by modules (e.g., procedures, functions, etc.) for performing the functions described in the present invention. The software code is stored in a memory and executed by a processor. The memory can be realized within the processor or external to the processor.

[0551] Optionally, as another embodiment, the first processor 1303 is configured to execute any of the methods in the above embodiments when executing a computer program.

[0552] In an embodiment of the present invention, an encoder is provided. The encoder can include a first decision unit, a first prediction unit, a first transformation unit and an encoding unit. In this way, in MIP mode, while ensuring the encoding performance, the complexity can be reduced, and at the same time, the storage space required for the encoding process can be reduced, and the encoding efficiency can be effectively improved. In addition, when LFNST technology is applied to MIP mode prediction, the introduction of MIP parameters makes LFNST more flexible, and the encoding efficiency can be further improved.

[0553] In another embodiment of the present invention, based on the same inventive idea as the above embodiment, refer to Fig. 14, which shows a schematic diagram of the structure of a decoder 140 according to an embodiment of the present invention. As shown in Fig. 14, the decoder 140 can include: a parsing unit 1401, a second determining unit 1402, a second predicting unit 1403 and a second transform unit 1404.

[0554] The parsing unit 1401 is configured to parse the bitstream to obtain prediction parameters and LFNST parameters of a current block, where the prediction parameters include a prediction mode parameter. The second determination unit 1402 is configured to obtain neighboring sample values ​​of the current block, and determine MIP input sample values ​​of the current block based on the neighboring sample values ​​of the current block when the prediction mode parameter indicates to use a MIP mode to determine an intra-prediction value of the chroma component of the current block. The second prediction unit 1403 is configured to determine a MIP predicted value of the chroma component of the current block based on the MIP input sample value, the MIP weighting matrix and the shift parameter, and determine an intra predicted value of the chroma component of the current block by filtering the MIP predicted value, where the MIP predicted value is a predicted value of some samples in the chroma component of the current block. The second transform unit 1404 is configured to, when the LFNST parameter indicates to perform LFNST on the current block, determine a reconstructed transform coefficient block of the current block, perform LFNST on at least some of the reconstructed transform coefficients in the reconstructed transform coefficient block to obtain a second transform coefficient block, and perform a first transform on the second transform coefficient block to obtain a reconstructed residual block of a chroma component of the current block. The second determining unit 1402 is further configured to determine, based on the intra predicted value of the chroma component of the current block and the reconstructed residual block, a reconstructed block of the chroma component of the current block.

[0555] In some embodiments, the neighboring sample values ​​of the current block include a left neighboring sample value and an above neighboring sample value of the current block.

[0556] In some embodiments, the shift parameters include a shift offset parameter and a shift number parameter. The second determination unit 1402 is specifically configured to determine a product of the sum of the MIP input sample values ​​and the shift offset parameter based on the value of a shift offset parameter which is a fixed constant, determine a first constant value based on the value of a shift number parameter which is a fixed constant, set the value of the first offset to the difference between the first constant value and the above product, determine a MIP weighting matrix of the current block based on the prediction parameters, and determine a MIP predicted value of the chroma component of the current block based on the MIP weighting matrix, the MIP input sample values, the shift number parameter and the first offset.

[0557] In some embodiments, the prediction parameters further include a size parameter of the current block.

[0558] In some embodiments, the second determining unit 1402 is configured to determine a block size index value of the current block based on a size parameter of the current block, and obtain the first temporary reference value by downsampling adjacent sample values ​​of the current block. The second determination unit 1402 is also configured to determine a second constant value based on the bit depth of the adjacent sample values ​​of the current block when the block size index value of the current block is within a predetermined range, set a value corresponding to index 0 in the MIP input sample value to a difference between the second constant value and a value corresponding to index 0 in the first temporary reference value, and set a value corresponding to index i in the MIP input sample value to a difference between a value corresponding to index i in the first temporary reference value and a value corresponding to index 0 in the first temporary reference value, where i is an integer greater than 0. In addition, the second determining unit 1402 is further configured to: when the block size index value of the current block is outside the preset range, set a value corresponding to index j in the MIP input sample value to a difference between a value corresponding to index (j+1) in the first temporary reference value and a value corresponding to index 0 in the first temporary reference value, where j is an integer greater than or equal to 0.

[0559] In some embodiments, the second determination unit 1402 is further configured to: determine a second constant value based on the bit depth of the adjacent sample values ​​of the current block when the block size index value of the current block is within a preset range; set a value corresponding to index 0 in the MIP input sample value to the difference between the value corresponding to index 0 in the first temporary reference value and the second constant value; and set a value corresponding to index i in the MIP input sample value to the difference between the value corresponding to index i in the first temporary reference value and the value corresponding to index 0 in the first temporary reference value, where i is an integer greater than 0.

[0560] In some embodiments, the second determining unit 1402 is further configured to: set a block size index value of the current block to 0 when the width and height of the current block are both equal to 4; set a block size index value of the current block to 1 when the width and height of the current block are both equal to 8 or one of the width and height of the current block is equal to 4; and set a block size index value of the current block to 2 when the width and height of the current block do not satisfy the above condition.

[0561] In some embodiments, the second determination unit 1402 is further configured to obtain a first temporary reference value by downsampling neighboring sample values ​​of the current block, determine a second constant value based on a bit depth of the neighboring sample values ​​of the current block, obtain the second temporary reference value by buffering the second constant value within one data unit after the first temporary reference value, and set a value corresponding to index j in the MIP input sample value to a difference between a value corresponding to index (j+1) in the second temporary reference value and a value corresponding to index 0 in the second temporary reference value, where j is an integer greater than or equal to 0.

[0562] In some embodiments, the second determining unit 1402 is further configured to set the second constant value to a power of two, where the exponent of the power is an integer and is equal to the bit depth of the adjacent sample values ​​of the current block minus one.

[0563] In some embodiments, the second determining unit 1402 is further configured to obtain the second constant value by binary left-shifting “1”, where the number of left-shifted bits is equal to the bit depth of the adjacent sample values ​​of the current block minus 1.

[0564] In some embodiments, the value of the shift number parameter is a fixed constant that is unrelated to the block size index value and the MIP mode index value.

[0565] In some embodiments, the value of the shift number parameter is set to six.

[0566] In some embodiments, the value of the shift offset parameter is a fixed constant that is unrelated to the block size index value and the MIP mode index value.

[0567] In some embodiments, the value of the shift offset parameter is set to 32.

[0568] In some embodiments, the second determining unit 1402 is further configured to set the first constant value to a power of two, the exponent of the power being an integer and equal to the value of the shift number parameter minus one.

[0569] In some embodiments, the second determination unit 1402 is further configured to obtain the first constant value by binary left-shifting a “1”, where the number of left-shifted bits is equal to the value of the shift number parameter minus 1.

[0570] 14, in some embodiments, the decoder 140 may further include a second calculation unit 1405. The second calculation unit 1405 is configured to calculate a first weighted sum of the MIP weighting matrix and the MIP input sample values, calculate a first sum of the first weighted sum and a first offset, and obtain a first right-shift value by binary right-shifting the first sum, where the number of right-shifted bits is equal to the value of the shift number parameter. The second determination unit 1402 is further configured to set the MIP predicted value of the chroma component of the current block to a sum of the first right-shifted value and a value corresponding to index 0 in the first temporary reference value, where the first temporary reference value is obtained by downsampling adjacent sample values ​​of the current block.

[0571] In some embodiments, the second determining unit 1402 is further configured to determine whether there is an LFNST parameter in the bitstream. The parsing unit 1401 is further configured to parse the bitstream to obtain the LFNST parameters, if any, in the bitstream.

[0572] In some embodiments, the second determining unit 1402 is further configured to: determine a minimum value among size parameters of the current block when determining an intra-prediction value of a chroma component of the current block using the MIP mode; and determine that there is an LFNST parameter in the bitstream if the minimum value is greater than or equal to a first preset threshold.

[0573] In some embodiments, referring to FIG. 14, the decoder 140 may further include an inverse quantization unit 1406 . The parsing unit 1401 is further configured to parse the bitstream to obtain quantized coefficients of the current block if the LFNST parameter indicates to perform LFNST on the current block. The inverse quantization unit 1406 is configured to obtain a reconstructed transform coefficient block of the current block by inverse quantizing the quantized coefficients.

[0574] In some embodiments, the inverse quantization unit 1406 is specifically configured to obtain a reconstructed transform coefficient block of the current block by scaling the quantized coefficients.

[0575] In some embodiments, the parsing unit 1401 is specifically configured to parse the bitstream to obtain a value of an LFNST index, and if the value of the LFNST index is greater than 0, determine to perform LFNST on the current block, and if the value of the LFNST index is equal to 0, determine not to perform LFNST on the current block.

[0576] In this embodiment, it can be understood that a "unit" can be a part of a circuit, a part of a processor, a part of a program, or a part of software. Of course, a "unit" can be a module or a non-module. In addition, each constituent unit according to this embodiment can be integrated into one processing unit, each unit can exist physically alone, or two or more units can be integrated into one unit. The integrated unit can be realized in the form of a hardware or software functional module.

[0577] The integrated unit may be realized as a software functional module and stored in a computer-readable recording medium when sold or used as an independent product. According to this understanding, in an embodiment of the present invention, a computer storage medium is provided which is applied to the decoder 140. The computer storage medium stores a computer program, which, when executed by the second processor, performs any method in the above embodiment.

[0578] Based on the above configuration of the decoder 140 and the computer storage medium, refer to FIG. 15, which shows a schematic diagram of a specific hardware structure of the decoder 140 according to an embodiment of the present invention. As shown in FIG. 15, the decoder 140 can include a second communication interface 1501, a second memory 1502 and a second processor 1503. Each component is coupled through a second bus system 1504. The second bus system 1504 is used to realize the connection and communication between these components. In addition to a data bus, the second bus system 1504 further includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, each type of bus is referred to as the second bus system 1504 in FIG. 15.

[0579] The second communication interface 1501 is configured to receive and transmit signals in the process of receiving and transmitting information to and from other external network elements.

[0580] The second memory 1502 is configured to store a computer program executable by the second processor 1503 .

[0581] The second processor 1503, when executing a computer program, is configured to perform the following:

[0582] Parse the bitstream to obtain prediction parameters and LFNST parameters of the current block. The prediction parameters include a prediction mode parameter. When the prediction mode parameter indicates that an MIP mode is used to determine an intra prediction value of a chroma component of the current block, neighboring sample values ​​of the current block are obtained, and an MIP input sample value of the current block is determined based on the neighboring sample values ​​of the current block. Based on the MIP input sample values, the MIP weighting matrix and the shift parameters, a MIP prediction value of the chroma component of the current block is determined, where the MIP prediction value is a prediction value of some samples in the chroma component of the current block. An intra prediction value of the chroma component of the current block is determined by filtering the MIP prediction value. If the LFNST parameter indicates to perform LFNST on the current block, a reconstructed transform coefficient block of the current block is determined, and a second transform coefficient block is obtained by performing LFNST on at least some of the reconstructed transform coefficients in the reconstructed transform coefficient block. A reconstructed residual block of the chroma component of the current block is obtained by performing a first transform on the second transform coefficient block. A reconstructed block of the chroma component of the current block is determined based on the intra predicted value of the chroma component of the current block and the reconstructed residual block.

[0583] Optionally, as another embodiment, the second processor 1503 is configured to execute any of the methods in the above embodiments when executing a computer program.

[0584] The hardware functions of the second memory 1502 are similar to those of the first memory 1302 , and the hardware functions of the second processor 1503 are similar to those of the first processor 1303 .

[0585] In this embodiment, a decoder is provided. The decoder can include an analysis unit, a second decision unit, a second prediction unit and a second transformation unit. In this way, in MIP mode, while ensuring the decoding performance, the complexity can be reduced, and at the same time, the memory space required for the decoding process can be reduced, and the decoding efficiency can be effectively improved. In addition, when LFNST technology is applied to MIP mode prediction, the introduction of MIP parameters makes LFNST more flexible, and the decoding efficiency can be further improved.

[0586] In another embodiment of the present invention, FIG. 16 shows a schematic diagram of the structure of a transmitting device according to an embodiment of the present invention. As shown in FIG. 16, an exemplary transmitting device 1600 is shown. The collecting unit 1601 acquires a video signal and sends the video signal to an encoder 1602. The collecting unit 1601 may be a device with one or more cameras (including depth cameras). The collecting unit 1601 may be a device that partially or fully decodes a bitstream to acquire a video. The collecting unit 1601 may further include one or more elements to capture an audio signal. One specific embodiment of the encoder 1602 is the encoder 100 or the encoder 120 described in the above embodiment, which decodes the video signal from the collecting unit 1601, which is its input video, and generates a video bitstream. The encoder 1602 may further include one or more audio encoders to generate an audio bitstream by encoding the audio signal. The storing / transmitting unit 1603 receives the video bitstream from the encoder 1602. The storage / transmission unit 1603 further receives the audio bitstream from the encoder 1602 and forms a media file (eg, ISO-based media file format) or a transmission stream by compressing the video and audio bitstreams together.

[0587] Optionally, the storage / transmission unit 1603 writes the media file or transmission stream to a storage unit, examples of which include a hard disk, a digital versatile disc (DVD) disc, a cloud disk, and a portable storage device.

[0588] Optionally, the storage / transmission unit 1603 sends the bitstream to a transport network, examples of which include the Internet, a wired network, a cellular network, a wireless local area network, and so on.

[0589] In another embodiment of the present invention, refer to FIG. 17, which shows a schematic diagram of a structure of a target device according to an embodiment of the present invention. As shown in FIG. 17, an exemplary target device 1700 is shown. The receiving unit 1701 receives a media file or a transmission stream from a network, or reads a media file or a transmission stream from a storage device. The receiving unit 1701 separates a video bitstream and an audio bitstream from the media file or the transmission stream. The receiving unit 1701 can further generate a new video bitstream by extracting the video bitstream. The receiving unit 1701 can further generate a new audio bitstream by extracting the audio bitstream. The decoder 1702 includes one or more video decoders, and one specific embodiment of the video decoder is the decoder 200 or the decoder 140 described in the above embodiment as an example.

[0590] The decoder 1702 may further include one or more audio decoders. The decoder 1702 decodes the video bitstream and the audio bitstream from the receiving unit 1701 to obtain a decoded video and one or more decoded audios corresponding to one or more sound channels. The rendering unit 1703 performs operations on the reconstructed video so that the reconstructed video is suitable for display. The operations may include one or more of perceptual quality enhancing operations such as denoising, synthesis, color space conversion, upsampling, downsampling, etc. The rendering unit 1703 may further perform operations on the decoded audio to enhance the perceptual quality of the displayed audio signal.

[0591] In another embodiment of the present invention, refer to FIG. 18, which shows a schematic diagram of the structure of a communication system according to an embodiment of the present invention. As shown in FIG. 18, an exemplary communication system 1800 is shown. The source device 1801 may be the transmitting device 1600 shown in FIG. 16. The output of the storage / transmission unit 1603 is processed by the storage medium / transport network 1802 to store / transmit the bitstream. The target device 1803 may be the target device 1700 shown in FIG. 17. Here, the receiving unit 1701 can obtain the bitstream from the storage medium / transport network 1802. The receiving unit 1701 can extract a new video bitstream from the media file or the transmission stream. The receiving unit 1701 can further extract a new audio bitstream from the media file or the transmission stream.

[0592] It should be noted that, in the present invention, terms such as "comprises," "having," or other variants are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that includes a set of elements may include not only those elements, but also other elements not expressly listed, or may include other elements inherent in the process, method, article, or apparatus. Unless otherwise limited, a process, method, article, or apparatus that includes an element defined by the phrase "comprises" does not exclude the presence of other identical elements.

[0593] The sequence numbers of the above embodiments of the present invention are not used to indicate superiority or inferiority of the embodiments, but are used for explanation purposes only.

[0594] The methods disclosed in the several method embodiments of the present invention can be combined in any manner to arrive at new method embodiments, unless there is a contradiction.

[0595] The features disclosed in the various product embodiments of the present invention can be combined in any manner, unless inconsistent, to arrive at new product embodiments.

[0596] The features disclosed in any of the method or apparatus embodiments according to the present invention may be combined in any manner, unless inconsistent, to arrive at new method or apparatus embodiments.

[0597] The above is only a specific embodiment of the present invention, and the protection scope of the present invention is not limited thereto. Any modifications or replacements that a person skilled in the art can easily think of within the technical scope disclosed in the present invention should be included in the scope of the present invention. Therefore, the protection scope of the present invention should be determined by the protection scope of the claims. [Industrial Applicability]

[0598] In an embodiment of the present invention, after determining the prediction parameters of the current block, the encoder obtains neighboring sample values ​​of the current block, and determines MIP input sample values ​​of the current block according to the neighboring sample values ​​of the current block when the prediction mode parameter indicates that the MIP mode is used to determine the intra predicted value of the chroma component of the current block; determines the MIP predicted value of the chroma component of the current block according to the MIP input sample values, the MIP weighting matrix and the shift parameter; determines the intra predicted value of the chroma component of the current block by filtering the MIP predicted value; determines the prediction residual value of the chroma component of the current block according to the intra predicted value of the chroma component of the current block; determines the LFNST parameters by performing LFNST on the prediction residual value; encodes the LFNST parameters and writes them into the bitstream. At the decoder side, after parsing the bitstream to obtain prediction parameters and LFNST parameters of the current block, obtain neighboring sample values ​​of the current block, and determine MIP input sample values ​​of the current block according to the neighboring sample values ​​of the current block, when the prediction mode parameter indicates that the MIP mode is used to determine the intra predicted value of the chroma component of the current block; determine a MIP predicted value of the chroma component of the current block according to the MIP input sample values, the MIP weighting matrix, and the shift parameter; determine an intra predicted value of the chroma component of the current block by filtering the MIP predicted value; determine a reconstructed transform coefficient block of the current block, and obtain a second transform coefficient block by performing LFNST on at least some reconstructed transform coefficients in the reconstructed transform coefficient block, when the LFNST parameter indicates that LFNST is performed on the current block; obtain a reconstructed residual block of the chroma component of the current block by performing a first transform on the second transform coefficient block; determine a reconstructed block of the chroma component of the current block according to the intra predicted value of the chroma component of the current block and the reconstructed residual block.In this way, in the MIP mode, while ensuring the encoding / decoding performance, the complexity is reduced, and at the same time, the memory space required for the encoding / decoding process is reduced, effectively improving the encoding / decoding efficiency. In addition, when the LFNST technology is applied to MIP mode prediction, the introduction of MIP parameters makes LFNST more flexible, which can further improve the encoding / decoding efficiency.

Claims

1. 1. An encoding method applied to an encoder, the encoding method comprising: determining prediction parameters for a current block, the prediction parameters including a prediction mode parameter; If the prediction mode parameter indicates that an intra prediction value of a color component of the current block is determined using matrix-based intra prediction (MIP), obtaining neighboring sample values ​​of the current block, and determining MIP input sample values ​​of the current block based on the neighboring sample values ​​of the current block; determining a MIP prediction value of the color component of the current block based on the MIP input sample values, a MIP weighting matrix and a shift parameter, the MIP prediction value being a prediction value of a portion of samples of the color component of the current block; determining the intra prediction value of the color component of the current block by filtering the MIP prediction value; determining a prediction residual value of the color component of the current block based on the intra prediction value of the color component of the current block; determining first transform coefficients by performing a first transform on the prediction residual values ​​of the current block; determining whether to perform a low-frequency non-separable transform (LFNST) on at least some of the first transform coefficients; when it is determined that the LFNST is performed on the at least some of the transform coefficients, determining an LFNST coefficient by performing the LFNST on the at least some of the transform coefficients of the first transform coefficient, and setting a value of an LFNST index to be greater than 0; setting a value of the LFNST index to 0 when it is determined that the LFNST is not to be performed on the at least some of the transform coefficients; encoding and writing the LFNST index into a bitstream; Determining whether to perform the LFNST on the at least some of the transform coefficients of the first transform coefficients includes: determining a minimum of size parameters of the current block; determining to perform the LFNST on the at least some transform coefficients of the first transform coefficients when the minimum value is equal to or greater than a first preset threshold value; 13. An encoding method comprising:

2. The shift parameters include a shift offset parameter and a shift number parameter, Determining the MIP predicted values ​​of the color components of the current block based on the MIP input sample values, the MIP weighting matrix, and the shift parameters includes: determining a product of the sum of the MIP input sample values ​​and the shift offset parameter, the value of the shift offset parameter being a fixed constant; determining a first offset value to be the difference between a first constant value and the multiplication product; determining a MIP weighting matrix for the current block based on the prediction parameters; calculating a first weighted sum of the MIP weighting matrix and the MIP input sample values; calculating a first sum of the first weighted sum and the first offset; determining a first right shift value by binary right shifting the first sum, the number of bits right shifted being equal to the value of the shift number parameter; setting the MIP predicted value of the color component of the current block to a sum of the first right-shifted value and a value corresponding to index 0 in a first temporary reference value, where the first temporary reference value is obtained by downsampling the neighboring sample values ​​of the current block.

2. The method of claim 1.

3. the prediction parameters further include a size parameter of the current block; Determining the MIP input sample values ​​of the current block based on the neighboring sample values ​​of the current block includes: determining a block size index value for the current block based on the size parameter of the current block; Obtaining a first temporary reference value by downsampling the neighboring sample values ​​of the current block; if the block size index value of the current block is within a preset range, setting a value corresponding to index 0 in the MIP input sample values ​​to a difference between a second constant value and a value corresponding to index 0 in the first temporary reference value, and setting a value corresponding to index i in the MIP input sample values ​​to a difference between a value corresponding to index i in the first temporary reference value and the value corresponding to index 0 in the first temporary reference value, where i is an integer greater than 0; if the block size index value of the current block is outside the preset range, setting a value corresponding to index j in the MIP input sample values ​​to a difference between a value corresponding to index (j+1) in the first temporary reference value and the value corresponding to index 0 in the first temporary reference value, where j is an integer greater than or equal to 0.

2. The method of claim 1.

4. the second constant value is equal to 1<<(BitDepth-1), where BitDepth is the bit depth of the color components of the current block.

4. The encoding method according to claim 3.

5. the value of the shift number parameter is a fixed constant that is not related to block size index values ​​and MIP mode index values; the value of the shift number parameter is set to 6; the value of the shift offset parameter is a fixed constant that is not related to the block size index value and the MIP mode index value; The value of the shift offset parameter is set to 32; 3. The encoding method according to claim 2.

6. the first constant value is obtained by binary left-shifting a "1", the number of bits left-shifted being equal to the value of the shift number parameter minus one; 3. The encoding method according to claim 2.

7. A decoding method applied to a decoder, the decoding method comprising: parsing the bitstream to obtain prediction parameters, the prediction parameters including a prediction mode parameter; If the prediction mode parameter indicates that an intra prediction value of a color component of the current block is determined using matrix-based intra prediction (MIP), obtaining neighboring sample values ​​of the current block, and determining MIP input sample values ​​of the current block based on the neighboring sample values ​​of the current block; determining a MIP prediction value of the color component of the current block based on the MIP input sample values, a MIP weighting matrix and a shift parameter, the MIP prediction value being a prediction value of a portion of samples of the color component of the current block; determining the intra prediction value of the color component of the current block by filtering the MIP prediction value; determining a value of a low-frequency non-separable transform (LFNST) index of the current block, wherein a value of the LFNST index greater than 0 indicates performing the LFNST on the current block, and a value of the LFNST index equal to 0 indicates not performing the LFNST on the current block; determining reconstructed transform coefficients of the current block when the LFNST index indicates that the LFNST is to be performed on the current block, and determining second transform coefficients by performing an LFNST on at least a portion of the reconstructed transform coefficients; determining reconstructed residual values ​​of the color components of the current block by performing a first transform on the second transform coefficients; determining reconstructed values ​​of the color components of the current block based on the intra predicted values ​​and the reconstructed residual values ​​of the color components of the current block; Determining the value of the LFNST index of the current block comprises: determining whether the LFNST index is present in the bitstream, where determining whether the LFNST index is present in the bitstream includes determining a minimum value of a size parameter of the current block, and determining that the LFNST index is present in the bitstream if the minimum value is greater than or equal to a first preset threshold; parsing the bitstream to determine a value of the LFNST index; 4. A decoding method comprising:

8. The shift parameters include a shift offset parameter and a shift number parameter, Determining the MIP predicted values ​​of the color components of the current block based on the MIP input sample values, the MIP weighting matrix, and the shift parameters includes: determining a product of the sum of the MIP input sample values ​​and the shift offset parameter, the value of the shift offset parameter being a fixed constant; determining a first offset value to be the difference between a first constant value and the multiplication product; determining a MIP weighting matrix for the current block based on the prediction parameters; calculating a first weighted sum of the MIP weighting matrix and the MIP input sample values; calculating a first sum of the first weighted sum and the first offset; determining a first right shift value by binary right shifting the first sum, the number of bits right shifted being equal to the value of the shift number parameter; setting the MIP predicted value of the color component of the current block to a sum of the first right-shifted value and a value corresponding to index 0 in a first temporary reference value, where the first temporary reference value is obtained by downsampling the neighboring sample values ​​of the current block.

8. The method of claim 7, wherein the decoding step is carried out in a step of decoding the encoded data.

9. the prediction parameters further include a size parameter of the current block; Determining the MIP input sample values ​​of the current block based on the neighboring sample values ​​of the current block includes: determining a block size index value for the current block based on the size parameter of the current block; Obtaining a first temporary reference value by downsampling the neighboring sample values ​​of the current block; if the block size index value of the current block is within a preset range, setting a value corresponding to index 0 in the MIP input sample values ​​to a difference between a second constant value and a value corresponding to index 0 in the first temporary reference value, and setting a value corresponding to index i in the MIP input sample values ​​to a difference between a value corresponding to index i in the first temporary reference value and the value corresponding to index 0 in the first temporary reference value, where i is an integer greater than 0; if the block size index value of the current block is outside the preset range, setting a value corresponding to index j in the MIP input sample values ​​to a difference between a value corresponding to index (j+1) in the first temporary reference value and the value corresponding to index 0 in the first temporary reference value, where j is an integer greater than or equal to 0.

8. The method of claim 7, wherein the decoding step is carried out in a step of decoding the encoded data.

10. the second constant value is equal to 1<<(BitDepth-1), where BitDepth is the bit depth of the color components of the current block.

10. The method of claim 9.

11. the value of the shift number parameter is a fixed constant that is not related to block size index values ​​and MIP mode index values; the value of the shift number parameter is set to 6; the value of the shift offset parameter is a fixed constant that is not related to the block size index value and the MIP mode index value; The value of the shift offset parameter is set to 32; 9. The method of claim 8.

12. the first constant value is obtained by binary left-shifting a "1", the number of left-shifted bits being equal to the value of the shift number parameter minus one; 9. The method of claim 8.

13. Determining the reconstructed transform coefficients of the current block comprises: Parsing the bitstream to obtain quantized coefficients of the current block; and obtaining the reconstructed transform coefficients of the current block by scaling the quantized coefficients.

8. The method of claim 7, wherein the decoding step is carried out in a step of decoding the encoded data.

14. 1. A decoder comprising at least one processor and a memory, said memory being connected to said at least one processor and storing at least one computer executable instruction, said at least one computer executable instruction being executed by said at least one processor to cause said at least one processor to perform the decoding method according to any one of claims 7 to 13.

23. A decoder comprising:

15. An encoder including at least one processor and a memory, said memory being connected to said at least one processor and storing at least one computer executable instruction, said at least one computer executable instruction being executed by said at least one processor to cause said at least one processor to perform the encoding method according to any one of claims 1 to 6.

16. An encoder comprising:

16. 1. A computer storage medium for storing a bitstream, comprising: The bitstream is generated according to the encoding method of any one of claims 1 to 6. A computer storage medium comprising:

Images