Encoding method, decoding method, encoder, decoder, and storage medium

Abstract

To provide an encoding method, a decoding method, an encoder, a decoder, and a storage medium, capable of improving encoding and decoding efficiency.SOLUTION: An encoding method includes the steps of: acquiring neighboring sample values of a current block by using a matrix-based intra prediction (MIP) mode by a prediction mode parameter of the current block; determining, on the basis the acquired values, MIP input sample values of the current block; determining an MIP prediction value of a chroma component of the current block on the basis of the MIP input sample values, a MIP weighting matrix and a shifting parameter; determining an intra prediction value of a chroma component of the current block. by filtering the MIP prediction value; determining a prediction residual value of the chroma component of the current block on the basis of the determined intra prediction value; determining a LFNST parameter by performing a low-frequency non-separable transform (LFNST) on the prediction residual value; encoding the LFNST parameter; and writing the encoded LFNST parameter into a bitstream.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] [CROSS-REFERENCE TO RELATED APPLICATIONS] This application claims priority to U.S. Provisional Application No. 62 / 958,582, filed on January 8, 2020, entitled "VIDEO ENCODING AND DECODING METHOD, APPARATUS AND COMMUNICATION SYSTEM", in the names of Junyan Huo, Shuai Wan, and Yanzhuo Ma, the entire contents of which are incorporated herein by reference.

[0002] Embodiments of the present invention relate to the technical field of video encoding, and in particular, to encoding methods, decoding methods, encoders, decoders, and storage media.

Background Art

[0003] As people's requirements for video display quality increase, new video application forms such as high-definition video and ultra-high-definition video have emerged. H.265 / High Efficiency Video Coding (HEVC) can no longer meet the needs of rapidly developing video applications. Therefore, the JVET (Joint Video Exploration Team) proposed H.266 / Versatile Video Coding (VVC), the next-generation video coding standard.

[0004] In H.266 / VVC, Matrix-based Intra Prediction (MIP) is one intra-prediction mode used to obtain the intra-predicted block for the current block. Next, the prediction residual for the current block is determined using the Low-Frequency Non-Separable Transform (LFNST) technique. However, a challenge exists in that the LFNST technique negatively impacts the coding efficiency of VVC when applied to MIP mode prediction. [Overview of the project]

[0005] Embodiments of the present invention provide an encoding method, a decoding method, an encoder, a decoder, and a storage medium that can improve encoding and decoding efficiency.

[0006] The technical solutions of the embodiments of the present invention can be realized as follows.

[0007] In a first embodiment of the present invention, an encoding method applicable to an encoder is provided. The method may include the following:

[0008] Determine the prediction parameters for the current block. These prediction parameters include the prediction mode parameters. If the prediction mode parameter indicates that the matrix-based intra prediction (MIP) mode is used to determine the intra-predicted value of the saturation component of the current block, the adjacent sample values ​​of the current block are obtained, and the MIP input sample values ​​of the current block are determined based on the adjacent sample values ​​of the current block. Based on the MIP input sample values, MIP weighting matrix, and shift parameters, the MIP predicted value for the saturation component of the current block is determined. The MIP predicted value is the predicted value for a subset of samples in the saturation component of the current block. By filtering the MIP prediction values, we determine the intra-predicted value for the saturation component of the current block. Based on the intra-predicted value of the current block's saturation component, the predicted residual value of the current block's saturation component is determined. The LFNST parameters are determined by applying a low-frequency non-separable transform (LFNST) to the predicted residual values. Encode the LFNST parameters and write them to the bitstream.

[0009] In a second embodiment of the present invention, a decoding method applied to a decoder is provided. This method may include the following:

[0010] The bitstream is analyzed to obtain the prediction parameters and low-frequency unseparated transform (LFNST) parameters for the current block. The prediction parameters include the prediction mode parameters. If the prediction mode parameter indicates that the matrix-based intra-prediction (MIP) mode is used to determine the intra-predicted value of the saturation component of the current block, the adjacent sample values ​​of the current block are obtained, and the MIP input sample values ​​of the current block are determined based on the adjacent sample values ​​of the current block. Based on the MIP input sample values, MIP weighting matrix, and shift parameters, the MIP predicted value for the saturation component of the current block is determined. The MIP predicted value is the predicted value for a subset of samples in the saturation component of the current block. By filtering the MIP prediction values, we determine the intra-predicted value for the saturation component of the current block. If the LFNST parameter indicates that LFNST should be performed on the current block, the reconstruction transformation coefficient block of the current block is determined, and a second transformation coefficient block is obtained by performing LFNST on at least some of the reconstruction transformation coefficients in the reconstruction transformation coefficient block. By performing the first transformation on the second transformation coefficient block, the reconstructed residual block of the saturation component of the current block is obtained. Based on the intra-predicted value of the current block's saturation component and the reconstructed residual block, the reconstructed block of the current block's saturation component is determined.

[0011] In a third aspect, an embodiment of the present invention provides an encoder comprising a first decision unit, a first prediction unit, a first transformation unit, and an encoding unit.

[0012] The first decision unit is configured to determine the prediction parameters for the current block, which include prediction mode parameters. The first decision unit is further configured to obtain the adjacent sample values ​​of the current block and to determine the MIP input sample values ​​of the current block based on the adjacent sample values ​​of the current block, when the prediction mode parameter indicates that the matrix-based intra-prediction (MIP) mode is used to determine the intra-predicted value of the saturation component of the current block. The first prediction unit is configured to determine the MIP predicted value of the saturation component of the current block based on the MIP input sample values, the MIP weighting matrix, and the shift parameters, and to determine the intra predicted value of the saturation component of the current block by filtering the MIP predicted value, the MIP predicted value being the predicted value of a subset of samples in the saturation component of the current block. The first conversion unit is configured to determine the predicted residual value of the saturation component of the current block based on the intra-predicted value of the saturation component of the current block, and to determine the LFNST parameters by performing a low-frequency non-separated conversion (LFNST) on the predicted residual value. The encoding unit is configured to encode the LFNST parameters and write them to the bitstream.

[0013] In a fourth aspect, an embodiment of the present invention provides an encoder, which includes a first memory and a first processor. The first memory is configured to store computer programs that can be executed by the first processor. The first processor is configured to execute the method described in the first mode when it runs a computer program.

[0014] In the fifth aspect, an embodiment of the present invention provides a decoder, which includes an analysis unit, a second decision unit, a second prediction unit, and a second transformation unit.

[0015] The analysis unit is configured to analyze the bitstream to obtain the prediction parameters and low-frequency non-separated transform (LFNST) parameters for the current block, with the prediction parameters including the prediction mode parameters. The second decision unit is configured to obtain the adjacent sample values ​​of the current block and to determine the MIP input sample values ​​of the current block based on the adjacent sample values ​​of the current block, when the prediction mode parameter indicates that the matrix-based intra-prediction (MIP) mode is used to determine the intra-predicted value of the saturation component of the current block. The second prediction unit is configured to determine the MIP predicted value of the saturation component of the current block based on the MIP input sample values, the MIP weighting matrix, and the shift parameters, and to determine the intra predicted value of the saturation component of the current block by filtering the MIP predicted value, the MIP predicted value being the predicted value of a subset of samples in the saturation component of the current block. The second transformation unit is configured to determine the reconstruction transformation coefficient block of the current block when the LFNST parameter indicates that LFNST should be performed on the current block, obtain a second transformation coefficient block by performing LFNST on at least some of the reconstruction transformation coefficients in the reconstruction transformation coefficient block, and obtain a reconstruction residual block of the saturation component of the current block by performing the first transformation on the second transformation coefficient block. The second decision unit is further configured to determine the reconstructed block of the saturation component of the current block, based on the intra-predicted value of the saturation component of the current block and the reconstructed residual block.

[0016] In the sixth aspect, an embodiment of the present invention provides a decoder, which includes a second memory and a second processor. The second memory is configured to store computer programs that can be executed by the second processor. The second processor is configured to execute the method described in the second mode when it runs a computer program.

[0017] In the seventh embodiment of the present invention, a computer storage medium for storing a computer program is provided. When the computer program is executed by a first processor, the method described in the first embodiment is performed, or when it is executed by a second processor, the method described in the second embodiment is performed.

[0018] Embodiments of the present invention provide an encoding method, a decoding method, an encoder, a decoder, and a storage medium. On the encoder side, after determining the prediction parameters of the current block, if the prediction mode parameter indicates that the matrix-based intra-prediction (MIP) mode is used to determine the intra-predicted value of the saturation component of the current block, the encoder obtains the adjacent sample values ​​of the current block and determines the MIP input sample values ​​of the current block based on the adjacent sample values ​​of the current block. Based on the MIP input sample values, the MIP weighting matrix, and the shift parameters, the MIP predicted value of the saturation component of the current block is determined. The intra-predicted value of the saturation component of the current block is determined by filtering the MIP predicted value. Based on the intra-predicted value of the saturation component of the current block, the predicted residual value of the saturation component of the current block is determined. The LFNST parameter is determined by performing a low-frequency non-separated transform (LFNST) on the predicted residual value. The LFNST parameter is encoded and written to a bitstream. On the decoder side, after analyzing the bitstream to obtain the prediction parameters and LFNST parameters for the current block, if the prediction mode parameter indicates that the intra-predicted value of the saturation component of the current block is determined using the MIP mode, the adjacent sample values ​​of the current block are obtained, and the MIP input sample values ​​of the current block are determined based on the adjacent sample values ​​of the current block. Based on the MIP input sample values, MIP weighting matrix, and shift parameters, the MIP predicted value of the saturation component of the current block is determined. The intra-predicted value of the saturation component of the current block is determined by filtering the MIP predicted value. If the LFNST parameter indicates that LFNST should be performed on the current block, the reconstruction transformation coefficient block of the current block is determined, and a second transformation coefficient block is obtained by performing LFNST on at least some of the reconstruction transformation coefficients in the reconstruction transformation coefficient block. The reconstruction residual block of the saturation component of the current block is obtained by performing the first transformation on the second transformation coefficient block.Based on the intra-prediction value and the reconstruction residual block of the chrominance component of the current block, a reconstruction block of the chrominance component of the current block is determined. Thus, in the MIP mode, while ensuring the performance of encoding and decoding, the complexity can be reduced, the storage space required for the encoding and decoding process can be reduced at the same time, and the encoding and decoding efficiency can be effectively improved. In addition, when the LFNST technology is applied to MIP mode prediction, the introduction of MIP parameters makes the LFNST more flexible and the encoding and decoding efficiency can be further improved.

Brief Description of the Drawings

[0019] [Figure 1] FIG. 1 is a schematic diagram showing the structure of an encoder according to an embodiment of the present invention. [Figure 2] FIG. 2 is a schematic diagram showing the structure of a decoder according to an embodiment of the present invention. [Figure 3] FIG. 3 is a flowchart showing an encoding method according to an embodiment of the present invention. [Figure 4] FIG. 4 is a flowchart showing another encoding method according to an embodiment of the present invention. [Figure 5] FIG. 5 is a flowchart showing a matrix-based intra-prediction (MIP) prediction process according to an embodiment of the present invention. [Figure 6] FIG. 6 is a flowchart showing obtaining an intra-prediction block using the MIP mode according to an embodiment of the present invention. [Figure 7] FIG. 7 is a schematic diagram showing a flow of performing low-frequency non-separable transform (LFNST) according to an embodiment of the present invention. [Figure 8] FIG. 8 is a flowchart showing the encoding of LFNST parameters according to an embodiment of the present invention. [Figure 9] FIG. 9 is a flowchart showing a decoding method according to an embodiment of the present invention. [Figure 10] FIG. 10 is a flowchart showing the analysis of LFNST parameters according to an embodiment of the present invention. [Figure 11] Figure 11 is a schematic diagram showing another flow of performing LFNST according to an embodiment of the present invention. [Figure 12] Figure 12 is a schematic diagram showing the structure of another encoder according to an embodiment of the present invention. [Figure 13] Figure 13 is a schematic diagram showing a specific hardware structure of an encoder according to an embodiment of the present invention. [Figure 14] Figure 14 is a schematic diagram showing the structure of another decoder according to an embodiment of the present invention. [Figure 15] Figure 15 is a schematic diagram showing a specific hardware structure of a decoder according to an embodiment of the present invention. [Figure 16] Figure 16 is a schematic diagram showing the structure of a transmitting device according to an embodiment of the present invention. [Figure 17] Figure 17 is a schematic diagram showing the structure of a target device according to an embodiment of the present invention. [Figure 18] Figure 18 is a schematic diagram showing the structure of a communication system according to an embodiment of the present invention. [Modes for carrying out the invention]

[0020] To better understand the features and technical details of the embodiments of the present invention, the realization of embodiments of the present invention will be described in detail below with reference to the drawings. The accompanying drawings are for illustrative purposes only and do not limit the embodiments of the present invention.

[0021] All technical and scientific terms used herein have the same meaning as those generally understood by those skilled in the art, unless otherwise defined. The terms used herein are for illustrative purposes only and are not intended to limit the invention.

[0022] In the following description, “several embodiments” describe a subset of all possible embodiments, but it should be understood that “several embodiments” may be the same subset or different subsets of all possible embodiments, and they may be combined with each other as long as they do not contradict each other. Also note that the terms “first / second / third” in embodiments of the present invention merely distinguish similar objects and do not imply a particular order of objects. “First / second / third” may be replaced with a specific order or priority where permitted, so that embodiments of the present invention described herein may be carried out in an order other than that illustrated or described herein.

[0023] With the advancement of video compression technology, the International Telecommunication Union-Telecommunication Sector (ITU-T) and the International Organization for Standardization (ISO) / International Electrotechnical Commission (IEC) have launched a standardization project titled "Vulnerable Video Coding (VVC)" to develop a new generation of video coding standards. The goal is to improve the performance of VVC by approximately 50% compared to the latest H.265 / HEVC standard when coding high-quality video with one or more of the following features: high resolution, high frame rate, high bit depth, high dynamic range, wide color gamut, and omnidirectional viewing angle. The ITU-T SG16WP3 and ISO / IEC JTC1 / SC29 / WG11's JVET (Joint Video Exploration Team) are responsible for this standardization project. Various intra-prediction and inter-prediction modes were adopted in the VVC Working Draft (WD) because they were verified to achieve high compression efficiency when encoding high-quality video.

[0024] The Matrix-Based Intra-Prediction (MIP) mode is one intra-prediction mode. In a VVC, the encoder or decoder can invoke the MIP mode to obtain the intra-prediction block of the current block. Then, the Low-Frequency Non-Separating Transform (LFNST) can be applied as a quadratic transform (or second transform) to the prediction residual of the current block.

[0025] In related technologies, LFNST can be applied to MIP mode predictions where the LFNST transform kernel candidate set and the LFNST transposition indication parameter are fixed, but this negatively impacts the encoding and decoding efficiency of VVCs.

[0026] Based on the above, an encoding method is provided in one embodiment of the present invention. The basic idea of ​​the encoding method is as follows: Determine the prediction parameters for the current block. The prediction parameters include prediction mode parameters. If the prediction mode parameters indicate that the intra-predicted value of the saturation component of the current block is determined using the MIP mode, obtain the adjacent sample values ​​of the current block and determine the MIP input sample values ​​of the current block based on the adjacent sample values ​​of the current block. Determine the MIP prediction value of the saturation component of the current block based on the MIP input sample values, the MIP weighting matrix and the shift parameters. Determine the intra-predicted value of the saturation component of the current block by filtering the MIP prediction values. Determine the predicted residual value of the saturation component of the current block based on the intra-predicted value of the saturation component of the current block. Determine the LFNST parameters by performing LFNST on the predicted residual values. Encode the LFNST parameters and write them to the bitstream. In another embodiment of the present invention, a decoding method is provided. The basic idea of ​​the decoding method is as follows: Analyze the bitstream to obtain the prediction parameters and LFNST parameters of the current block. The prediction parameters include prediction mode parameters. If the prediction mode parameter indicates that the intra-predicted value of the saturation component of the current block is determined using the MIP mode, the adjacent sample values ​​of the current block are obtained, and the MIP input sample values ​​of the current block are determined based on the adjacent sample values ​​of the current block. The MIP predicted value of the saturation component of the current block is determined based on the MIP input sample values, the MIP weighting matrix, and the shift parameter. The intra-predicted value of the saturation component of the current block is determined by filtering the MIP predicted value. If the LFNST parameter indicates that LFNST is performed on the current block, the reconstruction transformation coefficient block of the current block is determined, and a second transformation coefficient block is obtained by performing LFNST on at least some of the reconstruction transformation coefficients in the reconstruction transformation coefficient block.By performing the first transformation on the second transformation coefficient block, the reconstructed residual block of the saturation component of the current block is obtained. Based on the intra-predicted value of the saturation component of the current block and the reconstructed residual block, the reconstructed block of the saturation component of the current block is determined. In this way, MIP mode can reduce complexity while ensuring encoding and decoding performance, and at the same time reduce the memory space required for the encoding and decoding process, thereby effectively increasing encoding and decoding efficiency. Furthermore, when LFNST technology is applied to MIP mode prediction, the introduction of MIP parameters makes LFNST more flexible and can further improve encoding and decoding efficiency.

[0027] Hereinafter, each embodiment of the present invention will be described in detail with reference to the attached drawings.

[0028] Referring to Figure 1, Figure 1 shows a block diagram of an example of the structure of an encoder system according to an embodiment of the present invention. As shown in Figure 1, the encoder 100 includes a splitting unit 101, a prediction unit 102, a first adder 107, a transform unit 108, a quantization unit 109, an inverse quantization unit 110, an inverse transform unit 111, a second adder 112, a filtering unit 113, a decoded picture buffer (DPB) unit 114, and an entropy coding unit 115. Here, the input to the encoder 100 may be a video consisting of a series of images or a single still image, and the output to the encoder 100 may be a bitstream representing a compressed version of the input video.

[0029] The splitting unit 101 divides the image in the input video into one or more coding tree units (CTUs). The splitting unit 101 may divide the image into multiple tiles, and further divide one tile into one or more bricks. Here, one tile or one brick may contain one or more complete and / or partial CTUs. The splitting unit 101 may also form one or more slices. One slice may contain one or more tiles in the image arranged in raster order, or one or more tiles in the image covering a rectangular region. The splitting unit 101 may further form one or more subimages, which may contain one or more slices, tiles, or bricks.

[0030] In the encoding process of the encoder 100, the partitioning unit 101 sends the CTU to the prediction unit 102. The prediction unit 102 may generally include a block partitioning unit 103, a motion estimation (ME) unit 104, a motion compensation (MC) unit 105, and an intra-prediction unit 106. Specifically, the block partitioning unit 103 further partitions the input CTU into smaller coding units (CUs) by iteratively using quadtree partitioning, binary tree partitioning, and ternary tree partitioning. The prediction unit 102 can obtain inter-prediction blocks of CUs using the ME unit 104 and the MC unit 105. The intra-prediction unit 106 can obtain intra-prediction blocks of CUs using various intra-prediction modes, including the MIP mode. As an example, to obtain an interpretation block, a rate-distortion optimized motion estimation method can be called by ME unit 104 and MC unit 105, and to obtain an interpretation block, a rate-distortion optimized mode determination method can be called by intraprediction unit 106.

[0031] The prediction unit 102 outputs the prediction block of CU, and the first adder 107 calculates the difference between CU and the prediction block of CU at the output of the division unit 101, i.e., the residual CU. The transformation unit 108 reads the residual CU and performs one or more transformation operations on the residual CU to obtain coefficients. The quantization unit 109 quantizes the coefficients and outputs the quantization coefficients (i.e., levels). The inverse quantization unit 110 outputs the reconstruction coefficients by performing a scaling operation on the quantization coefficients. The inverse transformation unit 111 performs one or more inverse transformations corresponding to the transformation in the transformation unit 108 and outputs the reconstruction residual. The second adder 112 calculates the reconstruction CU by adding the reconstruction residual and the prediction block of CU from the prediction unit 102. The second adder 112 also sends its output to the prediction unit 102 to be used as a reference for intra-prediction. After all CUs in the image or sub-image have been reconstructed, the filtering unit 113 performs in-loop filtering on the reconstructed image or sub-image. Here, the filtering unit 113 includes one or more filters, such as a deblocking filter, a sample adaptive offset (SAO) filter, an adaptive loop filter (ALF), a luma mapping with chroma scaling (LMCS) filter, and a neural-network-based filter. Alternatively, the filtering unit 113 performs in-loop filtering on one or more target samples in a CU if it determines that the CU will not be used as a reference for encoding other CUs.

[0032] The output of the filtering unit 113 is a decoded image or sub-image, which is buffered in the DPB unit 114. The DPB unit 114 outputs the decoded image or sub-image based on timing and control information. The image stored in the DPB unit 114 can then be used as a reference for inter-prediction or intra-prediction by the prediction unit 102. Finally, the entropy coding unit 115 converts the parameters necessary for image decoding (such as control parameters and supplementary information) from the encoder 100 into binary format and writes this binary format to a bitstream based on the syntax structure of each data unit, so that is, the encoder 100 finally outputs a bitstream.

[0033] Furthermore, the encoder 100 may include a first processor and a first memory for storing a computer program. When the first processor reads and executes the computer program, the encoder 100 reads the input video and generates a corresponding bitstream. The encoder 100 may also be a computing device comprising one or more chips. The above unit, implemented as an integrated circuit on a chip, has connection and data exchange functions similar to the corresponding unit in Figure 1.

[0034] Referring to Figure 2, Figure 2 shows a block diagram of an example of the structure of a decoder system according to an embodiment of the present invention. As shown in Figure 2, the decoder 200 may include an analysis unit 201, a prediction unit 202, a scaling unit 205, a transformation unit 206, an adder 207, a filtering unit 208, and a decoded image buffer unit 209. Here, the input to the decoder 200 is a bitstream representing a compressed version of a video or a single still image, and the output to the decoder 200 may be a decoded video consisting of a series of images or a single decoded still image.

[0035] The input bitstream to the decoder 200 may be the bitstream generated by the encoder 100. The analysis unit 201 analyzes the input bitstream and obtains the values ​​of the syntax elements from the input bitstream. The analysis unit 201 converts the binary representation of the syntax elements into numerical values ​​and sends the numerical values ​​to the unit in the decoder 200 in order to obtain one or more decoded images. The analysis unit 201 can further analyze one or more syntax elements from the input bitstream in order to display the decoded images.

[0036] During the decoding process of the decoder 200, the analysis unit 201 sends the values ​​of the syntax elements and one or more variables set or determined based on the values ​​of the syntax elements to the unit in the decoder 200 for obtaining one or more decoded images.

[0037] The prediction unit 202 determines the prediction block for the current decoding block (e.g., CU). Here, the prediction unit 202 may include a motion compensation unit 203 and an intra-prediction unit 204. Specifically, if it is indicated that an inter-decoding mode will be used to decode the current decoding block, the prediction unit 202 sends the relevant parameters from the analysis unit 201 to the motion compensation unit 203 to obtain the inter-prediction block. If it is indicated that an intra-prediction mode (including an MIP mode indicated based on the MIP mode index value) will be used to decode the current decoding block, the prediction unit 202 sends the relevant parameters from the analysis unit 201 to the intra-prediction unit 204 to obtain the intra-prediction block.

[0038] The scaling unit 205 has the same function as the inverse quantization unit 110 in the encoder 100. The scaling unit 205 performs a scaling operation on the quantization coefficients (i.e., levels) from the analysis unit 201 in order to obtain reconstruction coefficients.

[0039] The conversion unit 206 has the same function as the inverse conversion unit 111 in the encoder 100. The conversion unit 206 performs one or more conversion operations (i.e., the inverse operations of one or more conversion operations performed by the inverse conversion unit 111 in the encoder 100) to obtain the reconstruction residual.

[0040] The adder 207 performs an addition operation on its inputs (the predicted block from the prediction unit 202 and the reconstructed residual from the transformation unit 206) to obtain the reconstructed block of the current decoded block. The reconstructed block is also sent to the prediction unit 202 to be used as a reference for other blocks to be encoded under intra-prediction mode.

[0041] After all CUs in an image or sub-image have been reconstructed, the filtering unit 208 performs in-loop filtering on the reconstructed image or sub-image. The filtering unit 208 includes one or more filters, such as a deblocking filter, a sample adaptive offset (SAO) filter, an adaptive loop filter (ALF), a chroma-scaling luma mapping (LMCS) filter, and a neural network-based filter. Alternatively, the filtering unit 208 performs in-loop filtering on one or more target samples in a reconstructed block if it determines that the reconstructed block will not be used as a reference for decoding other blocks. The output of the filtering unit 208 is the decoded image or sub-image, which is buffered in the DPB unit 209. The DPB unit 209 outputs the decoded image or sub-image based on timing and control information. The images stored in the DPB unit 209 can further be used as a reference for inter-prediction or intra-prediction by the prediction unit 202.

[0042] Furthermore, the decoder 200 may include a second processor and a second memory for storing a computer program. When the second processor reads and executes the computer program, the decoder 200 reads the input bitstream and generates the corresponding decoded video. The decoder 200 may also be a computing device comprising one or more chips. The above unit, implemented as an integrated circuit on a chip, has connectivity and data exchange functions similar to the corresponding unit in Figure 2.

[0043] It should be noted that the encoding method according to the embodiment of the present invention is mainly applied to the intra-prediction unit 106 and the conversion unit 108 of the encoder 100, and the decoding method according to the embodiment of the present invention is mainly applied to the intra-prediction unit 204 and the conversion unit 206 of the decoder 200. That is, embodiments of the present invention may be applied to an encoder, to a decoder, or to both an encoder and a decoder simultaneously. This specification is not limited thereto.

[0044] Furthermore, it should be noted that when the embodiments of the present invention are applied to the encoder 100, "current block" specifically refers to the image block currently to be encoded in intra prediction (which may also be called the "encoded block"), and when the embodiments of the present invention are applied to the decoder 200, "current block" specifically refers to the image block currently to be decoded in intra prediction (which may also be called the "decoded block").

[0045] If the encoder 100 can improve its encoding performance by achieving a better predictive effect through the encoding method according to the embodiment of the present invention, the decoder 200 can also improve its decoding performance by improving the video decoding quality through the decoding method according to the embodiment of the present invention.

[0046] Referring to Figure 3 in one embodiment of the present invention, Figure 3 shows a flowchart of an encoding method according to an embodiment of the present invention. As shown in Figure 3, the method may include the following:

[0047] S301: Determine the prediction parameters for the current block. The prediction parameters include the prediction mode parameters.

[0048] The image to be encoded can be divided into multiple image blocks, and each image block to be encoded can be called an encoded block. Here, each encoded block may contain a first image component, a second image component, and a third image component. The current block is the encoded block in which the prediction of the first, second, or third image component in the video image is currently being performed.

[0049] If we perform a prediction of the first image component for the current block, and assume that the first image component is the luminance component, that is, the image component to be predicted is the luminance component, then the current block can also be called the luminance block, and we obtain the predicted value of the luminance component for the current block. Alternatively, if we perform a prediction of the second image component for the current block, and assume that the second image component is the saturation component, that is, the image component to be predicted is the saturation component, then the current block can also be called the saturation block, and we obtain the predicted value of the saturation component for the current block.

[0050] Furthermore, the prediction parameters may include prediction mode parameters, which are used to indicate the prediction mode adopted by the current block, and it should be noted that different prediction modes correspond to different prediction mode parameters. Here, a simple decision strategy may be employed for determining the prediction mode parameters, for example, based on the magnitude of the distortion value. Alternatively, a more complex decision strategy may be employed, for example, based on the results of Rate Distortion Optimization (RDO). The embodiments of the present invention are not limited thereto. In general, the prediction mode parameters of the current block are determined by employing the RDO method, i.e., the prediction parameters of the current block are determined.

[0051] Specifically, in some embodiments, determining the prediction parameters of the current block may include the following: By pre-encoding the current block using multiple prediction modes, rate distortion cost values ​​corresponding to the multiple prediction modes are obtained. The optimal rate distortion cost value is selected from the multiple rate distortion cost values ​​obtained, and the prediction parameters under the prediction mode corresponding to the optimal rate distortion cost value are determined as the prediction parameters for the current block.

[0052] In other words, the encoder can pre-encode the current block using multiple prediction modes. Here, the multiple prediction modes generally include inter-prediction modes, conventional intra-prediction modes, and non-conventional intra-prediction modes. Conventional intra-prediction modes may include Direct Current (DC) mode, Planar mode, and angular intra-prediction mode. Non-conventional intra-prediction modes may include MIP mode, CCLM (Cross-component Linear Model Prediction) mode, Intra Block Copy (IBC) mode, and PLT (Palette) mode. Inter-prediction modes may include conventional inter-prediction modes and Geometrical (GEO) partitioning for inter blocks mode.

[0053] In this way, after pre-encoding the current block using multiple prediction modes, rate distortion cost values ​​corresponding to the multiple prediction modes can be obtained. Next, the optimal rate distortion cost value is selected from the obtained multiple rate distortion cost values ​​(generally, the minimum rate distortion cost value is the optimal rate distortion cost value), and the prediction parameters under the prediction mode corresponding to that optimal rate distortion cost value are determined as the prediction parameters for the current block.

[0054] Furthermore, by pre-encoding the current block using multiple prediction modes, distortion values ​​corresponding to multiple prediction modes can be obtained. Next, the optimal distortion value is selected from the obtained multiple distortion values, and the prediction parameters of the prediction mode corresponding to that optimal distortion value are determined as the prediction parameters for the current block. In this way, the encoder ultimately performs predictive coding on the current block using the prediction mode indicated by the prediction mode parameter in the determined prediction parameters. This allows for obtaining smaller prediction residuals and improves coding efficiency.

[0055] S302: If the prediction mode parameter indicates that the intra-predicted value of the saturation component of the current block is determined using matrix-based intra-prediction (MIP) mode, the adjacent sample value of the current block is obtained, and the MIP input sample value of the current block is determined based on the adjacent sample value of the current block.

[0056] Furthermore, in this embodiment of the present invention, intra-prediction is performed on the saturation component of the current block using MIP mode. In this process, first, the adjacent sample values ​​of the current block are obtained, and then, based on the adjacent sample values ​​of the current block, the MIP input sample values ​​of the current block are determined.

[0057] For MIP mode, it should be understood that the MIP core parameters must first be configured. Here, the MIP core parameters may include the current block type (represented by mipSizeId), the number of reference samples for each edge (represented by boundarySize), the number of MIP input samples (represented by inSize), and the size of the MIP predicted block output from matrix multiplication (arranged as predSize × predSize). In MIP mode, the current block can be classified into three types based on the width and height of the current block, and mipSizeId may be equal to 0, 1, or 2. In this specification, mipSizeId represents the current block type, i.e., the "block size index value of the current block" as shown in embodiments of the present invention. Different mipSizeIds result in different numbers of reference samples (boundySize is the number of reference samples required for each edge), numbers of MIP input samples (inSize), and sizes of the MIP predicted block output from matrix multiplication (arranged as predSize × predSize).

[0058] Furthermore, it should be noted that the prediction parameters may include, in addition to the prediction mode parameters, the current block size parameters. The current block size parameters may include the width (represented by nTbW) and height (represented by nTbH) of the current block. Also, the block size index value (i.e., mipSizeId) of the current block can be determined based on the current block size parameters.

[0059] In one possible embodiment, determining the block size index value of the current block based on the current block size parameters may include setting the block size index value of the current block to 0 if both the width and height of the current block are equal to 4; setting the block size index value of the current block to 1 if both the width and height of the current block are equal to 8, or if either the width or height of the current block is equal to 4; and setting the block size index value of the current block to 2 if the current block is of a different size.

[0060] In another possible embodiment, determining the block size index value of the current block based on the current block size parameters may include setting the block size index value of the current block to 0 if both the width and height of the current block are equal to 4; setting the block size index value of the current block to 1 if either the width or height of the current block is equal to 4; and setting the block size index value of the current block to 2 if the current block is of a different size.

[0061] Thus, based on the block size index value of the current block, the number of adjacent boundary reference samples (where the variable is boundarySize) and the size of the MIP prediction block (where the variable is predSize, and the size of the MIP prediction block is predSize × predSize) can be determined using the Look-Up Table (LUT) shown in Table 1, and the number of MIP input sample values ​​(represented by inSize) used in the MIP matrix multiplication process can be calculated. The formula for this calculation is as follows:

number

[0062] The operator arithmetic rules in expression (1) are the same as those defined in the ITU-TH.265 standard; for example, "==" is the logical "equal to" operator.

[0063] [Table 1]

[0064] Thus, according to Table 1, when the value of mipSizeId is equal to 0, the value of boundarySize is equal to 2 and the value of predSize is equal to 4. That is, in this case, the reference samples are the two samples selected at each edge, and the output of matrix multiplication is a 4x4 MIP prediction block.

[0065] When the value of mipSizeId is equal to 1, the value of boundarySize is equal to 4, and the value of predSize is equal to 4. That is, in this case, the reference samples are the four samples selected at each edge, and the output of the matrix multiplication is a 4x4 MIP prediction block.

[0066] When the value of mipSizeId is equal to 2, the value of boundarySize is equal to 4 and the value of predSize is equal to 8. That is, in this case, the reference samples are the four samples selected at each edge, and the output of the matrix multiplication is an 8x8 MIP prediction block.

[0067] Furthermore, based on the current block size index value, the values ​​of boundarySize, inSize, and predSize can be simultaneously determined using the lookup table shown in Table 2.

[0068] [Table 2]

[0069] Thus, according to Table 2, when the value of mipSizeId is equal to 0, the value of boundarySize is equal to 2, the value of inSize is equal to 4, and the value of predSize is equal to 4. In other words, in this case, the reference samples are the two samples selected at each edge, the number of input samples for matrix multiplication is 4, and the output of matrix multiplication is a 4x4 MIP prediction block.

[0070] When the value of mipSizeId is equal to 1, the value of boundarySize is equal to 4, the value of inSize is equal to 8, and the value of predSize is equal to 4. That is, in this case, the reference samples are the four samples selected at each edge, the number of input samples for matrix multiplication is 8, and the output of matrix multiplication is a 4x4 MIP prediction block.

[0071] When the value of mipSizeId is equal to 2, the value of boundarySize is equal to 4, the value of inSize is equal to 7, and the value of predSize is equal to 8. That is, in this case, the reference samples are the four samples selected at each edge, the number of input samples for matrix multiplication is 7, and the output of matrix multiplication is an 8x8 MIP prediction block.

[0072] Furthermore, after configuring the MIP core parameters, it is necessary to obtain a reference sample. Then, based on the reference sample and the MIP core parameters, the MIP input sample value is constructed. Here, the reference sample refers to the adjacent sample value of the current block and can include the left adjacent sample value and the upper adjacent sample value of the current block. That is, the MIP input sample value of the current block can be determined based on the left adjacent sample value and the upper adjacent sample value of the current block.

[0073] In some embodiments, determining the MIP input sample value of the current block based on the adjacent sample values ​​of the current block may include the following:

[0074] The block size index value of the current block is determined based on the current block's size parameters.

[0075] A first temporary reference value is obtained by downsampling the adjacent sample values ​​of the current block.

[0076] If the block size index value of the current block falls within a predetermined range, a second constant value is determined based on the bit depth of the adjacent sample values ​​of the current block. The value corresponding to index 0 in the MIP input sample value is set to the difference between the second constant value and the value corresponding to index 0 in the first temporary reference value. The value corresponding to index i in the MIP input sample value is set to the difference between the value corresponding to index i in the first temporary reference value and the value corresponding to index 0 in the first temporary reference value. i is an integer greater than 0.

[0077] If the block size index value of the current block is outside the pre-set range, the value corresponding to index j in the MIP input sample value is set to the difference between the value corresponding to index (j+1) in the first temporary reference value and the value corresponding to index 0 in the first temporary reference value. j is a non-negative integer.

[0078] Furthermore, if the block size index value of the current block is within a predetermined range, the above method may further include the following: A second constant value is determined based on the bit depth of the adjacent sample values ​​in the current block. The value corresponding to index 0 in the MIP input sample value is set to the difference between the value corresponding to index 0 in the first temporary reference value and the second constant value. The value corresponding to index i in the MIP input sample value is set to the difference between the value corresponding to index i in the first temporary reference value and the value corresponding to index 0 in the first temporary reference value. i is an integer greater than 0.

[0079] Note that a first temporary reference value can be obtained after downsampling the adjacent sample values ​​of the current block. Specifically, the first temporary reference value can be obtained by downsampling the adjacent sample values ​​of the current block and then buffering the filtered adjacent sample values ​​in a buffer (represented by pTemp). The value corresponding to index 0 in the first temporary reference value points to pTemp[0], and the value corresponding to index i in the first temporary reference value points to pTemp[i].

[0080] Furthermore, it should be noted that the current block's size parameter value can be determined based on whether the current block's block size index value (represented by mipSizeId) is within a predetermined range. Specifically, if mipSizeId=0 or 1, it indicates that the current block's block size index value is within the predetermined range, i.e., the current block's size parameter value is within the predetermined range. If mipSizeId=2, it indicates that the current block's block size index value is outside the predetermined range, i.e., the current block's size parameter value is not within the predetermined range.

[0081] In other words, the MIP input sample value is determined by the buffer (represented by pTemp), the block size index value of the current block (represented by MipSizeId), and the bit depth of the adjacent sample values ​​of the current block (represented by BitDepth). The number of input samples corresponding to the MIP input sample value is related to the block size index value of the current block. Finally, the value corresponding to index x in the MIP input sample value (represented by p[x]) can be obtained.

[0082] Furthermore, in some embodiments, determining a second constant value based on the bit depth of the adjacent sample values ​​of the current block may include the following: The second constant value is set to a power of 2. The exponent of the exponent is an integer, equal to the bit depth of the adjacent sample values ​​of the current block minus 1.

[0083] Alternatively, in some embodiments, determining a second constant value based on the bit depth of the adjacent sample values ​​of the current block may include the following: A second constant value is obtained by left-shifting "1" in binary. The number of bits left-shifted is equal to the bit depth of the adjacent sample value in the current block minus 1.

[0084] In other words, after obtaining the bit depth (represented by BitDepth) of the adjacent sample values ​​of the current block, the second constant value can be expressed as 1 << (BitDepth-1) or 2^(BitDepth-1). In this way, if the value of the current block's size parameter is within a predetermined range, the MIP input sample value of the current block can be determined by combining it with the second constant value.

[0085] Furthermore, it should be noted that the MIP input samples are matrix vectors used in matrix multiplication operations. The current related technical solution is as follows: Based on the buffer (represented by pTemp), the type of the current block (i.e., the block size index value of the current block, represented by mipSizeId), the bit depth of the adjacent sample values ​​of the current block (represented by BitDepth), and the number of MIP input samples, the value corresponding to index x in the MIP input sample value (represented by p[x]) is finally obtained.

[0086] Specifically, the process for constructing the x-th input sample value p[x] is as follows:

[0087] In one possible embodiment, if the block size index value of the current block is within a predetermined range, the construction process may include the following: The value corresponding to index 0 in the MIP input sample value is obtained by calculating the difference between the second constant value and the value corresponding to index 0 in the first temporary reference value. The value corresponding to index i in the MIP input sample value is obtained by performing a subtraction operation on the value corresponding to index i in the first temporary reference value and the value corresponding to index 0 in the first temporary reference value. i is a positive integer greater than 0 and less than N, and N represents the number of elements in the input sample matrix.

[0088] In an embodiment of the present invention, the minuend in the above difference calculation is set to be equal to a second constant value, and the minuend in the above difference calculation is set to be equal to the value corresponding to index 0 in the first temporary reference value.

[0089] In other words, when mipSizeId=0 or 1, p[0] is obtained by subtracting pTemp[0] from 1<<(BitDepth-1), and when x is not 0, p[x] is obtained by subtracting pTemp[0] from pTemp[x]. The specifics are as follows.

number

[0090] In another possible embodiment, if the block size index value of the current block is within a predetermined range, the construction process may include the following: The value corresponding to index 0 in the MIP input sample value is obtained by calculating the difference between the value corresponding to index 0 in the first temporary reference value and the second constant value. The value corresponding to index i in the MIP input sample value is obtained by performing a subtraction operation on the value corresponding to index i in the first temporary reference value and the value corresponding to index 0 in the first temporary reference value. i is a positive integer greater than 0 and less than N, and N represents the number of elements in the input sample matrix.

[0091] In an embodiment of the present invention, the minuend in the above difference calculation is set to be equal to the value corresponding to index 0 in the first temporary reference value, and the minuend in the above difference calculation is set to be equal to the second constant value.

[0092] In other words, when mipSizeId=0 or 1, if x is equal to 0, the value corresponding to index 0 in the MIP input sample value (represented by p[0]) is obtained by subtracting the second constant value (i.e., 1<<(BitDepth-1)) from the value corresponding to index 0 in the first temporary reference value (i.e., pTemp[0]). If x is not equal to 0, the value corresponding to index x in the MIP input sample value (represented by p[x]) is obtained by subtracting the value corresponding to index 0 in the first temporary reference value (i.e., pTemp[0]) from the value corresponding to index x in the first temporary reference value (i.e., pTemp[x]).

[0093] The specifics are as follows:

number

[0094] In another possible embodiment, if the block size index value of the current block is outside a predetermined range, the construction process may include the following: The value corresponding to index i in the MIP input sample value is obtained by performing a subtraction operation on the value corresponding to index (i+1) in the first temporary reference value and the value corresponding to index 0 in the first temporary reference value. i is a positive integer greater than or equal to 0 and less than N, where N represents the number of elements in the input sample matrix.

[0095] In other words, when mipSizeId=2, the value corresponding to index 0 in the first temporary reference value, i.e., pTemp[0], may be ignored. Next, the value corresponding to index x in the MIP input sample value (represented by p[x]) is obtained by subtracting the value corresponding to index 0 in the first temporary reference value (i.e., pTemp[0]) from the value corresponding to index (x+1) in the first temporary reference value (i.e., pTemp[x+1]). Here, x is a positive integer greater than or equal to 0. The specifics are as follows.

number

[0096] Here, still using the current 4x4 block as an example, four values ​​are stored in the buffer pTemp. That is, the first temporary reference value includes the value corresponding to index 0 (i.e., pTemp[0]), the value corresponding to index 1 (i.e., pTemp[1]), the value corresponding to index 2 (i.e., pTemp[2]), and the value corresponding to index 3 (i.e., pTemp[3]). In this case, four MIP input sample values ​​represented by p[x] can be determined by equation (2), equation (3), or equation (4), where x = 0, 1, 2, 3. Here, the four MIP input sample values ​​can also form a 1x4 MIP input sample matrix.

[0097] In addition to the above, in embodiments of the present invention, it is also possible to obtain the value of p[x] by a unified calculation method without determining the value of mipSizeId. In some embodiments, determining the MIP input sample value of the current block based on the adjacent sample values ​​of the current block may include the following:

[0098] A first temporary reference value is obtained by downsampling the adjacent sample values ​​of the current block. A second constant value is determined based on the bit depth of the adjacent sample values ​​in the current block, and the second temporary reference value is obtained by buffering the second constant value within one data unit after the first temporary reference value. The value corresponding to index j in the MIP input sample value is set to the difference between the value corresponding to index (j+1) in the second temporary reference value and the value corresponding to index 0 in the second temporary reference value. j is a non-negative integer.

[0099] In other words, in embodiments of the present invention, a second constant value (i.e., (1 << (BitDepth-1))) is added as an append element to the end of the buffer pTemp. In this case, there is no need to consider the size parameter of the current block (i.e., there is no need to consider the value of mipSizeId), and p[x] can be directly set to p[x] = pTemp[x+1] - pTemp[0], where x = 0, ..., inSize-1.

[0100] In this way, after determining the MIP input sample value of the current block based on the adjacent sample values ​​of the current block, the MIP prediction value of the current block can be further determined.

[0101] S303: Determine the MIP predicted value for the saturation component of the current block based on the MIP input sample values, MIP weighting matrix, and shift parameters.

[0102] The MIP prediction is a prediction of a subset of samples in the saturation component of the current block.

[0103] The shift parameters may include a shift offset parameter and a shift number parameter. In this case, as shown in Figure 4, S303 may include the following steps.

[0104] S303-1: Based on the value of the shift offset parameter, the product of the sum of the MIP input sample values ​​and the shift offset parameter is determined. The value of the shift offset parameter is a fixed constant.

[0105] The shift offset parameter may also be called the shifting compensation parameter or offset factor, and can be represented by fO. In embodiments of the present invention, the value of the shift offset parameter can be set to a fixed constant, such as 32, 46, 56, or 66. Furthermore, the value of the shift offset parameter can be set in relation to a shift offset parameter table, and the value of the shift offset parameter can be determined by a lookup table. This specification is not limited thereto.

[0106] S303-2: Determine a first constant value based on the value of the shift number parameter.

[0107] S303-3: Set the value of the first offset to the difference between the first constant value and the above product.

[0108] The shift number parameter may also be called the shifting factor, shift bit number, or weight shifting value, and can be represented as sW, shift, or weight shift. In embodiments of the present invention, the shift number parameter is represented as sW. The first offset can also be represented as oW, and the first offset is related to both the shift number parameter and the shift offset parameter.

[0109] In embodiments of the present invention, the value of the shift number parameter can be set to a fixed constant, such as 5, 6, or 7. Furthermore, the value of the shift number parameter can be set to be related to a shift number parameter table, and the value of the shift number parameter can be determined by a lookup table. This specification is not limited thereto.

[0110] S303-4: Determine the MIP weighting matrix for the current block based on the prediction parameters.

[0111] The prediction parameters may include prediction mode parameters and may also include the size parameter of the current block. If the prediction mode parameter indicates that the MIP mode is used to determine the intra-predicted value of the saturation component of the current block, a weighting matrix table may be established in advance and stored in memory or a storage unit. This memory or storage unit may be integrated into the encoder or provided separately. Thus, based on the block size index value (mipSizeId) and MIP mode index value (modelId) of the current block, the MIP weighting matrix (also called the MIP weight matrix or MIP matrix) that needs to be used for the current block can be determined by a lookup table and is represented by mWeight[x][y]. The block size index value (mipSizeId) of the current block is determined based on the size parameter of the current block, and the size of the MIP weighting matrix mWeight[x][y] is related to the block size index value of the current block, as shown in Table 3.

[0112] In the MIP weighting matrix shown in Table 3, the number of columns is equal to the number of input samples in the matrix multiplication equal to inSize, and the number of rows is equal to the number of output samples in the matrix multiplication equal to predSize × predSize, allowing the MIP weighting matrix for the current block to be determined.

[0113] [Table 3]

[0114] In an encoder, a shift number parameter table can be established in advance, and this shift number parameter table is also stored in memory or a storage unit. This memory or storage unit may be integrated into the encoder or may be provided independently. In embodiments of the present invention, the shift number parameter (sW) can be determined in several ways.

[0115] In one possible embodiment, the shift number parameter may differ for different block size index values ​​and different MIP mode index values. In some embodiments, the methods described above may be further included.

[0116] When determining the intra-predicted value of the saturation component of the current block using MIP mode, the MIP mode index value of the current block is determined. Based on the MIP mode index value, a value corresponding to the MIP mode index value is queried from a first pre-configured lookup table. The first pre-configured lookup table is used to record the correspondence between the MIP mode index value and the value of the shift number parameter. The queried value is determined as the value of the shift number parameter.

[0117] Furthermore, the value of the shift number parameter can be queried based on the current block's block size index value (represented by mipSizeId) and MIP mode index value (modelId). In the first pre-configured lookup table shown in Table 4, the shift number parameter required for matrix multiplication can be determined by the lookup table for different mipSizeId and modelId values.

[0118] [Table 4]

[0119] However, the encoder needs to store Table 4 in memory or a storage unit as a lookup table. However, storage is costly, and the lookup process is also costly. Since the values ​​of the shift number parameters in Table 4 are related to both the block size and the MIP mode index value of the current block, the amount of memory occupied increases, and the computational complexity also increases.

[0120] In an embodiment of the present invention, the method for determining the shift number parameter can be simplified in order to reduce memory usage and computational complexity.

[0121] In another possible embodiment, the value of the shift number parameter can be set to a fixed constant independent of the block size index value and the MIP mode index value. For example, the value of the shift number parameter can be set to 5 for different block size index values ​​and different MIP mode index values. Alternatively, the value of the shift number parameter can be set to 6 for different block size index values ​​and different MIP mode index values. Alternatively, the value of the shift number parameter can be set to 7 for different block size index values ​​and different MIP mode index values. In embodiments of the present invention, it is preferable, but not limited thereto, to set the value of the shift number parameter to 6.

[0122] In another possible embodiment, the above method further includes the following regarding the value of the shift number parameter: The block size index value of the current block is determined based on the current block's size parameters. The value of the shift number parameter is determined based on the block size index value of the current block mentioned above.

[0123] It should be noted that the block size index value of the current block can be determined based on the current block size parameters. In some embodiments, determining the block size index value of the current block based on the current block size parameters may include the following:

[0124] If the current block's width and height are both equal to 4, set the current block's block size index value to 0. The block size index value of the current block is set to 1 if both the width and height of the current block are equal to 8, or if either the width or height of the current block is equal to 4. If the current block's width and height do not meet the above conditions, the block size index value of the current block is set to 2.

[0125] In this way, after determining the block size index value of the current block, the value of the shift number parameter can be further determined based on the block size index value of the current block.

[0126] Selectively, in some embodiments, determining the value of the shift number parameter based on the block size index value of the current block may include the following: If the block size index value is equal to 0, 1, or 2, it is determined that the value of the shift number parameter corresponding to the block size index value of the current block is equal to 5, 6, or 5, respectively.

[0127] Selectively, in some embodiments, determining the value of the shift number parameter based on the block size index value of the current block may include the following: Set the value of the shift number parameter to equal to the ratio of the current block's width or height to a first preset value corresponding to the current block's block size index value.

[0128] Here, the first preset value represents the number of MIP input sample values ​​obtained from the current block boundary. In this case, the above method may also include the following: If the current block's block size index value is equal to 0, 1, or 2, then it is determined that the first preset value corresponding to the current block's block size index value is equal to 2, 4, or 4.

[0129] In other words, when the first preset value represents the number of MIP input sample values ​​obtained from the boundary of the current block, the corresponding first preset value is equal to 2 when the block size index value of the current block is equal to 0, the corresponding first preset value is equal to 4 when the block size index value of the current block is equal to 1, and the corresponding first preset value is equal to 4 when the block size index value of the current block is equal to 2. In this way, the value of the shift number parameter can be determined based on the ratio of the width or height of the current block to the corresponding first preset value.

[0130] Selectively, in some embodiments, determining the value of the shift number parameter based on the block size index value of the current block may include the following: Set the value of the shift number parameter to the ratio of the current block's width or height to a second preset value corresponding to the current block's block size index value.

[0131] Here, the second pre-set value represents the MIP predicted block size of the current block, obtained by directly using the MIP weighting matrix. In this case, the above method may also include the following: If the current block's block size index value is equal to 0, 1, or 2, then it is determined that the second preset value corresponding to the current block's block size index value is equal to 4, 4, or 8.

[0132] In other words, when the second preset value represents the MIP predicted block size of the current block obtained by directly using the MIP weighting matrix, the corresponding second preset value is equal to 4 when the block size index value of the current block is equal to 0, the corresponding second preset value is equal to 4 when the block size index value of the current block is equal to 1, and the corresponding second preset value is equal to 8 when the block size index value of the current block is equal to 2. In this way, the value of the shift number parameter can be determined based on the ratio of the width or height of the current block to the corresponding second preset value.

[0133] In another possible embodiment, the shift number parameter table can be minimized, and the value of the shift number parameter is still determined by a lookup table method. Selectively, in some embodiments, determining the value of the shift number parameter based on the block size index value of the current block may include the following: Based on the block size index value, a second pre-configured lookup table is queried for the value corresponding to the block size index value. The second pre-configured lookup table is used to record the correspondence between the block size index value and the value of the shift number parameter. The queried value is determined as the value of the shift number parameter.

[0134] Furthermore, the value of the shift number parameter can be queried based solely on the block size index value (represented by mipSizeId) of the current block. In the second pre-configured lookup table shown in Table 5, each block size index value can correspond to a fixed value. That is, the size of each block or each block size set can have a fixed shift number parameter value as shown in Table 5.

[0135] [Table 5]

[0136] According to Table 5, if the current block size index value is equal to 0, 1, or 2, then it can be determined that the value of the shift number parameter corresponding to the block size index value is equal to 5, 6, or 5, respectively.

[0137] In the above embodiment, by simplifying the method for determining the shift number parameter, particularly by minimizing the shift number parameter table or fixing the values ​​of the shift number parameters, the amount of memory in the lookup table can be minimized, thereby reducing the memory occupied by the original shift number parameter table in MIP mode without increasing computational complexity.

[0138] In an encoder, a shift-offset parameter table can be established in advance, and this shift-offset parameter table is also stored in memory or a storage unit. This memory or storage unit may be integrated into the encoder or may be provided independently. In embodiments of the present invention, the shift-offset parameter (fO) can be determined in several ways.

[0139] In one possible embodiment, the shift offset parameter may differ for different block size index values ​​and different MIP mode index values. In some embodiments, the methods described above may be further included. When determining the intra-predicted value of the saturation component of the current block using MIP mode, the MIP mode index value of the current block is determined. Based on the MIP mode index value, a value corresponding to the MIP mode index value is queried from a third pre-configured lookup table. The third pre-configured lookup table is used to record the correspondence between the MIP mode index value and the value of the shift offset parameter. The queried value is determined as the value of the shift offset parameter.

[0140] Furthermore, the value of the shift offset parameter can be queried based on the current block size index value (represented by mipSizeId) and MIP mode index value (modelId). In the third pre-configured lookup table shown in Table 6, the shift offset parameter required for matrix multiplication can be determined by the lookup table for different mipSizeId and modelId values.

[0141] [Table 6]

[0142] However, the encoder needs to store Table 6 in memory or a storage unit as a lookup table. However, storage is costly, and the lookup process is also costly. Since the shift offset parameter values ​​in Table 6 are related to both the block size and the MIP mode index value of the current block, the amount of memory occupied increases, and the computational complexity also increases.

[0143] In an embodiment of the present invention, the method for determining the shift offset parameter can be simplified in order to reduce memory usage and computational complexity.

[0144] In another possible embodiment, the value of the shift offset parameter can be set to a fixed constant independent of the block size index value and the MIP mode index value. Generally, this fixed constant is in the range of 0 to 100. For example, the value of the shift offset parameter can be set to 32 for different block size index values ​​and different MIP mode index values. Alternatively, the value of the shift offset parameter can be set to 46 for different block size index values ​​and different MIP mode index values. Alternatively, the value of the shift offset parameter can be set to 56 for different block size index values ​​and different MIP mode index values. Alternatively, the value of the shift offset parameter can be set to 66 for different block size index values ​​and different MIP mode index values. In embodiments of the present invention, it is preferable, but not limited to, setting the value of the shift offset parameter to 32.

[0145] In another possible embodiment, the above method may further include the following regarding the value of the shift offset parameter: The block size index value of the current block is determined based on the current block's size parameters. The value of the shift offset parameter is determined based on the block size index value of the current block mentioned above.

[0146] Note that the current block's block size index value can be determined based on the current block's size parameter. Then, based on the current block's block size index value, the value of the shift offset parameter can be further determined.

[0147] Selectively, in some embodiments, determining the value of the shift offset parameter based on the block size index value of the current block may include the following: If the block size index value is equal to 0, 1, or 2, respectively, it is determined that the value of the shift offset parameter corresponding to the block size index value of the current block is equal to 34, 23, or 46, respectively.

[0148] In some embodiments, selectively, the shift offset parameter table can be minimized, and the shift factor can still be determined by a lookup table method. In some embodiments, selectively, determining the value of the shift offset parameter based on the block size index value of the current block may include the following: Based on the block size index value, a value corresponding to the block size index value is queried from a fourth pre-configured lookup table. The fourth pre-configured lookup table is used to record the correspondence between the block size index value and the value of the shift offset parameter. The queried value is determined as the value of the shift offset parameter.

[0149] Furthermore, the value of the shift offset parameter can be queried based solely on the block size index value (represented by mipSizeId) of the current block. In the fourth pre-configured lookup table shown in Table 7, each block size index value can correspond to a fixed value. That is, the size of each block or each block size set can have a fixed shift offset parameter value as shown in Table 7.

[0150] [Table 7]

[0151] According to Table 7, when the current block size index value is equal to 0, 1, or 2, it can be determined that the value of the shift offset parameter corresponding to the block size index value is equal to 34, 23, or 46, respectively.

[0152] In the above embodiment, by simplifying the method for determining the shift offset parameter, particularly by minimizing the shift offset parameter table or fixing the values ​​of the shift offset parameters, the amount of memory in the lookup table can be minimized, thereby reducing the memory occupied by the shift offset parameter table under MIP mode without increasing computational complexity.

[0153] Thus, after determining the shift offset parameter (fO) and the shift number parameter (sW), selectively determining the first constant value based on the value of the shift number parameter in some embodiments may include the following: The first constant value is set to a power of 2. The exponent of the power is an integer, equal to the value of the shift number parameter minus 1.

[0154] Selectively, in some embodiments, determining a first constant value based on the value of the shift number parameter may include the following: The first constant value is obtained by left-shifting "1" in binary. The number of bits left-shifted is equal to the value of the shift number parameter minus 1.

[0155] In other words, after the shift number parameter (sW) is obtained, the first constant value may be written as 1<<(sW-1) or as 2^(sW-1). In this case, when the value of the shift number parameter is set to 6, a first constant value equal to 32 can be obtained.

[0156] JPEG0007860206000012.jpg37170

[0157] Thus, under MIP mode, the MIP weighting matrix, MIP input sample values, shift number parameters, and the first offset can be obtained, which then facilitates the determination of the MIP prediction value for the current block.

[0158] S303-5: Determine the MIP predicted value for the saturation component of the current block based on the MIP weighting matrix, MIP input sample values, shift number parameter, and first offset.

[0159] Here, the MIP prediction is the predicted value for a subset of samples in the saturation component of the current block.

[0160] Determining the MIP prediction for the current block based on the above MIP weighting matrix, MIP input sample values, shift number parameters, and first offset may include the following:

[0161] The first weighted sum of the MIP weighting matrix and the MIP input sample values ​​is calculated. Calculate the first sum of the first weighted sum and the first offset. The first right-shifted value is obtained by right-shifting the first sum into binary. The number of bits right-shifted is equal to the value of the shift number parameter. The MIP prediction value for the current block is set to the sum of the first right-shifted value and the value corresponding to index 0 in the first temporary reference value. The first temporary reference value is obtained by downsampling the adjacent sample values ​​of the current block.

[0162] Specifically, under MIP mode, the MIP weighting matrix (represented by mWeight), shift number parameter (represented by sW), and shift offset parameter (represented by fO) can be determined based on the current block size index value (represented by mipSizeId) and MIP mode index value (represented by modelId). Next, the MIP input sample value (represented by p[x]), mWeight, sW, and fO are input to a matrix multiplication, and the MIP predicted value (which can be represented by predMip[x][y]) output from the matrix multiplication is obtained. The samples in predMip[x][y] are arranged in matrix / array form based on predSize × predSize. The calculation formula is as follows:

number

[0163] [x][y] represent the position coordinates of the sample, where x is the horizontal direction and y is the vertical direction. inSize represents the number of input samples. p[i] represents the value corresponding to index i in the MIP input sample value. pTemp[0] represents the value corresponding to index 0 in the first temporary reference value. ">>" represents the binary right shift operator, and "<<" represents the binary left shift operator. x=0, ..., predSize-1, y=0, ..., predSize-1. Thus, by calculating predMip[x][y] based on the above equation (5), the MIP prediction block can be obtained.

[0164] In some embodiments, when the fixed value of sW is 6 and the fixed value of fO is 32, the calculation formula is as follows:

number

[0165] Furthermore, in order to obtain the MIP prediction block for the current block, it is necessary to clip the predicted values ​​of the samples in the MIP prediction block. Next, it is determined whether or not to transpose the MIP prediction block. If the result is Yes, it is necessary to transpose the predicted sample values ​​in the MIP prediction block, and the transposed MIP prediction block is determined to be the MIP prediction block for the current block. If the result is No, it is not necessary to transpose the predicted sample values ​​in the MIP prediction block, and the MIP prediction value for the current block can be obtained by directly determining the MIP prediction block as the MIP prediction block for the current block, and this MIP prediction value is the predicted value of a portion of the samples in the current block.

[0166] S304: Determine the intra-predicted value of the saturation component of the current block by filtering the MIP prediction values.

[0167] Filtering the above MIP prediction values ​​to determine the current block's intra-prediction value may include the following:

[0168] Determine whether the size parameters of the MIP prediction block and the size parameters of the current block are the same. If the size parameters of the MIP prediction block and the current block are the same, the intra prediction block of the current block is set to be equal to the MIP prediction block. The MIP prediction block contains the predicted sample values ​​for all sample positions in the current block. If the size parameter of an MIP prediction block differs from the size parameter of the current block, the MIP prediction block is filtered to obtain the filtered prediction block, and the filtered prediction block is set as the intra-prediction block of the current block.

[0169] Here, filtering can include upsampling or low-pass filtering.

[0170] Note that MIP prediction blocks are composed of MIP prediction values. After obtaining an MIP prediction block, since the size parameter of the MIP prediction block only contains two types: 4x4 and 8x8, the size parameter of the current block and the size parameter of the MIP prediction block may or may not be the same. In other words, the sample values ​​corresponding to the MIP prediction block do not necessarily fill the current block completely, so upsampling of the MIP prediction block may be necessary to generate the final prediction values. That is, the decision of whether or not to upsample the MIP prediction block is made by determining whether or not the size parameter of the MIP prediction block and the size parameter of the current block are the same.

[0171] Furthermore, it is shown that upsampling of the MIP prediction block is not necessary when the size parameters of the MIP prediction block and the size parameters of the current block are the same, that is, when both the width and height of the MIP prediction block are the same as those of the current block. In this case, the MIP prediction block can be directly embedded into the current block, meaning that there are no vacant samples in the current block after embedding. In this case, the intra-predicted value of each sample in the current block can be directly set to the predicted value of each sample in the MIP prediction block, as shown below.

number

[0172] [x][y] represents the position coordinates of the sample, where x is the horizontal direction and y is the vertical direction. predSamples[x][y] represents the intra-predicted value corresponding to the sample whose position coordinates are [x][y] in the current block, and predMip[x][y] represents the predicted value corresponding to the sample whose position coordinates are [x][y] in the MIP prediction block. Thus, based on equation (7), the MIP prediction block predMip[x][y] can be directly used as the intra-predicted block predSamples[x][y] for the current block, that is, as the intra-predicted value of at least one sample in the current block.

[0173] Furthermore, if the size parameter of the MIP prediction block differs from the size parameter of the current block, that is, if at least one of the width and height of the MIP prediction block differs from that of the current block, the MIP prediction block cannot completely fill the current block, meaning that there are empty samples in the current block after filling, and in this case, it is shown that the MIP prediction block needs to be filtered. In other words, when upsampling is required in both the horizontal and vertical directions, the first upsampled block, which can be represented by predSamples[x][y], is obtained by first upsampling the MIP prediction block horizontally and then vertically. Subsequently, the second upsampled block, which can be represented by predSamplesTemp[x][y], can be obtained by upsampling the MIP prediction block vertically and then horizontally. Finally, the intra-predicted value of the saturation component of the current block is obtained by weighting predSamples[x][y] and predSamplesTemp[x][y], i.e., the intra-predicted block of the current block is obtained.

[0174] Exemplary, if the edge length nTbS of the current block (where S can be replaced with W and H respectively) is equal to the edge length predSize of predMip (where predSize is related only to the blocksizeIdx of the current block), the MIP prediction block can be directly set as the intra-prediction block of the current block. Otherwise, the MIP prediction block must be filtered to obtain the intra-prediction block of the current block. Here, taking a 4x4 current block as an example, if the size parameter of the current block and the size parameter of the MIP prediction block are the same, there is no need to filter the MIP prediction block, and the MIP prediction block can be directly set as the intra-prediction block of the current block, and the intra-prediction value of at least one sample in the current block is obtained.

[0175] In short, for the prediction process under MIP mode, the input data for MIP mode can include the current block position (xTbCmp, yTbCmp), the MIP mode index value applied to the current block (which can be represented by modelId), the current block height (represented by nTbH), the current block width (represented by nTbW), and a transpose instruction flag indicating whether transposition is required (i.e., the MIP transpose instruction parameter (which can be represented by isTransposed)). The output data for MIP mode can include the intra-prediction block of the current block. The predicted values ​​corresponding to the sample coordinates [x][y] in that intra-prediction block are predSamples[x][y], where x=0, 1, ... nTbW-1, y=0, 1, ... nTbH-1.

[0176] As shown in Figure 5, the MIP prediction process specifically includes the following steps.

[0177] S501: Configure the core parameters.

[0178] For S501, the MIP core parameters can include the current block type (represented by mipSizeId), the number of reference samples for each edge (represented by boundarySize), the number of MIP input samples (represented by inSize), and the size of the MIP predicted block output from matrix multiplication (arrayed as predSize × predSize). Based on the size of the current block, the current block can be classified into three types, and the current block type is recorded in mipSizeId. Here, mipSizeId may be equal to 0, 1, or 2. Also, different current block types will have different numbers of reference samples and different sizes of MIP predicted blocks output from matrix multiplication.

[0179] S502: Obtain a reference sample.

[0180] Regarding S502, when predicting the current block, both the block above and the block to the left of the current block are encoded blocks, and the original reference sample in MIP mode is the reconstructed value of the sample in the row above and the column to the left adjacent to the current block. In other words, the process of obtaining the reference sample above (represented by refT) and the reference sample to the left (represented by refL) of the current block is the process of obtaining the reference sample.

[0181] S503: Build an input sample.

[0182] Regarding S503, this step is used for inputting matrix multiplication and may specifically include the following: S5031: Obtain a reference sample. S5032: Build a reference sample buffer. S5033: Derive input samples for matrix multiplication.

[0183] Regarding S5031, the process of obtaining a reference sample is a process of downsampling the reference sample. Regarding S5031, there are two methods for constructing the reference sample buffer, which specifically include the following: S5032-1: A method for filling buffers that does not require transposition. S5032-2: Method for filling buffers that require transposition.

[0184] S504: Generate intra-predicted values.

[0185] Regarding S504, this step is used to obtain the MIP prediction value for the current block and may specifically include the following: S5041: Construct the MIP prediction block for the output samples of matrix multiplication. S5042: Perform clipping on the MIP prediction block of the output samples of matrix multiplication. S5043: Transpose the MIP prediction block of the output sample from matrix multiplication. S5044: Generate MIP prediction values.

[0186] Regarding S5041, the process for constructing the MIP prediction block may specifically include the following: S5041-1: Obtain the weight matrix. S5041-2: Retrieve shift parameters. S5041-3: Perform matrix multiplication operations.

[0187] In other words, in the process of constructing the MIP prediction block, it is necessary to obtain not only the weighting matrix but also the shift parameters (including the shift offset parameter and the shift number parameter). Next, matrix multiplication is performed to obtain the MIP prediction block of the output samples of the matrix multiplication. For S5044, the generation of the final MIP prediction value involves two situations, specifically including the following: S5044-1: Generates predicted values ​​that do not require upsampling. S5044-2: Generates predicted values ​​that require upsampling.

[0188] Thus, after the four steps S501 to S504 described above, in this embodiment of the present invention, an intra-predicted value of at least one sample in the saturation component of the current block can be obtained.

[0189] S305: Determine the predicted residual value of the saturation component of the current block based on the intra-predicted value of the saturation component of the current block.

[0190] Furthermore, after obtaining the intra-predicted value of the saturation component of the current block, the predicted residual value of the saturation component of the current block is obtained by calculating the difference between the actual value of the saturation component of the current block and the intra-predicted value of the saturation component of the current block, thereby obtaining the residual block of the current block.

[0191] S306: The LFNST parameters are determined by performing a low-frequency non-separated transform (LFNST) on the predicted residual values.

[0192] Regarding the transformation unit 108 shown in Figure 1, the transformation unit 108 can perform a first transformation on the residual block of the current block, for example, an integer transformation initially designed based on the Discrete Cosine Transform (DCT). It can also perform a second transformation on the residual block of the current block, for example, a Low Frequency Non-Separable Transform (LFNST). Specifically, in some embodiments, determining the LFNST parameters by performing the LFNST on the predicted residual values ​​may include the following.

[0193] Based on the predicted residual values, the residual block of the current block is determined, and the first transformation coefficient block is obtained by performing the first transformation on the residual block. It is determined whether or not to perform LFNST on at least some of the conversion coefficients in the first conversion coefficient block. If the result of the decision is Yes, the LFNST parameters are determined by performing LFNST on at least some of the conversion coefficients in the first conversion coefficient block.

[0194] In embodiments of the present invention, the encoder obtains a first transformation coefficient block after the first transformation by performing a first transformation (also called the "core transformation," "first-order transformation," or "main transformation") on the predicted residual values ​​in the residual block. Next, the LFNST parameters are determined by performing LFNST (also called the "secondary transformation" or "quadratic transformation") on some or all of the transformation coefficients in the first transformation coefficient block. Note that the first transformation in this specification is a different transformation from LFNST.

[0195] Furthermore, it should be noted that LFNST is not performed on any current block. After the first transformation, several conditions (e.g., the minimum value of the size parameters of the current block, the block size index value of the current block, etc.) must be checked to determine whether or not to perform LFNST on at least some of the transformation coefficients in the first transformation coefficient block. Selectively, in some embodiments, determining whether or not to perform LFNST on at least some of the transformation coefficients in the first transformation coefficient block may include the following: When determining the intra-predicted value of the saturation component of the current block using MIP mode, the minimum value among the size parameters of the current block is determined. Based on the minimum value, it is determined whether or not to perform LFNST on at least some of the conversion coefficients in the first conversion coefficient block.

[0196] Furthermore, determining whether or not to perform LFNST on at least some of the conversion coefficients in the first conversion coefficient block based on the above minimum value may include the following: If the minimum value is greater than or equal to a first preset threshold, it is decided to perform LFNST on at least some of the conversion coefficients in the first conversion coefficient block.

[0197] Here, the first preset threshold can be used to represent a preset threshold for determining whether or not to perform LFNST. In embodiments of the present invention, the first preset threshold can be set to 8, but is not particularly limited.

[0198] In other words, when performing intraprediction on the saturation component of the current block using MIP mode, it is possible to determine the minimum value among the size parameters of the current block. For example, if the minimum values ​​of width and height are 8 or greater, it can be determined to perform LFNST on at least some of the transformation coefficients in the first transformation coefficient block.

[0199] In some embodiments, selectively determining whether or not to perform LFNST on at least some of the conversion coefficients in the first conversion coefficient block may include the following: When determining the intra-predicted value of the saturation component of the current block using MIP mode, the block size index value of the current block is determined based on the size parameter of the current block. Based on the block size index value, it is determined whether or not to perform LFNST on at least some of the conversion coefficients in the first conversion coefficient block.

[0200] Furthermore, determining whether or not to perform LFNST on at least some of the conversion coefficients in the first conversion coefficient block based on the block size index value may include deciding to perform LFNST on at least some of the conversion coefficients in the first conversion coefficient block if the block size index value is equal to a second preset threshold.

[0201] Here, the second preset threshold can be used to represent a preset threshold for determining whether or not to perform LFNST. In embodiments of the present invention, the second preset threshold can be set to one set value (e.g., 2) or one of a plurality of set values ​​(e.g., 1, 2), but is not particularly limited.

[0202] In other words, when performing intraprediction on the saturation component of the current block using MIP mode, the block size index value (mipSizeId) of the current block can be determined based on the size parameter of the current block. For example, if the value of mipSizeId is equal to 2, it can be determined that LFNST should be performed on at least some of the transformation coefficients in the first transformation coefficient block.

[0203] In some embodiments, selectively determining whether or not to perform LFNST on at least some of the conversion coefficients in the first conversion coefficient block may include the following: The first cost result is calculated when LFNST is not performed on at least some of the conversion coefficients in the first conversion coefficient block. The second cost result is calculated when LFNST is performed on at least some of the conversion coefficients in the first conversion coefficient block. Based on the first and second cost results, it is determined whether or not to perform LFNST on at least some of the conversion coefficients in the first conversion coefficient block.

[0204] Furthermore, determining whether to perform LFNST on at least some of the conversion coefficients in the first conversion coefficient block based on the first and second cost results may include deciding to perform LFNST on at least some of the conversion coefficients in the first conversion coefficient block if the first cost result is greater than the second cost result.

[0205] It should be noted that in embodiments of the present invention, a conventional rate distortion optimization method may be employed. First, a first cost result is calculated when LFNST is not performed on at least some of the conversion coefficients in the first conversion coefficient block. Next, a second cost result is calculated when LFNST is performed on at least some of the conversion coefficients in the first conversion coefficient block. The first cost result and the second cost result are compared, and if the first cost result is greater than the second cost result, it can be decided to perform LFNST on at least some of the conversion coefficients in the first conversion coefficient block.

[0206] Thus, when the result of the decision is Yes, that is, when LFNST needs to be performed on at least some of the conversion coefficients in the first conversion coefficient block, the LFNST parameters can be determined by performing LFNST on at least some of the conversion coefficients in the first conversion coefficient block.

[0207] S307: Encode the LFNST parameters and write them to the bitstream.

[0208] The LFNST parameter includes the LFNST index and can be represented as lfnst_index. In some embodiments, determining the LFNST parameter may include the following: If it is decided to perform LFNST on at least some of the conversion coefficients in the first conversion coefficient block, the value of the LFNST index is set to a value greater than 0. If it is determined that LFNST should not be performed on at least some of the conversion coefficients in the first conversion coefficient block, the value of the LFNST index is set to 0.

[0209] Furthermore, for S307, encoding the LFNST parameters and writing them to the bitstream may include encoding the LFNST index and writing it to the bitstream.

[0210] In other words, after determining the LFNST parameter, i.e., lfnst_index, it is shown that LFNST is not performed if lfnst_index is equal to 0. It is shown that LFNST is necessary if lfnst_index is greater than 0. In the coding process, lfnst_index can be coded using Context-based Adaptive Binary Arithmetic Coding (CABAC) with descriptor "ae(v)".

[0211] For illustrative purposes, see Table 8, which shows an example of a description of the syntax structure of LFNST parameters related to the relevant technology. In this case, the conditions for determining whether or not to perform LFNST on at least some of the transformation coefficient blocks in the first transformation coefficient block may include at least one of the following: treeType == DUAL_TREE_CHROMA, !IntraMipFlag[ x0 ][ y0 ], Min( lfnstWidth, lfnstHeight ) >= 16.

[0212] [Table 8]

[0213] Referring to Tables 9A and 9B, Tables 9A and 9B each show an example of describing the syntax structure of LFNST parameters according to an embodiment of the present invention. In this case, the condition for determining whether or not to perform LFNST on at least some of the conversion coefficient blocks in the first conversion coefficient block may include at least one of intra_mip_flag[x0][y0]==1&&Min(lfnstWidth, lfnstHeight) >=8 and intra_mip_flag[x0][y0]==1&& mipSizeId ==2.

[0214] Specifically, as shown in the syntax structure of Table 9A, the lfnst_index of the current block can be encoded using MIP mode (i.e., when intra_mip_flag[x0][y0]==1) when the minimum value of the current block's size parameter is 8 or greater (i.e., both the width and height are 8 or greater). Alternatively, the determination condition may be set to be that both the width and height are greater than or equal to a first value (e.g., 8), and the sum of the width and height is greater than or equal to a second value (e.g., 16). As shown in the syntax structure of Table 9B, the lfnst_index of the current block can be encoded using MIP mode (i.e., when intra_mip_flag[x0][y0]==1) when the block size index value (mipSizeId) of the current block is equal to 2. Alternatively, the determination condition may be set to be that mipSizeId is equal to one of several set values ​​(e.g., 1, 2).

[0215] [Table 9A]

[0216] [Table 9B]

[0217] Furthermore, if it is determined that LFNST can be performed on the current block, it is necessary to determine the LFNST transformation kernel (represented by kernel) to be used for the current block. In some embodiments, determining the LFNST parameters by performing LFNST on the predicted residual values ​​may include: determining the LFNST transformation kernel to be used for the current block, and determining the LFNST parameters by performing LFNST on the predicted residual values ​​using the LFNST transformation kernel.

[0218] In embodiments of the present invention, the LFNST has four candidate sets of translation kernels. These four candidate sets of translation kernels may include set 0, set 1, set 2, and set 3. Here, in embodiments of the present invention, the LFNST translation kernel candidate set can be determined using MIP parameters, and then an LFNST translation kernel to be used for the current block is selected from the LFNST translation kernel candidate set. Thus, in some embodiments, the above method may further include the following.

[0219] When determining the intra-predicted value of the saturation component of the current block using MIP mode, the MIP parameters are determined from the prediction parameters of the current block. Based on the MIP parameters, a candidate set of LFNST conversion kernels is determined. In the LFNST conversion kernel candidate set, the conversion kernel indicated by the LFNST index is set as the LFNST conversion kernel to be used for the current block. The LFNST index value is set to indicate that LFNST should be performed on the current block and to indicate the index of the LFNST transform kernel in the LFNST transform kernel candidate set. The LFNST conversion kernel candidate set includes two or more pre-configured LFNST conversion kernels.

[0220] The MIP parameter may include at least one of the following: the MIP mode index value (represented by modelId) and the MIP transpose instruction parameter (represented by isTransposed). For the LFNST transformation kernel used for the current block, if the value of the MIP transpose instruction parameter indicates that a transpose should be performed on the sample input vector used for the MIP mode, then a matrix transpose must be performed on the selected transformation kernel to obtain the LFNST transformation kernel used for the current block.

[0221] Furthermore, in some embodiments, determining a candidate set of LFNST conversion kernels based on the above MIP parameters may include the following: The value of the LFNST intra-predictive mode index is determined based on the MIP mode index value. Based on the LFNST intra-prediction mode index value, a fifth pre-configured lookup table determines the index value of the LFNST conversion kernel candidate set. Based on the index value of the LFNST conversion kernel candidate set, the LFNST conversion kernel candidate set is selected from multiple candidate LFNST conversion kernel candidate sets.

[0222] Here, the LFNST intra prediction mode index can be represented by predModeIntra, and the index value of the LFNST conversion kernel candidate set can be represented by lfnstTrSetIdx.

[0223] The fifth pre-configured lookup table is shown in Table 10. The value of predModeIntra can be determined based on the MIP mode index value (i.e., the value of modelId). Next, based on the value of predModeIntra, the value of lfnstTrSetIdx can be directly determined in conjunction with Table 10, that is, the LFNST conversion kernel candidate set selected for the current block can be determined. Here, the value of lfnstTrSetIdx indicates the conversion kernel candidate set used for LFNST. Since the value of modelId can include 0, 1, 2, 3, 4, and 5, the value of predModeIntra can also be 0, 1, 2, 3, 4, or 5. The correspondence between predModeIntra and lfnstTrSetIdx is as follows.

[0224] [Table 10]

[0225] In one specific embodiment, determining the value of the LFNST intra prediction mode index based on the MIP mode index value may include setting the value of the LFNST intra prediction mode index equal to the MIP mode index value.

[0226] In other words, in an embodiment of the present invention, the value of predModeIntra can be set equal to the value of modelId. Then, based on the value of predModeIntra, the value of lfnstTrSetIdx can be directly determined in conjunction with Table 10, that is, the set of LFNST transform kernel candidates selected for the current block is determined. Also, in another specific embodiment of the present invention, the value of modelId is directly mapped to the value of the PLANAR mode, and then, using the value of predModeIntra corresponding to the PLANAR mode, the value of lfnstTrSetIdx is determined, that is, the set of LFNST transform kernel candidates selected for the current block is determined.

[0227] In some embodiments, the LFNST parameter further includes LFNST coefficients. Determining the LFNST parameter may include obtaining the LFNST coefficients by performing LFNST on at least some of the transform coefficients in the first transform coefficient block.

[0228] Accordingly, encoding the LFNST parameter and writing it into the bitstream includes obtaining quantization coefficients by quantizing the LFNST coefficients, and encoding and writing the quantization coefficients into the bitstream.

[0229] Furthermore, if lfnst_index is not equal to 0, that is, if it is decided to perform LFNST on the current block, the LFNST coefficients can be obtained by performing LFNST on at least some of the conversion coefficients in the first conversion coefficient block. Next, the LFNST coefficients are quantized, encoded, and written to a bitstream.

[0230] Let us assume that d[x][y] is the first transformation coefficient block, where x=0, ..., nTbW-1, y=0, ..., nTbH-1, where nTbW and nTbH represent the width and height of the current block, respectively. Let us assume that v[x] represents the block containing the LFNST coefficient, where x=0, ..., nLfnstOutSize-1, nLfnstOutSize = (nTbW>=8 && nTbH>=8)? 48 : 16.

[0231] In one possible embodiment, to obtain LFNST coefficients, LFNST can be performed on at least some of the transformation coefficients in the first transformation coefficient block in the horizontal scan sequence. Specifically, this is done as follows:

number

[0232] log2LfnstSize is determined as follows:

number

[0233] In another possible embodiment, the scan order may include both a horizontal and a vertical scan order. Here, the determination of the scan order is related to the MIP transpose instruction parameter (represented by isTransposed).

[0234] Selectively, in some embodiments, obtaining LFNST coefficients by performing LFNST on at least some of the conversion coefficients in the first conversion coefficient block may include the following:

[0235] When the value of the MIP transpose instruction parameter indicates that a transpose should be performed on the sample input vector used in MIP mode, the LFNST coefficients are obtained by performing LFNST on at least some of the transformation coefficients in the first transformation coefficient block in the vertical scan sequence. If the value of the MIP transpose instruction parameter indicates that no transposition should be performed on the sample input vector used in MIP mode, the LFNST coefficients are obtained by performing LFNST on at least some of the transformation coefficients in the first transformation coefficient block in the horizontal scan sequence.

[0236] Specifically, if the value of isTransposed is equal to 0, no transposition is performed on the sample input vector used in MIP mode. Then, LFNST can be performed on at least some of the transformation coefficients in the first transformation coefficient block in the horizontal scan order. Specifically, this is as follows:

number

[0237] If the value of isTransposed is equal to 1, the sample input vector used in MIP mode is transposed. Then, LFNST can be performed on at least some of the transformation coefficients in the first transformation coefficient block in the vertical scan order. Specifically, this is done as follows:

number

[0238] log2LfnstSize is determined as shown in equation (9) above.

[0239] Optionally, in some embodiments, obtaining the LFNST coefficients by performing LFNST on at least some of the transform coefficients in the first transform coefficient block may include the following.

[0240] When the value of the MIP transpose indication parameter indicates that transposition is to be performed on the sample input vector used in the MIP mode, the LFNST coefficients are obtained by performing LFNST on at least some of the transform coefficients in the first transform coefficient block in the horizontal scan order. When the value of the MIP transpose indication parameter indicates that no transposition is to be performed on the sample input vector used in the MIP mode, the LFNST coefficients are obtained by performing LFNST on at least some of the transform coefficients in the first transform coefficient block in the vertical scan order.

[0241] Specifically, when the value of isTransposed is equal to 1, transposition is performed on the sample input vector used in the MIP mode. And LFNST can be performed on at least some of the transform coefficients in the first transform coefficient block in the horizontal scan order. Specifically, it is as shown in the above formula (10).

[0242] When the value of isTransposed is equal to 0, no transposition is performed on the sample input vector used in the MIP mode. And LFNST can be performed on at least some of the transform coefficients in the first transform coefficient block in the vertical scan order. Specifically, it is as shown in the above formula (11).

[0243] log2LfnstSize is determined as in the above formula (9).

[0244] Also, the LFNST in the embodiments of the present invention can be applied not only to the MIP mode but also to conventional intra prediction modes (such as the angular intra prediction mode, the DC mode, and the PLANAR mode, etc.). The specific implementation method is similar to the MIP mode.

[0245] In some embodiments, determining a candidate set of LFNST conversion kernels based on the above MIP parameters may include the following: The value of the LFNST intra-prediction mode index is determined based on the index value of the conventional intra-prediction mode. Based on the LFNST intra-prediction mode index value, a sixth pre-configured lookup table determines the index value of the LFNST conversion kernel candidate set. Based on the index value of the candidate LFNST conversion kernel set, a candidate LFNST conversion kernel set is selected from multiple candidate LFNST conversion kernel sets.

[0246] The sixth pre-configured lookup table is shown in Table 11. The value of predModeIntra can be determined based on the index value of the conventional intra prediction mode. Next, based on the value of predModeIntra, the value of lfnstTrSetIdx can be directly determined in conjunction with Table 11, that is, the LFNST conversion kernel candidate set selected for the current block can be determined. Here, in order to determine the value of lfnstTrSetIdx, the index value of the conventional intra prediction mode can be assigned to predModeIntra.

[0247] [Table 11]

[0248] The encoding method of an embodiment of the present invention will be described in detail below with reference to the encoder 100 shown in Figure 1.

[0249] It should be understood that the current block (or "encoded block") in embodiments of the present invention may be a CU or a partition of a CU (e.g., a transform block). Specifically, when the intra-prediction unit 106 decides to encode the current block using an MIP mode, or when the intra-prediction unit 106 evaluates an MIP mode for encoding the current block using a rate-distortion-optimized method, the intra-prediction unit 106 obtains an intra-prediction block of the current block. The intra-prediction block of the current block contains intra-prediction values ​​for at least one sample in the current block.

[0250] Specifically, the steps by which the intra prediction unit 106 acquires intra prediction blocks using MIP mode are as follows:

[0251] First, the intra-prediction unit 106 obtains one or more reference samples from the adjacent samples of the current block. This can be done, for example, by downsampling the adjacent samples or by directly extracting from the adjacent samples.

[0252] Next, the intra-prediction unit 106 uses the acquired reference sample, MIP matrix, and shift parameters to determine one or more partial prediction samples corresponding to the sample positions in the current block. Here, the sample positions can be preset sample positions in the current block, for example, having equal horizontal and vertical coordinate values. The shift parameters include a shift number parameter and a shift offset parameter, which can be used for offset operations in the process of acquiring intra-predicted values ​​for partial samples.

[0253] Finally, once a corresponding predicted sample has been obtained for a partial sample in the current block, the intra-prediction unit 106 needs to obtain further predicted samples for the remaining samples in the current block other than the partial sample. For example, the intra-prediction unit 106 can use an interpolation filter to obtain predicted samples for the remaining samples, and the input to the interpolation filter can be the partial sample and adjacent samples.

[0254] As shown in Figure 6, Figure 6 shows a flowchart for obtaining an intra-prediction block using the MIP mode according to an embodiment of the present invention. This flow can be implemented in the encoder 100, where "current block" refers to the "encoded block".

[0255] In step 601, the intra-prediction unit 106 obtains adjacent samples of the current block, for example, marked as gray squares adjacent to the current block in step 601 as shown in Figure 6. The intra-prediction unit 106 obtains one or more reference samples from the adjacent samples. In the example of step 601 shown in Figure 6, selectively, the intra-prediction unit 106 may calculate the mean of two adjacent samples and use that mean as a reference sample. Selectively, the intra-prediction unit 106 selects one adjacent sample as a reference sample for every other adjacent sample. For example, in the example of step 601 shown in Figure 6, the intra-prediction unit 106 selects four reference samples from the eight upper adjacent samples of the current block and another four reference samples from the eight left adjacent samples of the current block.

[0256] The specific process for Step 601 is shown below.

[0257] The intra-prediction unit 106 obtains the width and height of the current block from the block division unit 103, where the width and height are represented by the variables cbWidth and cbHeight, respectively. For example, the intra-prediction unit 106 determines the intra-prediction mode by calling the mode determination method for rate distortion optimization. The current block is divided into one or more transformation blocks. Let the variables nTbW and nTbH be the width and height of the transformation block, respectively. When obtaining the intra-prediction value of the current block using the MIP mode as the intra-prediction mode, the intra-prediction unit 106 determines the MIP block size index value, i.e., the variable represented by mipSizeId.

[0258] Selectively, the intra-prediction unit 106 determines the value of mipSizeId as follows: -If both nTbW and nTbH are equal to 4, set mipSizeId to equal to 0. - Otherwise, if cbWidth or cbHeight is equal to 4, set mipSizeId to equal to 1. - Otherwise, set mipSizeId to equal 2.

[0259] Specifically, if the current block size parameter is 8x8 (i.e., both cbWidth and cbHeight are equal to 8), then mipSizeId is set to equal to 2.

[0260] Selectively, the intra-prediction unit 106 determines the value of mipSizeId as follows: -If both nTbW and nTbH are equal to 4, set mipSizeId to equal to 0. - Otherwise, set mipSizeId to equal to 1 if cbWidth or cbHeight is equal to 4, or if both cbWidth and cbHeight are equal to 8. - Otherwise, set mipSizeId to equal 2.

[0261] Specifically, if the current block size parameter is 8x8, set mipSizeId to equal 1.

[0262] Furthermore, the intra-prediction unit 106 obtains the values ​​of boundarySize and predSize based on mipSizeId. Specifically, it does so as follows: -If mipSizeId is equal to 0, set boundarySize to equal to 2 and predSize to equal to 4. - Otherwise, if mipSizeId is equal to 1, set boundarySize to equal to 4 and predSize to equal to 4. - Otherwise (if mipSizeId is equal to 2), set boundarySize to equal to 4 and predSize to equal to 8. boundarySize represents the number of reference samples obtained from the top adjacent reference samples and left adjacent reference samples of the current block.

[0263] The intra-prediction unit 106 may further obtain the variable isTransposed to indicate the order of the reference samples stored in the buffer pTemp. For example, isTransposed being equal to 0 indicates that the intra-prediction unit 106 first presents the reference sample obtained from the upper neighbor reference sample of the current block, and then the reference sample obtained from the left neighbor reference sample. Otherwise, isTransposed being equal to 1 indicates that the intra-prediction unit 106 first presents the reference sample obtained from the left neighbor reference sample of the current block, and then the reference sample obtained from the upper neighbor reference sample. The value of isTransposed is sent to the entropy coding unit 115 as one of the MIP mode parameters that need to be encoded and written to the bitstream.

[0264] The intra-prediction unit 106 obtains the value of inSize to indicate the number of reference samples used in MIP mode, as shown in equation (1) above.

[0265] The intra-prediction unit 106 invokes the following process to obtain a set of reference samples (stored as an array p[x], where x is 0, ..., inSize-1) using adjacent reference samples from the current block.

[0266] The intra-prediction unit 106 obtains nTbW reference samples from the upper adjacent reference samples of the current block (for example, stored in array refT) and nTbH reference samples from the left adjacent reference samples of the current block (for example, stored in array refL). The intra-prediction unit 106 calls downsampling on refT to obtain boundarySize reference samples and stores the boundarySize reference samples in refT. The intra-prediction unit 106 calls downsampling on refL to obtain boundarySize reference samples and stores the boundarySize reference samples in refL.

[0267] The intra-prediction unit 106 may further obtain the variable isTransposed to indicate the order of the reference samples stored in the buffer pTemp. For example, isTransposed being equal to 0 (or FALSE) indicates that the intra-prediction unit 106 first presents the reference sample obtained from the upper neighbor reference sample of the current block, and then the reference sample obtained from the left neighbor reference sample. Otherwise, isTransposed being equal to 1 (or TRUE) indicates that the intra-prediction unit 106 first presents the reference sample obtained from the left neighbor reference sample of the current block, and then the reference sample obtained from the upper neighbor reference sample. The intra-prediction unit 106 may determine the value of isTransposed by using a rate-distortion optimization method or based on a comparison of the correlation between neighbor reference samples and the current block. The value of isTransposed is sent to the entropy coding unit 115 as one of the MIP mode parameters that need to be encoded and written to the bitstream.

[0268] The intra prediction unit 106 places the elements in refT and refL into buffer pTemp based on the order indicated by isTransposed.

[0269] Selectively, the intra-prediction unit 106 obtains p[x] as follows: x=0, ..., inSize-1. -When mipSizeId is equal to 2, p[x] = pTemp[x + 1] - pTemp[0]. - Otherwise (mipSizeId is less than 2), p[0] = pTemp[0] - (1 << (BitDepth-1)), p[x] = pTemp[x] - pTemp[0]. BitDepth is the bit depth of the color component of the sample in the current block. Here, the color component can be one of the RGB components, one of the YUV components, or one of the YCbCr components, for example, the Y component.

[0270] Selectively, the intra-prediction unit 106 can obtain p[x] as follows, where x=0, ..., inSize-1. -When mipSizeId is equal to 2, p[x] = pTemp[x + 1] - pTemp[0]. - Otherwise (mipSizeId is less than 2), p[0] = (1 << (BitDepth-1)) - pTemp[0], p[x] = pTemp[x] - pTemp[0].

[0271] Selectively, the intra-prediction unit 106 can obtain the value of p[x] using a unified calculation method without determining the value of mipSizeId. For example, (1 << (BitDepth-1)) is added as an additional element to the buffer pTemp. In this case, the intra-prediction unit 106 calculates p[x] as pTemp[x+1] - pTemp[0].

[0272] In step 602, the intra-prediction unit 106 obtains the MIP prediction value for the current block using the reference sample set and the MIP weighting matrix. The MIP weighting matrix is ​​selected from a predefined set of MIP weighting matrices based on the MIP mode index value (represented by ModeId) and the MIP block size index value (represented by mipSizeId) corresponding to the MIP mode.

[0273] The intra-prediction unit 106 obtains the MIP prediction values ​​(represented as predMip[x][y]) of partial prediction samples corresponding to one or more sample positions in the current block. In the example of step 602 shown in Figure 6, the partial prediction samples are the samples marked as gray squares in the current block. The input to the prediction module 601 is the reference sample p[x] obtained in step 601, and the prediction module 601 calculates the partial prediction samples using the MIP weighting matrix and shift parameters. The shift parameters include a shift number parameter and a shift offset parameter.

[0274] In one possible embodiment, the prediction module 601 obtains a prediction sample represented by predMip[x][y] by setting its coordinates to (x,y). The formula for calculating predMip[x][y] is as shown in equation (5) or equation (6).

[0275] In equation (5) or (6), mWeight[i][j] is the MIP weighting matrix, the matrix elements may be predetermined constants or may be adaptively updated using a training method or the like. The input to the training method is one or more encoded images or blocks, or images from another bitstream provided to the encoder 100 by an external device. fO is the shift offset parameter for determining oW, sW is the shift number parameter, p[i] is the MIP input sample value calculated using a reference sample, pTemp[0] is the value corresponding to index 0 in the first temporary reference value, and ">>" is the binary right shift operator defined in VVC. When it is finally decided to perform intra-prediction on the current block using MIP mode, the intra-prediction unit 106 can further send mWeight[i][j] to the entropy coding unit 115. The entropy coding unit 115 can write mWeight[i][j] to one or more data units in the bitstream.

[0276] The prediction module 601 can determine the values ​​of sW and fO based on the current block size and the MIP mode used for the current block. In one example, the prediction module 601 obtains the values ​​of sW and fO using a lookup table.

[0277] Selectively, the prediction module 601 can determine sW based on the current block size parameters and MIP mode by utilizing Table 4 above.

[0278] Selectively, the prediction module 601 can also determine sW based on the current block size parameter by utilizing Table 5 above.

[0279] Selectively, the prediction module 601 can also directly set sW to a constant value. For example, for blocks with various size parameters and different MIP modes, the prediction module 601 can set sW to 5. Alternatively, for blocks with various size parameters and different MIP modes, the prediction module 601 can set sW to 6. Alternatively, for blocks with various size parameters and different MIP modes, the prediction module 601 can set sW to 7.

[0280] Selectively, the prediction module 601 can determine fO based on the current block size parameters and MIP mode by utilizing Table 6 above.

[0281] Selectively, the prediction module 601 can also determine fO based on the current block size parameters by utilizing Table 7 above.

[0282] Selectively, the prediction module 601 can also directly set fO to a constant value (e.g., 0 to 100). For example, for blocks with various size parameters and different MIP modes, the prediction module 601 can set fO to 32. Or, the prediction module 601 can set fO to 46. Or, the prediction module 601 can set fO to 56. Or, the prediction module 601 can set fO to 66.

[0283] The intra prediction unit 106 can perform a clipping operation on the MIP prediction values ​​in predMip. When isTransposed is equal to 1 (or TRUE), the array predMip[x][y] (x = 0, ..., predSize-1, y = 0, ..., predSize-1) of predSize × predSize is converted to predTemp[y][x] = predMip[x][y], and then predMip = predTemp.

[0284] As an example, selectively, if the current block size parameter is 8x8 (i.e., both cbWidth and cbHeight are equal to 8), the intra-prediction unit 106 sets mipSizeId to equal 2 and obtains an 8x8 predMip.

[0285] Selectively, if the current block size parameter is 8x8 (i.e., both cbWidth and cbHeight are equal to 8), the intra-prediction unit 106 sets mipSizeId to equal 1 and obtains a 4x4 predMip.

[0286] Step 603, the intra-prediction unit 106 obtains the intra-predicted values ​​for the current block (stored in an array as predSamples[x][y], where x=0, ..., nTbW-1, y=0, ..., nTbH-1) as follows:

[0287] - When the intra-prediction unit 106 determines that nTbW is greater than predSize, or that nTbH is greater than predSize, it invokes the upsampling process to obtain predSamples using predMip. The intra-prediction unit 106 obtains predicted samples corresponding to the remaining samples in the current block, excluding the partial samples. In Figure 6, the intra-prediction unit 106 can use the filtering module 602 to obtain predicted samples corresponding to the remaining samples in the current block, excluding the partial samples. The input to the filtering module 602 is the samples marked as gray squares in step 602. The filtering module 602 can use one or more interpolation filters to obtain predicted samples corresponding to the remaining samples in the current block, excluding the partial samples. For example, the input may include a reference sample and partial predicted samples of one or more sample locations in the current block. Alternatively, the input may include adjacent samples and partial predicted samples of one or more sample locations in the current block. Alternatively, the input may include a reference sample, adjacent samples, and partial predicted samples of one or more sample locations in the current block. For example, selectively, if the current block size is 8x8 (i.e., both cbWidth and cbHeight are equal to 8), the intra-prediction unit 106 determines that mipSizeId is equal to 1, and the intra-prediction unit 106 obtains an 8x8 intra-prediction block of the current block by applying an upsampling process to a 4x4 predMip.

[0288] -Otherwise, the intra-prediction unit 106 sets the intra-prediction block of the current block to be equal to the MIP prediction block of the current block, that is, sets predSamples[x][y](x=0, ..., nTbW-1, y=0, ..., nTbH-1) to be equal to predMip[x][y]. For example, specifically, if the size parameter of the current block is 8x8 (i.e., both cbWidth and cbHeight are equal to 8), the intra-prediction unit 106 determines that mipSizeId is equal to 2 and obtains predSamples for the current block which is 8x8 in size (i.e., both cbWidth and cbHeight are equal to 8).

[0289] Thus, according to the block diagram of the flow shown in Figure 6, after step 603, the intra-prediction unit 106 can obtain the intra-prediction block for the current block (i.e., CU), that is, it can determine the intra-prediction value for at least one sample of the current block.

[0290] The prediction unit 102 outputs the intra-predicted block of the current block. The first adder 107 calculates the difference between the current block and the intra-predicted block of the current block at the output of the division unit 101, i.e., the residual block (i.e., residual CU). The transformation unit 108 reads the residual block and obtains coefficients by performing one or more transformation operations on the residual block. The quantization unit 109 quantizes the coefficients and outputs the quantization coefficients (i.e., levels).

[0291] The transformation unit 108 performs a first transformation on the residual block, for example, an integer transformation initially designed based on the DCT. The transformation unit 108 determines whether or not it is permitted to apply a quadratic transformation to the block. If it determines that it is permitted, the transformation unit 108 further determines whether or not to apply the quadratic transformation to the coefficients obtained after the first transformation. LFNST is an example of a quadratic transformation.

[0292] Referring to Figure 7, Figure 7 shows a schematic diagram of the flow of performing LFNST according to an embodiment of the present invention. In Figure 7, block 701 represents the residual block.

[0293] Step 701, the transformation unit 108 obtains block 702 containing the transformation coefficients corresponding to the first transformation (referred to as the "first transformation coefficients") by performing a first transformation (i.e., a core transformation) on block 701, for example, an integer transformation initially designed based on the DCT.

[0294] Step 702, the conversion unit 108 determines whether or not a secondary conversion such as LFNST is permitted for all or some of the first conversion coefficients in block 702.

[0295] As an example, selectively, the conversion unit 108 is permitted to perform LFNST on a conversion block if the intra-prediction mode of the conversion block including LFNST is MIP mode and the minimum width and height of the conversion block are 8 or greater.

[0296] Selectively, the conversion unit 108 is permitted to perform LFNST on a conversion block if the intra-prediction mode of the conversion block including LFNST is MIP mode and the value of mipSizeId is equal to a set value (e.g., 2) or one of several set values ​​(e.g., 1, 2).

[0297] If the conversion unit 108 is permitted to perform LFNST on the conversion block, the conversion unit 108 determines the LFNST parameters, for example, lfnst_index. An lfnst_index equal to 0 indicates that the conversion unit 108 will not perform LFNST on the conversion block. An lfnst_index greater than 0 indicates that LFNST will be performed on the conversion block using a conversion kernel from the LFNST conversion kernel candidate set.

[0298] The conversion unit 108 can determine the value of lfnst_index using a conventional rate distortion optimization method. First, the conversion unit 108 obtains a first cost result when LFNST is not performed on the conversion block.

[0299] The conversion unit 108 selects a candidate set of LFNST conversion kernels based on the MIP parameters. The MIP parameters may include one of the following: the MIP mode index value (i.e., modelId) and the MIP transpose instruction parameter (i.e., isTransposed).

[0300] The conversion unit 108 sets the value of the variable predModeIntra to equal the value of modelId. The conversion unit 108 selects a portion of the first conversion coefficients in block 702, for example, the coefficients in subblocks 7001, 7002, and 7003, and determines a conversion kernel for performing LFNST on the coefficients in the subblocks. The size of the subblocks can be a set value, for example, 8 × 8. Based on the value of predModeIntra, the conversion unit 108 determines the index value of the LFNST conversion kernel candidate set (i.e., lfnstTrSetIdx). For the original block coding in MIP mode, the conversion unit 108 can obtain the value of lfnstTrSetIdx using a lookup table as shown in Table 10.

[0301] The conversion unit 108 can determine the value of lfnst_index using a conventional rate-distortion optimization method (i.e., a value greater than 0 indicates that the conversion kernel used in the LFNST conversion kernel candidate set is indicated by lfnstTrSetIdx). The conversion unit 108 sets the value of lfnst_index to the index of the conversion kernel in the LFNST conversion kernel candidate set. This gives the cost function a minimum value and also obtains a second cost result.

[0302] If the first cost result is greater than the second cost result, the conversion unit 108 decides to perform LFNST on the conversion block and sends lfnst_index, which indicates the index of the conversion kernel, to the entropy coding unit 115. Otherwise, if the conversion unit 108 does not perform LFNST on the conversion block, it sets lfnst_value to equal 0 and sends lfnst_index, which indicates that LFNST will not be performed, to the entropy coding unit 115.

[0303] If lfnst_index is not equal to 0, the conversion unit 108 performs LFNST on the conversion block using coefficients from subblocks 7001, 7002, and 7003 to obtain LFNST coefficients, and places the LFNST coefficients into subblocks 7011, 7012, and 7013 of block 703. Assume that d[x][y] is the block of the first conversion coefficient, where x=0, ..., nTbW-1, y=0, ..., nTbH-1, where nTbW and nTbH are the width and height of block 703, respectively. Assume that v[x] is the block of the LFNST coefficient, where x=0, ..., nLfnstOutSize-1, nLfnstOutSize = (nTbW>=8 && nTbH>=8)? 48:16.

[0304] Selectively, the conversion unit 108 obtains the conversion coefficient block 703 by adopting the following instruction and sends the LFNST coefficient block 703 in d[x][y] to the quantization unit 109. Specifically, this is as shown in equation (8) above. log2LfnstSize is determined as shown in equation (9) above.

[0305] Selectively, if isTransposed is equal to 0, the conversion unit 108 adopts the following instruction to obtain block 703 of the LFNST coefficients and sends block 703 of the LFNST coefficients in d[x][y] to the quantization unit 109. Specifically, this is as shown in equation (10) above. If isTransposed is equal to 1, the conversion unit 108 obtains block 703 of the LFNST coefficients by adopting the following instruction and sends block 703 of the LFNST coefficients in d[x][y] to the quantization unit 109. Specifically, this is as shown in equation (11) above.

[0306] Selectively, if isTransposed is equal to 1, the conversion unit 108 adopts the following instruction to obtain block 703 of the LFNST coefficients and sends block 703 of the LFNST coefficients in d[x][y] to the quantization unit 109. Specifically, this is as shown in equation (10) above. If isTransposed is equal to 0, the conversion unit 108 obtains the LFNST coefficient block 703 by adopting the following instruction and sends the LFNST coefficient block 703 in d[x][y] to the quantization unit 109. Specifically, this is as shown in equation (11) above.

[0307] log2LfnstSize is determined as shown in equation (9) above.

[0308] Furthermore, the conversion unit 108 can determine the value of lfnst_index for a conversion block encoded under conventional intra-prediction modes (i.e., angle intra-prediction mode, DC mode, and PLANAR mode, etc.). Similar to the rate distortion optimization method used to determine the value of lfnst_index for a conversion block under MIP mode, it determines the first cost result without performing LFNST on the conversion block.

[0309] The conversion unit 108 uses a lookup table to determine the value of lfnstTrSetIdx based on the mode index value of the conventional intra prediction mode (assigned to predModeIntra), thereby determining the LFNST conversion kernel candidate set selected for the current block. Specifically, this is as shown in Table 11 above.

[0310] Next, the conversion unit 108 can determine the value of lfnst_index using a conventional rate-distortion optimization method (i.e., a value greater than 0 indicates that the conversion kernel used in the LFNST conversion kernel candidate set is indicated by lfnstTrSetIdx). The conversion unit 108 sets the value of lfnst_index to the index of the conversion kernel in the LFNST conversion kernel candidate set. This gives the cost function a minimum value, and a second cost result is obtained.

[0311] If the first cost result is greater than the second cost result, the conversion unit 108 decides to perform LFNST on the conversion block and sends lfnst_index, which indicates the index of the conversion kernel, to the entropy coding unit 115. Otherwise, if the conversion unit 108 does not perform LFNST on the conversion block, it sets lfnst_value to equal 0 and sends lfnst_index, which indicates that LFNST will not be performed, to the entropy coding unit 115.

[0312] If lfnst_index is not equal to 0, the conversion unit 108 performs LFNST on the conversion block using coefficients from subblocks 7001, 7002, and 7003 to obtain LFNST coefficients, and places the LFNST coefficients into subblocks 7011, 7012, and 7013 of block 703. Assume that d[x][y] is the block of the first conversion coefficient, where x=0, ..., nTbW-1, y=0, ..., nTbH-1, where nTbW and nTbH are the width and height of block 703, respectively. Assume that v[x] is the block of the LFNST coefficient, where x=0, ..., nLfnstOutSize-1, nLfnstOutSize = (nTbW>=8 && nTbH>=8)? 48:16.

[0313] Selectively, if predModeIntra is 34 or less, the conversion unit 108 adopts the following instruction to obtain block 703 of the LFNST coefficients and sends block 703 of the LFNST coefficients in d[x][y] to the quantization unit 109. Specifically, this is as shown in equation (10) above. Selectively, if predModeIntra is greater than 34, the conversion unit 108 adopts the following instruction to obtain block 703 of the LFNST coefficients and sends block 703 of the LFNST coefficients in d[x][y] to the quantization unit 109. Specifically, this is as shown in equation (11) above.

[0314] Selectively, if predModeIntra is greater than 34, the conversion unit 108 adopts the following instruction to obtain block 703 of the LFNST coefficients and sends block 703 of the LFNST coefficients in d[x][y] to the quantization unit 109. Specifically, this is as shown in equation (10) above. Selectively, if predModeIntra is 34 or less, the conversion unit 108 adopts the following instruction to obtain block 703 of the LFNST coefficients and sends block 703 of the LFNST coefficients in d[x][y] to the quantization unit 109. Specifically, this is as shown in equation (11) above.

[0315] log2LfnstSize is determined as shown in equation (9) above.

[0316] Furthermore, the inverse quantization unit 110 performs a scaling operation on the quantization coefficients to output reconstruction coefficients. The inverse transform unit 111 performs one or more inverse transforms corresponding to the transform in the transform unit 108 and outputs the reconstruction residual. The second adder 112 calculates the reconstruction CU by adding the reconstruction residual to the intra-prediction block of the current block from the prediction unit 102. The second adder 112 further sends its output to the prediction unit 102 to be used as an intra-prediction reference.

[0317] The output of the filtering unit 113 is a decoded image or sub-image, which is sent to the DPB unit 114. The DPB unit 114 outputs the decoded image based on timing and control information. The images stored in the DPB unit 114 can be used as a reference for the prediction unit 102 to perform interpretation or intraprediction.

[0318] The entropy coding unit 115 converts the parameters (necessary for obtaining the decoded image), control parameters, and supplementary information from the units in the encoder 100 into a binary representation, and writes the above binary representation to the generated bitstream (or "video bitstream") based on the syntax structure of each data unit. The specifics are as shown in Tables 9A and 9B.

[0319] Using the syntax structure in Table 9A, if the minimum width and height of the transformation block in the coding unit is 8 or greater (i.e., both width and height are 8 or greater), the entropy coding unit 115 encodes the lfnst_index of the coding unit using MIP mode. Another selectable condition may be set such that both width and height are greater than or equal to a first value (e.g., 8), and the sum of width and height is greater than or equal to a second value (e.g., 16).

[0320] Using the syntax structure in Table 9B, if the mipSizeId of the coding unit is equal to 2, the entropy coding unit 115 encodes the lfnst_index of the coding unit using MIP mode. Another selectable condition is that the mipSizeId of the coding unit is set to be equal to one of several values ​​(e.g., 1, 2).

[0321] Referring to Figure 8, Figure 8 shows a flowchart of the encoding of LFNST parameters according to an embodiment of the present invention. As shown in Figure 8, this flow may include the following:

[0322] S801: The entropy coding unit 115 checks the above conditions to determine whether or not to encode the LFNST parameters.

[0323] S802: The entropy coding unit 115 encodes the LFNST parameters using context-adaptive binary arithmetic coding (CABAC).

[0324] In step S801, the entropy coding unit 115 performs a check to determine the value of the LFNST parameter, for example, lfnst_index. Next, in step S802, the value of lfnst_index is coded using a CABAC whose descriptor shown in Tables 9A and 9B is "ae(v)".

[0325] In embodiments of the present invention, the encoder 100 may be a computing device comprising a processor and a storage medium for storing an encoding program. When the processor reads and executes the encoding program, the encoder 100 reads the input video and generates a corresponding bitstream. Alternatively, the encoder 100 may be a computing device comprising one or more chips. A unit implemented as an integrated circuit on a chip has connectivity and data exchange functions similar to the corresponding unit in Figure 1.

[0326] In this embodiment, an encoding method is provided. The prediction parameters for the current block are determined. The prediction parameters include prediction mode parameters. If the prediction mode parameters indicate that the intra-predicted value of the saturation component of the current block is determined using MIP mode, the adjacent sample values ​​of the current block are obtained, and the MIP input sample values ​​of the current block are determined based on the adjacent sample values ​​of the current block. The MIP predicted value of the saturation component of the current block is determined based on the MIP input sample values, the MIP weighting matrix, and the shift parameters. The intra-predicted value of the saturation component of the current block is determined by filtering the MIP predicted value. The predicted residual value of the saturation component of the current block is determined based on the intra-predicted value of the saturation component of the current block. The LFNST parameters are determined by performing LFNST on the predicted residual value. The LFNST parameters are encoded and written to the bitstream. In this way, MIP mode can reduce complexity while ensuring encoding performance, and at the same time reduce the memory space required for the encoding process, thereby effectively increasing encoding efficiency. Furthermore, when LFNST technology is applied to MIP mode prediction, the introduction of MIP parameters makes LFNST more flexible and can further improve coding efficiency.

[0327] In another embodiment of the present invention, referring to Figure 9, Figure 9 shows a flowchart of a decoding method according to an embodiment of the present invention. As shown in Figure 9, the method may include the following:

[0328] S901: Analyze the bitstream to obtain the prediction parameters and LFNST parameters for the current block. The prediction parameters include the prediction mode parameters.

[0329] The image to be decoded can be divided into multiple image blocks, and each image block to be decoded can be called a decoding block. Here, each decoding block may contain a first image component, a second image component, and a third image component. The current block is the decoding block in which the prediction of the first, second, or third image component in the video image is currently being performed.

[0330] Furthermore, prediction parameters may include prediction mode parameters, which are used to indicate the prediction mode adopted by the current block, and it should be noted that different prediction modes correspond to different prediction mode parameters. Prediction modes generally include inter-prediction modes, conventional intra-prediction modes, and non-conventional intra-prediction modes. Conventional intra-prediction modes may include DC mode, PLANAR mode, angular intra-prediction mode, etc. Non-conventional intra-prediction modes may include MIP mode, CCLM mode, IBC mode, PLT mode, etc. In other words, the encoder selects the optimal prediction mode to encode the current block. In this process, the prediction mode of the current block can be determined, and the prediction mode parameters can be obtained accordingly. Subsequently, the prediction parameters, including the prediction mode parameters, are written to the bitstream and sent from the encoder to the decoder.

[0331] Thus, in the decoder, the prediction parameters of the current block can be obtained by analyzing the bitstream, and the prediction mode parameters included in the prediction parameters obtained by the analysis can be used to determine whether or not the MIP mode is used for the current block.

[0332] S902: When the prediction mode parameter indicates that the intra-predicted value of the saturation component of the current block is determined using the MIP mode, the adjacent sample values ​​of the current block are obtained, and the MIP input sample values ​​of the current block are determined based on the adjacent sample values ​​of the current block.

[0333] Furthermore, in this embodiment of the present invention, intra-prediction is performed on the saturation component of the current block using MIP mode. In this process, first, the adjacent sample values ​​of the current block are obtained, and then, based on the adjacent sample values ​​of the current block, the MIP input sample values ​​of the current block are determined.

[0334] The adjacent sample values ​​of the current block may include the left adjacent sample values ​​and the upper adjacent sample values ​​of the current block. That is, the MIP input sample values ​​of the current block can be determined based on the left adjacent sample values ​​and the upper adjacent sample values ​​of the current block.

[0335] In some embodiments, determining the MIP input sample value of the current block based on the adjacent sample values ​​of the current block may include the following:

[0336] The block size index value of the current block is determined based on the current block's size parameters.

[0337] A first temporary reference value is obtained by downsampling the adjacent sample values ​​of the current block.

[0338] If the block size index value of the current block falls within a predetermined range, a second constant value is determined based on the bit depth of the adjacent sample values ​​of the current block. The value corresponding to index 0 in the MIP input sample value is set to the difference between the second constant value and the value corresponding to index 0 in the first temporary reference value. The value corresponding to index i in the MIP input sample value is set to the difference between the value corresponding to index i in the first temporary reference value and the value corresponding to index 0 in the first temporary reference value. i is an integer greater than 0.

[0339] If the block size index value of the current block is outside the pre-set range, the value corresponding to index j in the MIP input sample value is set to the difference between the value corresponding to index (j+1) in the first temporary reference value and the value corresponding to index 0 in the first temporary reference value. j is a non-negative integer.

[0340] Furthermore, if the block size index value of the current block is within a predetermined range, the above method may further include the following: A second constant value is determined based on the bit depth of the adjacent sample values ​​in the current block. The value corresponding to index 0 in the MIP input sample value is set to the difference between the value corresponding to index 0 in the first temporary reference value and the second constant value. The value corresponding to index i in the MIP input sample value is set to the difference between the value corresponding to index i in the first temporary reference value and the value corresponding to index 0 in the first temporary reference value. i is an integer greater than 0.

[0341] Note that a first temporary reference value can be obtained after downsampling the adjacent sample values ​​of the current block. Specifically, the filtered adjacent sample values ​​can be buffered into a buffer (represented by pTemp) for the first temporary reference value. The value corresponding to index 0 in the first temporary reference value points to pTemp[0], and the value corresponding to index i in the first temporary reference value points to pTemp[i].

[0342] In embodiments of the present invention, the prediction parameters may further include, in addition to the prediction mode parameters, a current block size parameter. The current block size parameter may include the width (represented by nTbW) and height (represented by nTbH) of the current block. Furthermore, the block size index value (i.e., mipSizeId) of the current block can be determined based on the current block size parameter.

[0343] Furthermore, it should be noted that the current block's size parameter value can be determined based on whether the current block's block size index value (represented by mipSizeId) is within a predetermined range. Specifically, if mipSizeId=0 or 1, it indicates that the current block's block size index value is within the predetermined range, i.e., the current block's size parameter value is within the predetermined range. If mipSizeId=2, it indicates that the current block's block size index value is outside the predetermined range, i.e., the current block's size parameter value is not within the predetermined range.

[0344] In other words, the MIP input sample value is determined by the buffer (represented by pTemp), the block size index value of the current block (represented by MipSizeId), and the bit depth of the adjacent sample values ​​of the current block (represented by BitDepth). The number of input samples corresponding to the MIP input sample value is related to the block size index value of the current block. Finally, the value corresponding to index x in the MIP input sample value (represented by p[x]) can be obtained.

[0345] Furthermore, in some embodiments, determining a second constant value based on the bit depth of the adjacent sample values ​​of the current block may include the following: The second constant value is set to a power of 2. The exponent of the exponent is an integer, equal to the bit depth of the adjacent sample values ​​of the current block minus 1.

[0346] Alternatively, in some embodiments, determining a second constant value based on the bit depth of the adjacent sample values ​​of the current block may include the following: A second constant value is obtained by left-shifting "1" in binary. The number of bits left-shifted is equal to the bit depth of the adjacent sample value in the current block minus 1.

[0347] In other words, after obtaining the bit depth (represented by BitDepth) of the adjacent sample values ​​of the current block, the second constant value can be expressed as 1 << (BitDepth-1) or 2^(BitDepth-1). In this way, if the value of the current block's size parameter is within a predetermined range, the MIP input sample value of the current block can be determined by combining it with the second constant value.

[0348] Furthermore, it should be noted that the MIP input samples are matrix vectors used in matrix multiplication operations. The current related technical solution is as follows: Based on the buffer (represented by pTemp), the type of the current block (i.e., the block size index value of the current block, represented by mipSizeId), the bit depth of the adjacent sample values ​​of the current block (represented by BitDepth), and the number of MIP input samples, the value corresponding to index x in the MIP input sample value (represented by p[x]) is finally obtained.

[0349] In one possible embodiment, when mipSizeId=0 or 1, p[0] is obtained by subtracting pTemp[0] from 1<<(BitDepth-1), and when x is not 0, p[x] is obtained by subtracting pTemp[0] from pTemp[x]. Specifically, this is as shown in equation (2) above.

[0350] In another possible embodiment, when mipSizeId=0 or 1, p[0] is obtained by subtracting 1<<(BitDepth-1) from pTemp[0], and when x is not 0, p[x] is obtained by subtracting pTemp[0] from pTemp[x]. Specifically, this is as shown in equation (3) above.

[0351] In another possible embodiment, when mipSizeId=2, p[x] is obtained by subtracting pTemp[0] from pTemp[x+1]. Specifically, this is as shown in equation (4) above.

[0352] In addition to the above, in embodiments of the present invention, it is also possible to obtain the value of p[x] using a unified calculation method without determining the value of mipSizeId. In this case, in embodiments of the present invention, (1<<(BitDepth-1)) is added to the end of the buffer pTemp as an additional element. p[x] can be directly set to p[x]=pTemp[x+1]-pTemp[0], where x=0, ..., inSize-1.

[0353] In this way, after determining the MIP input sample value of the current block based on the adjacent sample values ​​of the current block, the MIP prediction value of the current block can be further determined.

[0354] S903: Determine the MIP predicted value for the saturation component of the current block based on the MIP input sample values, MIP weighting matrix, and shift parameters.

[0355] The MIP prediction is a prediction of a subset of samples in the saturation component of the current block.

[0356] The shift parameters may include shift offset parameters and shift number parameters. For S903, determining the MIP predicted value of the saturation component of the current block based on the above MIP input sample values, MIP weighting matrix, and shift parameters may include the following:

[0357] Based on the value of the shift offset parameter, the product of the sum of the MIP input sample values ​​and the shift offset parameter is determined. The value of the shift offset parameter is a fixed constant. The first constant value is determined based on the value of the shift number parameter. The value of the shift number parameter is a fixed constant. The value of the first offset is set to the difference between the first constant value and the above product. Based on the prediction parameters, the MIP weighting matrix for the current block is determined. Based on the MIP weighting matrix, MIP input sample values, shift number parameter, and first offset, the MIP predicted value for the saturation component of the current block is determined.

[0358] The shift offset parameter may also be called the shift compensation parameter or offset factor, and can be represented by fO. In embodiments of the present invention, the value of the shift offset parameter can be set to a fixed constant, for example, 32, 46, 56, 66, or 32. Furthermore, the value of the shift offset parameter can be set in relation to a shift offset parameter table, and the value of the shift offset parameter can be determined by a lookup table. This specification is not limited thereto.

[0359] The shift number parameter may also be called the shift factor, shift bit number, or weight shift value, and can be represented as sW, shift, or weight shift. In embodiments of the present invention, the shift number parameter is generally represented as sW. In embodiments of the present invention, the value of the shift number parameter can be set to a fixed constant, for example, 5, 6, or 7. Furthermore, the value of the shift number parameter can be set in relation to a shift number parameter table, and the value of the shift number parameter can be determined by a lookup table. This specification is not limited thereto.

[0360] Furthermore, the first offset can be represented by oW, and the first offset is related to both the shift number parameter and the shift offset parameter.

[0361] The prediction parameters may include the prediction mode parameter and the current block size parameter. If the prediction mode parameter indicates that the intra-predicted value of the saturation component of the current block is determined using the MIP mode, a weighting matrix table may be established in advance and stored in memory or a storage unit. This memory or storage unit may be integrated into the decoder or provided independently. Thus, based on the block size index value (mipSizeId) and the MIP mode index value (modelId) of the current block, the MIP weighting matrix (also called the MIP weight matrix or MIP matrix) that needs to be used for the current block can be determined by a lookup table and is represented by mWeight[x][y].

[0362] Furthermore, in the decoder, a shift number parameter table can be established in advance, and this shift number parameter table is also stored in memory or a storage unit. This memory or storage unit may be integrated into the decoder or may be provided independently. In embodiments of the present invention, the shift number parameter (sW) can be determined in several ways.

[0363] In one possible embodiment, the shift number parameter may differ for different block sizes and different MIP mode index values. In some embodiments, the methods described above may be further included. When determining the intra-predicted value of the saturation component of the current block using MIP mode, the bitstream is parsed to obtain the MIP mode index value of the current block. Based on the MIP mode index value, a value corresponding to the MIP mode index value is queried from a first pre-configured lookup table. The first pre-configured lookup table is used to record the correspondence between the MIP mode index value and the value of the shift number parameter. The queried value is determined as the value of the shift number parameter.

[0364] Furthermore, the value of the shift number parameter can be queried based on the current block size index value (represented by mipSizeId) and MIP mode index value (modelId). In the first pre-configured lookup table shown in Table 4 above, the shift number parameter required for matrix multiplication can be determined by the lookup table for different mipSizeId and modelId values.

[0365] However, the decoder needs to store Table 4 in memory or a storage unit as a lookup table. However, storage is costly, and the lookup process is also costly. Since the shift factor in Table 4 is related to both the block size and the MIP mode index value of the current block, the amount of memory occupied increases, and the computational complexity also increases.

[0366] In an embodiment of the present invention, the method for determining the shift factor can be simplified in order to reduce memory usage and lower computational complexity.

[0367] In another possible embodiment, the value of the shift number parameter can be set to a fixed constant independent of the block size index value and the MIP mode index value. For example, the value of the shift number parameter can be set to 5 for different block size index values ​​and different MIP mode index values. Alternatively, the value of the shift number parameter can be set to 6 for different block size index values ​​and different MIP mode index values. Alternatively, the value of the shift number parameter can be set to 7 for different block size index values ​​and different MIP mode index values. In embodiments of the present invention, it is preferable, but not limited thereto, to set the value of the shift number parameter to 6.

[0368] In another possible embodiment, the above method further includes the following regarding the value of the shift number parameter: The block size index value of the current block is determined based on the current block's size parameters. The value of the shift number parameter is determined based on the block size index value of the current block mentioned above.

[0369] It should be noted that the block size index value of the current block can be determined based on the current block size parameters. In some embodiments, determining the block size index value of the current block based on the current block size parameters may include the following: If the current block's width and height are both equal to 4, set the current block's block size index value to 0. The block size index value of the current block is set to 1 if both the width and height of the current block are equal to 8, or if either the width or height of the current block is equal to 4. If the current block's width and height do not meet the above conditions, the block size index value of the current block is set to 2.

[0370] In this way, after determining the block size index value of the current block, the value of the shift number parameter can be further determined based on the block size index value of the current block.

[0371] Selectively, in some embodiments, determining the value of the shift number parameter based on the block size index value of the current block may include the following: If the block size index value is equal to 0, 1, or 2, it is determined that the value of the shift number parameter corresponding to the block size index value of the current block is equal to 5, 6, or 5, respectively.

[0372] Selectively, in some embodiments, determining the value of the shift number parameter based on the block size index value of the current block may include the following: Set the value of the shift number parameter to equal to the ratio of the current block's width or height to a first preset value corresponding to the current block's block size index value.

[0373] Here, the first preset value represents the number of MIP input sample values ​​obtained from the current block boundary. In this case, the above method may also include the following: If the current block's block size index value is equal to 0, 1, or 2, then it is determined that the first preset value corresponding to the current block's block size index value is equal to 2, 4, or 4.

[0374] In other words, when the first preset value represents the number of MIP input sample values ​​obtained from the boundary of the current block, the corresponding first preset value is equal to 2 when the block size index value of the current block is equal to 0, the corresponding first preset value is equal to 4 when the block size index value of the current block is equal to 1, and the corresponding first preset value is equal to 4 when the block size index value of the current block is equal to 2. In this way, the value of the shift number parameter can be determined based on the ratio of the width or height of the current block to the corresponding first preset value.

[0375] Selectively, in some embodiments, determining the value of the shift number parameter based on the block size index value of the current block may include the following: Set the value of the shift number parameter to the ratio of the current block's width or height to a second preset value corresponding to the current block's block size index value.

[0376] Here, the second pre-set value represents the MIP predicted block size of the current block, obtained by directly using the MIP weighting matrix. In this case, the above method may also include the following: If the current block's block size index value is equal to 0, 1, or 2, then it is determined that the second preset value corresponding to the current block's block size index value is equal to 4, 4, or 8.

[0377] In other words, when the second preset value represents the MIP predicted block size of the current block obtained by directly using the MIP weighting matrix, the corresponding second preset value is equal to 4 when the block size index value of the current block is equal to 0, the corresponding second preset value is equal to 4 when the block size index value of the current block is equal to 1, and the corresponding second preset value is equal to 8 when the block size index value of the current block is equal to 2. In this way, the value of the shift number parameter can be determined based on the ratio of the width or height of the current block to the corresponding second preset value.

[0378] In another possible embodiment, the shift number parameter table can be minimized, and the value of the shift number parameter is still determined by a lookup table method. Selectively, in some embodiments, determining the value of the shift number parameter based on the block size index value of the current block may include the following: Based on the block size index value, a second pre-configured lookup table is queried for the value corresponding to the block size index value. The second pre-configured lookup table is used to record the correspondence between the block size index value and the value of the shift number parameter. The queried value is determined as the value of the shift number parameter.

[0379] Furthermore, the value of the shift number parameter can be queried based solely on the block size index value (represented by mipSizeId) of the current block. In the second pre-configured lookup table shown in Table 5 above, each block size index value can correspond to a fixed value. That is, the size of each block or each block size set can have a fixed shift number parameter value as shown in Table 5.

[0380] According to Table 5 above, if the current block size index value is equal to 0, 1, or 2, then it can be determined that the value of the shift number parameter corresponding to the block size index value is equal to 5, 6, or 5, respectively.

[0381] In this way, by simplifying the method for determining the shift number parameter, and in particular by minimizing the shift number parameter table or fixing the values ​​of the shift number parameters, the amount of memory required for the lookup table can be minimized, and the memory occupied by the original shift factor table in MIP mode can be reduced without increasing computational complexity.

[0382] Furthermore, in the decoder, a shift-offset parameter table can be established in advance, and this shift-offset parameter table is also stored in memory or a storage unit. This memory or storage unit may be integrated into the decoder or may be provided independently. In embodiments of the present invention, the shift-offset parameter (fO) can be determined in several ways.

[0383] In one possible embodiment, the shift offset parameter may differ for different block sizes and different MIP mode index values. In some embodiments, the methods described above may be further included. When determining the intra-predicted value of the saturation component of the current block using MIP mode, the bitstream is parsed to obtain the MIP mode index value of the current block. Based on the MIP mode index value, a value corresponding to the MIP mode index value is queried from a third pre-configured lookup table. The third pre-configured lookup table is used to record the correspondence between the MIP mode index value and the value of the shift offset parameter. The queried value is determined as the value of the shift offset parameter.

[0384] Furthermore, the value of the shift offset parameter can be queried based on the current block's block size index value (represented by mipSizeId) and MIP mode index value (modelId). In the third pre-configured lookup table shown in Table 6 above, the shift offset parameter required for matrix multiplication can be determined by the lookup table for different mipSizeId and modelId values.

[0385] However, the decoder needs to store Table 6 in memory or a storage unit as a lookup table. However, storage is costly, and the lookup process is also costly. Since the shift offset parameter values ​​in Table 6 are related to both the block size and the MIP mode index value of the current block, the amount of memory occupied increases, and the computational complexity also increases.

[0386] In an embodiment of the present invention, the method for determining the shift offset parameter can be simplified in order to reduce memory usage and computational complexity.

[0387] In another possible embodiment, the value of the shift offset parameter can be set to a fixed constant independent of the block size index value and the MIP mode index value. Generally, this fixed constant is in the range of 0 to 100. For example, the value of the shift offset parameter can be set to 32 for different block size index values ​​and different MIP mode index values. Alternatively, the value of the shift offset parameter can be set to 46 for different block size index values ​​and different MIP mode index values. Alternatively, the value of the shift offset parameter can be set to 56 for different block size index values ​​and different MIP mode index values. Alternatively, the value of the shift offset parameter can be set to 66 for different block size index values ​​and different MIP mode index values. In embodiments of the present invention, it is preferable, but not limited to, setting the value of the shift offset parameter to 32.

[0388] In another possible embodiment, the above method may further include the following regarding the value of the shift offset parameter: The block size index value of the current block is determined based on the current block's size parameters. The value of the shift offset parameter is determined based on the block size index value of the current block mentioned above.

[0389] Note that the current block's block size index value can be determined based on the current block's size parameter. Then, based on the current block's block size index value, the value of the shift offset parameter can be further determined.

[0390] Selectively, in some embodiments, determining the value of the shift offset parameter based on the block size index value of the current block may include the following: If the block size index value is equal to 0, 1, or 2, respectively, it is determined that the value of the shift offset parameter corresponding to the block size index value of the current block is equal to 34, 23, or 46, respectively.

[0391] In some embodiments, selectively, the shift offset parameter table can be minimized, and the shift factor can still be determined by a lookup table method. In some embodiments, selectively, determining the value of the shift offset parameter based on the block size index value of the current block may include the following: Based on the block size index value, a value corresponding to the block size index value is queried from a fourth pre-configured lookup table. The fourth pre-configured lookup table is used to record the correspondence between the block size index value and the value of the shift offset parameter. The queried value is determined as the value of the shift offset parameter.

[0392] Furthermore, the value of the shift offset parameter can be queried based solely on the block size index value (represented by mipSizeId) of the current block. In the fourth pre-configured lookup table shown in Table 7 above, each block size index value can correspond to a fixed value. That is, the size of each block or each block size set can have a fixed shift offset parameter value as shown in Table 7.

[0393] According to Table 7 above, when the current block size index value is equal to 0, 1, or 2, it can be determined that the value of the shift offset parameter corresponding to the block size index value is equal to 34, 23, or 46, respectively.

[0394] In this way, by simplifying the method for determining the shift offset parameters, particularly by minimizing the shift offset parameter table or fixing the values ​​of the shift offset parameters, the amount of memory required for the lookup table can be minimized. Thus, the memory occupied by the original shift offset parameter table in MIP mode can be reduced without increasing computational complexity.

[0395] Thus, after determining the shift offset parameter (fO) and the shift number parameter (sW), selectively determining the first constant value based on the value of the shift number parameter in some embodiments may include the following: The first constant value is set to a power of 2. The exponent of the power is an integer, equal to the value of the shift number parameter minus 1.

[0396] Selectively, in some embodiments, determining a first constant value based on the value of the shift number parameter may include the following: The first constant value is obtained by left-shifting "1" in binary. The number of bits left-shifted is equal to the value of the shift number parameter minus 1.

[0397] In other words, after the shift number parameter (sW) is obtained, the first constant value may be written as 1<<(sW-1) or as 2^(sW-1). In this case, when the value of the shift number parameter is set to 6, a first constant value equal to 32 can be obtained.

[0398] JPEG0007860206000025.jpg36170

[0399] Thus, under MIP mode, the MIP weighting matrix, MIP input sample values, shift number parameters, and a first offset can be obtained, thereby facilitating the subsequent determination of the MIP prediction value for the current block. Specifically, in some embodiments, determining the MIP prediction value for the current block based on the above MIP weighting matrix, MIP input sample values, shift number parameters, and first offset may include the following:

[0400] The first weighted sum of the MIP weighting matrix and the MIP input sample values ​​is calculated. Calculate the first sum of the first weighted sum and the first offset. The first right-shifted value is obtained by right-shifting the first sum into binary. The number of bits right-shifted is equal to the value of the shift number parameter. The MIP prediction value for the current block is set to the sum of the first right-shifted value and the value corresponding to index 0 in the first temporary reference value. The first temporary reference value is obtained by downsampling the adjacent sample values ​​of the current block.

[0401] In other words, under MIP mode, the MIP weighting matrix (represented by mWeight), shift number parameter (represented by sW), and shift offset parameter (represented by fO) can be determined based on the current block size index value (represented by mipSizeId) and MIP mode index value (represented by modelId) of the block. Next, the MIP input sample value (represented by p[x]), mWeight, sW, and fO are input to a matrix multiplication to obtain the MIP predicted value (which can be represented by predMip[x][y]) output from the matrix multiplication. The samples in predMip[x][y] are arranged in matrix / array form based on predSize × predSize. The calculation formula is as shown in equation (5) or equation (6) above.

[0402] S904: Determine the intra-predicted value of the saturation component of the current block by filtering the MIP prediction values.

[0403] The MIP prediction block consists of MIP prediction values. After obtaining the MIP prediction block, the intra-prediction values ​​of the current block can be further determined by determining whether the size parameter of the MIP prediction block is the same as the size parameter of the current block. Specifically, based on the determination result, if the size parameter of the MIP prediction block is the same as the size parameter of the current block, the intra-prediction block of the current block is set to be equal to the MIP prediction block. In this case, the MIP prediction block contains the prediction values ​​for all sample positions in the current block. If the size parameter of the MIP prediction block is different from the size parameter of the current block, the filtered prediction block is obtained by filtering the MIP prediction block, and the filtered prediction block is set as the intra-prediction block of the current block. Here, filtering can include upsampling or low-pass filtering.

[0404] S905: When the LFNST parameter indicates that LFNST should be performed on the current block, the reconstruction transformation coefficient block of the current block is determined, and a second transformation coefficient block is obtained by performing LFNST on at least some of the reconstruction transformation coefficients in the reconstruction transformation coefficient block.

[0405] Furthermore, it should be noted that LFNST is not performed on any given current block. This means that the bitstream does not necessarily contain LFNST parameters. In this case, several conditions (e.g., the minimum value of the current block's size parameters, the block size index value of the current block, etc.) need to be checked to determine whether the bitstream contains LFNST parameters. Therefore, in some embodiments, parsing the bitstream to obtain LFNST parameters may include the following: Determine whether the bitstream contains the LFNST parameter. If the bitstream contains an LFNST parameter, the bitstream is parsed to extract the LFNST parameter.

[0406] In other words, first, we need to determine whether or not the bitstream contains the LFNST parameter. Only if the result of this determination is Yes, that is, only if the bitstream contains the LFNST parameter, can we proceed with the step of parsing the bitstream to obtain the LFNST parameter.

[0407] Selectively, in some embodiments, determining whether or not the bitstream has an LFNST parameter may include the following: When determining the intra-predicted value of the saturation component of the current block using MIP mode, the minimum value among the size parameters of the current block is determined. Based on the minimum value, it is determined whether or not the bitstream contains the LFNST parameter.

[0408] Furthermore, determining whether or not the bitstream has an LFNST parameter based on the above minimum value may include determining that the bitstream has an LFNST parameter if the minimum value is greater than or equal to a first preset threshold.

[0409] Here, the first preset threshold can be used to represent a preset threshold for determining whether or not to perform LFNST. In embodiments of the present invention, the first preset threshold can be set to 8, but is not particularly limited.

[0410] In other words, when performing intraprediction on the saturation component of the current block using MIP mode, it is possible to determine the minimum value among the size parameters of the current block. For example, if the minimum width and height are 8 or greater, it is determined that the bitstream has an LFNST parameter. In this case, the LFNST parameter, for example the value of lfnst_index, can be obtained by analyzing the bitstream.

[0411] Selectively, in some embodiments, determining whether or not the bitstream has an LFNST parameter may include the following: When determining the intra-predicted value of the saturation component of the current block using MIP mode, the block size index value of the current block is determined based on the size parameter of the current block. Based on the block size index value, it is determined whether or not the bitstream contains the LFNST parameter.

[0412] Furthermore, determining whether a bitstream has an LFNST parameter based on the above block size index value may include determining that a bitstream has an LFNST parameter if the block size index value is equal to a second preset threshold.

[0413] Here, the second preset threshold can be used to represent a preset threshold for determining whether or not to perform LFNST. In embodiments of the present invention, the second preset threshold can be set to one set value (e.g., 2) or one of a plurality of set values ​​(e.g., 1, 2), but is not particularly limited.

[0414] In other words, when performing intraprediction on the saturation component of the current block using MIP mode, the block size index value (mipSizeId) of the current block can be determined based on the size parameter of the current block. For example, if the value of mipSizeId is equal to 2, it is determined that the bitstream has an LFNST parameter. In this case, the value of the LFNST parameter, for example, lfnst_index, can be obtained by analyzing the bitstream.

[0415] In some embodiments, parsing the bitstream to obtain the LFNST parameters may include parsing the bitstream to obtain the value of the LFNST index.

[0416] The LFNST index can be represented as lfnst_index, and the value of the LFNST index is used to determine whether or not to perform LFNST on the current block. Specifically, in some embodiments, the method may further include the following: If the LFNST index value is greater than 0, it is decided to perform LFNST on the current block. If the value of the LFNST index is equal to 0, it is decided not to perform LFNST on the current block.

[0417] In other words, after obtaining the value of lfnst_index, if the value of lfnst_index is greater than 0, it is decided to perform LFNST on the current block. If the value of lfnst_index is equal to 0, it is decided not to perform LFNST on the current block.

[0418] Furthermore, after obtaining the value of an LFNST parameter, for example, lfnst_index, it is shown that if the value of lfnst_index is greater than 0, the LFNST parameter indicates that LFNST should be performed on the current block. In this case, in some embodiments, determining the reconstruction transformation coefficient block of the current block when the LFNST parameter indicates that LFNST should be performed on the current block may include the following: The bitstream is parsed to obtain the quantization coefficients of the current block. By inverse quantization of the quantization coefficients, we obtain the reconstruction transformation coefficient block of the current block.

[0419] Specifically, inverse quantization is scaling. In one specific example, obtaining the reconstruction transformation coefficient block of the current block by inverse quantization of the quantization coefficients can include obtaining the reconstruction transformation coefficient block of the current block by scaling the quantization coefficients.

[0420] Thus, when the LFNST parameter indicates that LFNST should be performed on the current block, a second transformation coefficient block can be obtained by performing LFNST on all or some of the reconstruction transformation coefficients in the reconstruction transformation coefficient block.

[0421] Furthermore, if it is determined that LFNST can be performed on the current block, it is necessary to determine the LFNST transformation kernel (represented by kernel) to be used for the current block. In some embodiments, obtaining a second transformation coefficient block by performing LFNST on at least some of the reconstructed transformation coefficients in the above reconstructed transformation coefficient block may include obtaining a second transformation coefficient block by performing LFNST on at least some of the reconstructed transformation coefficients in the reconstructed transformation coefficient block using an LFNST transformation kernel.

[0422] In embodiments of the present invention, the LFNST has four candidate sets of translation kernels. These four candidate sets of translation kernels may include set 0, set 1, set 2, and set 3. Here, in embodiments of the present invention, the LFNST translation kernel candidate set can be determined using MIP parameters, and then an LFNST translation kernel to be used for the current block is selected from the LFNST translation kernel candidate set. Thus, in some embodiments, the above method may further include the following.

[0423] When determining the intra-predicted value of the saturation component of the current block using MIP mode, the MIP parameters are determined from the prediction mode parameters of the current block. Based on the MIP parameters, a candidate set of LFNST conversion kernels is determined. In the LFNST conversion kernel candidate set, the conversion kernel indicated by the LFNST index value is determined as the LFNST conversion kernel to be used for the current block. The LFNST conversion kernel candidate set contains two or more pre-configured LFNST conversion kernels.

[0424] The MIP parameter may include at least one of the following: the MIP mode index value (represented by modelId) and the MIP transpose instruction parameter (represented by isTransposed). For the LFNST transformation kernel used for the current block, if the value of the MIP transpose instruction parameter indicates that a transpose should be performed on the sample input vector used for the MIP mode, then a matrix transpose must be performed on the selected transformation kernel to obtain the LFNST transformation kernel used for the current block.

[0425] Furthermore, in some embodiments, determining a candidate set of LFNST conversion kernels based on the above MIP parameters may include the following: The value of the LFNST intra-predictive mode index is determined based on the MIP mode index value. Based on the LFNST intra-prediction mode index value, a fifth pre-configured lookup table determines the index value of the LFNST conversion kernel candidate set. Based on the index value of the LFNST conversion kernel candidate set, the LFNST conversion kernel candidate set is selected from multiple candidate LFNST conversion kernel candidate sets.

[0426] Here, the LFNST intra prediction mode index can be represented by predModeIntra, and the index value of the LFNST conversion kernel candidate set can be represented by lfnstTrSetIdx.

[0427] The fifth pre-configured lookup table is as shown in Table 10 above. The value of predModeIntra can be determined based on the MIP mode index value (i.e., the value of modelId). Next, based on the value of predModeIntra, the value of lfnstTrSetIdx can be directly determined in conjunction with Table 10, that is, the LFNST conversion kernel candidate set selected for the current block can be determined. Here, the value of lfnstTrSetIdx indicates the conversion kernel candidate set used for LFNST. Since the value of modelId can include 0, 1, 2, 3, 4, and 5, the value of predModeIntra can also be 0, 1, 2, 3, 4, or 5. The correspondence between predModeIntra and lfnstTrSetIdx is as shown in Table 10.

[0428] In one specific embodiment, determining the value of the LFNST intra predictive mode index based on the above MIP mode index value may include setting the value of the LFNST intra predictive mode index equal to the MIP mode index value.

[0429] In other words, in an embodiment of the present invention, the value of predModeIntra can be set to be equal to the value of modelId, and then, based on the value of predModeIntra, the value of lfnstTrSetIdx is directly determined in conjunction with Table 10, i.e., the LFNST conversion kernel candidate set selected for the current block is determined. In another specific embodiment of the present invention, the MIP mode index value is directly mapped to the PLANAR mode value, and then, using the predModeIntra value corresponding to the PLANAR mode, the value of lfnstTrSetIdx is determined, i.e., the LFNST conversion kernel candidate set selected for the current block is determined.

[0430] Furthermore, in the LFNST process, the scan order can include both horizontal and vertical scan orders. Here, the determination of the scan order is related to the MIP transpose instruction parameter (represented by isTransposed).

[0431] Selectively, in some embodiments, obtaining a second transformation coefficient block by performing LFNST on at least some of the reconstruction transformation coefficients in the above reconstruction transformation coefficient block may include the following:

[0432] When the value of the MIP transpose instruction parameter indicates that a transpose should be performed on the sample input vector used in MIP mode, a second transformation coefficient block is obtained by performing LFNST on at least some of the reconstruction transformation coefficients in the above reconstruction transformation coefficient block in the vertical scan order. If the value of the MIP transpose instruction parameter indicates that no transposition should be performed on the sample input vector used in MIP mode, a second transformation coefficient block is obtained by performing LFNST on at least some of the reconstruction transformation coefficients in the above reconstruction transformation coefficient block in the horizontal scan order.

[0433] Specifically, if the value of isTransposed is equal to 0, no transposition is performed on the sample input vector used in MIP mode. Then, LFNST can be performed on at least some of the transformation coefficients in the first transformation coefficient block in the horizontal scan order. This is specifically as shown in equation (10) above.

[0434] If the value of isTransposed is equal to 1, the sample input vector used in MIP mode is transposed. Then, LFNST can be performed on at least some of the transformation coefficients in the first transformation coefficient block in the vertical scan order. Specifically, this is as shown in equation (11) above.

[0435] log2LfnstSize is determined as shown in equation (9) above.

[0436] Selectively, in some embodiments, obtaining a second transformation coefficient block by performing LFNST on at least some of the reconstruction transformation coefficients in the above reconstruction transformation coefficient block may include the following:

[0437] When the value of the MIP transpose instruction parameter indicates that a transpose should be performed on the sample input vector used in MIP mode, a second transformation coefficient block is obtained by performing LFNST on at least some of the reconstruction transformation coefficients in the above reconstruction transformation coefficient block in the horizontal scan sequence. If the value of the MIP transpose instruction parameter indicates that no transposition should be performed on the sample input vector used in MIP mode, a second transformation coefficient block is obtained by performing LFNST on at least some of the reconstruction transformation coefficients in the above reconstruction transformation coefficient block in the vertical scan order.

[0438] Specifically, when the value of isTransposed is equal to 1, the sample input vector used in MIP mode is transposed. Then, LFNST can be performed on at least some of the transformation coefficients in the first transformation coefficient block in the horizontal scan order. This is specifically as shown in equation (10) above.

[0439] If the value of isTransposed is equal to 0, no transposition is performed on the sample input vector used in MIP mode. Then, LFNST can be performed on at least some of the transformation coefficients in the first transformation coefficient block in the vertical scan order. Specifically, this is as shown in equation (11) above.

[0440] log2LfnstSize is determined as shown in equation (9) above.

[0441] S906: The reconstructed residual block of the saturation component of the current block is obtained by performing the first transformation on the second transformation coefficient block.

[0442] S907: Determine the reconstructed block of the saturation component of the current block based on the intra-predicted value of the saturation component of the current block and the reconstructed residual block.

[0443] In the decoder, the quantized values ​​of the LFNST coefficients can be obtained by analyzing the bitstream. The inverse quantization unit then inversely quantizes these quantized values ​​(which may be called "scaling") to obtain a reconstruction transformation coefficient block. A second transformation coefficient block can be obtained by performing an inverse LFNST on at least some of the reconstruction transformation coefficients in this reconstruction transformation coefficient block. Next, the reconstruction residual block of the saturation component of the current block can be recovered by the inverse first transformation. Here, the inverse first transformation is the inverse transformation corresponding to the "first transformation (core transformation)" in the encoder. Note that since the standard only defines the "inverse transformation" in the decoder, the "inverse LFNST" may be called "LFNST" in the standard.

[0444] Furthermore, after recovering the reconstruction residual block of the current block's saturation component, the reconstruction block of the current block's saturation component can be determined by adding the intra-predicted value of the current block's saturation component to the reconstruction residual block.

[0445] The decoding method of an embodiment of the present invention will be described in detail below with reference to the decoder 200 shown in Figure 2.

[0446] The input bitstream to the decoder 200 may be the bitstream generated by the encoder 100. The analysis unit 201 analyzes the input bitstream and obtains the values ​​of the syntax elements from the input bitstream. The analysis unit 201 converts the binary representation of the syntax elements into numerical values ​​and sends the numerical values ​​to the unit in the decoder 200 in order to obtain one or more decoded images. The analysis unit 201 can further analyze one or more syntax elements from the input bitstream in order to render the decoded images.

[0447] Referring to Figure 10, Figure 10 shows a flowchart of the analysis of LFNST parameters according to an embodiment of the present invention. As shown in Figure 10, the flowchart may include the following:

[0448] S1001: The analysis unit 201 performs a check to determine whether the input bitstream contains the LFNST parameter.

[0449] Note that the LFNST parameter can refer to the value of lfnst_index. Here, the example syntax structure for representing the LFNST parameter may be the same as the example syntax structure shown in Table 8 above, or it may be the same as the example syntax structure shown in Tables 9A and 9B above.

[0450] Specifically, when the intra-prediction mode of the current block is MIP mode, the analysis unit 201 obtains the encoded bits of lfnst_index from the input bitstream using the following method: Selectively, if the minimum width and height of the transformed block in the current block is 8 or greater (i.e., both width and height are 8 or greater), the analysis unit 201 reads the lfnst_index bits using MIP mode by utilizing the syntax structure in Table 9A. Another selectable condition may be set such that the width and height are equal to a first value (e.g., 8) and the sum of the width and height is greater than or equal to a second value (e.g., 16).

[0451] Selectively, if the mipSizeId of the current block is equal to 2, the analysis unit 201 reads the lfnst_index bit using the MIP mode by utilizing the syntax structure in Table 9B. Another selectable condition is that the mipSizeId of the current block is set to be equal to one of several values ​​(e.g., 1, 2).

[0452] S1002: The analysis unit 201 obtains the LFNST parameters by performing adaptive binary arithmetic decoding on the LFNST parameters.

[0453] Here, on the encoder 100 side, the entropy coding unit 115 encodes the LFNST parameters (for example, the value of lfnst_index) using CABAC whose descriptor shown in Tables 9A and 9B is "ae(v)". On the decoder 200 side, the analysis unit 201 obtains the value of lfnst_index from the lfnst_index bit by performing adaptive binary arithmetic decoding.

[0454] The analysis unit 201 sends the values ​​of the syntax elements and one or more variables set or determined based on the values ​​of the syntax elements to the unit in the decoder 200 for obtaining one or more decoded images. The prediction unit 202 determines the intra-prediction block for the current block (e.g., CU). If it is indicated that the inter-decoding mode will be used to decode the current decoded block, the prediction unit 202 sends the relevant parameters from the analysis unit 201 to the MC unit 203 to obtain the inter-prediction block. If it is indicated that the intra-prediction mode (including the MIP mode indicated based on the MIP mode index value) will be used to decode the current block, the prediction unit 202 sends the relevant parameters from the analysis unit 201 to the intra-prediction unit 204 to obtain the intra-prediction block.

[0455] It should be understood that the current block (or "decoded block") in embodiments of the present invention may be a CU or a partition of a CU (e.g., a transform block). Specifically, when it is shown that the current block is decoded using MIP mode, the intra-prediction unit 204 obtains an intra-prediction block of the current block. The intra-prediction block of the current block contains intra-prediction values ​​for at least one sample in the current block.

[0456] Specifically, the steps by which the intra-prediction unit 204 acquires intra-prediction blocks using MIP mode are as follows:

[0457] First, the intra-prediction unit 204 obtains one or more reference samples from the adjacent samples of the current block. This can be done, for example, by downsampling the adjacent samples or by directly extracting from the adjacent samples.

[0458] Next, the intra-prediction unit 204 uses the acquired reference sample, MIP matrix, and shift parameters to determine one or more partial prediction samples corresponding to the sample positions in the current block. Here, the sample positions can be preset sample positions in the current block, for example, having equal horizontal and vertical coordinate values. The shift parameters include a shift number parameter and a shift offset parameter, which can be used for offset operations in the process of acquiring intra-predicted values ​​for partial samples.

[0459] Finally, once a corresponding predicted sample has been obtained for a partial sample in the current block, the intra-prediction unit 204 needs to obtain further predicted samples for the remaining samples in the current block other than the partial sample. For example, the intra-prediction unit 204 can use an interpolation filter to obtain predicted samples for the remaining samples, and the input to the interpolation filter can be the partial sample and adjacent samples.

[0460] As shown in Figure 6, Figure 6 shows a flowchart for obtaining an intra-predicted block using MIP mode. This flow can be implemented in the decoder 200, where "current block" refers to the "decoded block".

[0461] In step 601, the intra-prediction unit 204 obtains adjacent samples of the current block, for example, marked as gray squares adjacent to the current block in step 601 as shown in Figure 6. The intra-prediction unit 204 obtains one or more reference samples from the adjacent samples. In the example of step 601 shown in Figure 6, selectively, the intra-prediction unit 204 may calculate the mean of two adjacent samples and use that mean as a reference sample. Selectively, the intra-prediction unit 204 selects one adjacent sample as a reference sample for every other adjacent sample. For example, in the example of step 601 shown in Figure 6, the intra-prediction unit 204 selects four reference samples from the eight upper adjacent samples of the current block and another four reference samples from the eight left adjacent samples of the current block.

[0462] The specific process for Step 601 is shown below.

[0463] The intra-prediction unit 204 obtains the width and height of the current block from the analysis unit 201, which are represented by the variables cbWidth and cbHeight, respectively. Based on the parameters from the analysis unit 201, the current block is divided into one or more transformed blocks. The variables nTbW and nTbH are the width and height of the transformed blocks, respectively. When obtaining a prediction of the current block using the MIP mode as the intra-prediction mode, the intra-prediction unit 204 determines the MIP block size index value, i.e., the variable represented by mipSizeId.

[0464] Selectively, the intra-prediction unit 204 determines the value of mipSizeId as follows: -If both nTbW and nTbH are equal to 4, set mipSizeId to equal to 0. - Otherwise, if cbWidth or cbHeight is equal to 4, set mipSizeId to equal to 1. - Otherwise, set mipSizeId to equal 2.

[0465] Specifically, if the current block size parameter is 8x8 (i.e., both cbWidth and cbHeight are equal to 8), then mipSizeId is set to equal to 2.

[0466] Selectively, the intra-prediction unit 204 determines the value of mipSizeId as follows: -If both nTbW and nTbH are equal to 4, set mipSizeId to equal to 0. - Otherwise, set mipSizeId to equal to 1 if cbWidth or cbHeight is equal to 4, or if both cbWidth and cbHeight are equal to 8. - Otherwise, set mipSizeId to equal 2.

[0467] Specifically, if the current block size parameter is 8x8, set mipSizeId to equal 1.

[0468] Furthermore, the intra-prediction unit 204 obtains the values ​​of boundarySize and predSize based on mipSizeId. Specifically, it does so as follows: -If mipSizeId is equal to 0, set boundarySize to equal to 2 and predSize to equal to 4. - Otherwise, if mipSizeId is equal to 1, set boundarySize to equal to 4 and predSize to equal to 4. - Otherwise (if mipSizeId is equal to 2), set boundarySize to equal to 4 and predSize to equal to 8. boundarySize represents the number of reference samples obtained from the top adjacent reference samples and left adjacent reference samples of the current block.

[0469] The intra-prediction unit 204 can acquire additional reference samples for calculating the MIP prediction value based on a parameter from the analysis unit 701 that indicates the order of reference samples stored in the buffer pTemp. The intra-prediction unit 204 sets the variable isTransposed equal to the parameter from the analysis unit 701 that indicates the order of reference samples stored in the buffer pTemp. For example, isTransposed being equal to 0 (or FALSE) indicates that the intra-prediction unit 204 first presents reference samples acquired from the upper neighbor reference samples of the current block, and then reference samples acquired from the left neighbor reference samples. Otherwise, isTransposed being equal to 1 (or TRUE) indicates that the intra-prediction unit 204 first presents reference samples acquired from the left neighbor reference samples of the current block, and then reference samples acquired from the upper neighbor reference samples.

[0470] The intra-prediction unit 204 obtains the value of inSize to indicate the number of reference samples used in MIP mode, as shown in equation (1) above.

[0471] The intra-prediction unit 204 invokes the following process to obtain a set of reference samples (stored as an array p[x], where x is 0, ..., inSize-1) using the adjacent reference samples of the current block.

[0472] The intra-prediction unit 204 obtains nTbW reference samples from the upper adjacent reference samples of the current block (for example, stored in array refT) and nTbH reference samples from the left adjacent reference samples of the current block (for example, stored in array refL). The intra-prediction unit 204 calls downsampling on refT to obtain boundarySize reference samples and stores the boundarySize reference samples in refT. The intra-prediction unit 204 calls downsampling on refL to obtain boundarySize reference samples and stores the boundarySize reference samples in refL.

[0473] The intra prediction unit 204 can further obtain the order indicated by isTransposed in order to place the elements in refT and refL into buffer pTemp.

[0474] Selectively, the intra-prediction unit 204 obtains p[x] as follows: x=0, ..., inSize-1. -When mipSizeId is equal to 2, p[x] = pTemp[x + 1] - pTemp[0]. - Otherwise (mipSizeId is less than 2), p[0] = pTemp[0] - (1 << (BitDepth-1)), p[x] = pTemp[x] - pTemp[0]. BitDepth is the bit depth of the color component of the sample in the current block. Here, the color component can be one of the RGB components, one of the YUV components, or one of the YCbCr components, for example, the Y component.

[0475] Selectively, the intra-prediction unit 204 can obtain p[x] as follows, where x=0, ..., inSize-1. -When mipSizeId is equal to 2, p[x] = pTemp[x + 1] - pTemp[0]. - Otherwise (mipSizeId is less than 2), p[0] = (1 << (BitDepth-1)) - pTemp[0], p[x] = pTemp[x] - pTemp[0].

[0476] Selectively, the intra-prediction unit 204 can obtain the value of p[x] using a unified calculation method without determining the value of mipSizeId. For example, (1 << (BitDepth-1)) is added as an additional element to the buffer pTemp. In this case, the intra-prediction unit 204 calculates p[x] as pTemp[x+1] - pTemp[0].

[0477] Step 602, the intra-prediction unit 204 obtains the MIP prediction value for the current block using the reference sample set and the MIP weighting matrix. The MIP weighting matrix is ​​selected from a predefined set of MIP weighting matrices based on the MIP mode index value (represented by ModeId) and the MIP block size index value (represented by mipSizeId) corresponding to the MIP mode. The intra-prediction unit 204 obtains the ModeId value from the analysis unit 201.

[0478] The intra-prediction unit 204 obtains the MIP prediction value (represented as predMip[x][y]) for a partial prediction sample corresponding to one or more sample locations in the current block. In the example of step 602 shown in Figure 6, the partial prediction sample is the sample marked as a gray square in the current block. The input to the prediction module 601 is the reference sample p[x] obtained in step 601, and the prediction module 601 calculates partial prediction samples using the MIP weighting matrix and shift parameters. The shift parameters include a shift number parameter and a shift offset parameter.

[0479] In one possible embodiment, the prediction module 601 can represent a sample in coordinates (x,y), and the predicted sample can be represented as predMip[x][y]. The formula for calculating predMip[x][y] is as shown in equation (5) or equation (6).

[0480] In equation (5) or (6), mWeight[i][j] is the MIP weighting matrix, the matrix elements may be predetermined constants or may be adaptively updated using a training method or the like. The input to the training method is one or more encoded images or blocks, or images from other bitstreams provided to the decoder 200 by an external device, or images obtained from the analysis unit 201 by analyzing special data units in the input bitstream containing the MIP weighting matrix. mWeight[i][j] is determined based on the MIP mode indicated by one or more corresponding parameters from the analysis unit 201. fO is the shift offset parameter for determining oW, sW is the shift number parameter, p[i] is calculated using a reference sample, i.e., the MIP input sample value in the embodiment of the present invention, pTemp[0] represents the first sample in the reference sample, i.e., the value corresponding to index 0 in the first temporary reference value in the embodiment of the present invention, and ">>" is the binary right shift operator as defined in VVC.

[0481] The prediction module 601 can determine the values ​​of sW and fO based on the current block size and the MIP mode used for the current block. In one example, the prediction module 601 obtains the values ​​of sW and fO using a lookup table.

[0482] Selectively, the prediction module 601 can determine sW based on the current block size parameters and MIP mode by utilizing Table 4 above.

[0483] Selectively, the prediction module 601 can also determine sW based on the current block size parameter by utilizing Table 5 above.

[0484] Selectively, the prediction module 601 can also directly set sW to a constant value. For example, for blocks with various size parameters and different MIP modes, the prediction module 601 can set sW to 5. Alternatively, for blocks with various size parameters and different MIP modes, the prediction module 601 can set sW to 6. Alternatively, for blocks with various size parameters and different MIP modes, the prediction module 601 can set sW to 7.

[0485] Selectively, the prediction module 601 can determine fO based on the current block size parameters and MIP mode by utilizing Table 6 above.

[0486] Selectively, the prediction module 601 can also determine fO based on the current block size parameters by utilizing Table 7 above.

[0487] Selectively, the prediction module 601 can also directly set fO to a constant value (e.g., 0 to 100). For example, for blocks with various size parameters and different MIP modes, the prediction module 601 can set fO to 32. Or, the prediction module 601 can set fO to 46. Or, the prediction module 601 can set fO to 56. Or, the prediction module 601 can set fO to 66.

[0488] The intra prediction unit 204 can perform a clipping operation on the MIP prediction values ​​in predMip. When isTransposed is equal to 1 (or TRUE), the array predMip[x][y] (x = 0, ..., predSize-1, y = 0, ..., predSize-1) of predSize × predSize is converted to predTemp[y][x] = predMip[x][y], and then predMip = predTemp.

[0489] As an example, selectively, if the current block size parameter is 8x8 (i.e., both cbWidth and cbHeight are equal to 8), the intra-prediction unit 204 sets mipSizeId to equal 2 and obtains an 8x8 predMip.

[0490] Selectively, if the current block size parameter is 8x8 (i.e., both cbWidth and cbHeight are equal to 8), the intra-prediction unit 204 sets mipSizeId to equal 1 and obtains a 4x4 predMip.

[0491] Step 603, the intra-prediction unit 204 obtains the intra-predicted values ​​for the current block (stored in an array as predSamples[x][y], where x=0, ..., nTbW-1, y=0, ..., nTbH-1) as follows:

[0492] - When the intra-prediction unit 204 determines that nTbW is greater than predSize, or nTbH is greater than predSize, it invokes the upsampling process to obtain predSamples using predMip. The intra-prediction unit 204 obtains predicted samples corresponding to the remaining samples in the current block, excluding the partial samples. In Figure 6, the intra-prediction unit 204 can use the filtering module 602 to obtain predicted samples corresponding to the remaining samples in the current block, excluding the partial samples. The input to the filtering module 602 is the samples marked as gray squares in step 602. The filtering module 602 can use one or more interpolation filters to obtain predicted samples corresponding to the remaining samples in the current block, excluding the partial samples. For example, the input may include a reference sample and partial predicted samples of one or more sample locations in the current block. Alternatively, the input may include adjacent samples and partial predicted samples of one or more sample locations in the current block. Alternatively, the input may include a reference sample, adjacent samples, and partial predicted samples of one or more sample locations in the current block. For example, selectively, if the size of the current block is 8x8 (i.e., both cbWidth and cbHeight are equal to 8), the intra-prediction unit 204 determines that mipSizeId is equal to 1, and the intra-prediction unit 204 obtains an 8x8 intra-prediction block of the current block by applying an upsampling process to a 4x4 predMip.

[0493] - Otherwise, the intra-prediction unit 204 sets the intra-prediction block of the current block to be equal to the MIP prediction block of the current block, i.e., sets predSamples[x][y](x=0, ..., nTbW-1, y=0, ..., nTbH-1) to be equal to predMip[x][y]. For example, if the size parameter of the current block is 8x8 (i.e., both cbWidth and cbHeight are equal to 8), the intra-prediction unit 204 obtains predSamples of the current block which is 8x8 in size (i.e., both cbWidth and cbHeight are equal to 8).

[0494] Thus, according to the block diagram of the flow shown in Figure 6, after step 603, the intra-prediction unit 204 can obtain the intra-prediction block for the current block (i.e., CU), that is, it can determine the intra-prediction value for at least one sample of the current block.

[0495] The scaling unit 205 has the same function as the inverse quantization unit 110 in the encoder 100. The scaling unit 205 performs a scaling operation on the quantization coefficients (i.e., levels) from the analysis unit 201 in order to obtain reconstruction coefficients.

[0496] The conversion unit 206 has the same function as the inverse conversion unit 111 in the encoder 100. The conversion unit 206 performs one or more conversion operations (i.e., the inverse operations of one or more conversion operations performed by the inverse conversion unit 111 in the encoder 100) to obtain the reconstruction residual.

[0497] If lfnst_index is not equal to 0, the transformation unit 206 performs a quadratic transformation on the reconstruction coefficients to obtain a quadratic transformation coefficient block. Next, to obtain a reconstruction residual block, it performs a first transformation on the quadratic transformation coefficient block, for example, an integer transformation initially designed based on DCT. LFNST is an example of a quadratic transformation.

[0498] Referring to Figure 11, Figure 11 shows a schematic diagram of another flow for performing LFNST according to an embodiment of the present invention. In Figure 11, block 1101 is a reconstruction transformation coefficient block containing reconstruction transformation coefficients.

[0499] Step 1101, the conversion unit 206 performs a secondary transformation such as LFNST on all or some of the reconstruction conversion coefficients in block 1101.

[0500] If the current block's intra-prediction mode is MIP mode, the conversion unit 206 selects a set of LFNST conversion kernel candidates based on the MIP parameters. The MIP parameters may include one of the following: the MIP mode index value (i.e., modelId) and the MIP transpose instruction parameter (i.e., isTransposed).

[0501] The transformation unit 206 sets the value of the variable predModeIntra to equal the value of modelId. The transformation unit 206 determines a transformation kernel for performing LFNST on the coefficients in subblocks 11001, 11002, and 11003. The size of the subblocks can be a set value, for example, 8x8. Based on the value of predModeIntra, the transformation unit 206 determines the index value (i.e., lfnstTrSetIdx) of the LFNST transformation kernel candidate set. For the original block decoding in MIP mode, the transformation unit 206 obtains the value of lfnstTrSetIdx using a lookup table as shown in Table 10. The transformation unit 206 determines the transformation kernel for performing LFNST as the transformation kernel indicated by lfnst_index in the LFNST transformation kernel candidate set indicated by lfnstTrSetIdx.

[0502] If lfnst_index is not equal to 0, the transformation unit 206 performs LFNST on the transformation block using coefficients from subblocks 11001, 11002, and 11003 to obtain LFNST coefficients, and places the LFNST coefficients into subblocks 11001, 11002, and 11003 of block 1102. Assume that d[x][y] is the block of the first transformation coefficient, where x=0, ..., nTbW-1, y=0, ..., nTbH-1, where nTbW and nTbH are the width and height of block 1102, respectively. Assume that v[x] is the block of the LFNST coefficient, where x=0, ..., nLfnstOutSize-1, nLfnstOutSize = (nTbW>=8 && nTbH>=8)? 48:16.

[0503] Selectively, the conversion unit 206 obtains the conversion coefficient block 1102 by employing the following instructions, specifically as shown in equation (8) above. log2LfnstSize is determined as shown in equation (9) above.

[0504] Selectively, if isTransposed is equal to 0, the conversion unit 206 obtains the conversion coefficient block 1102 by employing the following instruction, specifically as shown in equation (10) above. When isTransposed is equal to 1, the conversion unit 206 obtains the conversion coefficient block 1102 by employing the following instruction, specifically as shown in equation (11) above.

[0505] Selectively, if isTransposed is equal to 1, the conversion unit 206 obtains the conversion coefficient block 1102 by employing the following instruction, specifically as shown in equation (10) above. If isTransposed is equal to 0, the conversion unit 206 obtains the conversion coefficient block 1102 by employing the following instruction, specifically as shown in equation (11) above.

[0506] log2LfnstSize is determined as shown in equation (9) above.

[0507] Furthermore, if the current block's intra-prediction mode is a conventional intra-prediction mode (i.e., angle intra-prediction mode, DC mode, and PLANAR mode, etc.), the conversion unit 206 uses the lookup table shown in Table 11 to determine the value of lfnstTrSetIdx based on the mode index value of the conventional intra-prediction mode (assigned to predModeIntra), thereby determining the LFNST conversion kernel candidate set selected for the current block. The conversion unit 206 determines the conversion kernel for performing LFNST as the conversion kernel indicated by lfnst_index within the LFNST conversion kernel candidate set indicated by lfnstTrSetIdx.

[0508] If lfnst_index is not equal to 0, the transformation unit 206 performs LFNST on the transformation block using coefficients from subblocks 11001, 11002, and 11003 to obtain LFNST coefficients, and places the LFNST coefficients into subblocks 11001, 11002, and 11003 of block 1102. Assume that d[x][y] is the block of the first transformation coefficient, where x=0, ..., nTbW-1, y=0, ..., nTbH-1, where nTbW and nTbH are the width and height of block 1102, respectively. Assume that v[x] is the block of the LFNST coefficient, where x=0, ..., nLfnstOutSize-1, nLfnstOutSize = (nTbW>=8 && nTbH>=8)? 48:16.

[0509] Selectively, when predModeIntra is 34 or less, the conversion unit 206 obtains the conversion coefficient block 1102 by adopting the following instruction, specifically as shown in equation (10) above. Selectively, if predModeIntra is greater than 34, the conversion unit 206 obtains the conversion coefficient block 1102 by employing the following instruction, specifically as shown in equation (11) above.

[0510] Selectively, if predModeIntra is greater than 34, the conversion unit 206 obtains the conversion coefficient block 1102 by employing the following instruction, specifically as shown in equation (10) above. Selectively, if predModeIntra is 34 or less, the conversion unit 206 obtains the conversion coefficient block 1102 by adopting the following instruction, specifically as shown in equation (11) above.

[0511] log2LfnstSize is determined as shown in equation (9) above.

[0512] Step 1102, the transformation unit 206 obtains the reconstructed residual block 1103 of the current block by performing a first transformation (i.e., a core transformation), for example, an integer transformation initially designed based on the DCT, on the transformation coefficients in block 1102.

[0513] Furthermore, the adder 207 performs an addition operation on its inputs (the intra-prediction block from the prediction unit 202 and the reconstructed residual block from the transformation unit 206) to obtain the reconstructed block of the current block. The reconstructed block is also sent to the prediction unit 202 to be used as a reference for other blocks that are encoded under intra-prediction mode.

[0514] After all CUs in the image or sub-image have been reconstructed, the filtering unit 208 performs in-loop filtering on the reconstructed image or sub-image. The filtering unit 208 includes one or more filters, such as a deblocking filter, a sample adaptive offset (SAO) filter, an adaptive loop filter (ALF), a chroma-scaling luma mapping (LMCS) filter, and a neural network-based filter. Alternatively, the filtering unit 208 performs in-loop filtering on one or more target samples in a reconstructed block if it determines that the reconstructed block will not be used as a reference for decoding other blocks.

[0515] The output of the filtering unit 208 is a decoded image or sub-image, which is sent to the DPB unit 209. The DPB unit 209 outputs the decoded image based on timing and control information. The images stored in the DPB unit 209 can further be used as a reference for inter-prediction or intra-prediction by the prediction unit 202.

[0516] In embodiments of the present invention, the decoder 200 may be a computing device comprising a processor and a storage medium for recording a decoding program. When the processor reads and executes the decoding program, the decoder 200 reads the input bitstream and generates the corresponding decoded video. Alternatively, the decoder 200 may be a computing device comprising one or more chips. A unit implemented as an integrated circuit on a chip has connectivity and data exchange functions similar to the corresponding unit in Figure 2.

[0517] In embodiments of the present invention, a decoding method is provided. The bitstream is analyzed to obtain prediction parameters and LFNST parameters for the current block. The prediction parameters include prediction mode parameters. If the prediction mode parameters indicate that the intra-predicted value of the saturation component of the current block is determined using the MIP mode, the adjacent sample values ​​of the current block are obtained, and the MIP input sample values ​​of the current block are determined based on the adjacent sample values ​​of the current block. Based on the MIP input sample values, the MIP weighting matrix, and the shift parameters, the MIP prediction value of the saturation component of the current block is determined. The intra-predicted value of the saturation component of the current block is determined by filtering the MIP prediction values. If the LFNST parameters indicate that LFNST is performed on the current block, the reconstruction transformation coefficient block of the current block is determined, and a second transformation coefficient block is obtained by performing LFNST on at least some of the reconstruction transformation coefficients in the reconstruction transformation coefficient block. The reconstruction residual block of the saturation component of the current block is obtained by performing the first transformation on the second transformation coefficient block. Based on the intra-predicted value of the saturation component of the current block and the reconstruction residual block, the reconstruction block of the saturation component of the current block is determined. Thus, in MIP mode, decoding performance is ensured while reducing complexity and simultaneously decreasing the memory space required for the decoding process, thereby effectively increasing decoding efficiency. Furthermore, when LFNST technology is applied to MIP mode prediction, the introduction of MIP parameters makes LFNST more flexible, further improving decoding efficiency.

[0518] In another embodiment of the present invention, based on the same inventive concept as the above embodiments, and referring to Figure 12, Figure 12 shows a schematic diagram of the structure of an encoder 120 according to an embodiment of the present invention. As shown in Figure 12, the encoder 120 may include a first decision unit 1201, a first prediction unit 1202, a first conversion unit 1203, and an encoding unit 1204.

[0519] The first decision unit 1201 is configured to determine the prediction parameters for the current block, the prediction parameters including the prediction mode parameters. The first decision unit 1201 is further configured to obtain the adjacent sample values ​​of the current block and to determine the MIP input sample values ​​of the current block based on the adjacent sample values ​​of the current block, when the prediction mode parameter indicates that the MIP mode is used to determine the intra-predicted value of the saturation component of the current block. The first prediction unit 1202 is configured to determine the MIP predicted value of the saturation component of the current block based on the MIP input sample values, the MIP weighting matrix, and the shift parameters, and to determine the intra predicted value of the saturation component of the current block by filtering the MIP predicted value, the MIP predicted value being the predicted value of a subset of samples in the saturation component of the current block. The first conversion unit 1203 is configured to determine the predicted residual value of the saturation component of the current block based on the intra-predicted value of the saturation component of the current block, and to determine the LFNST parameters by performing LFNST on the predicted residual value. The encoding unit 1204 is configured to encode the LFNST parameters and write them to the bitstream.

[0520] In some embodiments, the adjacent sample values ​​of the current block include the left adjacent sample values ​​and the upper adjacent sample values ​​of the current block.

[0521] In some embodiments, the shift parameters include a shift offset parameter and a shift number parameter. Specifically, the first decision unit 1201 is configured to determine the product of the sum of MIP input sample values ​​and the shift offset parameter based on the value of a fixed constant shift offset parameter, to determine a first constant value based on the value of a fixed constant shift number parameter, to set the value of the first offset to the difference between the first constant value and the above product, to determine the MIP weighting matrix of the current block based on the prediction parameter, and to determine the MIP prediction value of the saturation component of the current block based on the MIP weighting matrix, MIP input sample values, shift number parameter and the first offset.

[0522] In some embodiments, the prediction parameters further include the current block size parameters.

[0523] In some embodiments, the first determination unit 1201 is configured to obtain a first temporary reference value by determining the block size index value of the current block based on the size parameters of the current block and by downsampling the adjacent sample value of the current block. Furthermore, the first decision unit 1201 is configured to determine a second constant value based on the bit depth of the adjacent sample values ​​of the current block when the block size index value of the current block is within a preset range, set the value corresponding to index 0 in the MIP input sample value to the difference between the second constant value and the value corresponding to index 0 in the first temporary reference value, and set the value corresponding to index i in the MIP input sample value to the difference between the value corresponding to index i in the first temporary reference value and the value corresponding to index 0 in the first temporary reference value, where i is an integer greater than 0. Furthermore, the first decision unit 1201 is configured to set the value corresponding to index j in the MIP input sample value to the difference between the value corresponding to index (j+1) in the first temporary reference value and the value corresponding to index 0 in the first temporary reference value, when the block size index value of the current block is outside a preset range, where j is a non-negative integer.

[0524] In some embodiments, the first determination unit 1201 is further configured to determine a second constant value based on the bit depth of the adjacent sample values ​​of the current block, when the block size index value of the current block is within a preset range, set the value corresponding to index 0 in the MIP input sample value to the difference between the value corresponding to index 0 in the first temporary reference value and the second constant value, and set the value corresponding to index i in the MIP input sample value to the difference between the value corresponding to index i in the first temporary reference value and the value corresponding to index 0 in the first temporary reference value, where i is an integer greater than 0.

[0525] In some embodiments, the first determination unit 1201 is further configured to set the block size index value of the current block to 0 if both the width and height of the current block are equal to 4, to set the block size index value of the current block to 1 if both the width and height of the current block are equal to 8, or if one of the width and height of the current block is equal to 4, and to set the block size index value of the current block to 2 if the width and height of the current block do not satisfy the above conditions.

[0526] In some embodiments, the first determination unit 1201 is further configured to obtain a first temporary reference value by downsampling the adjacent sample values ​​of the current block, to determine a second constant value based on the bit depth of the adjacent sample values ​​of the current block, to obtain a second temporary reference value by buffering the second constant value in one data unit after the first temporary reference value, and to set the value corresponding to index j in the MIP input sample value to the difference between the value corresponding to index (j+1) in the second temporary reference value and the value corresponding to index 0 in the second temporary reference value, where j is a non-negative integer.

[0527] In some embodiments, the first decision unit 1201 is further configured to set a second constant value to a power of 2, where the exponent of the power is an integer and equal to the bit depth of the adjacent sample values ​​of the current block minus 1.

[0528] In some embodiments, the first decision unit 1201 is further configured to obtain a second constant value by binary left-shifting "1", where the number of left-shifted bits is equal to the bit depth of the adjacent sample value of the current block minus 1.

[0529] In some embodiments, the value of the shift number parameter is a fixed constant independent of the block size index value and the MIP mode index value.

[0530] In some embodiments, the value of the shift number parameter is set to 6.

[0531] In some embodiments, the value of the shift offset parameter is a fixed constant independent of the block size index value and the MIP mode index value.

[0532] In some embodiments, the value of the shift offset parameter is set to 32.

[0533] In some embodiments, the first decision unit 1201 is further configured to set a first constant value to a power of 2, where the exponent of the power is an integer and equal to the value of the shift number parameter minus 1.

[0534] In some embodiments, the first determination unit 1201 is further configured to obtain a first constant value by binary left-shifting "1", such that the number of left-shifted bits is equal to the value of the shift number parameter minus 1.

[0535] In some embodiments, referring to Figure 12, the encoder 120 may further include a first calculation unit 1205. The first calculation unit 1205 is configured to calculate a first weighted sum of the MIP weighting matrix and the MIP input sample values, calculate a first sum of the first weighted sum and a first offset, and obtain a first right-shifted value by binary right-shifting the first sum, the number of right-shifted bits being equal to the value of the shift number parameter. The first decision unit 1201 is further configured to set the MIP prediction value of the saturation component of the current block to the sum of a first right-shifted value and a value corresponding to index 0 in a first temporary reference value, the first temporary reference value is obtained by downsampling the adjacent sample values ​​of the current block.

[0536] In some embodiments, the first conversion unit 1203 is configured to determine the residual block of the current block based on the predicted residual value and to obtain a first conversion coefficient block by performing a first conversion on the residual block. The first decision unit 1201 is further configured to determine whether or not to perform LFNST on at least some of the conversion coefficients in the first conversion coefficient block. The first transformation unit 1203 is further configured to determine the LFNST parameters by performing LFNST on at least some of the transformation coefficients in the first transformation coefficient block, when the result of the decision is to perform LFNST on at least some of the transformation coefficients.

[0537] In some embodiments, the first decision unit 1201 is further configured to determine the minimum value among the size parameters of the current block when determining the intra-predicted value of the saturation component of the current block using MIP mode, and to determine to perform LFNST on at least some of the conversion coefficients in the first conversion coefficient block if the minimum value is greater than or equal to a first preset threshold.

[0538] In some embodiments, the LFNST parameter includes an LFNST index. The first decision unit 1201 is further configured to set the value of the LFNST index to greater than 0 when it is determined that LFNST should be performed on at least some of the conversion coefficients in the first conversion coefficient block, and to set the value of the LFNST index to 0 when it is determined that LFNST should not be performed on at least some of the conversion coefficients in the first conversion coefficient block. The encoding unit 1204 is configured to encode the LFNST index and write it to the bitstream.

[0539] In some embodiments, the LFNST parameters further include LFNST coefficients. Referring to Figure 12, the encoder 120 may further include a quantization unit 1206. The first conversion unit 1203 is configured to obtain LFNST coefficients by performing LFNST on at least some of the conversion coefficients in the first conversion coefficient block. The quantization unit 1206 is configured to obtain quantization coefficients by quantizing the LFNST coefficients. The encoding unit 1204 is further configured to encode the quantization coefficients and write them to the bitstream.

[0540] In embodiments of the present invention, it can be understood that a "unit" may be part of a circuit, part of a processor, part of a program, or part of software. Of course, a "unit" may be a module or a non-module. Furthermore, each component unit according to this embodiment may be integrated into a single processing unit, each unit may exist physically independently, or two or more units may be integrated into a single unit. The integrated unit may be implemented in the form of a hardware or software functional module.

[0541] The integrated unit may be implemented as a software functional module and, if sold or used as an independent product, may be stored on a computer-readable recording medium. According to this understanding, the essential parts of the technical solution of this embodiment, or parts that contribute to the prior art, or all or part of the technical solution, may be expressed as a software product. This computer software product is stored on a storage medium and includes a plurality of instructions for causing a computer device (which may be a personal computer, server, or network device, etc.) or processor to perform all or part of the steps of the method described in this embodiment. The storage medium includes various types of media capable of storing program code, such as universal serial bus (USB) flash disks, mobile hard disks, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0542] Accordingly, an embodiment of the present invention provides a computer storage medium to be applied to the encoder 120. The computer storage medium stores a computer program, which, when executed by the first processor, performs any of the methods in the above embodiment.

[0543] Referring to Figure 13, based on the above configuration of the encoder 120 and the computer storage medium, Figure 13 shows a schematic diagram of the specific hardware structure of the encoder 120 according to an embodiment of the present invention. As shown in Figure 13, the encoder 120 may include a first communication interface 1301, a first memory 1302, and a first processor 1303. Each component is coupled via a first bus system 1304. The first bus system 1304 is used to enable connection and communication between these components. In addition to the data bus, the first bus system 1304 further includes a power bus, a control bus, and a status signal bus. However, for clarity, in Figure 13, each type of bus is referred to as the first bus system 1304.

[0544] The first communication interface 1301 is configured to send and receive signals in the process of sending and receiving information with other external network elements.

[0545] The first memory 1302 is configured to store a computer program that can be executed by the first processor 1303.

[0546] The first processor 1303 is configured to perform the following when executing a computer program:

[0547] Determine the prediction parameters for the current block. These prediction parameters include the prediction mode parameters. If the prediction mode parameter indicates that the MIP mode will be used to determine the intra-predicted value of the saturation component of the current block, the adjacent sample values ​​of the current block are obtained, and the MIP input sample values ​​of the current block are determined based on the adjacent sample values ​​of the current block. Based on the MIP input sample values, MIP weighting matrix, and shift parameters, the MIP predicted value for the saturation component of the current block is determined. The MIP predicted value is the predicted value for a subset of samples in the saturation component of the current block. By filtering the MIP prediction values, we determine the intra-predicted value for the saturation component of the current block. Based on the intra-predicted value of the current block's saturation component, the predicted residual value of the current block's saturation component is determined. The LFNST parameters are determined by performing LFNST on the predicted residual values. Encode the LFNST parameters and write them to the bitstream.

[0548] The first memory 1302 in the embodiments of the present invention may be a volatile memory or a non-volatile memory, or may include both a volatile memory and a non-volatile memory. The non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (Erasable PROM, EPROM), an electrically erasable programmable read-only memory (Electrically EPROM, EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM) that functions as an external high-speed cache. Examples of various RAMs available include, but are not limited to, static random access memory (Static RAM, SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (Synchronous DRAM, SDRAM), double data rate synchronous dynamic random access memory (Double Data Rate SDRAM, DDRSDRAM), enhanced synchronous dynamic random access memory (Enhanced SDRAM, ESDRAM), synchronous-link dynamic random access memory (Synch-link DRAM, SLDRAM), and direct rambus random access memory (Dierct Rambus RAM, DRRAM). The first memory 1302 of the systems and methods described herein may include, but is not limited to, these and any other suitable types of memory.

[0549] The first processor 1303 may be an integrated circuit chip having signal processing capabilities. In the implementation process, each step of the above method embodiment may be completed by an integrated logic circuit in hardware form or by instructions in software form of the first processor 1303. The first processor 1303 may be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component. The processor may implement or execute the various methods, steps and logic block diagrams disclosed in embodiments of the present invention. The general-purpose processor may be a microprocessor or any ordinary processor. The steps of the methods disclosed in embodiments of the present invention may be executed and completed directly by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software module may be located in a mature storage medium in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, or electrically erasable programmable memory, or registers. The storage medium is located in the first memory 1302. The first processor 1303 reads the information in the first memory 1302 and, in conjunction with the processor hardware, completes the steps of the method described above.

[0550] It can be understood that these embodiments described in the present invention can be implemented by hardware, software, firmware, middleware, microcode, or a combination thereof. When implemented by hardware, the processing unit can be one or more application-specific integrated circuits (ASICs), digital signal processing (DSPs), DSP devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers, microprocessors, other electronic units used to perform the functions described in the present invention, or a combination thereof. When implemented by software, the techniques described in the present invention can be implemented by modules (e.g., procedures, functions, etc.) for performing the functions described in the present invention. The software code is stored in memory and executed by the processor. The memory can be implemented within or outside the processor.

[0551] Selectively, in another embodiment, the first processor 1303 is configured to perform any of the methods in the above embodiment when executing a computer program.

[0552] In embodiments of the present invention, an encoder is provided. The encoder may include a first decision unit, a first prediction unit, a first transformation unit, and an encoding unit. In this way, in MIP mode, encoding performance can be ensured while reducing complexity and simultaneously reducing the memory space required for the encoding process, thereby effectively increasing encoding efficiency. Furthermore, when LFNST technology is applied to MIP mode prediction, the introduction of MIP parameters makes LFNST more flexible and further improves encoding efficiency.

[0553] In another embodiment of the present invention, and based on the same inventive concept as the above embodiments, with reference to Figure 14, Figure 14 shows a schematic diagram of the structure of a decoder 140 according to an embodiment of the present invention. As shown in Figure 14, the decoder 140 may include an analysis unit 1401, a second decision unit 1402, a second prediction unit 1403, and a second transformation unit 1404.

[0554] The analysis unit 1401 is configured to analyze the bitstream to obtain the prediction parameters and LFNST parameters of the current block, the prediction parameters including the prediction mode parameters. The second decision unit 1402 is configured to obtain the adjacent sample values ​​of the current block and to determine the MIP input sample values ​​of the current block based on the adjacent sample values ​​of the current block, when the prediction mode parameter indicates that the MIP mode is used to determine the intra-predicted value of the saturation component of the current block. The second prediction unit 1403 is configured to determine the MIP predicted value of the saturation component of the current block based on the MIP input sample values, the MIP weighting matrix, and the shift parameters, and to determine the intra predicted value of the saturation component of the current block by filtering the MIP predicted value, the MIP predicted value being the predicted value of a subset of samples in the saturation component of the current block. The second transformation unit 1404 is configured to determine the reconstruction transformation coefficient block of the current block when the LFNST parameter indicates that LFNST should be performed on the current block, obtain a second transformation coefficient block by performing LFNST on at least some of the reconstruction transformation coefficients in the reconstruction transformation coefficient block, and obtain a reconstruction residual block of the saturation component of the current block by performing the first transformation on the second transformation coefficient block. A second decision unit 1402 is further configured to determine the reconstructed block of the saturation component of the current block based on the intra-predicted value of the saturation component of the current block and the reconstructed residual block.

[0555] In some embodiments, the adjacent sample values ​​of the current block include the left adjacent sample values ​​and the upper adjacent sample values ​​of the current block.

[0556] In some embodiments, the shift parameters include a shift offset parameter and a shift number parameter. Specifically, the second decision unit 1402 is configured to determine the product of the sum of MIP input sample values ​​and the shift offset parameter based on the value of a fixed constant shift offset parameter, to determine a first constant value based on the value of a fixed constant shift number parameter, to set the value of the first offset to the difference between the first constant value and the above product, to determine the MIP weighting matrix of the current block based on the prediction parameter, and to determine the MIP prediction value of the saturation component of the current block based on the MIP weighting matrix, MIP input sample values, shift number parameter and first offset.

[0557] In some embodiments, the prediction parameters further include the current block size parameters.

[0558] In some embodiments, the second decision unit 1402 is configured to obtain a first temporary reference value by determining the block size index value of the current block based on the size parameters of the current block and by downsampling the adjacent sample value of the current block. Furthermore, the second decision unit 1402 is configured to determine a second constant value based on the bit depth of the adjacent sample values ​​of the current block, when the block size index value of the current block is within a preset range, set the value corresponding to index 0 in the MIP input sample value to the difference between the second constant value and the value corresponding to index 0 in the first temporary reference value, and set the value corresponding to index i in the MIP input sample value to the difference between the value corresponding to index i in the first temporary reference value and the value corresponding to index 0 in the first temporary reference value, where i is an integer greater than 0. Furthermore, the second decision unit 1402 is configured to set the value corresponding to index j in the MIP input sample value to the difference between the value corresponding to index (j+1) in the first temporary reference value and the value corresponding to index 0 in the first temporary reference value, when the block size index value of the current block is outside a preset range, where j is a non-negative integer.

[0559] In some embodiments, the second determination unit 1402 is further configured to determine a second constant value based on the bit depth of the adjacent sample values ​​of the current block, when the block size index value of the current block is within a preset range, set the value corresponding to index 0 in the MIP input sample value to the difference between the value corresponding to index 0 in the first temporary reference value and the second constant value, and set the value corresponding to index i in the MIP input sample value to the difference between the value corresponding to index i in the first temporary reference value and the value corresponding to index 0 in the first temporary reference value, where i is an integer greater than 0.

[0560] In some embodiments, the second determination unit 1402 is further configured to set the block size index value of the current block to 0 if both the width and height of the current block are equal to 4, to set the block size index value of the current block to 1 if both the width and height of the current block are equal to 8, or if either the width or height of the current block is equal to 4, and to set the block size index value of the current block to 2 if the width and height of the current block do not satisfy the above conditions.

[0561] In some embodiments, the second determination unit 1402 is further configured to obtain a first temporary reference value by downsampling the adjacent sample values ​​of the current block, to determine a second constant value based on the bit depth of the adjacent sample values ​​of the current block, to obtain a second temporary reference value by buffering the second constant value in one data unit after the first temporary reference value, and to set the value corresponding to index j in the MIP input sample value to the difference between the value corresponding to index (j+1) in the second temporary reference value and the value corresponding to index 0 in the second temporary reference value, where j is a non-negative integer.

[0562] In some embodiments, the second decision unit 1402 is further configured to set a second constant value to a power of 2, where the exponent of the power is an integer and equal to the bit depth of the adjacent sample values ​​of the current block minus 1.

[0563] In some embodiments, the second decision unit 1402 is further configured to obtain a second constant value by binary left-shifting "1", the number of bits left-shifted being equal to the bit depth of the adjacent sample value of the current block minus 1.

[0564] In some embodiments, the value of the shift number parameter is a fixed constant independent of the block size index value and the MIP mode index value.

[0565] In some embodiments, the value of the shift number parameter is set to 6.

[0566] In some embodiments, the value of the shift offset parameter is a fixed constant independent of the block size index value and the MIP mode index value.

[0567] In some embodiments, the value of the shift offset parameter is set to 32.

[0568] In some embodiments, the second decision unit 1402 is further configured to set the first constant value to a power of 2, where the exponent of the power is an integer and equal to the value of the shift number parameter minus 1.

[0569] In some embodiments, the second determination unit 1402 is further configured to obtain a first constant value by binary left-shifting "1", such that the number of left-shifted bits is equal to the value of the shift number parameter minus 1.

[0570] In some embodiments, referring to Figure 14, the decoder 140 may further include a second computing unit 1405. The second computing unit 1405 is configured to compute a first weighted sum of the MIP weighting matrix and the MIP input sample values, compute a first sum of the first weighted sum and a first offset, and obtain a first right-shifted value by binary right-shifting the first sum, the number of right-shifted bits being equal to the value of the shift number parameter. The second decision unit 1402 is further configured to set the MIP prediction value of the saturation component of the current block to the sum of the first right-shifted value and the value corresponding to index 0 in the first temporary reference value, which is obtained by downsampling the adjacent sample values ​​of the current block.

[0571] In some embodiments, a second decision unit 1402 is further configured to determine whether or not the bitstream has an LFNST parameter. The analysis unit 1401 is further configured to analyze the bitstream to obtain the LFNST parameter if the bitstream contains the LFNST parameter.

[0572] In some embodiments, the second decision unit 1402 is further configured to determine the minimum of the size parameters of the current block when determining the intra-predicted value of the saturation component of the current block using MIP mode, and to determine that the bitstream has an LFNST parameter if the minimum is greater than or equal to a first preset threshold.

[0573] In some embodiments, referring to Figure 14, the decoder 140 may further include an inverse quantization unit 1406. The analysis unit 1401 is further configured to analyze the bitstream to obtain the quantization coefficients of the current block when the LFNST parameter indicates that LFNST should be performed on the current block. The inverse quantization unit 1406 is configured to obtain the reconstruction transformation coefficient block of the current block by inverse quantization of the quantization coefficients.

[0574] In some embodiments, the inverse quantization unit 1406 is specifically configured to obtain a reconstruction transformation coefficient block of the current block by scaling the quantization coefficients.

[0575] In some embodiments, the analysis unit 1401 is configured to specifically analyze the bitstream to obtain the value of the LFNST index, to decide to perform an LFNST on the current block if the value of the LFNST index is greater than 0, and to decide not to perform an LFNST on the current block if the value of the LFNST index is equal to 0.

[0576] In this embodiment, it can be understood that a "unit" may be part of a circuit, part of a processor, part of a program, or part of software. Of course, a "unit" may be a module or a non-module. Furthermore, each component unit according to this embodiment may be integrated into a single processing unit, each unit may exist physically independently, or two or more units may be integrated into a single unit. The integrated unit may be implemented in the form of a hardware or software functional module.

[0577] The integrated unit may be implemented as a software functional module and, if sold or used as an independent product, may be stored on a computer-readable recording medium. According to this understanding, embodiments of the present invention provide a computer storage medium applicable to the decoder 140. The computer storage medium stores a computer program, which, when executed by a second processor, performs any of the methods in the above embodiments.

[0578] Referring to Figure 15, based on the above configuration of the decoder 140 and the computer storage medium, Figure 15 shows a schematic diagram of the specific hardware structure of the decoder 140 according to an embodiment of the present invention. As shown in Figure 15, the decoder 140 may include a second communication interface 1501, a second memory 1502, and a second processor 1503. Each component is connected via a second bus system 1504. The second bus system 1504 is used to enable connection and communication between these components. In addition to the data bus, the second bus system 1504 further includes a power bus, a control bus, and a status signal bus. However, for clarity, in Figure 15, each type of bus is referred to as the second bus system 1504.

[0579] The second communication interface 1501 is configured to send and receive signals in the process of sending and receiving information with other external network elements.

[0580] The second memory 1502 is configured to store computer programs that can be executed by the second processor 1503.

[0581] The second processor 1503 is configured to perform the following when executing a computer program:

[0582] The bitstream is parsed to obtain the prediction parameters and LFNST parameters for the current block. The prediction parameters include the prediction mode parameters. If the prediction mode parameter indicates that the MIP mode will be used to determine the intra-predicted value of the saturation component of the current block, the adjacent sample values ​​of the current block are obtained, and the MIP input sample values ​​of the current block are determined based on the adjacent sample values ​​of the current block. Based on the MIP input sample values, MIP weighting matrix, and shift parameters, the MIP predicted value for the saturation component of the current block is determined. The MIP predicted value is the predicted value for a subset of samples in the saturation component of the current block. By filtering the MIP prediction values, we determine the intra-predicted value for the saturation component of the current block. If the LFNST parameter indicates that LFNST should be performed on the current block, the reconstruction transformation coefficient block of the current block is determined, and a second transformation coefficient block is obtained by performing LFNST on at least some of the reconstruction transformation coefficients in the reconstruction transformation coefficient block. By performing the first transformation on the second transformation coefficient block, the reconstructed residual block of the saturation component of the current block is obtained. Based on the intra-predicted value of the current block's saturation component and the reconstructed residual block, the reconstructed block of the current block's saturation component is determined.

[0583] Selectively, in another embodiment, the second processor 1503 is configured to perform any of the methods in the above embodiment when executing a computer program.

[0584] The hardware functionality of the second memory 1502 is similar to that of the first memory 1302, and the hardware functionality of the second processor 1503 is similar to that of the first processor 1303.

[0585] In this embodiment, a decoder is provided. The decoder may include an analysis unit, a second decision unit, a second prediction unit, and a second transformation unit. In this way, in MIP mode, decoding performance can be ensured while reducing complexity, simultaneously reducing the memory space required for the decoding process, and effectively increasing decoding efficiency. Furthermore, when LFNST technology is applied to MIP mode prediction, the introduction of MIP parameters makes LFNST more flexible and further improves decoding efficiency.

[0586] In another embodiment of the present invention, Figure 16 shows a schematic diagram of the structure of a transmitting device according to an embodiment of the present invention. As shown in Figure 16, an exemplary transmitting device 1600 is shown. The acquisition unit 1601 acquires a video signal and sends the video signal to the encoder 1602. The acquisition unit 1601 may be a device comprising one or more cameras (including a depth camera). The acquisition unit 1601 may be a device that partially or completely decodes the bitstream to acquire video. The acquisition unit 1601 may further include one or more elements for capturing an audio signal. One specific embodiment of the encoder 1602 is the encoder 100 or encoder 120 described in the above embodiment, which decodes a video signal from the acquisition unit 1601, which is to be its input video, and generates a video bitstream. The encoder 1602 may further include one or more audio encoders to generate an audio bitstream by encoding an audio signal. The storage / transmitting unit 1603 receives the video bitstream from the encoder 1602. The storage / transmission unit 1603 further receives the audio bitstream from the encoder 1602 and compresses the video bitstream and audio bitstream together to form a media file (e.g., an ISO-based media file format) or transmission stream.

[0587] Selectively, the storage / transmission unit 1603 writes media files or transmission streams to a storage unit, examples of which include hard disks, digital video discs (DVDs), cloud disks, and portable storage devices.

[0588] Selectively, the storage / transmission unit 1603 sends the bitstream to a transport network, such as the Internet, a wired network, a cellular network, or a wireless local area network.

[0589] In another embodiment of the present invention, referring to Figure 17, which shows a schematic diagram of the structure of a target device according to an embodiment of the present invention. As shown in Figure 17, an exemplary target device 1700 is shown. The receiving unit 1701 receives a media file or transmission stream from a network or reads a media file or transmission stream from a storage device. The receiving unit 1701 separates the video bitstream and audio bitstream from the media file or transmission stream. The receiving unit 1701 may further generate a new video bitstream by extracting the video bitstream. The receiving unit 1701 may further generate a new audio bitstream by extracting the audio bitstream. The decoder 1702 includes one or more video decoders, one specific embodiment of the video decoder being, for example, the decoder 200 or decoder 140 described in the above embodiment.

[0590] Decoder 1702 may further include one or more audio decoders. Decoder 1702 decodes the video bitstream and audio bitstream from receiving unit 1701 to obtain decoded video and one or more decoded audio corresponding to one or more sound channels. Rendering unit 1703 performs operations on the reconstructed video to make it suitable for display. These operations may include one or more operations that improve perceptual quality, such as denoising, compositing, color space conversion, upsampling, and downsampling. Rendering unit 1703 may further perform operations on the decoded audio to improve the perceptual quality of the displayed audio signal.

[0591] In another embodiment of the present invention, referring to Figure 18, which shows a schematic diagram of the structure of a communication system according to an embodiment of the present invention. As shown in Figure 18, an exemplary communication system 1800 is shown. The source device 1801 may be the transmission device 1600 shown in Figure 16. To store / transmit a bitstream, the output of the storage / transmission unit 1603 is processed by the storage medium / transport network 1802. The target device 1803 may be the target device 1700 shown in Figure 17. Here, the receiving unit 1701 can acquire a bitstream from the storage medium / transport network 1802. The receiving unit 1701 can extract a new video bitstream from a media file or transmission stream. The receiving unit 1701 can further extract a new audio bitstream from a media file or transmission stream.

[0592] In this invention, terms such as “includes,” “equipment,” or other variants are intended to cover non-exclusive implications. Therefore, a process, method, article, or apparatus that includes a set of elements may include not only those elements but also other elements not explicitly listed, or other elements inherent to the process, method, article, or apparatus. Unless otherwise limited, the phrase “includes…” does not preclude the existence of other identical elements in a process, method, article, or apparatus that includes the elements limited by the phrase “...includes…”.

[0593] The sequence numbers in the embodiments of the present invention described above are not intended to indicate superiority or inferiority among the embodiments, but are used solely for illustrative purposes.

[0594] The methods disclosed in some embodiments of the present invention can be arbitrarily combined, insofar as they do not contradict each other, to obtain new embodiments of the method.

[0595] The features disclosed in some product embodiments of the present invention can be arbitrarily combined, insofar as they do not contradict each other, to obtain new product embodiments.

[0596] The features disclosed in some embodiments of the method or apparatus according to the present invention can be arbitrarily combined, insofar as they do not conflict, to obtain new embodiments of the method or apparatus.

[0597] The above are merely specific embodiments of the present invention, and the scope of protection of the present invention is not limited thereto. Any modifications or substitutions that a person skilled in the art could easily conceive within the scope of the art disclosed herein should be included within the scope of the present invention. Accordingly, the scope of protection of the present invention should be determined by the scope of protection of the claims. [Industrial applicability]

[0598] In an embodiment of the present invention, the encoder determines the prediction parameters for the current block, and if the prediction mode parameter indicates that the MIP mode is used to determine the intra-predicted value of the saturation component of the current block, it obtains the adjacent sample values ​​of the current block and determines the MIP input sample values ​​of the current block based on the adjacent sample values ​​of the current block. Based on the MIP input sample values, the MIP weighting matrix, and the shift parameters, the MIP prediction value of the saturation component of the current block is determined. The intra-predicted value of the saturation component of the current block is determined by filtering the MIP prediction value. Based on the intra-predicted value of the saturation component of the current block, the predicted residual value of the saturation component of the current block is determined. The LFNST parameter is determined by performing LFNST on the predicted residual value. The LFNST parameter is encoded and written to the bitstream. On the decoder side, after analyzing the bitstream to obtain the prediction parameters and LFNST parameters for the current block, if the prediction mode parameter indicates that the intra-predicted value of the saturation component of the current block is determined using the MIP mode, the adjacent sample values ​​of the current block are obtained, and the MIP input sample values ​​of the current block are determined based on the adjacent sample values ​​of the current block. Based on the MIP input sample values, MIP weighting matrix, and shift parameters, the MIP prediction value of the saturation component of the current block is determined. The intra-predicted value of the saturation component of the current block is determined by filtering the MIP prediction value. If the LFNST parameter indicates that LFNST should be performed on the current block, the reconstruction transformation coefficient block of the current block is determined, and a second transformation coefficient block is obtained by performing LFNST on at least some of the reconstruction transformation coefficients in the reconstruction transformation coefficient block. The reconstruction residual block of the saturation component of the current block is obtained by performing the first transformation on the second transformation coefficient block. Based on the intra-predicted value of the saturation component of the current block and the reconstruction residual block, the reconstruction block of the saturation component of the current block is determined.Thus, in MIP mode, while ensuring encoding and decoding performance, complexity is reduced, and the memory space required for the encoding and decoding process is simultaneously decreased, effectively improving encoding and decoding efficiency. Furthermore, when LFNST technology is applied to MIP mode prediction, the introduction of MIP parameters makes LFNST more flexible, further improving encoding and decoding efficiency.

Claims

1. An encoding method applied to an encoder, wherein the encoding method is Determining the prediction parameters of the current block, wherein the prediction parameters include prediction mode parameters, When the prediction mode parameter indicates that matrix-based intra prediction (MIP) is used to determine the intra-predicted value of the saturation component of the current block, Determining the adjacent sample values ​​of the current block, The MIP input sample value of the current block is determined based on the adjacent sample values ​​of the current block, The process involves determining the MIP predicted value of the saturation component of the current block based on the MIP input sample values, the MIP weighting matrix, the shift offset parameter, and the shift number parameter, wherein the MIP predicted value is a predicted value of a portion of the samples of the saturation component of the current block, the shift offset parameter has a first fixed value, and the shift number parameter has a second fixed value. By filtering the MIP predicted values, the intra-predicted value of the saturation component of the current block is determined, Based on the intra-predicted value of the saturation component of the current block, the predicted residual value of the saturation component of the current block is determined. The first transformation coefficient is determined by performing a first transformation on the predicted residual value, Based on the decision to perform low-frequency non-separable transformation (LFNST) on at least some of the first transformation coefficients, the LFNST coefficients are determined by performing LFNST on at least some of the first transformation coefficients. Determining the LFNST index value, The process includes encoding the LFNST index and the LFNST coefficient, and writing the encoded LFNST index and the encoded LFNST coefficient to a bitstream. An encoding method characterized by the following.

2. Determining the predicted MIP value of the saturation component of the current block based on the MIP input sample value, the MIP weighting matrix, the shift offset parameter, and the shift number parameter is: The product of the sum of the MIP input sample values ​​and the value of the shift offset parameter is determined, The first offset value is set to the difference between the first constant value and the product thereof, Based on the aforementioned prediction parameters, the MIP weighting matrix for the current block is determined, Calculate the first weighted sum of the MIP weighting matrix and the MIP input sample values, Calculate the first sum of the first weighted sum and the first offset, The first right-shift value is determined by performing a binary right-shift on the first sum, wherein the number of bits shifted to the right is equal to the value of the shift number parameter. The method of setting the MIP prediction value of the saturation component of the current block to the sum of the first right-shifted value and the value corresponding to index 0 in the first temporary reference value, wherein the first temporary reference value is obtained by downsampling the adjacent sample value of the current block, The encoding method according to feature 1.

3. The prediction parameters further include the size parameters of the current block, Determining the MIP input sample value of the current block based on the adjacent sample value of the current block means that Based on the size parameters of the current block, the block size index value of the current block is determined, Obtaining a first temporary reference value by downsampling the adjacent sample values ​​of the current block, When the block size index value of the current block is within a preset range, the value corresponding to index 0 in the MIP input sample value is set to the difference between a second constant value and the value corresponding to index 0 in the first temporary reference value, and the value corresponding to index i in the MIP input sample value is set to the difference between the value corresponding to index i in the first temporary reference value and the value corresponding to index 0 in the first temporary reference value, wherein i is an integer greater than 0. If the block size index value of the current block is outside the preset range, the value corresponding to index j in the MIP input sample value is set to the difference between the value corresponding to index (j+1) in the first temporary reference value and the value corresponding to index 0 in the first temporary reference value, wherein j is a non-negative integer. The encoding method according to feature 1.

4. The second constant value is equal to 1 << (BitDepth - 1), where BitDepth is the bit depth of the saturation component of the current block. The encoding method according to feature 3.

5. The value of the aforementioned shift number parameter is set to 6. The first constant value is set to 32, The value of the shift offset parameter is set to 32. The encoding method according to feature 2.

6. Encoding the LFNST coefficients and writing the encoded LFNST coefficients to a bitstream is, The quantization coefficient is determined by quantizing the LFNST coefficient, The process includes encoding the quantization coefficients and writing the encoded quantization coefficients to the bitstream, The encoding method according to feature 1.

7. Determining the value of the LFNST index is When it is decided to perform the LFNST on at least some of the conversion coefficients of the first conversion coefficient, the value of the LFNST index is set to be greater than 0, This includes setting the value of the LFNST index to 0 when it is determined that the LFNST will not be performed on at least some of the conversion coefficients of the first conversion coefficient, The encoding method according to feature 6.

8. A decoding method applied to a decoder, wherein the decoding method is The bitstream is analyzed to obtain the prediction parameters and low-frequency non-separable transform (LFNST) index of the current block, wherein the prediction parameters include prediction mode parameters. When the prediction mode parameter indicates that matrix-based intra prediction (MIP) is used to determine the intra-predicted value of the saturation component of the current block, Determining the adjacent sample values ​​of the current block, The MIP input sample value of the current block is determined based on the adjacent sample values ​​of the current block, The process involves determining the MIP predicted value of the saturation component of the current block based on the MIP input sample values, the MIP weighting matrix, the shift offset parameter, and the shift number parameter, wherein the MIP predicted value is a predicted value of a portion of the samples of the saturation component of the current block, the shift offset parameter has a first fixed value, and the shift number parameter has a second fixed value. By filtering the MIP predicted values, the intra-predicted value of the saturation component of the current block is determined, The reconstruction transformation coefficients of the current block are determined, When the LFNST index indicates that the LFNST should be performed on the current block, the second transformation coefficient is determined by performing the LFNST on at least some of the reconstruction transformation coefficients. The reconstruction residual value of the saturation component of the current block is determined by performing the first transformation on the second transformation coefficient, This includes determining the reconstructed value of the saturation component of the current block based on the intra-predicted value and the reconstructed residual value of the saturation component of the current block, A decryption method characterized by the following:

9. Determining the predicted MIP value of the saturation component of the current block based on the MIP input sample value, the MIP weighting matrix, the shift offset parameter, and the shift number parameter is: The product of the sum of the MIP input sample values ​​and the value of the shift offset parameter is determined, The first offset value is set to the difference between the first constant value and the product thereof, Based on the aforementioned prediction parameters, the MIP weighting matrix for the current block is determined, Calculate the first weighted sum of the MIP weighting matrix and the MIP input sample values, Calculate the first sum of the first weighted sum and the first offset, The first right-shift value is determined by performing a binary right-shift on the first sum, wherein the number of bits shifted to the right is equal to the value of the shift number parameter. The method of setting the MIP prediction value of the saturation component of the current block to the sum of the first right-shifted value and the value corresponding to index 0 in the first temporary reference value, wherein the first temporary reference value is obtained by downsampling the adjacent sample value of the current block, The decryption method according to feature 8.

10. The prediction parameters further include the size parameters of the current block, Determining the MIP input sample value of the current block based on the adjacent sample value of the current block means that Based on the size parameters of the current block, the block size index value of the current block is determined, Obtaining a first temporary reference value by downsampling the adjacent sample values ​​of the current block, When the block size index value of the current block is within a preset range, the value corresponding to index 0 in the MIP input sample value is set to the difference between a second constant value and the value corresponding to index 0 in the first temporary reference value, and the value corresponding to index i in the MIP input sample value is set to the difference between the value corresponding to index i in the first temporary reference value and the value corresponding to index 0 in the first temporary reference value, wherein i is an integer greater than 0. If the block size index value of the current block is outside the preset range, the value corresponding to index j in the MIP input sample value is set to the difference between the value corresponding to index (j+1) in the first temporary reference value and the value corresponding to index 0 in the first temporary reference value, wherein j is a non-negative integer. The decryption method according to feature 8.

11. The second constant value is equal to 1 << (BitDepth - 1), where BitDepth is the bit depth of the saturation component of the current block. The decryption method according to feature 10.

12. The value of the aforementioned shift number parameter is set to 6. The first constant value is set to 32, The value of the shift offset parameter is set to 32. The decryption method according to feature 9.

13. Analyzing the bitstream to obtain the LFNST index is If the value of the LFNST index is greater than 0, it is decided to perform the LFNST on the current block. This includes deciding not to perform the LFNST on the current block if the value of the LFNST index is equal to 0. The decryption method according to feature 8.

14. Determining the reconstruction transformation coefficient of the current block is: Analyzing the bitstream to obtain the quantization coefficient of the current block, This includes obtaining the reconstruction transformation coefficients of the current block by scaling the quantization coefficients, The decryption method according to feature 8.

15. A decoder including at least one processor and memory, The memory is connected to the at least one processor and stores at least one computer-executable instruction, and when the at least one computer-executable instruction is executed by the at least one processor, it causes the at least one processor to execute the decoding method according to any one of claims 8 to 14. A decoder characterized by the following features.

16. An encoder comprising at least one processor and memory, The memory is connected to the at least one processor and stores at least one computer-executable instruction, and the at least one computer-executable instruction is executed by the at least one processor, causing the at least one processor to execute the encoding method according to any one of claims 1 to 7. An encoder characterized by the following features.

17. A method for transmitting a bitstream, The bitstream is generated by performing the encoding method described in any one of claims 1 to 7, The transmission of the aforementioned bitstream, including, A method for transmitting a bitstream characterized by the following.

Images