Decoding method, encoding method, decoder and encoder

Abstract

The embodiments of the present application provide a decoding method, an encoding method, a decoder, and an encoder. The decoder in the present application predicts a current block based on an optimal MIP mode and a first intra prediction mode. The optimal MIP mode is determined based on a distortion cost corresponding to a plurality of MIP modes and is designed to be an optimal MIP mode for predicting the current block. The first intra prediction mode is determined based on a distortion cost corresponding to a plurality of MIP modes and is designed to include at least one of a sub-optimal MIP mode for predicting the current block, an intra prediction mode derived from a DIMD mode, and an intra prediction mode derived from a TIMD mode. The method according to the present application can improve decompression efficiency and decompression performance.

Description

[Technical field]

[0001] TECHNICAL FIELD The embodiments of the present application relate to the technical field of image video coding, and more particularly to a decoding method, an encoding method, a decoder and an encoder. [Background technology]

[0002] Digital video compression technology is a technology that mainly compresses huge digital image video data for transmission, storage, etc. With the proliferation of Internet videos and people's increasing demands for video clarity, although the existing digital video compression standard can realize video decompression technology, a better digital video decompression technology is needed to improve compression efficiency. Summary of the Invention

[0003] SUMMARY OF THE DISCLOSURE The embodiments of the present application provide a decoding method, an encoding method, a decoder and an encoder, which can improve compression efficiency.

[0004] In a first aspect, the present application provides a decoding method, which includes: analyzing a bitstream to obtain a residual block of a current block in a current sequence; determining an optimal intra prediction (MIP) mode for predicting the current block based on distortion costs corresponding to a plurality of matrix-based MIP modes; the distortion costs corresponding to the plurality of MIP modes include distortion costs obtained by predicting samples in a template region adjacent to the current block using the plurality of MIP modes; determining a first intra prediction mode; the first intra prediction mode is determined based on the distortion costs corresponding to the plurality of MIP modes, and includes at least one of a sub-optimal MIP mode for predicting the current block, an intra prediction mode derived from a decoder-side intra mode derivation (DIMD) mode, and an intra prediction mode derived from a template-based intra mode derivation (TIMD) mode; predicting the current block based on the optimal MIP mode and the first intra prediction mode to obtain a predicted block of the current block; obtaining a reconstructed block of the current block based on the residual block of the current block and the predicted block of the current block.

[0005] In a second aspect, the present application provides an encoding method, which includes: determining an optimal intra prediction (MIP) mode for predicting a current block in a current sequence based on distortion costs corresponding to a plurality of matrix-based MIP modes; the distortion costs corresponding to the plurality of MIP modes include distortion costs obtained by predicting samples in a template region adjacent to the current block using the plurality of MIP modes; determining a first intra prediction mode; the first intra prediction mode is determined based on the distortion costs corresponding to the plurality of MIP modes, and includes at least one of a sub-optimal MIP mode for predicting the current block, an intra prediction mode derived from a decoder-side intra mode derivation (DIMD) mode, and an intra prediction mode derived from a template-based intra mode derivation (TIMD) mode; predicting the current block based on the optimal MIP mode and the first intra prediction mode to obtain a predicted block of the current block; obtaining a residual block of the current block based on the predicted block of the current block and an original block of the current block; encoding the residual block of the current block to obtain a bitstream of the current sequence.

[0006] In a third aspect, the present application provides a decoder, the decoder comprising: an analysis unit, a prediction unit, and a reconstruction unit. The analysis unit is configured to analyze the bitstream to obtain a residual block of a current block in a current sequence. The prediction unit is configured to determine an optimal intra prediction (MIP) mode for predicting the current block based on distortion costs corresponding to a plurality of matrix-based MIP modes, the distortion costs corresponding to the plurality of MIP modes including distortion costs obtained by predicting samples in a template region neighboring the current block using the plurality of MIP modes. The prediction unit is configured to determine a first intra prediction mode, the first intra prediction mode is determined based on the distortion costs corresponding to the plurality of MIP modes, and includes at least one of a sub-optimal MIP mode for predicting the current block, an intra prediction mode derived from a decoder-side intra mode derivation (DIMD) mode, and an intra prediction mode derived from a template-based intra mode derivation (TIMD) mode. The prediction unit is configured to predict the current block based on the optimal MIP mode and the first intra prediction mode to obtain a prediction block of the current block. The reconstruction unit is configured to obtain a reconstructed block of the current block based on the residual block of the current block and the prediction block of the current block.

[0007] In a fourth aspect, the present application provides an encoder, the encoder comprising: a prediction unit, a residual unit, and an encoding unit. The prediction unit is configured to determine an optimal intra-prediction (MIP) mode for predicting a current block in a current sequence based on distortion costs corresponding to a plurality of matrix-based MIP modes, the distortion costs corresponding to the plurality of MIP modes including distortion costs obtained by predicting samples in a template region neighboring the current block using the plurality of MIP modes. The prediction unit is configured to determine a first intra-prediction mode, the first intra-prediction mode is determined based on the distortion costs corresponding to the plurality of MIP modes, and includes at least one of a sub-optimal MIP mode for predicting the current block, an intra-prediction mode derived from a decoder-side intra-mode derivation (DIMD) mode, and an intra-prediction mode derived from a template-based intra-mode derivation (TIMD) mode. The prediction unit is configured to predict the current block based on the optimal MIP mode and the first intra-prediction mode to obtain a prediction block of the current block. The residual unit is configured to obtain a residual block of a current block based on the predicted block of the current block and the original block of the current block, and the encoding unit is configured to encode the residual block of the current block to obtain a bitstream of a current sequence.

[0008] In a fifth aspect, the present application provides a decoder. The decoder comprises a processor and a computer readable storage medium. The processor is configured to execute computer instructions. The computer readable storage medium stores computer instructions, which are loaded by the processor to execute the decoding method of the first aspect or each embodiment thereof. In an embodiment, the processor is one or more, and the memory is one or more. In an embodiment, the computer readable storage medium may be integrated with the processor, or the computer readable storage medium may be located separately from the processor.

[0009] In a sixth aspect, the present application provides an encoder. The encoder comprises a processor and a computer readable storage medium. The processor is configured to execute computer instructions. The computer readable storage medium stores computer instructions, which are loaded by the processor to execute the encoding method of the second aspect or any embodiment thereof. In an embodiment, the processor is one or more, and the memory is one or more. In an embodiment, the computer readable storage medium may be integrated with the processor, or the computer readable storage medium may be located separately from the processor.

[0010] In a seventh aspect, the present application provides a computer-readable storage medium configured to store computer instructions which, when read and executed by a processor of a computing device, cause the computing device to perform the decoding method according to the first aspect or the encoding method according to the second aspect.

[0011] In an eighth aspect, the present application provides a bitstream, the bitstream being a bitstream according to the first aspect or a bitstream according to the second aspect.

[0012] Based on the above technical proposal, the decoder predicts the current block according to an optimal MIP mode and a first intra prediction mode. The optimal MIP mode is determined based on the distortion costs corresponding to a plurality of MIP modes and is designed to be an optimal MIP mode for predicting the current block. The first intra prediction mode is determined based on the distortion costs corresponding to a plurality of MIP modes and is designed to include at least one of a sub-optimal MIP mode for predicting the current block, an intra prediction mode derived from a DIMD mode, and an intra prediction mode derived from a TIMD mode. It helps the decoder to avoid obtaining the MIP mode by parsing the bitstream. Compared with the conventional MIP technology, the present application can effectively reduce the bit overhead at the coding unit level, thereby improving the decompression efficiency.

[0013] In addition, by combining the optimal MIP mode with the first intra prediction mode, it is possible to avoid completely replacing the optimal prediction mode calculated based on the rate-distortion cost with the optimal MIP mode, thereby achieving both prediction accuracy and prediction diversity, and further improving decompression performance. [Brief description of the drawings]

[0014] [Figure 1] FIG. 1 is a block diagram illustrating an encoding framework according to an embodiment of the present application. [Diagram 2] FIG. 2 is a schematic diagram illustrating a MIP mode according to an embodiment of the present application. [Diagram 3] FIG. 3 is a schematic diagram illustrating deriving a prediction mode based on DIDM according to an embodiment of the present application. [Figure 4] FIG. 4 is a schematic diagram illustrating deriving a prediction block based on DIMD according to an embodiment of the present application. [Diagram 5] FIG. 5 is a schematic diagram illustrating a template used in a TIMD according to an embodiment of the present application. [Figure 6] FIG. 6 is a block diagram illustrating a decoding framework according to an embodiment of the present application. [Figure 7] FIG. 7 is a flow chart illustrating a decoding method according to an embodiment of the present application. [Figure 8] FIG. 8 is a flow chart illustrating an encoding method according to an embodiment of the present application. [Figure 9] FIG. 9 is a block diagram illustrating a decoder according to an embodiment of the present application. [Figure 10] FIG. 10 is a block diagram illustrating an encoder according to an embodiment of the present application. [Figure 11] FIG. 11 is a block diagram showing an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0015] Hereinafter, technical solutions in the embodiments of the present application will be described with reference to the drawings.

[0016] The solution according to the embodiment of the present application may be applied to the technical field of digital video coding, for example, including but not limited to image coding, video coding, hardware video coding, dedicated circuit video coding, and real-time video coding. The solution according to the embodiment of the present application may be combined with the audio video coding standard (AVS), the second generation AVS standard (AVS2), or the third generation AVS standard (AVS3). Examples include, but are not limited to, the H.264 / audio video coding (AVC) standard, the H.265 / high efficiency video coding (HEVC) standard, and the H.266 / versatile video coding (VVC) standard. The solution according to the embodiment of the present application may perform lossy compression or lossless compression on an image. The lossless compression may be visually lossless compression or mathematically lossless compression.

[0017] A mixed coding framework based on blocks is used in video coding standards. Each image (frame) in a video is divided into largest coding units (LCUs) or coding tree units (CTUs) of the same size (e.g., 128x128, 64x64, etc.). Each largest coding unit or coding tree unit can also be divided into rectangular coding units (CUs) based on rules. A coding unit can be further divided into prediction units (PUs), transform units (TUs), etc. The mixed coding framework includes modules such as prediction, transform, quantization, entropy coding, and loop filter. The prediction module includes intra prediction and inter prediction. The inter prediction includes motion estimation and motion compensation. Since there is a strong correlation between adjacent samples in the image of a video, the method of intra prediction is used in the video coding technology to eliminate spatial redundancy between adjacent samples. Intra prediction refers only to the information of the image of the same frame to predict the sample information in the current divided block. Since there is a strong similarity between adjacent images in a video, the method of inter prediction can be used in the video coding technology to eliminate temporal redundancy between adjacent images and improve coding efficiency. In inter prediction, by referring to the image information of different frames, motion estimation can be used to search for the motion vector information that is most matched to the current divided block. The predicted image block is transformed into the frequency domain and energy is redistributed. The combination of transformation and quantization can remove information that is not sensitive to the human eye, and is used to eliminate visual redundancy.Entropy coding allows removing character redundancies based on the current context model and probability information of the binary bitstream.

[0018] In the process of digital video encoding, an encoder can first read a black-and-white image or a color image from an original video sequence, and then encode the black-and-white image or the color image. Here, the black-and-white image can include samples of a luma component, and the color image can include samples of a chroma component. Optionally, the color image can further include samples of the luma component. The color format of the original video sequence can be a luma-chroma (YCbCr, YUV) format or a red-green-blue (RGB) format, etc. In particular, after reading a black-and-white image or a color image, an encoder divides it into blocks, performs intra prediction or inter prediction on a current block to generate a prediction block of the current block, subtracts the prediction block from an original block of the current block to obtain a residual block, transforms and quantizes the residual block to obtain a quantization coefficient matrix, and entropy codes the quantization coefficient matrix to output it to a bitstream. In the process of digital video decoding, a decoding side generates a prediction block of the current block by performing intra prediction or inter prediction on the current block. In addition, the decoding side analyzes the bitstream to obtain a quantization coefficient matrix, inverse quantizes and inverse transforms the quantization coefficient matrix to obtain a residual block, and adds the prediction block and the residual block to obtain a reconstructed block. The reconstructed block forms a reconstructed image. The decoding side loop filters the reconstructed image based on the image or block to obtain a decoded image.

[0019] The current block may be a current coding unit (CU) or a current prediction unit (PU), for example.

[0020] In addition, the encoding side also needs to perform a similar process to the decoding side to obtain a decoded image. The decoded image can be a reference image for inter-prediction of a subsequent image. The block partition information, prediction, transform, quantization, entropy coding, loop filtering, and other mode or parameter information determined by the encoding side are output to a bitstream as necessary. The decoding side analyzes and parses existing information to determine the same block partition information, prediction, transform, quantization, entropy coding, loop filtering, and other mode or parameter information as the encoding side. This ensures that the decoded image obtained by the encoding side is the same as the decoded image obtained by the decoding side. The decoded image obtained by the encoding side is also usually called a reconstructed image. During prediction, the current block may be divided into prediction units, and during transformation, the current block may be divided into transform units, and the division of the prediction units and transform units may be the same or different. The above is the basic flow of video coding in a block-based mixed coding framework. With the development of technology, some modules or steps in the framework may be optimized. This application applies to the basic flow of video codecs in this block-based mixed coding framework.

[0021] For ease of understanding, we first provide a brief overview of the encoding framework of this application.

[0022] FIG. 1 is a block diagram illustrating an encoding framework 100 according to an embodiment of the present application.

[0023] As shown in Fig. 1, the encoding framework 100 may include an intra prediction unit 180, an inter prediction unit 170, a residual unit 110, a transform and quantization unit 120, an entropy encoding unit 130, an inverse transform and inverse quantization unit 140, and a loop filtering unit 150. Optionally, the encoding framework 100 may further include a decoding image buffer unit 160. The encoding framework 100 is also referred to as a mixed framework encoding mode.

[0024] The intra prediction unit 180 or the inter prediction unit 170 may predict an image block to be coded and output a prediction block. The residual unit 110 may calculate a residual block, i.e., a difference between the prediction block and the image block to be coded, based on the prediction block and the image block to be coded. The transform and quantization unit 120 is used to perform operations such as transform and quantization on the residual block, thereby removing information that is not sensitive to human eyes and eliminating visual redundancy. Alternatively, the residual block before being transformed and quantized by the transform and quantization unit 120 may be referred to as a time-domain residual block, and the time-domain residual block after being transformed and quantized by the transform and quantization unit 120 may be referred to as a frequency residual block or a frequency-domain residual block. After receiving the quantized transform coefficients output by the transform and quantization unit 120, the entropy encoding unit 130 may output a bitstream based on the quantized transform coefficients. For example, the entropy encoding unit 130 may remove character redundancy based on a target context model and probability information of a binary bitstream. For example, the entropy encoding unit 130 may be used for context-based adaptive binary arithmetic coding (CABAC). The entropy encoding unit 130 may also be referred to as a header information encoding unit. Optionally, in this application, the image block to be coded may also be referred to as an original image block or a target image block. The prediction block may also be referred to as a predicted image block or an image prediction block, and may also be referred to as a prediction signal or prediction information.The reconstruction block may be called a reconstruction image block or an image reconstruction block, or may be called a reconstruction signal or reconstruction information. Furthermore, on the encoding side, the image block to be coded may be called an encoding block or an encoding image block. On the decoding side, the image block to be coded may be called a decoding block or a decoding image block. The image block to be coded may be a CTU or a CU.

[0025] In the encoding framework 100, a difference between a prediction block and an image block to be coded is calculated to obtain a residual block, and processes such as transform and quantization are performed on the residual block, and the residual block is transmitted to a decoding side. Accordingly, the decoding side receives and analyzes a bitstream, obtains a residual block through steps such as inverse transform and inverse quantization, and obtains a reconstructed block by adding the residual block to a prediction block obtained by prediction by the decoding side.

[0026] In addition, the inverse transform and inverse quantization unit 140, the loop filtering unit 150, and the decoding image buffer unit 160 in the encoding framework 100 can be used to form a decoder. The intra prediction unit 180 or the inter prediction unit 170 can predict an image block to be coded based on an existing reconstructed block, thereby ensuring that the encoding side uses a reference frame in the same manner as the decoding side. In other words, the encoder can replicate the processing loop of the decoder, thereby generating the same prediction as the decoding side. Specifically, the quantized transform coefficients are inverse transformed and inverse quantized by the inverse transform and inverse quantization unit 140 to replicate an approximate residual block of the decoding side. After the prediction block is added to the approximate residual block, the loop filtering unit 150 can smooth and filter the effects of block-based processing and blocking artifacts caused by quantization. The image blocks output from the loop filtering unit 150 may be stored in a decoding image buffer unit 160 for use in subsequent image prediction.

[0027] It should be understood that FIG. 1 is merely an example of the present application and should not be construed as a limitation of the present application.

[0028] For example, the loop filtering unit 150 in the encoding framework 100 may include a deblocking filter (DBF) and a sample adaptive offset (SAO). The role of the DBF is to deblock artifacts, and the role of the SAO is to remove ringing effects. In another embodiment of the present application, a neural-network-based loop filtering algorithm may be used in the encoding framework 100 to improve video compression efficiency. Or, the encoding framework 100 may be a deep learning neural network-based video coding mixed framework. In one embodiment, a convolutional neural network-based model may be used to calculate the result after sample filtering based on the deblocking filter and SAO. The network structure in the luma component of the loop filtering unit 150 and the network structure in the chroma component may be the same or different. Considering that the luma component contains more visual information, the luma component may be used to guide the filtering of the chroma component to improve the reconstruction quality of the chroma component.

[0029] The following describes the contents related to intra prediction.

[0030] In intra prediction, sample information in an image block waiting to be coded is predicted by referring only to information of the same image in order to eliminate spatial redundancy. The image used in intra prediction may be an I frame. For example, according to the coding order from left to right and top to bottom, the image block at the top left, the image block above, and the image block at the left can be used as reference information to predict the image block waiting to be coded. The image block waiting to be coded is also used as reference information for the next image block. In this way, the entire image can be predicted. If the input digital video is a color format such as YUV4:2:0 format, each of the four pixels of each image frame of the digital video consists of four Y components and two UV components. In the encoding framework, the Y component (i.e., the luma block) and the UV component (i.e., the chroma block) can be encoded respectively. Similarly, the decoding side can decode according to the format.

[0031] Other intra prediction modes according to the present application will be introduced below.

[0032] (1) Matrix based Intra Prediction (MIP) mode

[0033] The process related to the MIP model can be divided into three main steps: downsampling process, matrix multiplication process, and upsampling process. Specifically, first, the downsampling process downsamples spatially adjacent reconstructed samples. Then, the obtained downsampled sample sequence is used as the input vector of the matrix multiplication process, i.e., the output vector of the downsampling process is used as the input vector of the matrix multiplication process, the input vector of the matrix multiplication process is multiplied by a preset matrix, and the result is added with a bias vector, and the calculated sample vector is output. Finally, the output vector of the matrix multiplication process is used as the input vector of the upsampling process, and the final prediction block is obtained by upsampling.

[0034] FIG. 2 is a schematic diagram illustrating a MIP mode according to an embodiment of the present application.

[0035] JPEG2025511314000002.jpg69151

[0036] In other words, to predict a block with width W and height H, MIP requires H reconstructed samples in the left column of the current block and W reconstructed samples in the row above the current block as input. MIP generates a predicted block based on three steps: averaging of reference samples, matrix vector multiplication, and interpolation. The core of MIP is matrix vector multiplication, which can be considered as a process of generating a predicted block using input samples (reference samples) in a matrix vector multiplication manner. Various matrices are provided in MIP, and differences in prediction methods can be reflected in differences in matrices, and different results can be obtained when different matrices are used for the same input sample. In addition, the process of averaging and interpolating reference samples is designed with a trade-off between performance and complexity. For blocks with a large size, the averaging of reference samples can achieve an effect that is close to downsampling, so that the input can be adapted to a relatively small matrix. The effect of upsampling can be achieved by interpolation. In this way, it is no longer necessary to provide a MIP matrix for each size of block, and instead, only one or several matrices having a specific size need to be provided. As compression performance needs increase and hardware capabilities improve, more complex MIPs are likely to appear in future standards.

[0037] In the MIP mode, the MIP mode can be simplified from the neural network, for example, the matrix used in the MIP mode can be obtained based on training. Therefore, the MIP mode has a relatively strong generalization ability and a prediction effect that cannot be reached by conventional prediction modes. The MIP mode is a model obtained by performing multiple simplifications of hardware and software complexity for the intra prediction model based on the neural network. Based on a large number of training samples, multiple prediction modes can represent multiple models and parameters and better cover the texture of natural sequences.

[0038] MIP mode is a bit similar to Planar mode, but obviously, MIP mode is more complex and flexible than Planar mode.

[0039] It should be noted that for coding units of different block sizes, the number of MIP modes can be different. For example, for a coding unit of 4×4 size, the MIP mode has 16 prediction modes. For a coding unit of 8×8 size, a coding unit with a width equal to 4 or a height equal to 4, the MIP mode has 8 prediction modes. For coding units of other sizes, the MIP mode has 6 prediction modes. In addition, the MIP mode has a transpose function. For a prediction mode matching the current size, the MIP mode can try a transpose calculation on the encoder side. Therefore, the MIP mode requires a flag indicating whether the MIP mode is used for the current coding unit. In addition, if the MIP mode is used for the current coding unit, a transpose flag needs to be further transmitted to the decoder.

[0040] The MIP transposition flag is binarized by the Fixed Length (FL) coding method and has a length of 1. The MIP mode index is binarized by the Truncated Binary (TB) coding method.

[0041] (2) Decoder side Intra Mode Derivation (DIMD) prediction mode

[0042] The main point of the DIMD prediction mode is to derive the intra prediction mode at the decoder side using the same method as the encoder, thereby avoiding transmitting the intra prediction mode index of the current coding unit in the bitstream and achieving the purpose of saving bit overhead.

[0043] The specific process of the DIDM prediction mode can be mainly divided into the following two steps.

[0044] Step 1: Derive the prediction mode.

[0045] The encoding side uses the Sobel operator to calculate the histogram of gradients for each prediction mode. The region of action is the reconstructed samples of the three adjacent rows above the current block, the reconstructed samples of the three adjacent columns to the left, and the corresponding adjacent reconstructed samples to the top left. By calculating the gradient histogram within the aforementioned L-shaped region, we can obtain the prediction mode corresponding to the largest amplitude (also called magnitude) in the gradient histogram and the prediction mode corresponding to the second largest amplitude.

[0046] FIG. 3 is a schematic diagram illustrating deriving a prediction mode based on DIDM according to an embodiment of the present application.

[0047] As shown in (a) of FIG. 3, in DIMD, a prediction mode is derived using samples in a template in a reconstruction region (reconstructed samples on the left and above the current block). For example, the template may include reconstructed samples of three adjacent rows above the current block, reconstructed samples of three adjacent columns to the left, and the corresponding adjacent reconstructed sample on the top left. Based on this, multiple gradient values ​​are determined in the template according to a window (e.g., the window shown in (a) of FIG. 3 or (b) of FIG. 3). Each gradient value is used to obtain one intra prediction mode (ipm) that matches its gradient direction. Based on this, the encoder may determine the prediction mode corresponding to the largest gradient value and the prediction mode corresponding to the second largest gradient value among the multiple gradient values ​​as the derived prediction modes. For example, as shown in (b) of FIG. 3, for a block of size 4×4, all samples that need to determine gradient values ​​are analyzed to obtain corresponding gradient histograms. For example, as shown in (c) of FIG. 3, for blocks of other sizes, all samples that need to determine gradient values ​​are analyzed to obtain corresponding gradient histograms. Finally, the prediction mode corresponding to the largest gradient in the gradient histogram and the prediction mode corresponding to the second largest gradient are determined as the derived prediction modes.

[0048] Of course, the gradient histogram in this application is merely an example for determining the derived prediction mode, and may be implemented in various simple forms when specifically implemented, and this application is not particularly limited thereto. Furthermore, this application does not limit the method for obtaining the gradient histogram, and for example, the gradient histogram can be obtained by the Sobel operator or other methods.

[0049] Step 2: Derive the prediction block.

[0050] FIG. 4 is a schematic diagram illustrating deriving a prediction block based on DIMD according to an embodiment of the present application.

[0051] As shown in Figure 4, the encoder can weight the prediction values ​​corresponding to the three intra prediction modes (planar mode and two intra prediction modes derived based on DIMD). The codec uses the same prediction block derivation method to obtain the prediction block of the current block. Assuming that the prediction mode corresponding to the largest gradient value is prediction mode 1 and the prediction mode corresponding to the second largest gradient value is prediction mode 2, the encoder determines the following two conditions: 1. The gradient value of prediction mode 2 is not 0. 2. Neither prediction mode 1 nor prediction mode 2 is a planar mode or a direct current (DC) prediction mode.

[0052] If the above two conditions are not met at the same time, only prediction mode 1 is used to calculate the predicted sample value of the current block, that is, the normal prediction process is applied to prediction mode 1. Otherwise, that is, if the above two conditions are met at the same time, a weighted average method is used to derive the predicted block of the current block. The specific method is as follows: the planar mode accounts for 1 / 3 of the weight, and the remaining 2 / 3 is the weight of the sum of prediction mode 1 and prediction mode 2. For example, the gradient amplitude value of prediction mode 1 is divided by the sum of the gradient amplitude value of prediction mode 1 and the gradient amplitude value of prediction mode 2, and the result is the weight of prediction mode 1, and the gradient amplitude value of prediction mode 2 is divided by the sum of the gradient amplitude value of prediction mode 1 and the gradient amplitude value of prediction mode 2, and the result is the weight of prediction mode 2. Finally, the predicted blocks obtained based on the above three prediction modes, i.e., the predicted block 1, the predicted block 2, and the predicted block 3 obtained based on the planar mode, the prediction mode 1, and the prediction mode 2, respectively, are weighted averaged to obtain the predicted block of the current coding unit. The decoder also obtains the predicted block in the same steps.

[0053] In other words, the specific weights in step 2 above are calculated as follows: Weight(PLANAR) = 1 / 3, Weight(mode1) = 2 / 3 * (amp1 / (amp1+amp2)), Weight(mode2) = 1 - Weight(PLANAR) - Weight(mode1). mode1 and mode2 respectively represent prediction mode 1 and prediction mode 2, and amp1 and amp2 respectively represent the gradient amplitude value of prediction mode 1 and the gradient amplitude value of prediction mode 2. In the DIMD prediction mode, a flag needs to be transmitted to the decoder, which is used to indicate whether the DIDM prediction mode is used for the current coder unit.

[0054] Of course, the above weighted average method is merely an example of the present application and should not be understood as a limitation of the present application.

[0055] In summary, DIMD allows to select an intra prediction mode using gradient analysis of the reconstructed samples and weight the two intra prediction modes and the planar mode according to the analysis results. The advantage of DIMD is that if a DIMD mode is selected for the current block, there is no need to specifically indicate in the bitstream which intra prediction mode is used, but it is derived by the decoder itself through the above process, thus saving some overhead.

[0056] (3) Template based Intra Mode Derivation (TIMD) prediction mode

[0057] In the TIMD prediction mode, the codec uses the same operation to derive the prediction mode, thereby saving the overhead of transmitting the mode index. The TIMD prediction mode can be understood mainly in two parts. First, the cost information of each prediction mode is calculated based on the template, and the prediction mode corresponding to the smallest cost and the prediction mode corresponding to the second smallest cost are selected. The prediction mode corresponding to the smallest cost is denoted as prediction mode 1, and the prediction mode corresponding to the second smallest cost is denoted as prediction mode 2. If the ratio between the second smallest cost (costMode2) and the smallest cost (costMode1) satisfies a preset condition, such as costMode2<2*costMode1, the prediction block corresponding to prediction mode 1 and the prediction block corresponding to prediction mode 2 are weighted fused according to the weight corresponding to prediction mode 1 and the weight corresponding to prediction mode 2 to obtain the final prediction block.

[0058] Exemplarily, the weight corresponding to prediction mode 1 and the weight corresponding to prediction mode 2 are determined according to the following manner. weight1 = costMode2 / (costMode1+ costMode2), weight2 = 1 - weight1. weight1 is the weight of the prediction block corresponding to prediction mode 1, and weight2 is the weight of the prediction block corresponding to prediction mode 2. If the ratio between the second smallest cost costMode2 and the smallest cost costMode1 does not satisfy a preset condition, weighted fusion between prediction blocks is not performed, and the prediction block corresponding to prediction mode 1 becomes the prediction block of TIMD.

[0059] In addition, when performing intra prediction on a current block using the TIMD prediction mode, if the reconstructed sample template of the current block does not include available neighboring reconstructed samples, the TIMD prediction mode selects the planar mode to perform intra prediction on the current block, i.e., no weighted fusion is performed. Similar to the DIMD prediction mode, the TIMD prediction mode requires a flag to be transmitted to the decoder, which is used to indicate whether the TIMD prediction mode is used for the current coder unit.

[0060] The process of the encoder or decoder calculating the cost information of each prediction mode is mainly as follows: Based on the reconstructed sample adjacent to the top side and the reconstructed sample adjacent to the left side of the template region, intra-mode prediction is performed on the samples in the template region, and the prediction process is the same as the original intra-prediction mode. For example, when intra-mode prediction is performed on the samples in the template region using DC mode, the average value of the entire coding unit is calculated. As another example, when intra-mode prediction is performed on the samples in the template region using an angle prediction mode, a corresponding interpolation filter is selected according to the mode, and a predicted sample is obtained by interpolation according to a rule. In this case, based on the predicted sample and the reconstructed sample in the template region, the distortion between the predicted sample and the reconstructed sample in the region, i.e., the cost information of the current prediction mode, can be calculated.

[0061] FIG. 5 is a schematic diagram illustrating a template used in a TIMD according to an embodiment of the present application.

[0062] As shown in FIG. 5, if the current block is a coding unit with a width equal to M and a height equal to N, the codec may select reconstructed samples in a reference template of the current block from a coding unit with a width equal to 2(M+L1)+1 and a height equal to 2(N+L2)+1 to predict samples in the template region of the current block. Of course, if the template region of the current block does not include available neighboring reconstructed samples, the TIMD prediction mode selects a planar mode to perform intra prediction on the current block. For example, the available neighboring reconstructed samples may be samples adjacent to the left side and the upper side of the current CU in FIG. 5, i.e., there are no available reconstructed samples in the shaded region. That is, if there are no available reconstructed samples in the shaded region, the TIMD prediction mode selects a planar mode to perform intra prediction on the current block.

[0063] It should be noted that, except for boundary cases, when coding the current block, theoretically, the reconstructed values ​​can be obtained from the left and top of the current block, i.e., the template of the current block includes the available neighboring reconstructed samples. In a specific embodiment, the decoder predicts the template using an intra prediction mode, and compares the predicted value with the reconstructed value to obtain the cost of the intra prediction mode on the template. For example, Sum of Absolute Differences (SAD), Sum of Absolute Transformed Difference (SATD), Sum of Squared Error (SSE), etc. The template and the current block are adjacent to each other, so the reconstructed samples in the template and the samples in the current block are correlated. Therefore, the behavior of the prediction mode on the template can be used to estimate the behavior of this prediction mode on the current block. In TIMD, several candidate intra prediction modes are used to predict the template to obtain the costs of the candidate intra prediction modes on the template, and the predicted values ​​of one or two intra prediction modes with the lowest cost are taken as the intra prediction values ​​of the current block. If the difference between the two costs corresponding to the two intra prediction modes on the template is not large, the predicted values ​​of the two intra prediction modes can be weighted averaged to improve compression performance. Optionally, the weights of the predicted values ​​of the two prediction modes are related to the costs, for example, the weights are inversely proportional to the costs.

[0064] In summary, in TIMD, it is possible to use the prediction effect of the intra prediction mode on the template to select the intra prediction mode, and weight the two intra prediction modes according to the cost on the template. The advantage of TIMD is that when the TIMD mode is selected for the current block, it is not necessary to specifically indicate which intra prediction mode is used in the bitstream, but is derived by the decoder itself through the above process, which can save some overhead.

[0065] Through the brief introduction of the above several intra prediction modes, it can be seen that the technical principles of the DIMD prediction mode and the TIMD prediction mode are similar. Both of them utilize the decoder performing the same operation as the encoder to estimate the prediction mode of the current coding unit. In such prediction modes, if the complexity is acceptable, the transmission of the prediction mode index can be omitted, saving overhead and improving compression efficiency. However, due to the limited information that can be referenced and the lack of ability to improve prediction quality by itself, the DIMD prediction mode and the TIMD prediction mode are more effective in large areas with consistent texture characteristics. When the texture changes slightly or the template area cannot be covered, the prediction effect of such prediction modes is poor.

[0066] In addition, in both the DIMD prediction mode and the TIMD prediction mode, prediction blocks obtained based on multiple conventional prediction modes are fused or weighted. The fusion of prediction blocks can produce effects that cannot be achieved by a single prediction mode. In the DIMD prediction mode, the planar mode can be introduced as an additional weighted prediction mode to enhance the spatial correlation between adjacent reconstruction samples and prediction samples, thereby improving the prediction effect of intra prediction. However, since the prediction principle of the planar mode is relatively simple, it may have an adverse effect to use the planar mode as an additional weighted prediction mode for a prediction block with obvious differences between the upper right corner and the lower left corner.

[0067] FIG. 6 is a block diagram illustrating a decoding framework 200 according to an embodiment of the present application.

[0068] As shown in FIG. 6, the decoding framework 200 may include an entropy decoding unit 210, an inverse transform and inverse quantization unit 220, a residual unit 230, an intra prediction unit 240, an inter prediction unit 250, a loop filtering unit 260, and a decoding image buffer unit 270.

[0069] The entropy decoding unit 210 receives and analyzes the bitstream to obtain a prediction block and a frequency domain residual block. The frequency domain residual block is subjected to steps such as inverse transformation and inverse quantization through the inverse transformation and inverse quantization unit 220 to obtain a time domain residual block. The residual unit 230 adds the prediction block obtained by prediction through the intra prediction unit 240 or the inter prediction unit 250 to the time domain residual block obtained after inverse transformation and inverse quantization through the inverse transformation and inverse quantization unit 220 to obtain a reconstructed block. For example, the intra prediction unit 240 or the inter prediction unit 250 can obtain a prediction block by decoding header information of the bitstream.

[0070] 7 is a flow chart illustrating a decoding method 300 according to an embodiment of the present application. The decoding method 300 may be implemented by a decoder. For example, the decoding method 300 may be applied to the decoding framework 200 shown in FIG. 6. For ease of explanation, the following description will be given taking a decoder as an example.

[0071] As shown in FIG. 7, the decoding method 300 may include some or all of the following: S310: Parse the bitstream to obtain a residual block of a current block in a current sequence. S320: Determine an optimal intra-prediction (MIP) mode for predicting the current block based on distortion costs corresponding to a plurality of matrix-based MIP modes, including distortion costs obtained by predicting samples in a template region adjacent to the current block using the plurality of MIP modes. S330: Determine a first intra prediction mode, where the first intra prediction mode is determined based on distortion costs corresponding to a plurality of MIP modes, and includes at least one of a sub-optimal MIP mode for predicting a current block, an intra prediction mode derived from a decoder-side intra mode derivation (DIMD) mode, and an intra prediction mode derived from a template-based intra mode derivation (TIMD) mode. S340: Predict the current block based on the optimal MIP mode and the first intra prediction mode to obtain a predicted block of the current block. S350: Obtain a reconstructed block of the current block based on the residual block of the current block and the predicted block of the current block.

[0072] Exemplarily, in the present application, S320 to S340 (i.e., a process in which a decoder determines an optimal MIP mode based on distortion costs corresponding to a plurality of MIP modes, and predicts a current block based on the optimal MIP mode and a first intra-prediction mode) may be abbreviated as a template matching MIP (TMMIP) technology, a TMMIP-based prediction mode derivation method, or a TMMIP fusion enhancement technology. That is, after obtaining a residual block of a current block, a decoder can enhance performance for a prediction process of the current block based on the derived optimal MIP mode and the first intra-prediction mode. In other words, in the TMMIP technology, performance can be enhanced for a prediction process of a current block by using at least one of a sub-optimal MIP prediction mode, an intra-prediction mode derived from a TIMD mode, and an intra-prediction mode derived from a DIMD mode, and the optimal MIP prediction mode.

[0073] In this embodiment, the decoder predicts the current block based on an optimal MIP mode and a first intra prediction mode. The optimal MIP mode is determined based on a distortion cost corresponding to a plurality of MIP modes and is designed to be an optimal MIP mode for predicting the current block. The first intra prediction mode is determined based on a distortion cost corresponding to a plurality of MIP modes and is designed to include at least one of a sub-optimal MIP mode for predicting the current block, an intra prediction mode derived from a DIMD mode, and an intra prediction mode derived from a TIMD mode. It helps the decoder to avoid obtaining the MIP mode by parsing the bitstream. Compared with the conventional MIP technology, the present application can effectively reduce the bit overhead at the coding unit level, thereby improving the decompression efficiency.

[0074] In other words, the bit overhead of the conventional MIP technique is more than that of other intra prediction tools, and not only a flag indicating whether the MIP technique is used, but also a flag indicating whether the MIP is transposed is required, and finally, the largest overhead is that truncated binary coding must be used to represent the MIP prediction mode. The MIP technique is a simplified technique based on neural network technology, and is significantly different from conventional interpolation filtering prediction techniques. For some special textures, the MIP prediction mode is more effective than the conventional intra prediction mode, but the large flag overhead is a defect of the MIP technique. For example, a coding unit of 4×4 size has a total of 16 prediction samples, but its bit overhead includes one MIP use flag, one MIP transposition flag, and a 5-bit or 6-bit truncated binary flag. In view of this, in this application, by using a method in which the decoder independently determines an optimal MIP mode for predicting a current block, and determines the intra prediction mode of the current block based on the optimal MIP mode, it is possible to save up to 5 or 6 bits of overhead, effectively reduce bit overhead at the coding unit level, and thereby improve decompression efficiency.

[0075] Also, saving up to 5 or 6 bits of overhead per block presupposes that the template matching-based prediction mode derivation algorithm must be very accurate. If the MIP prediction mode calculated by the template matching-based fast algorithm is different from the MIP prediction mode calculated by the encoding side using the rate-distortion optimized and original sequence, it may not be the optimal choice. Therefore, the selection of the template matching-based MIP prediction mode depends on the degree of matching accuracy, i.e., the accuracy. However, both the conventional template-based derivation algorithm in the intra prediction mode and the template matching-based derivation algorithm in the inter prediction mode have not reached the expectations in accuracy, and although the performance has been significantly improved, the additional large amount of bit overhead at the block level brought by the template matching-based algorithm cannot allow subsequent technologies to obtain higher performance by relying only on the template matching-based algorithm. In view of this, in the present application, by fusing the optimal MIP mode and the first intra prediction mode, i.e., by performing fusion prediction for the current block based on the optimal MIP mode and the first intra prediction mode, it is possible to avoid completely replacing the optimal prediction mode calculated based on the rate-distortion cost with the optimal MIP mode, thereby achieving both prediction accuracy and prediction diversity, and further improving decompression performance.

[0076] In addition, the distortion cost of the decoder in this application is different from the rate-distortion cost (RDcost) of the encoder. The rate-distortion cost is a distortion cost used when the encoding side determines a specific intra-prediction technique from multiple intra-prediction techniques, and the rate-distortion cost can be a cost obtained by comparing a distorted image with an original image. Since the decoder cannot obtain the original image, the distortion cost of the decoder can be a distortion cost between a reconstructed sample and a predicted sample, such as a SATD cost between a reconstructed sample and a predicted sample, or a cost that can be used to calculate a difference between a reconstructed sample and a predicted sample.

[0077] The beneficial effects of the scheme of the present application are described below with reference to the test results in Tables 1 and 2. Table 1 shows the results obtained by testing the scheme of the present application when the first intra prediction mode is determined based on the distortion costs corresponding to multiple MIP modes and is designed to be a sub-optimal MIP mode for predicting the current block. Table 2 shows the results obtained by testing the scheme of the present application when the first intra prediction mode is designed to be an intra prediction mode derived from a TIMD mode.

[0078] [Table 1]

[0079] [Table 2]

[0080] JPEG2025511314000005.jpg82152

[0081] In some embodiments, S320 may include: Analyzing a bitstream of a current sequence to obtain a first flag, and if the first flag is used to indicate that an optimal MIP mode and a first intra-prediction mode are allowed to be used to predict an image block in the current sequence, determining an optimal MIP mode based on distortion costs corresponding to the multiple MIP modes.

[0082] Exemplarily, when the value of the first flag is a first value, the first flag is used to indicate that the image block in the current sequence is allowed to be predicted using the optimal MIP mode and the first intra-prediction mode. When the value of the first flag is a second value, the first flag is used to indicate that the image block in the current sequence is not allowed to be predicted using the optimal MIP mode and the first intra-prediction mode. In one embodiment, the first value is 1 and the second value is 0. In another embodiment, the first value is 0 and the second value is 1. Of course, the first value and the second value may be other values, and the present application is not limited thereto.

[0083] Exemplarily, when the first flag is true, the first flag is used to indicate that the image block in the current sequence is permitted to be predicted using the optimal MIP mode and the first intra-prediction mode, and when the first flag is false, the first flag is used to indicate that the image block in the current sequence is not permitted to be predicted using the optimal MIP mode and the first intra-prediction mode.

[0084] Exemplarily, the decoder analyzes a block-level flag. If an intra-prediction mode is used for the current block, the decoder analyzes or obtains a first flag. If the first flag is true, the decoder determines an optimal MIP mode based on distortion costs corresponding to multiple MIP modes.

[0085] Exemplarily, the first flag is denoted as sps_timd_enable_flag, in which case the decoder parses or obtains sps_timd_enable_flag. If sps_timd_enable_flag is true, the decoder determines an optimal MIP mode based on distortion costs corresponding to the multiple MIP modes.

[0086] Illustratively, the first flag is a sequence-level flag.

[0087] It should be noted that the statement that the first flag is used to indicate that the image block in the current sequence is permitted to be predicted using the optimal MIP mode and the first intra prediction mode may be replaced with a statement having a similar or identical meaning. For example, in another alternative embodiment, the statement that the first flag is used to indicate that the image block in the current sequence is permitted to be predicted using the optimal MIP mode and the first intra prediction mode may be replaced with any of the following: The first flag is used to indicate that the intra prediction mode of the image block in the current sequence is permitted to be determined using the TMMIP technique. The first flag is used to indicate that intra prediction is permitted for the image block in the current sequence using the TMMIP technique. The first flag is used to indicate that the TMMIP technique is permitted to be used for the image block in the current sequence. The first flag is used to indicate that the image block in the current sequence is permitted to be predicted using the MIP mode determined based on the multiple MIP modes.

[0088] Furthermore, in another alternative embodiment, when the TMMIP technology is combined with another technology, the enable flag of the other technology may indirectly indicate whether the TMMIP technology is permitted to be used in the current sequence. For example, taking the TIMD technology as an example, if the first flag is used to indicate that the TIMD technology is permitted to be used in the current sequence, it indicates that the TMMIP technology is also permitted to be used in the current sequence. In other words, if the first flag is used to indicate that the TIMD technology is permitted to be used in the current sequence, it indicates that both the TIMD technology and the TMMIP technology are permitted to be used in the current sequence. This further saves bit overhead.

[0089] In some embodiments, if the first flag is used to indicate that it is permitted to predict an image block in the current sequence using an optimal MIP mode and a first intra prediction mode, the bitstream is parsed to obtain a second flag. If the second flag is used to indicate that it is permitted to predict the current block using the optimal MIP mode and the first intra-prediction mode, the optimal MIP mode is determined based on distortion costs corresponding to the multiple MIP modes.

[0090] Exemplarily, the decoder analyzes a block-level flag. If an intra-prediction mode is used for the current block, the decoder analyzes or obtains a first flag. If the first flag is true, the decoder analyzes or obtains a second flag. If the second flag is true, the decoder determines an optimal MIP mode based on distortion costs corresponding to multiple MIP modes.

[0091] Exemplarily, when the value of the second flag is a third value, the second flag is used to indicate that the current block is allowed to be predicted using the optimal MIP mode and the first intra prediction mode. When the value of the second flag is a fourth value, the second flag is used to indicate that the current block is not allowed to be predicted using the optimal MIP mode and the first intra prediction mode. In one embodiment, the third value is 1 and the fourth value is 0. In another embodiment, the third value is 0 and the fourth value is 1. Of course, the third value and the fourth value may be other values, and the present application is not limited thereto.

[0092] Exemplarily, when the second flag is true, the second flag is used to indicate that the current block is permitted to be predicted using the optimal MIP mode and the first intra-prediction mode, and when the second flag is false, the second flag is used to indicate that the current block is not permitted to be predicted using the optimal MIP mode and the first intra-prediction mode.

[0093] Exemplarily, the first flag is denoted as sps_timd_enable_flag, and the second flag is denoted as cu_timd_enable_flag, in which case the decoder parses or obtains sps_timd_enable_flag. If sps_timd_enable_flag is true, the decoder parses or obtains cu_timd_enable_flag. If cu_timd_enable_flag is true, the decoder determines an optimal MIP mode based on distortion costs corresponding to multiple MIP modes.

[0094] Exemplarily, the second flag is a block-level flag or a coding unit-level flag.

[0095] It should be noted that the statement that the second flag is used to indicate that the current block is permitted to be predicted using the optimal MIP mode and the first intra prediction mode may be replaced with a statement having a similar or identical meaning. For example, in another alternative embodiment, the statement that the second flag is used to indicate that the current block is permitted to be predicted using the optimal MIP mode and the first intra prediction mode may be replaced with any of the following: The second flag is used to indicate that the current block is permitted to be determined as an intra prediction mode using the TMMIP technique. The second flag is used to indicate that the current block is permitted to be intra prediction using the TMMIP technique. The second flag is used to indicate that the TMMIP technique is permitted to be used for the image block of the current block. The second flag is used to indicate that the current block is permitted to be predicted using a MIP mode determined based on multiple MIP modes.

[0096] Furthermore, in another alternative embodiment, when the TMMIP technology is combined with other technologies, the permission flag of the other technology may indirectly indicate whether the TMMIP technology is permitted to be used for the current block. For example, taking the TIMD technology as an example, if the second flag is used to indicate that the TIMD technology is permitted to be used for the current block, it indicates that the TMMIP technology is also permitted to be used for the current block. In other words, if the second flag is used to indicate that the TIMD technology is permitted to be used for the current block, it indicates that both the TIMD technology and the TMMIP technology are permitted to be used for the current block. This further saves bit overhead.

[0097] In addition, when the decoding side analyzes the second flag, the decoding side may analyze the second flag before analyzing the residual block of the current block, or may analyze the second flag after analyzing the residual block of the current block, but this application is not particularly limited thereto.

[0098] In some embodiments, the method 300 may further include: determining an ordering of the multiple MIP modes based on distortion costs corresponding to the multiple MIP modes; determining a coding scheme to be used for an optimal MIP mode based on the ordering of the multiple MIP modes; and decoding a bitstream of the current sequence based on the coding scheme to be used for the optimal MIP mode to obtain an index of the optimal MIP mode.

[0099] Exemplarily, before determining an optimal MIP mode for predicting a current block based on the distortion costs corresponding to the multiple MIP modes, the decoder needs to calculate the distortion cost corresponding to each of the multiple MIP modes and sort the multiple MIP modes based on the distortion cost corresponding to each MIP mode, and the MIP mode with the smallest cost is the optimal prediction result.

[0100] It should be noted that in the conventional MIP technology, the index of the MIP mode is usually binarized by selecting a truncated binary scheme similar to equal probability coding. That is, in the truncated binary scheme, all prediction modes are divided into two segments, one segment is represented by N code words, and the other segment is represented by N+1 code words. In view of this, in this application, the decoder can calculate the distortion cost corresponding to each of the multiple MIP modes and sort the multiple MIP modes based on the distortion cost corresponding to each MIP mode before determining the optimal MIP mode for predicting the current block based on the distortion cost corresponding to the multiple MIP modes. Finally, the decoder selects and uses a more flexible variable-length coding scheme according to the arrangement order of the multiple MIP modes. Compared with equal probability coding, the bit overhead of the index of the MIP mode can be saved by flexibly setting the coding scheme of the MIP mode.

[0101] In some embodiments, the codeword length of the coding scheme used for the first n MIP modes in the ordering is smaller than the codeword length of the coding scheme used for the MIP mode following the nth MIP mode in the ordering, and / or a variable length coding scheme is used for the first n MIP modes and a truncated binary coding scheme is used for the MIP mode following the nth MIP mode.

[0102] Illustratively, N can be any value equal to or greater than 1.

[0103] Exemplarily, the ordering is an ordering obtained by a decoder arranging the multiple MIP modes according to ascending order of distortion cost, where the codeword length of the coding scheme used for the first n MIP modes in the ordering is smaller than the codeword length of the coding scheme used for the MIP mode following the nth MIP mode in the ordering, and / or a variable length coding scheme is used for the first n MIP modes and a truncated binary coding scheme is used for the MIP mode following the nth MIP mode in the ordering.

[0104] In this embodiment, the smaller the distortion cost corresponding to a MIP mode, the higher the probability that the encoder uses the MIP mode to perform intra prediction on the current block, so the codeword length of the coding scheme used for the first n MIP modes in the ordering is designed to be smaller than the codeword length of the coding scheme used for the MIP mode following the nth MIP mode in the ordering, and / or the coding scheme used for the first n MIP modes is designed to be a variable length coding scheme, and the coding scheme used for the MIP mode following the nth MIP mode is designed to be a truncated binary coding scheme. In this way, the MIP mode used by the encoder with a high probability uses a relatively short codeword length or a variable length coding scheme. This can save the bit overhead of the MIP mode index and improve the decompression performance.

[0105] In some embodiments, before S330, the method 300 may further include the following content: If the first intra prediction mode is a sub-optimal MIP mode, the decoder determines whether to adopt a sub-optimal MIP mode to predict the current block based on a distortion cost corresponding to the optimal MIP mode and a distortion cost corresponding to the sub-optimal MIP mode. If it is determined not to adopt the sub-optimal MIP mode, the decoder can directly predict the current block based on the optimal MIP mode. If it is determined to adopt the sub-optimal MIP mode, the decoder can predict the current block based on the optimal MIP mode and the sub-optimal MIP mode to obtain a predicted block of the current block.

[0106] Exemplarily, when the first intra prediction mode is a suboptimal MIP mode, if the ratio of the distortion cost corresponding to the optimal MIP mode to the distortion cost corresponding to the suboptimal MIP mode is equal to or less than a preset ratio, the decoder can directly predict the current block based on the optimal MIP mode to obtain a prediction block of the current block. Or, when the first intra prediction mode is a suboptimal MIP mode, if the ratio of the distortion cost corresponding to the suboptimal MIP mode to the distortion cost corresponding to the optimal MIP mode is equal to or greater than a preset ratio, the decoder can directly predict the current block based on the optimal MIP mode to obtain a prediction block of the current block. For example, if the distortion cost corresponding to the suboptimal MIP mode is equal to or greater than a certain multiple (e.g., twice) of the distortion cost corresponding to the optimal MIP mode, it can be interpreted that the suboptimal MIP mode already has a large distortion and is not suitable for the current block, i.e., it is possible to predict the current block using only the optimal MIP mode without the fusion enhancement technique.

[0107] In this embodiment, the decoder determines whether to adopt a suboptimal MIP mode to predict a current block according to the distortion cost corresponding to the optimal MIP mode and the distortion cost corresponding to the suboptimal MIP mode, which corresponds to the decoder determining whether to adopt a suboptimal MIP mode to improve the performance of the optimal MIP mode according to the distortion cost corresponding to the optimal MIP mode and the distortion cost corresponding to the suboptimal MIP mode, thereby avoiding carrying a flag for determining whether to adopt a suboptimal MIP mode to improve the performance of the optimal MIP mode in the bitstream, saving bit overhead and thus improving the decompression performance.

[0108] In some embodiments, S340 may include the following content: The decoder predicts the current block based on an optimal MIP mode to obtain a first predicted block; The decoder also predicts the current block based on a first intra-prediction mode to obtain a second predicted block; Then, the decoder weights the first predicted block and the second predicted block based on a weight of the optimal MIP mode and a weight of the first intra-prediction mode to obtain a predicted block of the current block.

[0109] Exemplarily, the decoder may directly perform intra prediction on the current block based on the optimal MIP mode to obtain a first predicted block. Furthermore, the decoder may directly obtain an optimal prediction mode and a suboptimal prediction mode based on the TIMD mode to predict the current block to obtain a second predicted block. For example, if the optimal prediction mode and the suboptimal prediction mode are neither a DC mode (which may also be called an average mode) nor a planar mode (which may also be called a planar mode), and the distortion cost corresponding to the suboptimal prediction mode is less than twice the distortion cost corresponding to the optimal prediction mode, it is necessary to fuse the predicted blocks. That is, the decoder may first perform intra prediction on the current block based on the optimal prediction mode to obtain an optimal predicted block. Then, the decoder may perform intra prediction on the current block based on the suboptimal prediction mode to obtain a suboptimal predicted block. Furthermore, the decoder may use a ratio between the distortion cost corresponding to the optimal prediction mode and the distortion cost corresponding to the suboptimal prediction mode to calculate a weight value of the optimal predicted block and a weight value of the suboptimal predicted block. Finally, the decoder may weight and fuse the optimal predicted block and the suboptimal predicted block to obtain a second predicted block. In another example, if the optimal prediction mode or the suboptimal prediction mode is a planar mode or a DC mode, or if the distortion cost corresponding to the suboptimal prediction mode is greater than twice the distortion cost corresponding to the optimal prediction mode, there is no need to fuse the prediction blocks. That is, only the optimal prediction block obtained according to the optimal prediction mode can be directly used as the second prediction block. After obtaining the first prediction block and the second prediction block, the decoder performs a weighting process on the first prediction block and the second prediction block to obtain a prediction block of the current block.

[0110] In some embodiments, S340 may include the following: If the first intra-prediction mode includes an intra-prediction mode derived from a sub-optimal MIP mode or a TIMD mode, the decoder determines an optimal MIP mode weight and a weight of the first intra-prediction mode based on a distortion cost corresponding to the optimal MIP mode and a distortion cost corresponding to the first intra-prediction mode. If the first intra-prediction mode includes an intra-prediction mode derived from a DIMD mode, the decoder determines both the optimal MIP mode weight and the first intra-prediction mode weight to a preset value.

[0111] In some embodiments, S320 may include the following: The decoder predicts samples in the template region based on the third flag and the multiple MIP modes to obtain distortion costs corresponding to the multiple MIP modes under each state of the third flag; The third flag is used to indicate whether to transpose the input vector and the output vector corresponding to the MIP mode; The decoder determines an optimal MIP mode based on the distortion costs corresponding to the multiple MIP modes under each state of the third flag.

[0112] As mentioned above, the bit overhead of the conventional MIP technique is more than that of other intra prediction tools, and not only a flag indicating whether the MIP technique is used is required, but also a flag indicating whether the MIP is transposed is required. Finally, the largest overhead is that the truncated binary coding needs to be used to represent the MIP prediction mode. The MIP technique is a simplified technique based on neural network technology, and is significantly different from the conventional interpolation filtering prediction technique. For some special textures, the MIP prediction mode is more effective than the conventional intra prediction mode, but the large flag overhead is a defect of the MIP technique. For example, a coding unit of 4×4 size has a total of 16 prediction samples, and the bit overhead includes one MIP use flag, one MIP transposition flag, and a 5-bit or 6-bit truncated binary flag. In view of this, in the present application, when determining the optimal MIP mode, the overhead of one MIP transposition flag can be saved by traversing each state of the third flag, taking into account the transposition function of the MIP mode, and the decompression efficiency can be further improved.

[0113] For example, the decoder traverses each state of the third flag and a plurality of MIP modes, determines distortion costs corresponding to the plurality of MIP modes under each state of the third flag, and determines an optimal MIP mode according to the distortion costs corresponding to the plurality of MIP modes under each state of the third flag. Or, the decoder traverses each state of the third flag and a plurality of MIP modes, determines distortion costs corresponding to the plurality of MIP modes under each state of the third flag, and determines an optimal MIP mode according to the distortion costs corresponding to the plurality of MIP modes under each state of the third flag. That is, the decoding side may traverse the plurality of MIP modes first, or may traverse the state of the third flag first.

[0114] Exemplarily, when the value of the third flag is a fifth value, the third flag is used to indicate that the input vector and the output vector corresponding to the MIP mode are transposed. When the value of the third flag is a sixth value, the third flag is used to indicate that the input vector and the output vector corresponding to the MIP mode are not transposed. In this case, each state of the third flag can be replaced by each value of the third flag. In one embodiment, the fifth value is 1 and the sixth value is 0. In another embodiment, the fifth value is 0 and the sixth value is 1. Of course, the fifth value and the sixth value may be other values, and the present application is not limited thereto.

[0115] Exemplarily, when the third flag is true, the third flag is used to indicate that the input vector and the output vector corresponding to the MIP mode are transposed. When the third flag is false, the third flag is used to indicate that the input vector and the output vector corresponding to the MIP mode are not transposed. In this case, whether the third flag is true or false is the state of the third flag.

[0116] Exemplarily, the third flag is a sequence-level flag, a block-level flag, or a coding unit-level flag.

[0117] Illustratively, the third flag may be referred to as a transposition message, a transposition flag, or a MIP transposition flag.

[0118] It should be noted that the description that the third flag is used to indicate whether the input vector and the output vector corresponding to the MIP mode are transposed or not can be replaced with a description having a similar or identical meaning. For example, in another alternative embodiment, the third flag is used to indicate whether the input and the output corresponding to the MIP mode need to be transposed or not. The third flag is used to indicate whether the input and the output vector corresponding to the MIP mode are transposed vectors or not. The third flag is used to indicate whether to transpose or not.

[0119] In some embodiments, S320 may include the following: If the size of the current block is a preset size, the decoder determines an optimal MIP mode based on distortion costs corresponding to multiple MIP modes.

[0120] Exemplarily, the preset size may include a size whose width is the preset width and whose height is the preset height. In other words, if the width of the current block is the preset width and the height is the preset height, the decoder determines an optimal MIP mode based on distortion costs corresponding to the multiple MIP modes.

[0121] Exemplarily, the preset size may be realized by pre-storing a corresponding code, table, or other scheme that can be used to indicate related information in an apparatus (including, for example, a decoder and an encoder). The present application is not limited to the specific embodiment. For example, the preset size may refer to a size defined in a protocol. Alternatively, the "protocol" may refer to a standard protocol in the technical field of coding, and may include related protocols such as the VCC protocol or the ECM protocol.

[0122] Of course, in other alternative embodiments, the decoder may determine whether to determine the optimal MIP mode based on the distortion costs corresponding to the multiple MIP modes based on the preset size through other manners, of which the present application is not specifically limited.

[0123] For example, the decoder may determine whether to determine the optimal MIP mode based on the distortion costs corresponding to the multiple MIP modes based only on the width or height of the current block. In one embodiment, if the width of the current block is a preset width or the height is a preset height, the decoder determines the optimal MIP mode based on the distortion costs corresponding to the multiple MIP modes. In another example, the decoder may determine whether to determine the optimal MIP mode based on the distortion costs corresponding to the multiple MIP modes by comparing the size of the current block with the preset size. In one embodiment, if the size of the current block is larger or smaller than the preset size, the decoder determines the optimal MIP mode based on the distortion costs corresponding to the multiple MIP modes. In another embodiment, if the width of the current block is larger or smaller than the preset width, the decoder determines the optimal MIP mode based on the distortion costs corresponding to the multiple MIP modes. In another embodiment, if the height of the current block is larger or smaller than the preset height, the decoder determines the optimal MIP mode based on the distortion costs corresponding to the multiple MIP modes.

[0124] In some embodiments, S320 may include the following content: If the image frame in which the current block is located is an I-frame, and the size of the current block is a preset size, the decoder determines an optimal MIP mode based on distortion costs corresponding to multiple MIP modes.

[0125] Exemplarily, if the image frame in which the current block is located is an I frame, the width of the current block is a preset width, and the height of the current block is a preset height, the decoder determines the optimal MIP mode based on the distortion costs corresponding to the multiple MIP modes, that is, only if the image frame in which the current block is located is an I frame, the decoder determines whether to determine the optimal MIP mode based on the distortion costs corresponding to the multiple MIP modes based on the size of the current block.

[0126] In some embodiments, S320 may include the following: If the image frame in which the current block is located is a B frame, the decoder determines an optimal MIP mode based on distortion costs corresponding to multiple MIP modes.

[0127] For example, when the image frame in which the current block is located is a B frame, the decoder can directly determine the optimal MIP mode based on the distortion costs corresponding to multiple MIP modes, that is, when the image frame in which the current block is located is a B frame, regardless of the size of the current block, the decoder can directly determine the optimal MIP mode based on the distortion costs corresponding to multiple MIP modes.

[0128] In some embodiments, before S320, the method 300 may further include: The decoder obtains a MIP mode used for a neighboring block adjacent to the current block. The decoder determines the MIP mode used for the neighboring block to be a plurality of MIP modes.

[0129] Exemplarily, the neighboring block may be an image block adjacent to at least one of the upper side, the left side, the lower left side, the upper right side, and the upper left side of the current block. For example, the decoder may determine the image blocks obtained in the order of the upper side, the left side, the lower left side, the upper right side, and the upper left side of the current block as the neighboring blocks. Optionally, the multiple MIP modes may be used to build an available MIP mode or a list of available MIP modes to be determined by the decoder and to predict the current block. Thereby, the decoder determines an optimal MIP mode by predicting samples in the template region from the available MIP mode or the list of available MIP modes.

[0130] In some embodiments, before S320, the method 300 may further include: the decoder performs reconstructed sample padding on a reference region adjacent to an outside of the template region to obtain a reference row and a reference column of the template region; the decoder takes the reference row and the reference column as input and predicts samples in the template region using each of the multiple MIP modes to obtain multiple prediction blocks corresponding to the multiple MIP modes; and the decoder determines distortion costs corresponding to the multiple MIP modes based on the multiple prediction blocks and the reconstructed blocks in the template region.

[0131] For example, the decoder pads with reference reconstructed samples needed for predicting the template.

[0132] Exemplarily, the width of the region adjacent to the top of the template region in the reference region is equal to the width of the template region. The height of the region adjacent to the left of the template region in the reference region is equal to the width of the template region. If the width of the region adjacent to the top of the template region in the reference region is greater than the width of the template region, the decoder can perform a downsampling or dimensionality reduction process on the region adjacent to the top of the template region in the reference region to obtain a reference row. If the height of the region adjacent to the left of the template region in the reference region is greater than the width of the template region, the decoder can perform a downsampling or dimensionality reduction process on the region adjacent to the left of the template region in the reference region to obtain a reference column.

[0133] Exemplarily, the template region may be a template region used in the above-mentioned TIMD mode, and the reference region may be a reference template used in the TIMD mode. For example, referring to FIG. 5, if the current block is a coding unit with a width equal to M and a height equal to N, the decoder pads a reference region consisting of coding units with a width equal to 2(M+L1)+1 and a height equal to 2(N+L2)+1 with reconstructed samples, performs a downsampling or dimensionality reduction process on the padded reference region to obtain reference rows and reference columns, and further constructs an input vector corresponding to the MIP mode according to the reference rows and reference columns.

[0134] Exemplarily, after obtaining the reference row and the reference column, the decoder predicts samples in the template region using each of the multiple MIP modes with the reference row and the reference column as inputs to obtain multiple predicted blocks corresponding to the multiple MIP modes. That is, the decoder predicts samples in the template region of the current block by traversing the multiple MIP modes based on the reconstructed samples in the reference template of the current block. Taking the currently traversed MIP mode as an example, the decoder obtains a predicted block corresponding to the currently traversed MIP mode with the reference row, the reference column, the index of the currently traversed MIP mode, and the third flag as inputs. The reference row and the reference column are used to construct an input vector corresponding to the currently traversed MIP mode. The index of the currently traversed MIP mode is used to determine a matrix and / or a bias vector corresponding to the currently traversed MIP mode. The third flag is used to indicate whether to transpose the input vector and the output vector corresponding to the MIP mode. For example, if the third flag is used to indicate that the input vector and the output vector corresponding to the MIP mode are not transposed, the reference row is spliced ​​after the reference row to form the input vector corresponding to the currently traversed MIP mode. If the third flag is used to indicate that the input vector and the output vector corresponding to the MIP mode are transposed, the reference row is spliced ​​after the reference row to form the input vector corresponding to the currently traversed MIP mode. Correspondingly, if the third flag is used to indicate that the input vector and the output vector corresponding to the MIP mode are transposed, the decoder transposes the output of the currently traversed MIP mode to obtain a predicted block in the template region. After the decoder obtains a plurality of predicted blocks corresponding to the multiple MIP modes by traversing the multiple MIP modes, the decoder can select a MIP mode with a minimum cost according to the principle of minimum distortion cost based on the distortion costs between the multiple predicted blocks and the reconstructed samples in the template region, and determine it as the optimal MIP mode of the current block under the template matching-based MIP mode.

[0135] In some embodiments, when predicting samples in the template region using each of the multiple MIP modes, the decoder first downsamples a reference row and a reference column to obtain an input vector, then predicts samples in the template region by traversing the multiple MIP modes using the input vector as input to obtain output vectors corresponding to the multiple MIP modes, and finally upsamples the output vectors corresponding to the multiple MIP modes to obtain prediction blocks corresponding to the multiple MIP modes.

[0136] Exemplarily, the reference row and the reference column satisfy the input conditions of multiple MIP modes. If the reference row and the reference column do not satisfy the input conditions of multiple MIP modes, the reference row and / or the reference column are first processed to become input samples that satisfy the input conditions of multiple MIP modes, and then, based on the input samples that satisfy the input conditions of multiple MIP modes, input vectors corresponding to multiple MIP modes may be determined. For example, taking the input condition as an example that a specified number of input samples is used, if the reference row and the reference column do not satisfy the number of input samples of the MIP mode, the decoder performs Haar-downsampling, etc., on the reference row and / or the reference column to reduce the dimension of the reference row and / or the reference column to a specified number of input samples, and determines an input vector corresponding to multiple MIP modes based on the specified number of input samples that have been reduced in dimension.

[0137] In some embodiments, S320 may include the following: The decoder determines an optimal MIP mode based on a sum of absolute difference transforms (SATD) corresponding to a plurality of MIP modes in the template region.

[0138] In this embodiment, when the decoder determines an optimal MIP mode based on the distortion costs corresponding to the multiple MIP modes in the template region, the distortion costs corresponding to the multiple MIP modes are designed into the SATD corresponding to the multiple MIP modes. Compared with directly calculating the rate-distortion costs corresponding to the multiple MIP modes, it can not only realize determining an optimal MIP mode based on the distortion costs corresponding to the multiple MIP modes in the template region, but also simplify the calculation complexity of the distortion costs corresponding to the multiple MIP modes, so as to improve the decoding performance of the decoder.

[0139] In summary, the scheme of the present application puts forward the idea of ​​fusion enhancement based on optimal MIP mode. That is, the decoder not only needs to determine the optimal MIP mode for predicting the current block, but also needs to fuse different prediction blocks to achieve different prediction effects. This not only saves bit overhead, but also creates new prediction techniques. In fact, because the optimal MIP mode cannot completely replace the optimal prediction mode calculated by the encoding side based on the rate-distortion cost, the fusion method is used to balance prediction accuracy and prediction diversity.

[0140] Exemplarily, the main idea of ​​the template matching based MIP mode derivation method by the decoder can be divided into several parts as follows:

[0141] First, a reference region (e.g., the reference template shown in FIG. 5) is padded with reconstruction samples, i.e., reference reconstruction samples required for predicting samples in the template region (e.g., the template shown in FIG. 5). Optionally, the width and height of the reference region do not need to exceed the width and height of the template region. If the width and height of the reference region padded with samples exceeds the width and height of the template region, downsampling or other dimensionality reduction is required until the input dimension requirement of the MIP is achieved.

[0142] Next, the decoder predicts the samples in the template domain using the reference reconstruction samples in the reference domain, the indices of the multiple MIP modes, and the MIP transpose flag as inputs to obtain prediction blocks corresponding to the multiple MIP modes. Optionally, the reference reconstruction samples in the reference domain need to satisfy the input conditions of the MIP modes, and perform dimension reduction, such as Haar downsampling, until a specified number of input samples. The indices of the multiple MIP modes are used to determine the matrix indexes of the MIP technique, and further to obtain the MIP prediction matrix coefficients. The MIP transpose flag is used to indicate whether the input and output need to be transposed.

[0143] Then, for a prediction block corresponding to multiple MIP modes, all combinations of MIP modes and whether to transpose MIP can be traversed to obtain prediction samples in the template region under each MIP mode and each state of the MIP transposition flag, and calculate the distortion between the prediction samples and the reconstructed samples in the template region and record the cost information thereof. Finally, according to the principle of least distortion, select the MIP mode with the least cost and its corresponding MIP transposition information, and the MIP mode with the least cost is the optimal MIP mode of the current block under the template matching-based MIP prediction derivation mode.

[0144] Finally, the decoder predicts the current block using the optimal MIP prediction mode and the first intra prediction mode, respectively, to obtain a first prediction block and a second prediction block, and weights the first prediction block and the second prediction block according to the weight of the optimal MIP prediction mode and the weight of the first intra prediction mode to obtain a prediction block of the current block.

[0145] It should be noted that some of the calculations in the present application can be replaced by a lookup table or shift method. The lookup table method may result in some errors compared with direct division, but it is beneficial for hardware implementation and coding cost control. Examples of the above calculations include calculations related to distortion cost or calculations related to determining the optimal MIP mode.

[0146] Having described above in detail the decoding method according to an embodiment of the present application from the decoder's perspective, the encoding method according to an embodiment of the present application will now be described from the encoder's perspective with reference to FIG.

[0147] 8 is a flow chart illustrating an encoding method 400 according to an embodiment of the present application. The encoding method 400 may be implemented by an encoder. For example, the encoding method 400 may be applied to the encoding framework 100 shown in FIG. 1. For ease of explanation, the following description will be given taking the encoder as an example.

[0148] As shown in FIG. 8, the encoding method 400 may include the following: S410: Determine an optimal intra-prediction (MIP) mode for predicting a current block in a current sequence based on distortion costs corresponding to a plurality of matrix-based intra-prediction (MIP) modes, including distortion costs obtained by predicting samples in a template region adjacent to the current block using the plurality of MIP modes. S420: Determine a first intra prediction mode, where the first intra prediction mode is determined based on distortion costs corresponding to a plurality of MIP modes, and includes at least one of a sub-optimal MIP mode for predicting a current block, an intra prediction mode derived from a decoder-side intra mode derivation (DIMD) mode, and an intra prediction mode derived from a template-based intra mode derivation (TIMD) mode. S430: Predict the current block based on the optimal MIP mode and the first intra prediction mode to obtain a predicted block of the current block. S440: Obtain a residual block of the current block based on the predicted block of the current block and the original block of the current block. S450: Encode a residual block of the current block to obtain a bitstream of the current sequence.

[0149] In some embodiments, S410 may include the following: Obtain a first flag, and if the first flag is used to indicate that the image block in the current sequence is allowed to be predicted using the optimal MIP mode and the first intra-prediction mode, determine an optimal MIP mode based on distortion costs corresponding to the multiple MIP modes. S450 may include encoding the residual block and the first flag of the current block to obtain a bitstream.

[0150] In some embodiments, S430 may include the following: If the first flag is used to indicate that it is permitted to predict an image block in the current sequence using the optimal MIP mode and the first intra prediction mode, the current block is predicted based on the optimal MIP mode and the first intra prediction mode to obtain a first rate-distortion cost. The current block is predicted based on at least one intra prediction mode to obtain at least one rate-distortion cost. If the first rate-distortion cost is less than or equal to the minimum value of the at least one rate-distortion cost, a prediction block obtained by predicting the current block based on the optimal MIP mode and the first intra-prediction mode is determined as the prediction block of the current block. S450 may include the following: The residual block of the current block, the first flag, and the second flag are encoded to obtain a bitstream. If the first rate-distortion cost is less than or equal to the minimum of the at least one rate-distortion cost, the second flag is used to indicate that predicting the current block using the optimal MIP mode and the first intra-prediction mode is permitted, and if the first rate-distortion cost is greater than the minimum of the at least one rate-distortion cost, the second flag is used to indicate that predicting the current block using the optimal MIP mode and the first intra-prediction mode is not permitted.

[0151] In some embodiments, S450 may include the following: An ordering of the multiple MIP modes is determined based on distortion costs corresponding to the multiple MIP modes. A coding scheme to be used for the optimal MIP mode is determined based on the arrangement order of the multiple MIP modes. A residual block of the current block is encoded, and an index of an optimal MIP mode is encoded based on a coding scheme used for the optimal MIP mode to obtain a bitstream.

[0152] In some embodiments, the codeword length of the coding scheme used for the first n MIP modes in the ordering is smaller than the codeword length of the coding scheme used for the MIP mode following the nth MIP mode in the ordering, and / or A variable length coding scheme is used for the first n MIP modes, and a truncated binary (TB) coding scheme is used for the MIP modes following the nth MIP mode.

[0153] In some embodiments, S430 may include the following: The current block is predicted based on the optimal MIP mode to obtain a first predicted block. The current block is predicted based on the first intra-prediction mode to obtain a second predicted block. The first prediction block and the second prediction block are weighted according to the optimal MIP mode weight and the first intra-prediction mode weight to obtain a prediction block of the current block.

[0154] In some embodiments, S430 may include the following: If the first intra prediction mode includes an intra prediction mode derived from a sub-optimal MIP mode or a TIMD mode, a weight of the optimal MIP mode and a weight of the first intra prediction mode are determined based on a distortion cost corresponding to the optimal MIP mode and a distortion cost corresponding to the first intra prediction mode. When the first intra prediction mode includes an intra prediction mode derived from a DIMD mode, the weight of the optimal MIP mode and the weight of the first intra prediction mode are both determined to be preset values.

[0155] In some embodiments, S410 may include the following: Predict the samples in the template region according to the third flag and the multiple MIP modes to obtain distortion costs corresponding to the multiple MIP modes under each state of the third flag, where the third flag is used to indicate whether to transpose the input vector and the output vector corresponding to the MIP mode. An optimal MIP mode is determined based on distortion costs corresponding to the multiple MIP modes under each state of the third flag.

[0156] In some embodiments, S410 may include the following: If the size of the current block is a preset size, determine an optimal MIP mode based on distortion costs corresponding to a plurality of MIP modes.

[0157] In some embodiments, S410 may include the following content: If the image frame in which the current block is located is an I-frame and the size of the current block is a preset size, determine an optimal MIP mode based on distortion costs corresponding to multiple MIP modes.

[0158] In some embodiments, S410 may include the following: If the image frame in which the current block is located is a B frame, determine an optimal MIP mode based on distortion costs corresponding to a plurality of MIP modes.

[0159] In some embodiments, before S410, the method 400 may further include: Get the MIP mode used for the neighboring blocks adjacent to the current block. The MIP modes used for the neighboring blocks are determined to be multiple MIP modes.

[0160] In some embodiments, before S410, the method 400 may further include: The reconstruction sample padding is performed on the reference region adjacent to the outside of the template region to obtain the reference row and the reference column of the template region. Using a reference row and a reference column as input, samples in the template region are predicted using each of a plurality of MIP modes to obtain a plurality of predicted blocks corresponding to the plurality of MIP modes. Distortion costs corresponding to a plurality of MIP modes are determined based on the plurality of predicted blocks and the reconstructed block in the template region.

[0161] In some embodiments, when predicting samples in a template region using each of multiple MIP modes, first, a reference row and a reference column are downsampled to obtain an input vector, then, using the input vector as input, the samples in the template region are predicted by traversing the multiple MIP modes to obtain output vectors corresponding to the multiple MIP modes, and finally, the output vectors corresponding to the multiple MIP modes are upsampled to obtain prediction blocks corresponding to the multiple MIP modes.

[0162] In some embodiments, S410 may include: determining an optimal MIP mode based on a sum of absolute difference transforms (SATD) corresponding to a plurality of MIP modes in the template region;

[0163] It should be noted that the encoding method can be understood as the reverse process of the decoding method. Therefore, for a specific scheme of the encoding method 400, reference can be made to the relevant content of the decoding method 300, and for convenience of explanation, the detailed description will not be given in this application.

[0164] The scheme of the present application will now be described with reference to specific examples.

[0165] <Example 1>

[0166] In this embodiment, the first intra prediction mode is a sub-optimal prediction mode, that is, the encoder or decoder can perform intra prediction on the current block according to the optimal MIP mode and the sub-optimal MIP mode to obtain a prediction block of the current block.

[0167] The encoder traverses the prediction modes. If intra mode is used for the current block, the encoder gets a sequence-level enable flag, such as sps_tmmip_enable_flag. The sequence-level enable flag is used to indicate whether the template matching-based MIP mode derivation technique is allowed to be used for the current sequence. If any of the tmmip enable flags is true, it indicates that the encoder is currently allowed to use the TMMIP technique.

[0168] Exemplarily, the process of the encoder can be realized as the following process.

[0169] Step 1: If sps_tmmip_enable_flag is true, the encoder tries the TMMIP technique, i.e., executes step 2. If sps_tmmip_enable_flag is false, the encoder does not try the TMMIP technique, i.e., skips step 2 and executes step 3 directly.

[0170] Step 2: First, the encoder performs reconstructed sample padding on adjacent rows and columns outside the template region. The padding process is the same as the padding method of the original intra prediction process. For example, the encoder can traverse from the lower left corner to the upper right corner to pad. If all reconstructed samples are available, pad with all available reconstructed samples in order. If all reconstructed samples are unavailable, pad with all average values. If some reconstructed samples are available, pad with available reconstructed samples first, and for the remaining unavailable reconstructed samples, the encoder traverses from the lower left corner to the upper right corner in order until the first available reconstructed sample appears, and uses the first available reconstructed sample to pad the previous unavailable position. The encoder then takes the reconstructed samples outside the padded template region as input and predicts samples inside the template region using any MIP mode that is allowed to be used.

[0171] Exemplarily, for a block of size 4x4, there are 16 MIP modes that are allowed to be used. For a block of width or height equal to 4, or a block of size 8x8, there are 8 MIP modes that are allowed to be used. For blocks of other sizes, there are 6 MIP modes that are allowed to be used. Also, for blocks of any size, the MIP transpose function can be used, and the TMMIP prediction modes are the same as the MIP technique.

[0172] Exemplarily, a specific prediction calculation process includes the following contents. First, the encoder performs Haar downsampling on the reconstructed sample, for example, the encoder determines a downsampling step size based on the block size. Next, the encoder adjusts the splice order of the upper downsampled reconstructed sample and the left downsampled reconstructed sample according to the information on whether to transpose. If transposition is not required, the left downsampled reconstructed sample is spliced ​​after the upper downsampled reconstructed sample, and the resulting vector is input. If transposition is required, the upper downsampled reconstructed sample is spliced ​​after the left downsampled reconstructed sample, and the resulting vector is input. Next, the encoder obtains MIP matrix coefficients using the traversed prediction mode as an index, and obtains an output vector by calculating the MIP matrix coefficients and the input. Finally, the encoder upsamples the output vector according to the number of samples of the output vector and the size of the current template. If upsampling is not required, the vectors are arranged in order according to the horizontal direction and output as a prediction block in the template area. If upsampling is required, first upsample according to the horizontal direction, then downsample according to the vertical direction, and so on until the size is the same as the template, and output the predicted block within the template region.

[0173] Next, the encoder calculates a distortion cost based on the predicted block in the template region and the reconstructed sample in the template region obtained by traversing each MIP mode, and records the distortion cost under each prediction mode and transposition information. After traversing all permitted prediction modes and transposition information, the encoder selects an optimal MIP mode and its corresponding transposition information and a suboptimal MIP mode and its corresponding transposition information according to the principle of minimum cost. The encoder determines whether fusion enhancement is required based on the relationship between the cost of the optimal MIP mode and the cost of the suboptimal MIP mode. If the cost of the suboptimal MIP mode is less than twice the cost of the optimal MIP mode, the optimal MIP predicted block and the suboptimal MIP predicted block need to be fusion enhanced. If the cost of the suboptimal predicted mode is more than twice the cost of the optimal MIP mode, fusion enhancement is not required.

[0174] Finally, if fusion enhancement is required, the encoder obtains a prediction block corresponding to the optimal MIP mode and a prediction block corresponding to the suboptimal MIP mode according to the optimal MIP mode, the suboptimal MIP mode, the transposition information of the optimal MIP mode, and the transposition information of the suboptimal MIP mode. Specifically, the encoder first downsamples the reconstructed samples adjacent to the upper and left sides of the current block as needed, splices them according to the transposition information to obtain an input vector, reads the matrix coefficients under the current mode using the MIP mode as an index, and then calculates the input vector and the matrix coefficients to obtain an output vector. The encoder transposes the output according to the transposition information, and upsamples the output vector according to the size of the current block and the number of samples of the output vector to obtain an optimal MIP prediction block and a suboptimal MIP prediction block of the same size as the current block, and performs a weighted average on the optimal MIP prediction block and the suboptimal MIP prediction block according to the calculated weight values ​​of the optimal MIP mode and the suboptimal MIP mode to obtain a new prediction block as the final prediction block of the current block. If no fusion enhancement is required, the encoder can calculate the optimal MIP prediction block based on the optimal MIP mode and its transposition information, and the calculation process is the same as that described above. Finally, the encoder takes the optimal MIP prediction block as the prediction block of the current block. Furthermore, the encoder obtains the rate-distortion cost of the current block and denote it as cost1.

[0175] Step 3: The encoder continues to traverse other intra prediction techniques and calculates the corresponding rate-distortion costs, denoted as cost2...costN.

[0176] Step 4: If cost1 is the smallest among all rate-distortion costs, use the TMMIP technique for the current block, and the encoder sets the TMMIP usage flag of the current block to true and writes it into the bitstream. If cost1 is not the smallest rate-distortion cost, use another intra prediction technique for the current block, and the encoder sets the TMMIP usage flag of the current block to false and writes it into the bitstream. Note that information such as the flag or index of the other intra prediction technique is transmitted based on the definition and is not detailed here.

[0177] Step 5: The encoder determines the residual block of the current block according to the prediction block of the current block and the original block of the current block, and performs operations such as transform and quantization, entropy coding, and loop filtering on the residual block of the current block. For the specific process, please refer to the above related content, and in order to avoid duplication, it will not be described in detail here.

[0178] The relevant scheme of the decoder in this embodiment is described below.

[0179] The decoder analyzes the block-level flag. If intra mode is used for the current block, the decoder analyzes or obtains the sequence-level enable flag, such as sps_tmmip_enable_flag. The sequence-level enable flag is used to indicate whether the template matching-based MIP mode derivation technique is allowed to be used for the current sequence. If all of the tmmip enable flags are true, it indicates that the decoder is currently allowed to use the TMMIP technique.

[0180] Exemplarily, the decoder process can be realized as the following process.

[0181] Step 1: If sps_tmmip_enable_flag is true, the decoder parses the TMMIP usage flag of the current block. Otherwise, the current decoding process does not require the block-level TMMIP usage flag to be decoded, and the block-level TMMIP usage flag is defaulted to false. If the TMMIP usage flag of the current block is true, step 2 is performed. Otherwise, step 3 is performed.

[0182] Step 2: First, the decoder performs reconstructed sample padding on adjacent rows and columns outside the template region. The padding process is the same as the padding method of the original intra prediction process. For example, the decoder can traverse from the lower left corner to the upper right corner to pad. If all reconstructed samples are available, pad with all available reconstructed samples in order. If all reconstructed samples are unavailable, pad with all average values. If some reconstructed samples are available, pad with available reconstructed samples first, and for the remaining unavailable reconstructed samples, the decoder traverses from the lower left corner to the upper right corner in order until the first available reconstructed sample appears, and uses the first available reconstructed sample to pad the previous unavailable position. The decoder then takes the reconstructed samples outside the padded template region as input and predicts samples inside the template region using any MIP mode that is allowed to be used.

[0183] Exemplarily, for a block of size 4x4, there are 16 MIP modes that are allowed to be used. For a block of width or height equal to 4, or a block of size 8x8, there are 8 MIP modes that are allowed to be used. For blocks of other sizes, there are 6 MIP modes that are allowed to be used. Also, for blocks of any size, the MIP transpose function can be used, and the TMMIP prediction modes are the same as the MIP technique.

[0184] Exemplarily, a specific prediction calculation process includes the following contents. First, the decoder performs Haar downsampling on the reconstructed sample, for example, the decoder determines the downsampling step size based on the block size. Next, the decoder adjusts the splice order of the reconstructed sample after downsampling on the upper side and the reconstructed sample after downsampling on the left side according to the information on whether to transpose. If transposition is not required, the reconstructed sample after downsampling on the left side is spliced ​​after the reconstructed sample after downsampling on the upper side, and the resulting vector is input. If transposition is required, the reconstructed sample after downsampling on the upper side is spliced ​​after the reconstructed sample after downsampling on the left side, and the resulting vector is input. Next, the decoder obtains MIP matrix coefficients using the traversed prediction mode as an index, and obtains an output vector by calculating the MIP matrix coefficients and the input. Finally, the decoder upsamples the output vector according to the number of samples of the output vector and the size of the current template. If upsampling is not required, the vectors are arranged in order according to the horizontal direction and output as a predicted block in the template area. If upsampling is required, first upsample according to the horizontal direction, then downsample according to the vertical direction, and so on until the size is the same as the template, and output the predicted block within the template region.

[0185] Then, the decoder calculates a distortion cost based on the predicted block in the template region and the reconstructed sample in the template region obtained by traversing each MIP mode, and records the distortion cost under each prediction mode and transposition information. After traversing all the allowed prediction modes and transposition information, the decoder selects an optimal MIP mode and its corresponding transposition information and a suboptimal MIP mode and its corresponding transposition information according to the principle of minimum cost. The decoder determines whether fusion enhancement is required based on the relationship between the cost of the optimal MIP mode and the cost of the suboptimal MIP mode. If the cost of the suboptimal MIP mode is less than twice the cost of the optimal MIP mode, the optimal MIP predicted block and the suboptimal MIP predicted block need to be fusion enhanced. If the cost of the suboptimal predicted mode is more than twice the cost of the optimal MIP mode, fusion enhancement is not required.

[0186] Finally, if fusion enhancement is required, the decoder obtains a prediction block corresponding to the optimal MIP mode and a prediction block corresponding to the suboptimal MIP mode according to the optimal MIP mode, the suboptimal MIP mode, the transposition information of the optimal MIP mode, and the transposition information of the suboptimal MIP mode. Specifically, the decoder first downsamples the reconstructed samples adjacent to the upper and left sides of the current block as needed, and splices them according to the transposition information to obtain an input vector, and reads the matrix coefficients under the current mode using the MIP mode as an index, and then obtains an output vector by calculating the input vector and the matrix coefficients. The decoder transposes the output according to the transposition information, and upsamples the output vector according to the size of the current block and the number of samples of the output vector to obtain an optimal MIP prediction block and a suboptimal MIP prediction block of the same size as the current block, and performs a weighted average on the optimal MIP prediction block and the suboptimal MIP prediction block according to the calculated weight values ​​of the optimal MIP mode and the suboptimal MIP mode to obtain a new prediction block as the final prediction block of the current block. If no fusion enhancement is required, the decoder can calculate the optimal MIP prediction block based on the optimal MIP mode and its transposition information, and the calculation process is the same as that described above. Finally, the decoder takes the optimal MIP prediction block as the prediction block of the current block.

[0187] Step 3: The decoder continues to analyze information such as the use flag or index of other intra prediction techniques, and obtains the final prediction block of the current block according to the analyzed information.

[0188] Step 4: The decoder parses the bitstream to obtain a frequency domain residual block (also called frequency domain residual information) of the current block, and performs inverse quantization and inverse transform on the frequency domain residual block of the current block to obtain a residual block (also called time domain residual block or time domain residual information) of the current block. Then, the decoder adds the prediction block of the current block to the residual block of the current block to obtain a reconstructed sample block.

[0189] Step 5: After techniques such as loop filtering are performed on all reconstructed sample blocks in the current image, the final reconstructed image is obtained.

[0190] Optionally, the reconstructed image may be the video output or may serve as a reference for subsequent decoding.

[0191] In this embodiment, the size of the template region used by the encoder or decoder in the TMMIP technique can be predefined according to the size of the current block. For example, the width of the region adjacent to the top of the current block in the template region is equal to the width of the current block, and its height is equal to the height of two rows of samples. The height of the region adjacent to the left of the current block in the template region is equal to the height of the current block, and its width is equal to the width of two rows of samples. Of course, other alternative embodiments can be implemented as template regions of other sizes, and the present application is not specifically limited thereto.

[0192] <Example 2>

[0193] In this embodiment, the first intra prediction mode is an intra prediction mode derived from a TIMD mode, that is, the encoder or decoder can perform intra prediction on the current block according to the optimal MIP mode and the intra prediction mode derived from the TIMD mode to obtain a predicted block of the current block.

[0194] That is, the template matching-based MIP mode derivation fusion enhancement technique can not only fuse two derived MIP prediction blocks, but also fuse with prediction blocks generated by other template matching-based derivation techniques. In this application, the TMMIP technique and the TIMD technique are fused to obtain a method for fusing a derived conventional prediction block with a matrix-based prediction block. In the TIMD technique, the coding side utilizes the idea of ​​template matching to derive an optimal conventional intra prediction mode, and the TIMD technique can also perform offset expansion on the prediction mode to obtain an updated intra prediction mode. In addition, the TMMIP technique utilizes the idea of ​​template matching on the coding side to derive an optimal MIP mode. By fusing these two optimal prediction modes, the directionality of the conventional prediction block and the unique texture characteristics of the MIP prediction can be compatible, and a completely new prediction block can be generated, thereby improving the coding efficiency.

[0195] The encoder traverses the prediction modes. If intra mode is used for the current block, the encoder gets a sequence-level enable flag such as sps_tmmip_enable_flag. The sequence-level enable flag is used to indicate whether the template matching-based MIP mode derivation technique is allowed to be used for the current sequence. If any of the tmmip enable flags is true, it indicates that the encoder is currently allowed to use the TMMIP technique.

[0196] Exemplarily, the process of the encoder can be realized as the following process.

[0197] Step 1: If sps_tmmip_enable_flag is true, the encoder tries the TMMIP technique, i.e., executes step 2. If sps_tmmip_enable_flag is false, the encoder does not try the TMMIP technique, i.e., skips step 2 and executes step 3 directly.

[0198] Step 2: First, the encoder performs reconstructed sample padding on adjacent rows and columns outside the template region. The padding process is the same as the padding method of the original intra prediction process. For example, the encoder can traverse from the lower left corner to the upper right corner to pad. If all reconstructed samples are available, pad with all available reconstructed samples in order. If all reconstructed samples are unavailable, pad with all average values. If some reconstructed samples are available, pad with available reconstructed samples first, and for the remaining unavailable reconstructed samples, the encoder traverses from the lower left corner to the upper right corner in order until the first available reconstructed sample appears, and uses the first available reconstructed sample to pad the previous unavailable position. The encoder then takes the reconstructed samples outside the padded template region as input and predicts samples inside the template region using any MIP mode that is allowed to be used.

[0199] Exemplarily, for a block of size 4x4, there are 16 MIP modes that are allowed to be used. For a block of width or height equal to 4, or a block of size 8x8, there are 8 MIP modes that are allowed to be used. For blocks of other sizes, there are 6 MIP modes that are allowed to be used. Also, for blocks of any size, the MIP transpose function can be used, and the TMMIP prediction modes are the same as the MIP technique.

[0200] Exemplarily, a specific prediction calculation process includes the following contents. First, the encoder performs Haar downsampling on the reconstructed sample, for example, the encoder determines the downsampling step size based on the block size. Then, the encoder adjusts the splice order of the upper downsampled reconstructed sample and the left downsampled reconstructed sample according to the information on whether to transpose. If transposition is not required, the left downsampled reconstructed sample is spliced ​​after the upper downsampled reconstructed sample, and the resulting vector is input. If transposition is required, the upper downsampled reconstructed sample is spliced ​​after the left downsampled reconstructed sample, and the resulting vector is input. Next, the encoder obtains MIP matrix coefficients using the traversed prediction mode as an index, and obtains an output vector by calculating the MIP matrix coefficients and the input. Finally, the encoder upsamples the output vector according to the number of samples of the output vector and the size of the current template. If upsampling is not required, the vectors are arranged in order according to the horizontal direction and output as a prediction block in the template area. If upsampling is required, first upsample according to the horizontal direction, then downsample according to the vertical direction, and so on until the size is the same as the template, and output the predicted block within the template region.

[0201] In addition, the encoder needs to try the template matching calculation process of TIMD, obtain different interpolation filters based on different prediction mode indexes, and obtain predicted samples in the template by interpolating the reference samples.

[0202] Next, the encoder calculates distortion costs based on the prediction samples in the template region and the reconstructed samples in the template region obtained by traversing each MIP mode, and records the distortion costs under each prediction mode and transposition information. Based on the distortion costs under each prediction mode and transposition information, the encoder selects an optimal MIP mode and its corresponding transposition information according to the principle of least cost. In addition, the encoder needs to traverse all allowed intra prediction modes in TIMD, calculate prediction samples in the template, calculate distortion costs using the prediction samples in the template and the reconstructed samples in the template, and record the optimal prediction mode, suboptimal prediction mode, distortion cost corresponding to the optimal prediction mode, and distortion cost corresponding to the suboptimal prediction mode derived from the TIMD technology according to the principle of least cost.

[0203] Finally, the encoder may downsample the reconstructed samples adjacent to the upper and left sides of the current block according to the obtained optimal MIP mode and transposition information, splice them according to the transposition information to obtain an input vector, read the matrix coefficients under the current mode using the MIP mode as an index, and then obtain an output vector by calculating the input vector and the matrix coefficients. The encoder may transpose the output according to the transposition information, and upsample the output vector according to the size of the current block and the number of samples of the output vector to obtain an output with the same size as the current block, which is the optimal MIP prediction block of the current block.

[0204] For the optimal prediction mode and the suboptimal prediction mode derived from the TIMD technology, if the optimal prediction mode and the suboptimal prediction mode are neither a DC mode nor a planar mode, and the distortion cost corresponding to the suboptimal prediction mode is less than twice the distortion cost corresponding to the optimal prediction mode, the encoder needs to fuse the prediction blocks. First, the encoder obtains an interpolation filtering coefficient based on the optimal prediction mode, and performs an interpolation filtering on the reconstructed samples adjacent to the upper side and the left side to obtain prediction samples at all positions in the current block, which is marked as an optimal prediction block. Next, the encoder obtains an interpolation filtering coefficient based on the suboptimal prediction mode, and performs an interpolation filtering on the reconstructed samples adjacent to the upper side and the left side to obtain prediction samples at all positions in the current block, which is marked as a suboptimal prediction block. Furthermore, the encoder uses a ratio between the cost corresponding to the optimal prediction mode and the cost corresponding to the suboptimal prediction mode to calculate a weight value of the optimal prediction block and a weight value of the suboptimal prediction block. Finally, the encoder performs a weighted fusion of the optimal prediction block and the suboptimal prediction block to obtain a prediction block of the current block as an output. In addition, if the optimal prediction mode or the suboptimal prediction mode is average mode (DC) or planar mode (PLANAR), or if the cost corresponding to the suboptimal prediction mode is greater than twice the cost corresponding to the optimal prediction mode, the encoder does not need to fuse prediction blocks, and the optimal prediction block obtained by performing interpolation filtering on the adjacent reconstructed samples above and to the left using only the optimal prediction mode is determined to be the optimal TIMD prediction block for the current block.

[0205] Finally, the encoder performs a weighted average on the optimal MIP prediction block and the optimal TIMD prediction block based on the calculated optimal MIP mode weight value and the prediction mode weight value derived from the TIMD technique to obtain a new prediction block, which is the prediction block of the current block. Furthermore, the encoder obtains the rate-distortion cost of the current block and denotes it as cost1.

[0206] In addition, the template region in the TIMD technology and the template region in the TMMIP technology can be set to be the same, that is, the region for calculating the distortion cost corresponding to the template region is the same, so the cost information of the template region in the two technologies can be equivalent and are at the same comparison level. In this case, whether or not fusion enhancement is performed can be determined based on the cost information, which is not specifically limited in the present application.

[0207] Step 3: The encoder continues to traverse other intra prediction techniques and calculates the corresponding rate-distortion costs, denoted as cost2...costN.

[0208] Step 4: If cost1 is the smallest among all rate-distortion costs, use the TMMIP technique for the current block, and the encoder sets the TMMIP usage flag for the current block to true and writes it into the bitstream. If cost1 is not the smallest rate-distortion cost, use another intra prediction technique for the current block, and the encoder sets the TMMIP usage flag for the current block to false and writes it into the bitstream. Note that information such as the flag or index of the other intra prediction technique is transmitted based on the definition and will not be detailed here.

[0209] Step 5: The encoder determines the residual block of the current block according to the prediction block of the current block and the original block of the current block, and performs operations such as transform and quantization, entropy coding, and loop filtering on the residual block of the current block. For the specific process, please refer to the above related content, and in order to avoid duplication, it will not be described in detail here.

[0210] The relevant scheme of the decoder in this embodiment is described below.

[0211] The decoder analyzes the block-level flag. If intra mode is used for the current block, the decoder analyzes or obtains the sequence-level enable flag, such as sps_tmmip_enable_flag. The sequence-level enable flag is used to indicate whether the template matching-based MIP mode derivation technique is allowed to be used for the current sequence. If all of the tmmip enable flags are true, it indicates that the decoder is currently allowed to use the TMMIP technique.

[0212] Exemplarily, the decoder process can be realized as the following process.

[0213] Step 1: If sps_tmmip_enable_flag is true, the decoder parses the TMMIP usage flag of the current block. Otherwise, the current decoding process does not require the block-level TMMIP usage flag to be decoded, and the block-level TMMIP usage flag is defaulted to false. If the TMMIP usage flag of the current block is true, step 2 is performed. Otherwise, step 3 is performed.

[0214] Step 2: First, the decoder performs reconstructed sample padding on adjacent rows and columns outside the template region. The padding process is the same as the padding method of the original intra prediction process. For example, the decoder can traverse from the lower left corner to the upper right corner to pad. If all reconstructed samples are available, pad with all available reconstructed samples in order. If all reconstructed samples are unavailable, pad with all average values. If some reconstructed samples are available, pad with available reconstructed samples first, and for the remaining unavailable reconstructed samples, the decoder traverses from the lower left corner to the upper right corner in order until the first available reconstructed sample appears, and uses the first available reconstructed sample to pad the previous unavailable position. The decoder then takes the reconstructed samples outside the padded template region as input and predicts samples inside the template region using any MIP mode that is allowed to be used.

[0215] Exemplarily, for a block of size 4x4, there are 16 MIP modes that are allowed to be used. For a block with width or height equal to 4, or a block of size 8x8, there are 8 MIP modes that are allowed to be used. For blocks of other sizes, there are 6 MIP modes that are allowed to be used. Also, for blocks of any size, the MIP transpose function can be used, and the TMMIP prediction modes are the same as the MIP technique.

[0216] Exemplarily, a specific prediction calculation process includes the following contents. First, the decoder performs Haar downsampling on the reconstructed sample, for example, the decoder determines the downsampling step size based on the block size. Next, the decoder adjusts the splice order of the reconstructed sample after downsampling on the upper side and the reconstructed sample after downsampling on the left side according to the information on whether to transpose. If transposition is not required, the reconstructed sample after downsampling on the left side is spliced ​​after the reconstructed sample after downsampling on the upper side, and the resulting vector is input. If transposition is required, the reconstructed sample after downsampling on the upper side is spliced ​​after the reconstructed sample after downsampling on the left side, and the resulting vector is input. Next, the decoder obtains MIP matrix coefficients using the traversed prediction mode as an index, and obtains an output vector by calculating the MIP matrix coefficients and the input. Finally, the decoder upsamples the output vector according to the number of samples of the output vector and the size of the current template. If upsampling is not required, the vectors are arranged in order according to the horizontal direction and output as a predicted block in the template area. If upsampling is required, first upsample according to the horizontal direction, then downsample according to the vertical direction, and so on until the size is the same as the template, and output the predicted block within the template region.

[0217] In addition, the decoder needs to try the template matching calculation process of the TIMD, obtain different interpolation filters based on different prediction mode indexes, and obtain predicted samples in the template by interpolating the reference samples.

[0218] Next, the decoder calculates distortion costs based on the predicted samples in the template region and the reconstructed samples in the template region obtained by traversing each MIP mode, and records the distortion costs under each prediction mode and transposition information. Based on the distortion costs under each prediction mode and transposition information, the decoder selects an optimal MIP mode and its corresponding transposition information according to the principle of least cost. Furthermore, the decoder needs to traverse all allowed intra prediction modes in TIMD, calculate predicted samples in the template, calculate distortion costs using the predicted samples in the template and the reconstructed samples in the template, and record the optimal prediction mode, suboptimal prediction mode, distortion cost corresponding to the optimal prediction mode, and distortion cost corresponding to the suboptimal prediction mode derived from the TIMD technology according to the principle of least cost.

[0219] Finally, the decoder may downsample the reconstructed samples adjacent to the upper and left sides of the current block according to the obtained optimal MIP mode and transposition information, splice according to the transposition information to obtain an input vector, read the matrix coefficients under the current mode using the MIP mode as an index, and then obtain an output vector by calculating the input vector and the matrix coefficients. The decoder may transpose the output according to the transposition information, and upsample the output vector according to the size of the current block and the number of samples of the output vector to obtain an output with the same size as the current block, which is the optimal MIP predicted block of the current block.

[0220] For the optimal prediction mode and the suboptimal prediction mode derived from the TIMD technology, if the optimal prediction mode and the suboptimal prediction mode are neither a DC mode nor a planar mode, and the distortion cost corresponding to the suboptimal prediction mode is less than twice the distortion cost corresponding to the optimal prediction mode, the decoder needs to fuse the prediction block. First, the decoder obtains an interpolation filtering coefficient based on the optimal prediction mode, and performs an interpolation filtering on the reconstructed samples adjacent to the upper side and the left side to obtain prediction samples at all positions in the current block, which is marked as an optimal prediction block. Next, the decoder obtains an interpolation filtering coefficient based on the suboptimal prediction mode, and performs an interpolation filtering on the reconstructed samples adjacent to the upper side and the left side to obtain prediction samples at all positions in the current block, which is marked as a suboptimal prediction block. Furthermore, the decoder uses a ratio between the cost corresponding to the optimal prediction mode and the cost corresponding to the suboptimal prediction mode to calculate a weight value of the optimal prediction block and a weight value of the suboptimal prediction block. Finally, the decoder performs a weighted fusion of the optimal prediction block and the suboptimal prediction block to obtain a prediction block of the current block as an output. In addition, if the optimal prediction mode or the suboptimal prediction mode is average mode (DC) or planar mode (PLANAR), or if the cost corresponding to the suboptimal prediction mode is greater than twice the cost corresponding to the optimal prediction mode, the decoder does not need to fuse prediction blocks, and the optimal prediction block obtained by performing interpolation filtering on the adjacent reconstructed samples above and to the left using only the optimal prediction mode is determined to be the optimal TIMD prediction block for the current block.

[0221] Finally, the decoder performs a weighted average on the optimal MIP prediction block and the optimal TIMD prediction block according to the calculated optimal MIP mode weight value and the prediction mode weight value derived from the TIMD technique to obtain a new prediction block, which is the prediction block of the current block.

[0222] Step 3: The decoder continues to analyze information such as the use flag or index of other intra prediction techniques, and obtains the final prediction block of the current block according to the analyzed information.

[0223] Step 4: The decoder parses the bitstream to obtain a frequency domain residual block (also called frequency domain residual information) of the current block, and performs inverse quantization and inverse transform on the frequency domain residual block of the current block to obtain a residual block (also called time domain residual block or time domain residual information) of the current block. Then, the decoder adds the prediction block of the current block to the residual block of the current block to obtain a reconstructed sample block.

[0224] Step 5: After techniques such as loop filtering are performed on all reconstructed sample blocks in the current image, the final reconstructed image is obtained.

[0225] Optionally, the reconstructed image may be the video output or may serve as a reference for subsequent decoding.

[0226] In this embodiment, the process of calculating the weight value for the weighted fusion of the TIMD prediction block may refer to the above description of the TIMD technology, and will not be described in detail here to avoid duplication. Furthermore, the encoder or decoder may determine whether fusion enhancement is performed based on the optimal prediction mode derived from the TIMD. For example, if the optimal prediction mode derived from the TIMD is DC mode or PLANAR mode, the encoder or decoder may not use fusion enhancement. That is, the encoder or decoder may only use the prediction block generated by the optimal MIP mode derived from the TIMD technology as the prediction block of the current block. Furthermore, the size of the template region used by the encoder or decoder in the TMMIP technology may be predefined according to the size of the current block. For example, the definition of the template region in the TMMIP technology may be consistent with or different from the definition of the template region in the TIMD technology. For example, if the width of the current block is 8 or less, the height of the region adjacent to the upper side of the current block in the template region is equal to the height of two rows of samples. Otherwise, the height is equal to the height of four rows of samples. Similarly, if the height of the current block is less than or equal to 8, the width of the region within the template region adjacent to the left of the current block is equal to the height of 2 columns of samples, otherwise the width is equal to the height of 4 columns of samples.

[0227] <Example 3>

[0228] In this embodiment, the first intra prediction mode is an intra prediction mode derived from a DIMD mode, that is, the encoder or decoder can perform intra prediction on the current block according to the optimal MIP mode and the intra prediction mode derived from the DIMD mode to obtain a predicted block of the current block.

[0229] As in the second embodiment, the TMMIP technology can be integrated with the DIMD technology for enhanced performance.

[0230] It should be noted that the prediction mode derived from the DIMD technology and the prediction mode derived from the TIMD technology are both conventional intra prediction modes, but the derivation methods are different, so the prediction modes obtained by both are not necessarily the same. In addition, the method of enhancing the fusion between the TMMIP technology and the DIMD technology is different from the method of enhancing the fusion between the TMMIP technology and the TIMD technology. For example, in the TMMIP technology and the TIMD technology, the size of the template region is generally the same, and the calculated cost information is basically SATD (also called the distortion cost based on the Hadamard transform), so in the TMMIP technology and the TIMD technology, the fusion weight can be calculated directly based on this cost information. However, the size of the template region in the DIMD technology is generally different from the size of the template region in the TMMIP technology (or the DIMD technology), and the rule of the DIMD derived prediction mode is according to the gradient amplitude value. Because the gradient amplitude value is not equivalent to the SATD cost, the weight cannot be calculated by simply referring to the scheme when fusing the TMMIP technology and the TIMD technology.

[0231] The encoder traverses the prediction modes. If intra mode is used for the current block, the encoder gets a sequence-level enable flag such as sps_tmmip_enable_flag. The sequence-level enable flag is used to indicate whether the template matching-based MIP mode derivation technique is allowed to be used for the current sequence. If any of the tmmip enable flags is true, it indicates that the encoder is currently allowed to use the TMMIP technique.

[0232] Exemplarily, the process of the encoder can be realized as the following process.

[0233] Step 1: If sps_tmmip_enable_flag is true, the encoder tries the TMMIP technique, i.e., executes step 2. If sps_tmmip_enable_flag is false, the encoder does not try the TMMIP technique, i.e., skips step 2 and executes step 3 directly.

[0234] Step 2: First, the encoder performs reconstructed sample padding on adjacent rows and columns outside the template region. The padding process is the same as the padding method of the original intra prediction process. For example, the encoder can traverse from the lower left corner to the upper right corner to pad. If all reconstructed samples are available, pad with all available reconstructed samples in order. If all reconstructed samples are unavailable, pad with all average values. If some reconstructed samples are available, pad with available reconstructed samples first, and for the remaining unavailable reconstructed samples, the encoder traverses from the lower left corner to the upper right corner in order until the first available reconstructed sample appears, and uses the first available reconstructed sample to pad the previous unavailable position. The encoder then takes the reconstructed samples outside the padded template region as input and predicts samples inside the template region using any MIP mode that is allowed to be used.

[0235] Exemplarily, for a block of size 4x4, there are 16 MIP modes that are allowed to be used. For a block with width or height equal to 4, or a block of size 8x8, there are 8 MIP modes that are allowed to be used. For blocks of other sizes, there are 6 MIP modes that are allowed to be used. Also, for blocks of any size, the MIP transpose function can be used, and the TMMIP prediction modes are the same as the MIP technique.

[0236] Exemplarily, a specific prediction calculation process includes the following contents. First, the encoder performs Haar downsampling on the reconstructed sample, for example, the encoder determines the downsampling step size based on the block size. Next, the encoder adjusts the splice order of the upper downsampled reconstructed sample and the left downsampled reconstructed sample according to the information on whether to transpose. If transposition is not required, the left downsampled reconstructed sample is spliced ​​after the upper downsampled reconstructed sample, and the resulting vector is input. If transposition is required, the upper downsampled reconstructed sample is spliced ​​after the left downsampled reconstructed sample, and the resulting vector is input. Next, the encoder obtains MIP matrix coefficients using the traversed prediction mode as an index, and obtains an output vector by calculating the MIP matrix coefficients and the input. Finally, the encoder upsamples the output vector according to the number of samples of the output vector and the size of the current template. If upsampling is not required, the vectors are arranged in order according to the horizontal direction and output as a prediction block in the template area. If upsampling is required, first upsample according to the horizontal direction, then downsample according to the vertical direction, and so on until the size is the same as the template, and output the predicted block within the template region.

[0237] In addition, the encoder uses a DIMD technique to derive an optimal intra prediction mode, i.e., derive an optimal DIMD mode, in which the gradient values ​​of the reconstructed samples in the template region are calculated based on a Sobel operator, and the gradient values ​​are transformed based on angle values ​​corresponding to different prediction modes to obtain amplitude values ​​under the corresponding prediction modes.

[0238] Next, the encoder calculates the distortion cost according to the prediction block of the template obtained by traversing each MIP mode and the reconstructed sample in the template, and records the optimal MIP mode and transposition information according to the principle of minimum cost.Furthermore, the encoder traverses all intra prediction modes that are allowed to be used, calculates the amplitude value under each intra prediction mode, and records the optimal DIMD prediction mode according to the principle of maximum amplitude.

[0239] Finally, the encoder may downsample the upper and left adjacent reconstructed samples of the current block according to the obtained optimal MIP mode and transposition information, splice them according to the transposition information to obtain an input vector, use the MIP mode as an index to read the matrix coefficients under the current mode, and then obtain an output vector by calculating the input vector and the matrix coefficients. The encoder may transpose the output according to the transposition information, and upsample the output vector according to the size of the current block and the number of samples of the output vector to obtain an output of the same size as the current block, which is the optimal MIP predicted block of the current block. Furthermore, for the optimal DIMD prediction mode, the encoder obtains the corresponding interpolation filtering coefficients, and performs interpolation filtering on the upper and left adjacent reconstructed samples to obtain predicted samples at all positions in the current block, which is noted as the optimal DIMD predicted block. For each predicted sample, the encoder performs a weighted average of the optimal MIP predicted block and the optimal DIMD predicted block according to a preset weight to obtain a new predicted block. The new predicted block is the predicted block of the current block. Furthermore, the encoder obtains the rate-distortion cost of the current block, which is noted as cost1.

[0240] Step 3: The encoder continues to traverse other intra prediction techniques and calculates the corresponding rate-distortion costs, denoted as cost2...costN.

[0241] Step 4: If cost1 is the smallest among all rate-distortion costs, use the TMMIP technique for the current block, and the encoder sets the TMMIP usage flag of the current block to true and writes it into the bitstream. If cost1 is not the smallest rate-distortion cost, use another intra prediction technique for the current block, and the encoder sets the TMMIP usage flag of the current block to false and writes it into the bitstream. Note that information such as the flag or index of the other intra prediction technique is transmitted based on the definition and is not detailed here.

[0242] Step 5: The encoder determines the residual block of the current block according to the prediction block of the current block and the original block of the current block, and performs operations such as transform and quantization, entropy coding, and loop filtering on the residual block of the current block. For the specific process, please refer to the above related content, and in order to avoid duplication, it will not be described in detail here.

[0243] The relevant scheme of the decoder in this embodiment is described below.

[0244] The decoder analyzes the block-level flag. If intra mode is used for the current block, the decoder analyzes or obtains the sequence-level enable flag, such as sps_tmmip_enable_flag. The sequence-level enable flag is used to indicate whether the template matching-based MIP mode derivation technique is allowed to be used for the current sequence. If all of the tmmip enable flags are true, it indicates that the decoder is currently allowed to use the TMMIP technique.

[0245] Exemplarily, the decoder process can be realized as the following process.

[0246] Step 1: If sps_tmmip_enable_flag is true, the decoder parses the TMMIP usage flag of the current block. Otherwise, the current decoding process does not require the block-level TMMIP usage flag to be decoded, and the block-level TMMIP usage flag is defaulted to false. If the TMMIP usage flag of the current block is true, step 2 is performed. Otherwise, step 3 is performed.

[0247] Step 2: First, the decoder performs reconstructed sample padding on adjacent rows and columns outside the template region. The padding process is the same as the padding method of the original intra prediction process. For example, the decoder can traverse from the lower left corner to the upper right corner to pad. If all reconstructed samples are available, pad with all available reconstructed samples in order. If all reconstructed samples are unavailable, pad with all average values. If some reconstructed samples are available, pad with available reconstructed samples first, and for the remaining unavailable reconstructed samples, the decoder traverses from the lower left corner to the upper right corner in order until the first available reconstructed sample appears, and uses the first available reconstructed sample to pad the previous unavailable position. The decoder then takes the reconstructed samples outside the padded template region as input and predicts samples inside the template region using any MIP mode that is allowed to be used.

[0248] Exemplarily, for a block of size 4x4, there are 16 MIP modes that are allowed to be used. For a block with width or height equal to 4, or a block of size 8x8, there are 8 MIP modes that are allowed to be used. For blocks of other sizes, there are 6 MIP modes that are allowed to be used. Also, for blocks of any size, the MIP transpose function can be used, and the TMMIP prediction modes are the same as the MIP technique.

[0249] Exemplarily, a specific prediction calculation process includes the following contents. First, the decoder performs Haar downsampling on the reconstructed sample, for example, the decoder determines the downsampling step size based on the block size. Next, the decoder adjusts the splice order of the reconstructed sample after downsampling on the upper side and the reconstructed sample after downsampling on the left side according to the information on whether to transpose. If transposition is not required, the reconstructed sample after downsampling on the left side is spliced ​​after the reconstructed sample after downsampling on the upper side, and the resulting vector is input. If transposition is required, the reconstructed sample after downsampling on the upper side is spliced ​​after the reconstructed sample after downsampling on the left side, and the resulting vector is input. Next, the decoder obtains MIP matrix coefficients using the traversed prediction mode as an index, and obtains an output vector by calculating the MIP matrix coefficients and the input. Finally, the decoder upsamples the output vector according to the number of samples of the output vector and the size of the current template. If upsampling is not required, the vectors are arranged in order according to the horizontal direction and output as a predicted block in the template area. If upsampling is required, first upsample according to the horizontal direction, then downsample according to the vertical direction, and so on until the size is the same as the template, and output the predicted block within the template region.

[0250] In addition, the decoder uses a DIMD technique to derive an optimal intra prediction mode, i.e., derive an optimal DIMD mode, in which the gradient values ​​of the reconstructed samples in the template region are calculated based on a Sobel operator, and the gradient values ​​are transformed according to angle values ​​corresponding to different prediction modes to obtain amplitude values ​​under the corresponding prediction modes.

[0251] Next, the decoder calculates the distortion cost according to the prediction block of the template obtained by traversing each MIP mode and the reconstructed sample in the template, and records the optimal MIP mode and transposition information according to the principle of minimum cost.Furthermore, the decoder traverses all intra prediction modes that are allowed to be used, calculates the amplitude value under each intra prediction mode, and records the optimal DIMD prediction mode according to the principle of maximum amplitude.

[0252] Finally, the decoder may downsample the upper and left adjacent reconstructed samples of the current block according to the obtained optimal MIP mode and transposition information, splice them according to the transposition information to obtain an input vector, use the MIP mode as an index to read the matrix coefficients under the current mode, and then obtain an output vector by calculating the input vector and the matrix coefficients. The decoder may transpose the output according to the transposition information, and upsample the output vector according to the size of the current block and the number of samples of the output vector to obtain an output of the same size as the current block, which is the optimal MIP predicted block of the current block. Furthermore, for the optimal DIMD prediction mode, the decoder obtains the corresponding interpolation filtering coefficients, and performs interpolation filtering on the upper and left adjacent reconstructed samples to obtain predicted samples at all positions in the current block, which is noted as the optimal DIMD predicted block. For each predicted sample, the decoder performs a weighted average of the optimal MIP predicted block and the optimal DIMD predicted block according to a preset weight to obtain a new predicted block. The new predicted block is the predicted block of the current block.

[0253] Step 3: The decoder continues to analyze information such as the use flag or index of other intra prediction techniques, and obtains the final prediction block of the current block according to the analyzed information.

[0254] Step 4: The decoder parses the bitstream to obtain a frequency domain residual block (also called frequency domain residual information) of the current block, and performs inverse quantization and inverse transform on the frequency domain residual block of the current block to obtain a residual block (also called time domain residual block or time domain residual information) of the current block. Then, the decoder adds the prediction block of the current block to the residual block of the current block to obtain a reconstructed sample block.

[0255] Step 5: After techniques such as loop filtering are performed on all reconstructed sample blocks in the current image, the final reconstructed image is obtained.

[0256] Optionally, the reconstructed image may be the video output or may serve as a reference for subsequent decoding.

[0257] In this embodiment, the calculation process of the optimal DIMD prediction block can refer to the above description of the DIMD technology, and will not be described in detail here to avoid duplication.Furthermore, the fusion weight of the optimal MIP prediction block and the optimal DIMD prediction block can be a preset value, for example, the optimal MIP prediction block accounts for 5 / 9, and the optimal DIMD prediction block accounts for 4 / 9.Of course, in other alternative embodiments, the fusion weight of the optimal MIP prediction block and the optimal DIMD prediction block can be other values, and this application is not specifically limited thereto.

[0258] Although the preferred embodiments of the present application have been described in detail above with reference to the accompanying drawings, the present application is not limited to the detailed contents of the above embodiments, and various simple modifications can be made to the technical solutions of the present application within the scope of the technical ideas of the present application, and all such simple modifications fall within the scope of protection of the present application. For example, the specific technical features described in the above specific embodiments may be combined by any suitable means if there is no contradiction, and various possible combinations are not described again in this application to avoid unnecessary duplication. In addition, for example, any combination between various different embodiments of the present application should be considered as disclosed in the present application as long as it does not contradict the ideas of the present application. It should be understood that in various method embodiments of the present application, the magnitude of the sequence numbers of each of the above processes does not mean the order of execution. The order of execution of each process should be determined by its function and internal logic, and should not constitute any restriction on the implementation process of the embodiments of the present application.

[0259] The method embodiment of the present application has been described in detail above. Hereinafter, the apparatus embodiment of the present application will be described in detail with reference to FIGS.

[0260] FIG. 9 is a block diagram illustrating a decoder 500 according to an embodiment of the present application.

[0261] As shown in FIG. 9, the decoder 500 may include a parsing unit 510, a prediction unit 520, and a reconstruction unit 530. The parsing unit 510 is configured to parse the bitstream to obtain a residual block of a current block in a current sequence. The prediction unit 520 is configured to determine an optimal matrix-based intra-prediction (MIP) mode for predicting the current block based on distortion costs corresponding to a plurality of MIP modes, the distortion costs corresponding to the plurality of MIP modes including distortion costs obtained by predicting samples in a template region adjacent to the current block using the plurality of MIP modes. The prediction unit 520 is configured to determine a first intra-prediction mode, where the first intra-prediction mode is determined based on distortion costs corresponding to a plurality of MIP modes and includes at least one of a sub-optimal MIP mode for predicting the current block, an intra-prediction mode derived from a decoder-side intra-mode derivation (DIMD) mode, and an intra-prediction mode derived from a template-based intra-mode derivation (TIMD) mode. The prediction unit 520 is configured to predict the current block based on the optimal MIP mode and the first intra-prediction mode to obtain a prediction block of the current block. The reconstruction unit 530 is configured to obtain a reconstructed block of the current block based on the residual block of the current block and the prediction block of the current block.

[0262] In some embodiments, the prediction unit 520 specifically: Parse the bit stream of the current sequence to obtain the first flag; When the first flag is used to indicate that it is permitted to predict an image block in the current sequence using the optimal MIP mode and the first intra prediction mode, the method is configured to determine the optimal MIP mode based on distortion costs corresponding to the multiple MIP modes.

[0263] In some embodiments, the prediction unit 520 specifically: If the first flag is used to indicate that the optimal MIP mode and the first intra prediction mode are used to predict the image block in the current sequence, parsing the bitstream to obtain a second flag; If the second flag is used to indicate that it is permitted to predict the current block using the optimal MIP mode and the first intra-prediction mode, the optimal MIP mode is determined based on distortion costs corresponding to the multiple MIP modes.

[0264] In some embodiments, the analysis unit 510 further comprises: determining an arrangement order of the plurality of MIP modes based on distortion costs corresponding to the plurality of MIP modes; determining a coding scheme to be used for an optimal MIP mode based on an arrangement order of the plurality of MIP modes; The method is configured to decode a bitstream of the current sequence based on a coding scheme used for the optimal MIP mode to obtain an index of the optimal MIP mode.

[0265] In some embodiments, the codeword length of the coding scheme used for the first n MIP modes in the ordering is smaller than the codeword length of the coding scheme used for the MIP mode following the nth MIP mode in the ordering, and / or A variable length coding scheme is used for the first n MIP modes, and a truncated binary (TB) coding scheme is used for the MIP modes following the nth MIP mode.

[0266] In some embodiments, the prediction unit 520 specifically: Predicting the current block based on an optimal MIP mode to obtain a first predicted block; predicting the current block based on the first intra-prediction mode to obtain a second predicted block; The method is configured to weight the first prediction block and the second prediction block based on the optimal MIP mode weight and the first intra-prediction mode weight to obtain a prediction block for the current block.

[0267] In some embodiments, the prediction unit 520 weights the first prediction block and the second prediction block based on the optimal MIP mode weight and the first intra-prediction mode weight before obtaining the prediction block of the current block: If the first intra prediction mode includes an intra prediction mode derived from a sub-optimal MIP mode or a TIMD mode, determining an optimal MIP mode weight and a first intra prediction mode weight based on a distortion cost corresponding to the optimal MIP mode and a distortion cost corresponding to the first intra prediction mode; If the first intra-prediction mode includes an intra-prediction mode derived from a DIMD mode, the method is further configured to determine both an optimal MIP mode weight and a weight of the first intra-prediction mode to a pre-set value.

[0268] In some embodiments, the prediction unit 520 specifically: The method is configured to predict samples in the template region based on a third flag and a plurality of MIP modes to obtain distortion costs corresponding to the plurality of MIP modes under each state of the third flag, and the third flag is used to indicate whether to transpose the input vector and the output vector corresponding to the MIP mode. The prediction unit 520 is configured to determine an optimal MIP mode based on distortion costs corresponding to the multiple MIP modes under each state of the third flag.

[0269] In some embodiments, the prediction unit 520 specifically: It is configured to determine an optimal MIP mode based on distortion costs corresponding to the multiple MIP modes when the size of the current block is a preset size.

[0270] In some embodiments, the prediction unit 520 specifically: When the image frame in which the current block is located is an I frame and the size of the current block is a preset size, the method is configured to determine an optimal MIP mode based on distortion costs corresponding to a plurality of MIP modes.

[0271] In some embodiments, the prediction unit 520 specifically: If the image frame in which the current block is located is a B frame, it is configured to determine an optimal MIP mode based on distortion costs corresponding to the multiple MIP modes.

[0272] In some embodiments, the prediction unit 520 may, before determining an optimal MIP mode for predicting the current block based on distortion costs corresponding to the multiple MIP modes, perform: Get the MIP mode used for the neighboring blocks adjacent to the current block, The method is further configured to determine the MIP mode used for the neighboring blocks to a plurality of MIP modes.

[0273] In some embodiments, the prediction unit 520 may, before determining an optimal MIP mode for predicting the current block based on distortion costs corresponding to the multiple MIP modes, perform: performing reconstruction sample padding on the reference region adjacent to the outside of the template region to obtain a reference row and a reference column of the template region; Using the reference row and the reference column as input, predicting samples in the template region using each of a plurality of MIP modes to obtain a plurality of predicted blocks corresponding to the plurality of MIP modes; The method is further configured to determine distortion costs corresponding to the plurality of MIP modes based on the plurality of predicted blocks and the reconstructed block in the template region.

[0274] In some embodiments, the prediction unit 520 specifically: Downsample the reference row and the reference column to obtain the input vector, Taking an input vector as input, predicting samples in the template region by traversing a plurality of MIP modes to obtain an output vector corresponding to the plurality of MIP modes; The method is further configured to upsample the output vector corresponding to the multiple MIP modes to obtain a prediction block corresponding to the multiple MIP modes.

[0275] In some embodiments, the prediction unit 520 specifically: The method is configured to determine an optimal MIP mode based on a sum of absolute difference transforms (SATD) corresponding to a plurality of MIP modes within a template region.

[0276] FIG. 10 is a block diagram illustrating an encoder 600 according to an embodiment of the present application.

[0277] As shown in FIG. 10, the encoder 600 may include a prediction unit 610, a residual unit 620, and an encoding unit 630. The prediction unit 610 is configured to determine an optimal matrix-based intra-prediction (MIP) mode for predicting a current block in a current sequence based on distortion costs corresponding to a plurality of MIP modes, the distortion costs corresponding to the plurality of MIP modes including distortion costs obtained by predicting samples in a template region adjacent to the current block using the plurality of MIP modes. The prediction unit 610 is configured to determine a first intra-prediction mode, where the first intra-prediction mode is determined based on distortion costs corresponding to a plurality of MIP modes and includes at least one of a sub-optimal MIP mode for predicting the current block, an intra-prediction mode derived from a decoder-side intra-mode derivation (DIMD) mode, and an intra-prediction mode derived from a template-based intra-mode derivation (TIMD) mode. The prediction unit 610 is configured to predict the current block based on the optimal MIP mode and the first intra-prediction mode to obtain a prediction block of the current block. Residual unit 620 is configured to obtain a residual block for the current block based on the predicted block for the current block and the original block for the current block. The encoding unit 630 is configured to encode the residual block of the current block to obtain a bitstream of the current sequence.

[0278] In some embodiments, the prediction unit 610 specifically: Get the first flag, When the first flag is used to indicate that it is permitted to predict an image block in the current sequence using the optimal MIP mode and the first intra prediction mode, the method is configured to determine the optimal MIP mode based on distortion costs corresponding to the multiple MIP modes. Specifically, the encoding unit 630: The method is configured to encode the residual block and the first flag of the current block to obtain a bitstream.

[0279] Predicting the current block based on the optimal MIP mode and the first intra prediction mode to obtain the predicted block of the current block includes: In some embodiments, the prediction unit 610 specifically: If the first flag is used to indicate that predicting an image block in the current sequence using the optimal MIP mode and the first intra-prediction mode is permitted, predicting the current block based on the optimal MIP mode and the first intra-prediction mode to obtain a first rate-distortion cost; predicting the current block based on at least one intra prediction mode to obtain at least one rate-distortion cost; If the first rate-distortion cost is less than or equal to the minimum value of the at least one rate-distortion cost, a prediction block obtained by predicting the current block based on the optimal MIP mode and the first intra-prediction mode is determined to be the prediction block of the current block. Specifically, the encoding unit 630: The residual block, the first flag, and the second flag of the current block are encoded to obtain a bitstream. If the first rate-distortion cost is less than or equal to the minimum of the at least one rate-distortion cost, the second flag is used to indicate that predicting the current block using the optimal MIP mode and the first intra-prediction mode is permitted, and if the first rate-distortion cost is greater than the minimum of the at least one rate-distortion cost, the second flag is used to indicate that predicting the current block using the optimal MIP mode and the first intra-prediction mode is not permitted.

[0280] In some embodiments, the encoding unit 630 specifically: determining an arrangement order of the plurality of MIP modes based on distortion costs corresponding to the plurality of MIP modes; determining a coding scheme to be used for an optimal MIP mode based on an arrangement order of the plurality of MIP modes; The method is configured to encode a residual block of the current block, and encode an index of the optimal MIP mode based on a coding scheme used for the optimal MIP mode to obtain a bitstream.

[0281] In some embodiments, the codeword length of the coding scheme used for the first n MIP modes in the ordering is smaller than the codeword length of the coding scheme used for the MIP mode following the nth MIP mode in the ordering; and / or A variable length coding scheme is used for the first n MIP modes, and a truncated binary (TB) coding scheme is used for the MIP modes following the nth MIP mode.

[0282] In some embodiments, the prediction unit 610 specifically: Predicting the current block based on an optimal MIP mode to obtain a first predicted block; predicting the current block based on the first intra-prediction mode to obtain a second predicted block; The method is configured to weight the first prediction block and the second prediction block based on the optimal MIP mode weight and the first intra-prediction mode weight to obtain a prediction block for the current block.

[0283] In some embodiments, the prediction unit 610 weights the first prediction block and the second prediction block based on the optimal MIP mode weight and the first intra-prediction mode weight before obtaining the prediction block of the current block: If the first intra prediction mode includes an intra prediction mode derived from a sub-optimal MIP mode or a TIMD mode, determining an optimal MIP mode weight and a first intra prediction mode weight based on a distortion cost corresponding to the optimal MIP mode and a distortion cost corresponding to the first intra prediction mode; If the first intra-prediction mode includes an intra-prediction mode derived from a DIMD mode, the method is further configured to determine both an optimal MIP mode weight and a weight of the first intra-prediction mode to a pre-set value.

[0284] In some embodiments, the prediction unit 610 is specifically configured to predict samples in the template region based on a third flag and a plurality of MIP modes to obtain distortion costs corresponding to the plurality of MIP modes under each state of the third flag, and the third flag is used to indicate whether to transpose the input vector and the output vector corresponding to the MIP mode. The prediction unit 610 is configured to determine an optimal MIP mode based on distortion costs corresponding to the multiple MIP modes under each state of the third flag.

[0285] In some embodiments, the prediction unit 610 specifically: It is configured to determine an optimal MIP mode based on distortion costs corresponding to the multiple MIP modes when the size of the current block is a preset size.

[0286] In some embodiments, the prediction unit 610 specifically: When the image frame in which the current block is located is an I frame and the size of the current block is a preset size, the method is configured to determine an optimal MIP mode based on distortion costs corresponding to a plurality of MIP modes.

[0287] In some embodiments, the prediction unit 610 specifically: If the image frame in which the current block is located is a B frame, it is configured to determine an optimal MIP mode based on distortion costs corresponding to the multiple MIP modes.

[0288] In some embodiments, the prediction unit 610 may, before determining an optimal MIP mode for predicting a current block in a current sequence based on distortion costs corresponding to the multiple MIP modes, perform: Get the MIP mode used for the neighboring blocks adjacent to the current block, The method is further configured to determine the MIP mode used for the neighboring blocks to a plurality of MIP modes.

[0289] In some embodiments, the prediction unit 610 determines an optimal MIP mode for predicting a current block in a current sequence based on distortion costs corresponding to the multiple MIP modes by: performing reconstruction sample padding on the reference region adjacent to the outside of the template region to obtain a reference row and a reference column of the template region; Using the reference row and the reference column as input, predicting samples in the template region using each of a plurality of MIP modes to obtain a plurality of predicted blocks corresponding to the plurality of MIP modes; The method is configured to determine distortion costs corresponding to a plurality of MIP modes based on the plurality of predicted blocks and the reconstructed block in the template region.

[0290] In some embodiments, the prediction unit 610 specifically: Downsample the reference row and the reference column to obtain the input vector, Taking an input vector as input, predicting samples in the template region by traversing a plurality of MIP modes to obtain an output vector corresponding to the plurality of MIP modes; The output vector corresponding to the multiple MIP modes is upsampled to obtain a prediction block corresponding to the multiple MIP modes.

[0291] In some embodiments, the prediction unit 610 specifically: The method is configured to determine an optimal MIP mode based on a sum of absolute difference transforms (SATD) corresponding to a plurality of MIP modes within a template region.

[0292] It should be noted that the apparatus embodiment may correspond to the method embodiment, and similar descriptions may refer to the method embodiment. In order to avoid repetition, the description is omitted here. Specifically, the decoder 500 shown in FIG. 9 may correspond to an entity that performs the method 300 in the embodiment of the present application. In addition, the above-mentioned and other operations and / or functions of each unit in the decoder 500 are respectively used to realize the corresponding process in each method, such as the method 300. Similarly, the encoder 600 shown in FIG. 10 may correspond to an entity that performs the method 400 in the embodiment of the present application. That is, the above-mentioned and other operations and / or functions of each unit in the encoder 600 are respectively used to realize the corresponding process in each method, such as the method 400.

[0293] Furthermore, each unit in the decoder 500 or the encoder 600 according to the embodiment of the present application may be integrated into one or several other units, or some of the units may be further divided into several smaller units in terms of function, so that the same operation can be realized without affecting the realization of the technical effect of the embodiment of the present application. The above units are divided based on logic functions. In practical applications, the function of one unit can be realized by several units, or the functions of several units can be realized by one unit. In other embodiments of the present application, the decoder 500 or the encoder 600 may include other units, and in practical applications, these functions may be realized by other units working together, or may be realized by several units working together. According to another embodiment of the present application, a decoder 500 or an encoder 600 according to an embodiment of the present application is constructed by executing a computer program (including program code) capable of executing each step of a corresponding method on a general-purpose computer device, such as a general-purpose computer having processing elements and storage elements, such as a central processing unit (CPU), a random access memory (RAM), and a read only memory (ROM), to realize the encoding method or the decoding method according to an embodiment of the present application. The computer program can be recorded in, for example, a computer-readable storage medium, and can be installed in an electronic device via the computer-readable storage medium and operated therein to realize the corresponding method in the embodiment of the present application.

[0294] In other words, the above-mentioned units may be realized in the form of hardware, may be realized by instructions in the form of software, or may be realized by a combination of hardware and software. Specifically, each step of the method embodiment in the embodiment of the present application can be completed by an integrated logic circuit of hardware in a processor or an instruction in the form of software. The steps of the method disclosed in the embodiment of the present application can be directly performed and completed by a hardware decoding processor, or can be performed and completed by a combination of hardware and software in a decoding processor. Alternatively, the software can be located in a mature storage medium in the technical field, such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory. The processor reads the information in the memory and completes the steps of the above-mentioned method embodiment together with the hardware of the processor.

[0295] FIG. 11 is a block diagram illustrating an electronic device 700 according to an embodiment of the present application.

[0296] As shown in FIG. 11, the electronic device 700 includes at least a processor 710 and a computer-readable storage medium 720. The processor 710 and the computer-readable storage medium 720 may be connected by a bus or other methods. The computer-readable storage medium 720 is used to store a computer program 721 including computer instructions. The processor 710 is used to execute the computer instructions stored in the computer-readable storage medium 720. The processor 710 is a calculation core and a control core of the electronic device 700, and is suitable for implementing one or more computer instructions, specifically, for implementing a process of a corresponding method or a corresponding function by loading and executing one or more computer instructions.

[0297] By way of example, processor 710 may also be referred to as a CPU. Processor 710 may include, but is not limited to, a general purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, or the like.

[0298] By way of example, the computer-readable storage medium 720 may be a high-speed RAM memory or a non-volatile memory, such as at least one magnetic disk storage device. Alternatively, the computer-readable storage medium 720 may be at least one computer-readable storage medium remote from the processor 710. Specifically, the computer-readable storage medium 720 includes, but is not limited to, a volatile memory and / or a non-volatile memory. The non-volatile memory may be a ROM, a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a RAM that functions as an external high-speed cache. By way of illustrative, but non-limiting example, various RAMs are available, such as static random access memory (static RAM, SRAM), dynamic random access memory (dynamic RAM, DRAM), synchronous dynamic random access memory (synchronous DRAM, SDRAM), double data rate synchronous dynamic random access memory (double data rate SDRAM, DDRSDRAM), enhanced synchronous dynamic random access memory (enhanced SDRAM, ESDRAM), synchronous link dynamic random access memory (synch link DRAM, SLDRAM), and direct rambus random access memory (direct rambus RAM, DRRAM).

[0299] In one embodiment, the electronic device 700 may be an encoder or encoding frame according to an embodiment of the present application. The computer-readable storage medium 720 stores a first computer instruction. The processor 710 loads and executes the first computer instruction stored in the computer-readable storage medium 720 to realize the corresponding step in the encoding method according to an embodiment of the present application. In other words, the first computer instruction in the computer-readable storage medium 720 is loaded and executed by the processor 710 to perform the corresponding step. To avoid repetition, the description is omitted here.

[0300] In one embodiment, the electronic device 700 may be a decoder or a decoding framework according to an embodiment of the present application. The computer-readable storage medium 720 stores a second computer instruction. The processor 710 loads and executes the second computer instruction stored in the computer-readable storage medium 720 to realize the corresponding step in the decoding method according to an embodiment of the present application. In other words, the second computer instruction in the computer-readable storage medium 720 is loaded and executed by the processor 710 to perform the corresponding step. To avoid repetition, the description is omitted here.

[0301] According to another aspect of the present application, in an embodiment of the present application, there is provided a coding system, the coding system including an encoder and a decoder according to the above.

[0302] According to another aspect of the present application, in an embodiment of the present application, a computer-readable storage medium (Memory) is provided. The computer-readable storage medium is a storage device in the electronic device 700, and is used to store programs and data. For example, the computer-readable storage medium may be a computer-readable storage medium 720. As can be understood, the computer-readable storage medium 720 here may include a built-in storage medium in the electronic device 700, and of course may include an extended storage medium supported by the electronic device 700. The computer-readable storage medium provides a storage space in which an operating system of the electronic device 700 is stored. Furthermore, the storage space stores one or more computer instructions suitable for being loaded and executed by the processor 710, and these computer instructions may be one or more computer programs 721 (including program codes).

[0303] According to another aspect of the present application, a computer program product or computer program is provided, which includes computer instructions, which are stored in a computer-readable storage medium, for example, the computer instructions may be a computer program 721. In this case, the data processing device 700 may be a computer, and the processor 710 reads the computer instructions from the computer-readable storage medium 720, and the processor 710 executes the computer instructions to cause the computer to perform the encoding method or the decoding method according to the various selectable schemes.

[0304] In other words, when implemented by software, all or part of the above embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the process described in the embodiments of the present application is executed, or the function described in the embodiments of the present application is realized. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by wire (e.g., coaxial cable, fiber optic, digital subscriber line (DSL), etc.) or wirelessly (e.g., infrared, radio, microwave, etc.).

[0305] It is obvious to those skilled in the art that the present application can be realized by electronic hardware or a combination of computer software and electronic hardware in combination with each exemplary unit and process step described in the embodiments disclosed herein. Whether these functions are performed by hardware or software is determined by the specific application of the technical solution and the design constraints. Those skilled in the art can realize the described functions using different methods for each specific application, but these realizations should not be considered as going beyond the scope of the present application.

[0306] Finally, the above content is only a specific embodiment of the present application, and the protection scope of the present application is not limited thereto. Any modifications or replacements that a person skilled in the art can easily think of within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be determined by the protection scope of the claims.

Claims

1. A decoding method applied to a decoder, the method comprising: Parsing the bitstream to obtain a residual block of a current block in a current sequence; determining an optimal matrix-based intra-prediction (MIP) mode for predicting the current block based on distortion costs corresponding to a plurality of MIP modes, the distortion costs corresponding to the plurality of MIP modes including distortion costs obtained by predicting samples in a template region neighboring the current block using the plurality of MIP modes; determining a first intra prediction mode, the first intra prediction mode being determined based on distortion costs corresponding to the plurality of MIP modes, and including at least one of a sub-optimal MIP mode for predicting the current block, an intra prediction mode derived from a decoder-side intra mode derivation (DIMD) mode, and an intra prediction mode derived from a template-based intra mode derivation (TIMD) mode; predicting the current block based on the optimal MIP mode and the first intra prediction mode to obtain a predicted block of the current block; obtaining a reconstructed block of the current block based on a residual block of the current block and a predicted block of the current block; Including, 4. A decoding method comprising:

2. determining an optimal MIP mode for predicting the current block based on distortion costs corresponding to the plurality of MIP modes, Parsing the bitstream of the current sequence to obtain a first flag; determining the optimal MIP mode based on distortion costs corresponding to the plurality of MIP modes, if the first flag is used to indicate that it is permitted to predict an image block in the current sequence using the optimal MIP mode and the first intra-prediction mode; Including, 2. The method of claim 1 .

3. When the first flag is used to indicate that it is permitted to predict an image block in the current sequence using the optimal MIP mode and the first intra prediction mode, determining the optimal MIP mode based on distortion costs corresponding to the plurality of MIP modes includes: parsing the bitstream to obtain a second flag if the first flag is used to indicate that predicting an image block in the current sequence using the optimal MIP mode and the first intra prediction mode is permitted; if the second flag is used to indicate that predicting the current block using the optimal MIP mode and the first intra-prediction mode is permitted, determining the optimal MIP mode based on distortion costs corresponding to the multiple MIP modes; Including, 3. The method of claim 2 .

4. The method comprises: determining an ordering of the plurality of MIP modes based on distortion costs corresponding to the plurality of MIP modes; determining a coding scheme to be used for the optimal MIP mode based on an arrangement order of the plurality of MIP modes; Decoding a bitstream of the current sequence based on a coding scheme used for the optimal MIP mode to obtain an index of the optimal MIP mode; Further comprising:

2. The method of claim 1 .

5. The codeword length of the coding scheme used for the first n MIP modes in the arrangement order is smaller than the codeword length of the coding scheme used for the MIP mode following the n-th MIP mode in the arrangement order; and / or a variable length coding scheme is used for the first n MIP modes, and a truncated binary (TB) coding scheme is used for MIP modes following the nth MIP mode; 5. The method of claim 4.

6. predicting the current block based on the optimal MIP mode and the first intra prediction mode to obtain a predicted block of the current block, predicting the current block based on the optimal MIP mode to obtain a first predicted block; predicting the current block based on the first intra-prediction mode to obtain a second predicted block; weighting the first prediction block and the second prediction block based on the optimal MIP mode weight and the first intra-prediction mode weight to obtain a prediction block of the current block; Including, 6. The method according to claim 1, wherein the first and second electrodes are arranged in a first direction.

7. Before weighting the first prediction block and the second prediction block based on the optimal MIP mode weight and the first intra prediction mode weight to obtain a prediction block of the current block, the method further comprises: if the first intra prediction mode includes an intra prediction mode derived from the sub-optimal MIP mode or the TIMD mode, determining a weight of the optimal MIP mode and a weight of the first intra prediction mode based on a distortion cost corresponding to the optimal MIP mode and a distortion cost corresponding to the first intra prediction mode; When the first intra prediction mode includes an intra prediction mode derived from the DIMD mode, determining a weight of the optimal MIP mode and a weight of the first intra prediction mode to be both preset values; Further comprising:

7. The method of claim 6.

8. determining an optimal MIP mode for predicting the current block based on distortion costs corresponding to the plurality of MIP modes, predicting samples in the template region based on a third flag and the plurality of MIP modes to obtain distortion costs corresponding to the plurality of MIP modes under each state of the third flag, the third flag being used to indicate whether to transpose input and output vectors corresponding to a MIP mode; determining the optimal MIP mode based on distortion costs corresponding to the plurality of MIP modes under each state of the third flag; Including, The method according to any one of claims 1 to 7.

9. determining an optimal MIP mode for predicting the current block based on distortion costs corresponding to the plurality of MIP modes, determining the optimal MIP mode based on distortion costs corresponding to the plurality of MIP modes when the size of the current block is a preset size; Including, The method according to any one of claims 1 to 8.

10. When a size of the current block is a preset size, determining the optimal MIP mode based on distortion costs corresponding to the plurality of MIP modes includes: determining the optimal MIP mode based on distortion costs corresponding to the plurality of MIP modes when the image frame in which the current block is located is an I frame and the size of the current block is the preset size; Including, 10. The method of claim 9.

11. determining an optimal MIP mode for predicting the current block based on distortion costs corresponding to the plurality of MIP modes, if the image frame in which the current block is located is a B frame, determining the optimal MIP mode based on distortion costs corresponding to the plurality of MIP modes; Including, The method according to any one of claims 1 to 8.

12. Prior to determining an optimal MIP mode for predicting the current block based on distortion costs corresponding to the plurality of MIP modes, the method further comprises: Obtaining a MIP mode used for a neighboring block adjacent to the current block; determining the MIP modes to be used for the adjacent blocks to be the plurality of MIP modes; Further comprising: The method according to any one of claims 1 to 11.

13. Prior to determining an optimal MIP mode for predicting the current block based on distortion costs corresponding to the plurality of MIP modes, the method further comprises: performing reconstruction sample padding on a reference region adjacent to the outside of the template region to obtain a reference row and a reference column of the template region; predicting samples in the template region using each of the plurality of MIP modes using the reference row and the reference column as input to obtain a plurality of predicted blocks corresponding to the plurality of MIP modes; determining distortion costs corresponding to the plurality of MIP modes based on the plurality of predicted blocks and reconstructed blocks within the template region; Further comprising: The method according to any one of claims 1 to 12.

14. predicting samples in the template region using each of the plurality of MIP modes using the reference row and the reference column as input to obtain a plurality of predicted blocks corresponding to the plurality of MIP modes; downsampling the reference row and the reference column to obtain an input vector; taking the input vector as an input, predicting samples in the template region by traversing the plurality of MIP modes to obtain an output vector corresponding to the plurality of MIP modes; upsampling the output vectors corresponding to the plurality of MIP modes to obtain prediction blocks corresponding to the plurality of MIP modes; Including, 14. The method of claim 13.

15. determining an optimal MIP mode for predicting the current block based on distortion costs corresponding to the plurality of MIP modes, determining the optimal MIP mode based on a sum of absolute transformed differences (SATD) corresponding to the plurality of MIP modes within the template region; Including, 15. The method according to any one of claims 1 to 14.

16. 1. An encoding method applied to an encoder, the method comprising: determining an optimal matrix-based intra-prediction (MIP) mode for predicting a current block in a current sequence based on distortion costs corresponding to a plurality of MIP modes, the distortion costs corresponding to the plurality of MIP modes including distortion costs obtained by predicting samples in a template region neighboring the current block using the plurality of MIP modes; determining a first intra prediction mode, the first intra prediction mode being determined based on distortion costs corresponding to the plurality of MIP modes, and including at least one of a sub-optimal MIP mode for predicting the current block, an intra prediction mode derived from a decoder-side intra mode derivation (DIMD) mode, and an intra prediction mode derived from a template-based intra mode derivation (TIMD) mode; predicting the current block based on the optimal MIP mode and the first intra prediction mode to obtain a predicted block of the current block; Obtaining a residual block of the current block based on the predicted block of the current block and the original block of the current block; encoding a residual block of the current block to obtain a bitstream of the current sequence; Including, 13. An encoding method comprising:

17. determining an optimal MIP mode for predicting a current block in a current sequence based on distortion costs corresponding to the plurality of MIP modes, comprising: Obtaining a first flag; and if the first flag is used to indicate that it is permitted to predict an image block in the current sequence using the optimal MIP mode and the first intra-prediction mode, determining the optimal MIP mode based on distortion costs corresponding to the multiple MIP modes; encoding a residual block of the current block to obtain a bitstream of the current sequence, encoding a residual block of the current block and the first flag to obtain the bitstream.

17. The method of claim 16.

18. predicting the current block based on the optimal MIP mode and the first intra prediction mode to obtain a predicted block of the current block, if the first flag is used to indicate that predicting an image block in the current sequence using the optimal MIP mode and the first intra prediction mode is permitted, predicting the current block based on the optimal MIP mode and the first intra prediction mode to obtain a first rate-distortion cost; predicting the current block based on at least one intra prediction mode to obtain at least one rate-distortion cost; determining a prediction block obtained by predicting the current block based on the optimal MIP mode and the first intra prediction mode as a prediction block of the current block when the first rate-distortion cost is equal to or less than a minimum value of the at least one rate-distortion cost; Including, encoding a residual block of the current block and the first flag to obtain the bitstream, encoding a residual block of the current block, the first flag, and the second flag to obtain the bitstream; If the first rate-distortion cost is less than or equal to a minimum of the at least one rate-distortion cost, the second flag is used to indicate that predicting the current block using the optimal MIP mode and the first intra-prediction mode is permitted, and if the first rate-distortion cost is greater than a minimum of the at least one rate-distortion cost, the second flag is used to indicate that predicting the current block using the optimal MIP mode and the first intra-prediction mode is not permitted.

20. The method of claim 17 .

19. encoding a residual block of the current block to obtain a bitstream of the current sequence, determining an ordering of the plurality of MIP modes based on distortion costs corresponding to the plurality of MIP modes; determining a coding scheme to be used for the optimal MIP mode based on an arrangement order of the plurality of MIP modes; encoding a residual block of the current block and encoding an index of the optimal MIP mode based on a coding scheme used for the optimal MIP mode to obtain the bitstream; Including, 17. The method of claim 16.

20. The codeword length of the coding scheme used for the first n MIP modes in the arrangement order is smaller than the codeword length of the coding scheme used for the MIP mode following the n-th MIP mode in the arrangement order; and / or a variable length coding scheme is used for the first n MIP modes, and a truncated binary (TB) coding scheme is used for MIP modes following the nth MIP mode; 20. The method of claim 19 .

21. predicting the current block based on the optimal MIP mode and the first intra prediction mode to obtain a predicted block of the current block, predicting the current block based on the optimal MIP mode to obtain a first predicted block; predicting the current block based on the first intra-prediction mode to obtain a second predicted block; weighting the first prediction block and the second prediction block based on the optimal MIP mode weight and the first intra-prediction mode weight to obtain a prediction block of the current block; Including, 21. The method according to any one of claims 16 to 20.

22. Before weighting the first prediction block and the second prediction block based on the optimal MIP mode weight and the first intra prediction mode weight to obtain a prediction block of the current block, the method further comprises: if the first intra prediction mode includes an intra prediction mode derived from the sub-optimal MIP mode or the TIMD mode, determining a weight of the optimal MIP mode and a weight of the first intra prediction mode based on a distortion cost corresponding to the optimal MIP mode and a distortion cost corresponding to the first intra prediction mode; When the first intra prediction mode includes an intra prediction mode derived from the DIMD mode, determining a weight of the optimal MIP mode and a weight of the first intra prediction mode to be both preset values; Further comprising:

22. The method of claim 21 .

23. determining an optimal MIP mode for predicting a current block in a current sequence based on distortion costs corresponding to the plurality of MIP modes, comprising: predicting samples in the template region based on a third flag and the plurality of MIP modes to obtain distortion costs corresponding to the plurality of MIP modes under each state of the third flag, the third flag being used to indicate whether to transpose input and output vectors corresponding to a MIP mode; determining the optimal MIP mode based on distortion costs corresponding to the plurality of MIP modes under each state of the third flag; Including, 23. The method according to any one of claims 16 to 22.

24. determining an optimal MIP mode for predicting a current block in a current sequence based on distortion costs corresponding to the plurality of MIP modes, comprising: determining the optimal MIP mode based on distortion costs corresponding to the plurality of MIP modes when the size of the current block is a preset size; Including, 24. The method according to any one of claims 16 to 23.

25. When a size of the current block is a preset size, determining the optimal MIP mode based on distortion costs corresponding to the plurality of MIP modes includes: determining the optimal MIP mode based on distortion costs corresponding to the plurality of MIP modes when the image frame in which the current block is located is an I frame and the size of the current block is the preset size; Including, 25. The method of claim 24.

26. determining an optimal MIP mode for predicting a current block in a current sequence based on distortion costs corresponding to the plurality of MIP modes, comprising: if the image frame in which the current block is located is a B frame, determining the optimal MIP mode based on distortion costs corresponding to the plurality of MIP modes; Including, 24. The method according to any one of claims 16 to 23.

27. Prior to determining an optimal MIP mode for predicting a current block in a current sequence based on distortion costs corresponding to the plurality of MIP modes, the method further comprises: Obtaining a MIP mode used for a neighboring block adjacent to the current block; determining the MIP modes to be used for the adjacent blocks to be the plurality of MIP modes; Further comprising:

27. The method according to any one of claims 16 to 26.

28. Prior to determining an optimal MIP mode for predicting a current block in a current sequence based on distortion costs corresponding to the plurality of MIP modes, the method further comprises: performing reconstruction sample padding on a reference region adjacent to the outside of the template region to obtain a reference row and a reference column of the template region; predicting samples in the template region using each of the plurality of MIP modes using the reference row and the reference column as input to obtain a plurality of predicted blocks corresponding to the plurality of MIP modes; determining distortion costs corresponding to the plurality of MIP modes based on the plurality of predicted blocks and reconstructed blocks within the template region; Further comprising:

28. The method according to any one of claims 16 to 27.

29. predicting samples in the template region using each of the plurality of MIP modes using the reference row and the reference column as input to obtain a plurality of predicted blocks corresponding to the plurality of MIP modes; downsampling the reference row and the reference column to obtain an input vector; taking the input vector as an input, predicting samples in the template region by traversing the plurality of MIP modes to obtain an output vector corresponding to the plurality of MIP modes; upsampling the output vectors corresponding to the plurality of MIP modes to obtain prediction blocks corresponding to the plurality of MIP modes; Including, 29. The method of claim 28.

30. determining an optimal MIP mode for predicting a current block in a current sequence based on distortion costs corresponding to the plurality of MIP modes, comprising: determining the optimal MIP mode based on a sum of absolute transformed differences (SATD) corresponding to the plurality of MIP modes within the template region; Including, 30. The method according to any one of claims 16 to 29.

31. A decoder comprising: An analysis unit, a prediction unit, and a reconstruction unit, The parsing unit is configured to parse a bitstream to obtain a residual block of a current block in a current sequence; The prediction unit is configured to determine an optimal matrix-based intra-prediction (MIP) mode for predicting the current block based on distortion costs corresponding to a plurality of MIP modes, the distortion costs corresponding to the plurality of MIP modes including distortion costs obtained by predicting samples in a template region neighboring the current block using the plurality of MIP modes; The prediction unit is configured to determine a first intra prediction mode, the first intra prediction mode being determined based on distortion costs corresponding to the plurality of MIP modes, and comprising at least one of a sub-optimal MIP mode for predicting the current block, an intra prediction mode derived from a decoder-side intra mode derivation (DIMD) mode, and an intra prediction mode derived from a template-based intra mode derivation (TIMD) mode; the prediction unit is configured to predict the current block based on the optimal MIP mode and the first intra prediction mode to obtain a prediction block of the current block; the reconstruction unit is configured to obtain a reconstructed block of the current block based on a residual block of the current block and a prediction block of the current block; A decoder characterized by:

32. An encoder comprising: A prediction unit, a residual unit, and an encoding unit, The prediction unit is configured to determine an optimal matrix-based intra-prediction (MIP) mode for predicting a current block in a current sequence based on distortion costs corresponding to a plurality of MIP modes, the distortion costs corresponding to the plurality of MIP modes including distortion costs obtained by predicting samples in a template region neighboring the current block using the plurality of MIP modes; The prediction unit is configured to determine a first intra prediction mode, the first intra prediction mode being determined based on distortion costs corresponding to the plurality of MIP modes, and comprising at least one of a sub-optimal MIP mode for predicting the current block, an intra prediction mode derived from a decoder-side intra mode derivation (DIMD) mode, and an intra prediction mode derived from a template-based intra mode derivation (TIMD) mode; the prediction unit is configured to predict the current block based on the optimal MIP mode and the first intra prediction mode to obtain a prediction block of the current block; The residual unit is configured to obtain a residual block of the current block based on a prediction block of the current block and an original block of the current block; the encoding unit is configured to encode a residual block of the current block to obtain a bitstream of the current sequence. An encoder characterized by:

33. An electronic device, A processor and a computer readable storage medium, the processor is configured to execute a computer program; The computer program is stored on the computer-readable storage medium, and when the computer program is executed by the processor, the method according to any one of claims 1 to 15 or the method according to any one of claims 16 to 30 is performed.

1. An electronic device comprising:

34. 1. A computer-readable storage medium, comprising: The computer readable storage medium is configured to store a computer program, the computer program causing a computer to perform the method according to any one of claims 1 to 15 or the method according to any one of claims 16 to 30. A computer-readable storage medium comprising:

35. 1. A computer program product comprising: The computer program product comprises computer programs / instructions which, when executed by a processor, perform the method according to any one of claims 1 to 15 or the method according to any one of claims 16 to 30.

23. A computer program product comprising:

36. A bitstream comprising: The bitstream is a bitstream in the method according to any one of claims 1 to 15 or a bitstream generated according to the method according to any one of claims 16 to 30. A bitstream comprising:

Images